Greenmail is the use of the threat of a hostile takeover to coerce a company into buying off a shareholder at a premium.
The use of greenmail is fairly straightforward: the buying of a stake that is large enough to be a takeover threat, without triggering any requirement to make a mandatory offer. The company is then offered an opportunity to buy back the greenmailer's share at a premium, or to otherwise pay off the greenmailer.
Greenmail has virtually disappeared in well regulated markets as the combinations of stricter takeover rules, activism by other shareholders at whose expense the greenmailer gains, and, in some countries, specific anti-greenmail regulation.
Buying off a greenmailer essentially lets the management keep their jobs by buying off the “raider” at the expense of other shareholders — especially given that hostile takeovers are, at least in part, a mechanism for removing poor management.
The impact of greater regulation has been positive with the focus, including that of former greenmailers, shifting to shareholder activism that benefits all shareholders. Rather than seeking to be bought off, a large shareholding can instead be used as a base to demand changes in management, the selling of assets (or even a complete break-up of a group) to release shareholder value better treatment for minority shareholders, etc.