The second of Mandelbrot's ten heresies of finance, in The (Mis)behaviour of Markets, is that markets are much riskier than financial theory suggests. In discussing this Mandelbrot notes that "people intuitively the market is very risky". This explains why real life asset allocation seems unnecessarily conservative by standard measures, and why the equity risk premium is so big.
This actually suggests that the market is more efficient in performing asset allocation than it would be if it followed modern portfolio theory. Of course, I am talking here about efficiency in the sense of the efficient frontier, but this is closely linked market efficiency, and it does not seem plausible that a market in which portfolios are (in aggregate) inefficient could be itself efficient.
So, how could markets actually be more efficient than the best financial theory available to investors? Is the invisible hand capable of magic. The answer lies in the nature of financial markets. Despite the many flaws introduced my insider trading and other distortions, capital markets are the jewel of capitalism, approaching perfection in a way that other markets very rarely do.
Investors adapt to what works. Those who persist in strategies that are too risky are likely to lose their money (or their jobs, if they are professional investors) and therefore their ability to influence market prices. Many investors copy the strategies of those who are successful (copying Warren Buffet is almost an industry in itself).
So, most money ends up being run according to strategies that work, which are those that best assess the price of securities on the basis of all available information.
An obvious objection is that these strategies are likely to be too short term, and take too little account of events that occur one in decades, such as crashes. This, in fact, corresponds with what we can observe. The effect is also ameliorated, if Mandelbrot is right, by the fractal nature of markets, which ensures similarities in the patterns of long term and short term behaviour.
There is no magic, but financial markets tend to correct their own inefficiencies.
Comments
Ravi"capital markets are the jewel of capitalism, approaching perfection in a way that other markets very rarely do."
I would clarify this by saying developed capital markets, which is obviously what you mean.
However, given the recent events perhaps it would be better to say that in the long run the markets are efficient, with poor strategies being exposed. In the short run, many a crook and half-wit traders seem to have made quite a return.