Behavioral finance has exploded in popularity not just because it’s interesting—regular finance is interesting, too—but because it combines that interest factor with an abundance of what one scholar called “descriptive charm.”

There is something strangely entertaining in reading about the economic foibles of others: one part schadenfreude,  plus one part abashed recognition of one’s own past mistakes—mixed with quiet relief to find oneself  with lots of company in making those mistakes.

Plus, it gives many of us readers great stories that to tell at dinner parties. Being able to talk about the myth of the “hot hand” in basketball, or investor preferences for cash versus stock dividends, can make for a pretty entertaining evening in some circles.

Reading behavioral finance can also provides a sense of Finally Understanding How Things Work when it comes to money and markets. Learning about common cognitive errors in economic decision making, such as “anchoring,” and “availability bias,” feels like getting a peek inside the flawed machines that are our brains, making it seem possible to predict the behavior (particularly the mistakes) of others, and guard against them in oneself. In short, behavioral finance seems to offer insight and a sense of control, imposing order on what otherwise appears chaotic and unpredictable.

But a dozen years after my own personal infatuation with behavioral finance began—by devouring Richard Thaler‘s classics, Advances in Behavioral Finance and The Winner’s Curse—a number of limitations have become obvious.

All research programs have limitations, of course. But those of behavioral finance undermine its purpose: that is, to enhance understanding of financial markets and investor behavior. Others have written eloquently on this subject, particularly Daniel Beunza and David Stark, and John Y. Campbell. What I have to add to this ongoing debate boils down to two points and their consequences:

1. Failure to acknowledge the findings of the allied social sciences
One of the cardinal laws in scholarship is to acknowledge the work of others and avoid reinventing the wheel. But when you read works of behavioral finance, you’d never know that deviations from rational, self-maximizing behavior are old news in psychology, political science, sociology and anthropology. Check the references section of a behavioral finance article or book and see how many citations from those fields you can find. Chances are, there will be zero. Behavioral finance scholars generally cite each other, or work from mainstream economics and finance.
This is particularly strange since so much of contemporary behavioral finance depends on the contributions of social psychologists like Daniel Kahneman (a 2002 Nobel laureate for his work in behavioral finance) and the late Amos Tversky, as well as much older work by people like Herb Simon, who was a Professor of Political Science and Industrial Administration at various points in his career. Simon won the Nobel Prize in economics for work done half a century ago on “bounded rationality“—a concept closely tied to many of the key phenomena examined by behavioral finance, but which is virtually ignored in their publications.
While a lot of academic research speaks to a rather small group of other academics, behavioral finance is distinctive within the social sciences for restricting its scholarly conversation so tightly. This isn’t the case in the closely-related disciplines of economic sociology, neo-institutionalist political science, and economic anthropology, all of which regularly cite and engage with one another, to the enrichment of all.
In the case of behavioral finance, reluctance to acknowledge the many research interests it shares with the allied social sciences may be part of the larger project of rigid separation and boundary enforcement that has been carried out by economists since the time of Pareto. This has created what Schumpeter described as a regime of “mutual vituperation” which has kept economics and finance in a state of self-imposed incommunicado with sociology, limiting the advancement of knowledge on our shared interests. Behavioral finance, it seems, is sticking to the party line on this point.

2. A narrow, limited critique of economic theory
Cataloging the many ways humans fail to think rationally about money, investments and risk is a good start. But in most ways, behavioral finance leaves intact the problematic assumptions of traditional finance, pulling its punches, so to speak. Among the most noteworthy examples:

a. Behavioral finance remains stuck at the individual level of analysis
As in traditional finance and economics, the object of inquiry in behavioral finance is the individual—despite rafts of evidence going back decades that individuals don’t make decisions about money, risk or investing in a vacuum, but as a result of social influences. Of course, this evidence comes from those allied social sciences that are being so studiously ignored. For example, economic psychologist George Katona showed 35 years ago that most people choose investments based on word of mouth recommendations from their friends and neighbors. This influence of social forces in economic decision-making has been demonstrated with equal or greater impact among finance professionals—for instance, in a study of Wall Street pension fund managers by economic anthropologists O’Barr and Conley, and more recently in a sociological study of arbitrage traders by Beunza and Stark.
In the past, there were encouraging signs that behavioral finance might break through the limitations imposed by sticking to the individual level of analysis, most strikingly in Robert Shiller’s 1993 statement that “Investing in speculative assets is a social activity.” But thus far, the implications of such statements, and the plethora of evidence supporting them, remain unexplored.

b. Behavioral finance limits itself to pointing out failures of cognition and calculation
As important as those factors are in distorting financial decision-making, there are a host of others that we know about—based on research in those allied social sciences that behavioral finance doesn’t acknowledge—that are excluded from research in behavioral finance. This includes emotions, and social phenomena like status competition, both of which play a significant role in the findings of economic sociology, psychology and anthropology. The cognitive/calculative failures may interact with the socio-emotional phenomena, but we won’t know as long as behavioral finance pretends the latter don’t exist. That’s a loss for all of us interested in markets, money and investing.

c. Behavioral finance doesn’t explain how individual acts and decisions produce aggregate outcomes
As a consequence of keeping the analytical focus on individuals—avoiding the social and interactive aspects of economic activity—behavioral finance doesn’t have the theoretical means to address mechanisms through which individual acts and decisions aggregate. That means it can’t explain institutions and other manifestations of collective behavior which form the context for all the individual behavior it examines. Of course behavioral finance can’t answer all questions about money and markets, but it ought to be able to explain what happens when hundreds, or hundreds of millions, of people fall prey to the “hot hand” fallacy or availability bias? If behavioral finance won’t touch questions like that, who will?

The consequences for ignoring the other social sciences and mounting a very narrow critique of traditional finance and economics, include:

• Limited predictive power
  Behavioral finance tells us more about what people won’t do (e.g., behave according to notions of rationality outlined in economic theory) than what they will do.

• Contradictory implications
  Are investors risk-averse or overconfident? How should we reconcile seemingly contradictory findings like these? Behavioral finance doesn’t tell us, because of its…

• Failure to offer a viable alternative to the theories it challenges
  Pointing out all the ways that real life behavior doesn’t bear out the predictions of traditional economics and finance is interesting—even fascinating, at times—but it’s not an alternative theory. “People aren’t rational” isn’t a theory: it’s an empirical observation. An alternative theory would need to offer an explanation, including causal processes, underlying mechanisms and testable propositions.

All this keeps behavioral finance dependent on traditional economics and finance rather than allowing it to grow into a robust theoretical realm in its own right. Perhaps someday the field will develop into something more truly challenging to economic orthodoxy. Until then, behavioral finance will have to play Statler and Waldorf to the Muppet Show of mainstream finance—providing entertaining critique, but not replacing the marquee acts. (Now if economics would just substitute Milton Berle for Milton Friedman….)

Can you guess the decade in which the following words were written? How about the century?

…a certain class of dishonesty, dishonesty magnificent in its proportions, and climbing into high places, has become at the same time so rampant and so splendid that men and women will be taught to feel that dishonesty, if it can become splendid, will cease to be abominable.

This quotation is from the 50s. The 1850s, that is. And the author was prolific British novelist (and postal service employee!) Anthony Trollope. The reflections on dishonesty come from his 1856 Autobiography, but he is probably best-known for his novel of greed and fraud, The Way We Live Now.

Trollope’s most famous creation is undoubtedly the flamboyant con artist Augustus Melmotte, to whom Bernard Madoff was often compared in recent years. In fact, a comparison between the two is illuminating. Like Madoff, who built his hedge fund on a “big lie,” Melmotte conjured a fortune out of smoke and mirrors, through moves like selling shares in a railway that never existed. And both parlay wealth into access to positions of social prominence and public trust—Melmotte buys a seat in Parliament, while Madoff chaired NASDAQ for several years during the 1990s. Life imitating art, or just tragedy following farce?

This kind of financial chicanery, “dishonesty magnificent in its proportions,” was very much on Trollope’s mind in 1856 because English law had recently re-legalized and expanded the rights of the joint stock company—the ancestor of the modern corporation, and a form of business enterprise so tarnished by fraud that it was outlawed for over a century in the UK.

What, you might ask, was the big deal? Why get so exercised about a change in the legal status of an organizational form?

Four words: the South Sea Bubble. Also known as the world’s first great financial fraud, the Bubble was so destructive to England’s economy and society that Parliament banned all new incorporations (with a very few exceptions) for more than a century.

The Bubble arose around the stock of the South Sea Company, an entity created by act of Parliament to trade exclusively with the Spanish colonies in South America—and, not incidentally, to rid the British Crown of a crippling national debt by raising capital through the sale of shares.

Even though, as UBC law professor Joel Bakan has written, “the South Sea Company was a scam from the very start,” it was wildly successful—until it wasn’t.

Investors flocked to buy the company’s stock, which rose dramatically, by sixfold in one year, and then quickly plummeted as shareholders, realizing that the company was worthless, panicked and sold. In 1720 — the year a major plague hit Europe, public anxiety about which “was heightened,” according to one historian, “by a superstitious fear that it had been sent as a judgment on human materialism” — the South Sea Company collapsed. Fortunes were lost, lives were ruined, one of the company’s directors, John Blunt, was shot by an angry shareholder, mobs crowded Westminster, and the king hastened back to London from his country retreat to deal with the crisis. The directors of the South Sea Company were called before Parliament, where they were fined, and some of them jailed, for “notorious fraud and breach of trust.” Though one parliamentarian demanded they be sewn up in sacks, along with snakes and monies, and then drowned, they were, for the most part, spared harsh punishment. As for the corporation itself, in 1720 Parliament passed the Bubble Act, which made it a criminal offense to create a company “presuming to be a corporate body,” and to issue “transferable stocks without legal authority.”

It was possible to outlaw the corporate form of organizations because at that time, they required a  government charter to exist. If a corporation broke the law, it could be punished by revocation of its charter—sort of like the death penalty for juris persons (“legal persons,” which distinguished the status of corporate actors from that of “natural persons,” i.e., human beings).

This tight legal control was driven by deep suspicion of the corporate form which long predated the South Sea Bubble. In contrast to business partnerships, where the managers were also the owners of the firm, the corporation innovated by separating management from ownership, with the latter belonging to shareholders. Even Adam Smith thought this was a bad idea, warning in The Wealth of Nations that

The directors of such companies, however, being the managers rather of other people’s money rather than of their own, it cannot well be expected that they should watch over it with the same anxious vigilance with which the partners in a private copartnery would watch over their own….Negligence and profusion, therefore, must always prevail, more or less, in the management of the affairs of such a company…They have, accordingly, very seldom succeeded without an exclusive privilege, and frequently have not succeeded with one (1776: 439).

In other words, the corporate form had trouble written all over it. Specifically, it presented a built-in problem of moral hazard by given managers authority over “other people’s money.” Among other things, this created tempting conditions for financial fraud.

All these suspicions were borne out and more by the South Sea Bubble. But by the time of Trollope’s reflections, the memory of the disaster was fading, and the Industrial Revolution was generating tremendous wealth. By 1856, Parliament had not only repealed the Bubble Act, but expanded the powers of juris persons, clearing the way for super-charged economic growth—and for a return to the “magnificent” frauds perpetrated by corporations.

Yet, as Bakan writes, the corporate frauds that have come to light since the re-legalization of the corporation have not provoked anything remotely like the “Bubble Act” in response:

Today, in the wake of corporate scandals similar to and every bit as nefarious as the South Sea bubble, it is unthinkable that a government would ban the corporate form…over the last three hundred years, corporations have amassed such great power as to weaken government’s ability to control them. A fledgling institution that could be banned with the stroke of a legislative pen in 1720, the corporation now dominates society and government.

To Bakan’s power-based explanation—that the power of the corporation has simply outgrown that of the state—we might add Trollope’s notion of public tolerance for fraud in high places. That is, the state doesn’t crack down on corporations because it lacks the popular mandate to do so.

Indeed, the platform of Tea Party—a new political group predicted to win an influential number of Congressional seats in the upcoming midterm elections—seems to validate Trollope’s pessimism: the Tea Partiers want to repeal even the relatively limited regulatory reforms that were made in the wake of the 2008 subprime mortgage crisis. Because the answer to a crisis in which fraud flourished in the absence of proper oversight is…even less oversight!

Trollope would probably have appreciated John Kenneth Galbraith’s rueful observation that “There can be few fields of human endeavor in which history counts for so little as in the world of finance.”