The Replication Crisis and ESG’s Throne of Lies

“The trouble with our liberal friends is not that they’re ignorant; it’s just that they know so much that isn’t so.” -Ronald Reagan, “A Time for Choosing”

For more than two decades now, a handful of academics, led by John Ioannidis, a professor of medicine at Stanford University, have been documenting what has come to be called “the replication crisis.” In brief, a considerable amount of academic scientific research fails one of the key tests of the scientific method: it cannot be reproduced by other researchers.

In 2023, four German neuropsychologists attempted to quantify the incidence of “fraud” in the scientific research business, and what they found was disheartening to say the least. As Jeffrey Brainard at Science magazine put it, “After screening some 5000 papers, [lead researcher Bernhard Sabel] estimates up to 34% of neuroscience papers published in 2020 were likely made up or plagiarized; in medicine, the figure was 24%.”

Even allowing some margin of error in Sabel’s methodology, those are shocking numbers.

For obvious reasons, this is a massive scandal in the scientific community. In the hard sciences—medicine and neuroscience, especially—academic malfeasance can have serious real-world implications. If this research is fraudulent, if it is not replicable and, therefore, not “scientific” at all, then the consequences can be quite serious, even deadly.

A similar condition exists in the social sciences as well and is likely even more prevalent. Nevertheless, this “crisis” is hardly even referred to as such and has received far less public attention. This is largely the result of the perception that social sciences don’t matter as much as hard sciences, and therefore, research malfeasance isn’t as big of a scandal.

In many cases, this perception is true. In an academic universe where “publish or perish” remains dogma and which strives relentlessly to “create” “new” knowledge, the results of research are often absolutely irrelevant except in the battle for promotion and tenure.

In other cases, however, that perception is false, and the lack of attention paid to the replication crisis has had widespread deleterious effects on markets, businesses, the economy, and society more broadly. Indeed, the accumulated evidence suggests that the irreproducible research has strongly influenced the rise over the last decade of ESG—environmental, social, and governance—investing.

Consider, for example, a 2014 paper titled “The Impact of Corporate Sustainability on Organizational Processes and Performance,” written by Robert Eccles, Ioannis Ioannou, and George Serafeim, three extremely prominent and well-respected finance researchers. Published by the prestigious journal Management Science, this study is one of the foundational studies underpinning the practice of ESG.

Eccles, Ioannou, and Serafeim claim to have found that “high sustainability companies significantly outperform their counterparts over the long term, both in terms of stock market and accounting performance,” which, of course, suggests that sustainability is a characteristic investors should look for when choosing where to place their capital, as well as a behavioral outcome towards which all corporations should strive.

If sustainability means better returns, then it’s great for investors; if it means better accounting performance, then corporate executives have a fiduciary obligation to adopt those practices. Case closed: ESG works—or so we were told.

Andrew A. King, the Allen and Kelli Questrom Professor in Strategy and Innovation at the Questrom School of Business at Boston University, has noted that this study by Eccles and his co-authors is one of the most important and most cited studies in ESG history.

The article’s publication, King writes, “preceded a massive growth in ‘sustainable’ investing,” and it “was used to promote these new investment strategies.”  He notes that Allison Herren Lee, the former acting chair of the SEC, cited the study “to support her claim that corporate sustainability assessment has become ‘a core risk management strategy for portfolio construction.’”  It’s even been referenced by the former vice president of the United States and one of the early prophets of climate change, Al Gore.

King says that the study “quickly became a touchstone for the research community” and “has been used widely in testimony on policymaking.”  The study turned Eccles and his co-authors into ESG superstars. It made them heroes and made ESG acceptable—practically undeniable.

But there’s a catch. The study is irreproducible.

Actually, make that two catches, because it’s also wrong, as King proved and as its authors eventually had to concede (emphasis in original):

“The article helped ignite the ESG boom. … But when I replicated the study, I discovered their method did not, and could not, deliver a sufficient number of closely matched firms. … Now, in a comment to JOMSR, Eccles, Ioannou, and Serafeim acknowledge that I am correct. … With this new correction, the article’s causal claims collapse, and all its estimates become ambiguous.

“So, we now know:
 1. ESG’s most influential paper does not replicate.
 2. A “typo” caused a key result to be reported as ‘significant’ when it was ‘NOT significant.’ …
 3. Its text misrepresented its empirical method as far stronger than it was.”

The good news is that Andrew King is not alone in his fight to expose the shoddy and irreproducible research that has become part of the “legend” of ESG. Other researchers—notably Alex Edmans, a professor of Finance at London Business School—have also made it their mission to push back against poorly constructed studies that mislead and promote unproven ideas.

The bad news is that much of the damage from these irreproducible claims has already been done. The “myths” that sustainability and diversity inevitably produce better business results are embedded in certain segments of the corporate and investment worlds, despite the unreliability of the evidence supporting them.

Those of us who believe that corporations have an obligation to focus exclusively on their business missions are fighting an uphill battle. Again, the consequences of poor social science research here might not be life-or-death, as they are in the hard sciences, but they are serious.

ESG has become standard practice across Western capital markets, in Europe especially, and throughout much of American business. Trillions of dollars in capital are now allocated based on a study whose authors have conceded its central claims aren’t valid. That is very serious.

We publish a variety of perspectives. Nothing written here is to be construed as representing the views of the Daily Signal.