Thursday, October 8, 2015

MIke Bird — Two Fed economists tried to replicate some top economic studies — and the results are dreadful

A new economics paper is making the rounds this week, and it's got some pretty damning conclusions for the state of the whole subject.
Federal Reserve economists Andrew Chang and Phillip Li set about researching how many of the results published in top economics journals could be replicated — repeating the study and finding the same results.
They looked at 67 papers in 13 solid academic journals.
It's a shocking result — without the help of the authors, only a third of the results could be independently replicated by the researchers. Even with their help, only 49% could.
That leads them to a pretty blunt conclusion: "Because we are able to replicate less than half of the papers in our sample even with help from the authors, we assert that economics research is usually not replicable."…
Business Insider UK

5 comments:

Ignacio said...

Who needs the scientific method, not the economists!

Random said...

http://www.theguardian.com/world/2015/oct/08/pinochet-directly-ordered-washington-killing-diplomat-documents-orlando-letelier-declassified
Promote to post please.

Anonymous said...

Since by and large economists do not run experiments or surveys, shouldn't replicability be much higher than in other social sciences?

John said...

As high as a third? Are they massaging the figures to save the academic economic community's collective face?

Ryan Harris said...

Shocking. Shocking I say! Keep using those spreadsheets for analysis, Economists, top work! Ideology trumps reality.

When the data doesn't fit the ideology... something has to change, and in the case of economic and climate models, the data gets changes rather than the models! At least Climatologists report their adjustments to data. Economists don't even remember they've done it. Truly sad.

Economists that took government money or public university money should have wages and research dollars clawed back to give them nothing more than minimum wage for their faked and bunk work paid for with tax-payer dollars. In cases where they intentionally deceived to influence policy, they should serve prison time for fraud. I've no doubt that the economists would clean up their act after a few high profile economists are sent down for a few years.

I seriously blame the spreadsheets more than the economists though. The software encourages fiddling data, hiding operations, forgetting what has been done, snooping through data to find patterns all sorts of behavior that makes analysis meaningless. Much of the work done in spreadsheets is simple incompetence and negligence more than a nefarious ideological intent.