Tag: time-series data

An excellent learning exercise for students of economics is to take a journal article that uses data to model the economy and then try to replicate the authors’ results. You may well be given an assignment like this in future years of your degree.

One such exercise is used on the University of Massachusetts Amherst’s doctoral programme in economics. Thomas Herndon is a student on that degree and chose to examine a well-known and highly influential paper, Growth in a Time of Debt by Carmen Reinhart then of the University of Maryland and Kenneth Rogoff of Harvard University and former chief economist of the IMF. Professors Reinhart and Rogoff used new data on 44 countries spanning about 200 years.

A key finding of their paper, published in 2010 in the American Economic Review Papers and Proceedings, is that once a country’s government debt exceeds 90% of GDP, growth rates fall considerably: the median across countries by about 1% and the mean considerably more.

The paper has been hugely influential. It has been used to justify the austerity programmes being pursued in many countries, including the UK and the eurozone. Cutting the government deficit to GDP ratio, and ultimately the government debt to GDP ratio, has been seen as a way of achieving higher growth over the longer term, and justifies the adverse effect on short-term growth from the dampening of aggregate demand.

Well, this seemed an interesting paper for Thomas Herndon to examine, and he was keen to show just how Reinhart and Rogoff’s data led to their conclusions. But try as he might, he could not replicate their results. His initial reaction was to think he had made an error, but each time he checked he came back with the same conclusion: they must have made errors in their calculations.

His supervisor at Amherst, Professor Michael Ash, after Thomas had checked and checked again, realised that something was wrong. He encouraged Thomas to write to Reinhart and Rogoff to request sight of their dataset. They duly obliged and it was then that Thomas spotted various errors. These are explained in the articles below, but the overall effect was to alter the conclusion. Although high debt may undermine growth to some extent, the effect is much less than Reinhart and Rogoff concluded, and there are several exceptions to this rule.

On 15 April 2013, Thomas, along with his supervisor, Michael Ash and his colleague, Robert Pollin, published a response to the Reinhart and Rogoff paper. In the abstract to their paper, Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff they state that:

… coding errors, selective exclusion of available data, and unconventional weighting of summary statistics lead to serious errors that inaccurately represent the relationship between public debt and GDP growth among 20 advanced economies in the post-war period. They find that when properly calculated, the average real GDP growth rate for countries carrying a public-debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not –0:1 percent as published in Reinhart and Rogoff. That is, contrary to RR, average GDP growth at public debt/GDP ratios over 90 percent is not dramatically different than when debt/GDP ratios are lower.

The authors also show how the relationship between public debt and GDP growth varies significantly by time period and country. Overall, the evidence we review contradicts Reinhart and Rogoff’s claim to have identified an important stylized fact, that public debt loads greater than 90 percent of GDP consistently reduce GDP growth.

So could this be you in the future? Will you take a famous paper and, by re-examining and reworking the data, find that its conclusions are wrong? Could you end up changing the world? Exciting stuff!

Podcasts

Austerity: A Spreadsheet Error? BBC, More or Less, Tim Harford (20/4/13)
Austerity justification study ‘inaccurate’ BBC Today Programme, Robert Pollin (18/4/13)

Articles

UMass Student Exposes Serious Flaws in Harvard Economists’ Influential Study The Atlantic Wire, J.K. Trotter (18/4/13)
Shocking Paper Claims That Microsoft Excel Coding Error Is Behind The Reinhart-Rogoff Study On Debt Business Insider, Mike Konczal (16/4/13)
How a student took on eminent economists on debt issue – and won Economic Times of India (19/4/13)
Meet the 28-Year-Old Grad Student Who Just Shook the Global Austerity Movement New York Magazine, Kevin Roose (19/4/13)
An economist’s mea culpa: I relied on Reinhart and Rogoff Confessions of a Supply-Side Liberal blog, Miles Kimball (22/4/13)
The Rogoff-Reinhart data scandal reminds us economists aren’t gods The Guardian, Heidi Moore (18/4/13)
Reinhart, Rogoff… and Herndon: The student who caught out the profs BBC News Magazine, Ruth Alexander (20/4/13)
George Osborne’s case for austerity has just started to wobble The Guardian, Polly Toynbee (18/4/13)
The error that could subvert George Osborne’s austerity programme The Guardian, Charles Arthur and Phillip Inman (18/4/13)
The Excel depression Sydney Morning Herald, Paul Krugman (19/4/13)
Europe: Retreat from austerity BBC News, Gavin Hewitt (23/4/13)

Guest post by Thomas Herndon
The Grad Student Who Took Down Reinhart And Rogoff Explains Why They’re Fundamentally Wrong Business Insider, Thomas Herndon (22/4/13)

Papers
Growth in a Time of Debt NBER working paper, Carmen M. Reinhart and Kenneth S. Rogoff (January 2010)
Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff PERI Working Paper 322, Thomas Herndon, Michael Ash and Robert Pollin (April 2013)

Questions