High levels of government debt and the adverse effect this has on the economy has been a key influencing factor in the fiscal consolidation efforts across the world. A key factor providing evidence in support of the connection between high government debts and low economic growth was a paper by two Harvard economists. However, the data used in their research has been called into question.
As we saw in a previous post, It could be you, Carmen Reinhart and Kenneth Rogoff presented a paper back in January 2010. Their research suggested that when a country’s debt increases above 90% of GDP, economic growth will slow considerably. (Click here for a PowerPoint of the above chart.) As you might expect, given the timing of this research, policymakers were intrigued. For those governments in favour of cuts in government spending and increases in taxation to bring the government debt down, this research was dynamite. It seemed to provide the evidence needed to confirm that if left to grow, government debt will have a significantly adverse effect on growth. Here was evidence in favour of austerity.
But, did a simple error create misleading information? A student at the University of Massachusetts Amherst was trying to replicate the results found by Reinhart and Rogoff, but was unable to do so. Thomas Herndon contacted the Harvard professors and they sent him the spreadsheets they had used in their calculations. Looking through it, an error in calculating the average GDP was spotted. However, the student and his supervisors also engaged in further research and came across other inconsistencies. This led to a draft working paper being published in April. The paper did find the same correlation between high debt levels and low growth, but the outstanding results found by Reinhart and Rogoff disappeared. Responding to the error, the Harvard professors said:
We are grateful to Herndon et al. for the careful attention to our original Growth in a Time of Debt AER paper and for pointing out an important correction to Figure 2 of that paper. It is sobering that such an error slipped into one of our papers despite our best efforts to be consistently careful. We will redouble our efforts to avoid such errors in the future. We do not, however, believe this regrettable slip affects in any significant way the central message of the paper or that in our subsequent work.
So, how might this correction and the implications affect government policy? Are we likely to see a reversal in austerity measures? Only time will tell.
Seminal economic paper on debt draws criticism Wall Street Journal, Brenda Cronin (16/4/13)
Reinhart, Rogoff … and Herndon: The student who caught out the Profs BBC News, Ruth Alexander (20/4/13)
Reinhart and Rogoff publish formal correction Financial Times, Robin Harding (8/5/13)
The 90% question The Economist (20/4/13)
Reinhart and Rogoff correct austerity research error BBC News (9/5/13)
Harvard’s Reinhart and Rogoff publish formal collection CNBC, Robin Harding (9/5/13)
Rogoff and Reinhart should show some remorse and reconsider austerity The Guardian, Heidi Moore (26/4/13)
The buck does not stop with Reinhart and Rogoff Financial Times, Lawrence Summers (5/5/13)
Meet Carmen Reinhart and Kenneth Rogoff, the Harvard professors who thought they had austerity licked – and Thomas Herndon, the student who proved them wrong Independent, Tim Walker (22/4/13)
Growth in a time of debt American Economic Review (May 2010)
Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff Political Economy Research Institute, Herndon, Ash and Pollin (April 2013)
- How do high government debts arise?
- In order to reduce government debts, cuts in government spending and increases in taxation are advocated. How does theory suggest that these changes in fiscal policy will affect economic growth?
- What are the arguments (a) in favour of and (b) against austerity measures?
- How might the correction made by Reinhart and Rogoff affect policymakers and their austerity plans?
- What are the key messages from Reinhart and Rogoff’s paper?
An excellent learning exercise for students of economics is to take a journal article that uses data to model the economy and then try to replicate the authors’ results. You may well be given an assignment like this in future years of your degree.
One such exercise is used on the University of Massachusetts Amherst’s doctoral programme in economics. Thomas Herndon is a student on that degree and chose to examine a well-known and highly influential paper, Growth in a Time of Debt by Carmen Reinhart then of the University of Maryland and Kenneth Rogoff of Harvard University and former chief economist of the IMF. Professors Reinhart and Rogoff used new data on 44 countries spanning about 200 years.
A key finding of their paper, published in 2010 in the American Economic Review Papers and Proceedings, is that once a country’s government debt exceeds 90% of GDP, growth rates fall considerably: the median across countries by about 1% and the mean considerably more.
The paper has been hugely influential. It has been used to justify the austerity programmes being pursued in many countries, including the UK and the eurozone. Cutting the government deficit to GDP ratio, and ultimately the government debt to GDP ratio, has been seen as a way of achieving higher growth over the longer term, and justifies the adverse effect on short-term growth from the dampening of aggregate demand.
Well, this seemed an interesting paper for Thomas Herndon to examine, and he was keen to show just how Reinhart and Rogoff’s data led to their conclusions. But try as he might, he could not replicate their results. His initial reaction was to think he had made an error, but each time he checked he came back with the same conclusion: they must have made errors in their calculations.
His supervisor at Amherst, Professor Michael Ash, after Thomas had checked and checked again, realised that something was wrong. He encouraged Thomas to write to Reinhart and Rogoff to request sight of their dataset. They duly obliged and it was then that Thomas spotted various errors. These are explained in the articles below, but the overall effect was to alter the conclusion. Although high debt may undermine growth to some extent, the effect is much less than Reinhart and Rogoff concluded, and there are several exceptions to this rule.
On 15 April 2013, Thomas, along with his supervisor, Michael Ash and his colleague, Robert Pollin, published a response to the Reinhart and Rogoff paper. In the abstract to their paper, Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff they state that:
… coding errors, selective exclusion of available data, and unconventional weighting of summary statistics lead to serious errors that inaccurately represent the relationship between public debt and GDP growth among 20 advanced economies in the post-war period. They find that when properly calculated, the average real GDP growth rate for countries carrying a public-debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not –0:1 percent as published in Reinhart and Rogoff. That is, contrary to RR, average GDP growth at public debt/GDP ratios over 90 percent is not dramatically different than when debt/GDP ratios are lower.
The authors also show how the relationship between public debt and GDP growth varies significantly by time period and country. Overall, the evidence we review contradicts Reinhart and Rogoff’s claim to have identified an important stylized fact, that public debt loads greater than 90 percent of GDP consistently reduce GDP growth.
So could this be you in the future? Will you take a famous paper and, by re-examining and reworking the data, find that its conclusions are wrong? Could you end up changing the world? Exciting stuff!
Austerity: A Spreadsheet Error? BBC, More or Less, Tim Harford (20/4/13)
Austerity justification study ‘inaccurate’ BBC Today Programme, Robert Pollin (18/4/13)
UMass Student Exposes Serious Flaws in Harvard Economists’ Influential Study The Atlantic Wire, J.K. Trotter (18/4/13)
Shocking Paper Claims That Microsoft Excel Coding Error Is Behind The Reinhart-Rogoff Study On Debt Business Insider, Mike Konczal (16/4/13)
How a student took on eminent economists on debt issue – and won Economic Times of India (19/4/13)
Meet the 28-Year-Old Grad Student Who Just Shook the Global Austerity Movement New York Magazine, Kevin Roose (19/4/13)
An economist’s mea culpa: I relied on Reinhart and Rogoff Confessions of a Supply-Side Liberal blog, Miles Kimball (22/4/13)
The Rogoff-Reinhart data scandal reminds us economists aren’t gods The Guardian, Heidi Moore (18/4/13)
Reinhart, Rogoff… and Herndon: The student who caught out the profs BBC News Magazine, Ruth Alexander (20/4/13)
George Osborne’s case for austerity has just started to wobble The Guardian, Polly Toynbee (18/4/13)
The error that could subvert George Osborne’s austerity programme The Guardian, Charles Arthur and Phillip Inman (18/4/13)
The Excel depression Sydney Morning Herald, Paul Krugman (19/4/13)
Europe: Retreat from austerity BBC News, Gavin Hewitt (23/4/13)
Guest post by Thomas Herndon
The Grad Student Who Took Down Reinhart And Rogoff Explains Why They’re Fundamentally Wrong Business Insider, Thomas Herndon (22/4/13)
Growth in a Time of Debt NBER working paper, Carmen M. Reinhart and Kenneth S. Rogoff (January 2010)
Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff PERI Working Paper 322, Thomas Herndon, Michael Ash and Robert Pollin (April 2013)
- What were the particular errors made by Reinhart and Rogoff?
- How has their paper been used as a basis for the design of macroeconomic policy?
- What are the limitations of using even accurate time-series data as the basis for policy measures?
- How might the work of Herndon change the direction of future macroeconomic policy?
- In his guest post in Business Insider (see link above), Herndon wrote: ‘The implication for policy is that, under particular circumstances, public debt can play a key role in overcoming a recession.’ What might this role be?
- Why might we have to be cautious in drawing policy conclusions from Herndon’s work?