An excellent learning exercise for students of economics is to take a journal article that uses data to model the economy and then try to replicate the authors’ results. You may well be given an assignment like this in future years of your degree.
One such exercise is used on the University of Massachusetts Amherst’s doctoral programme in economics. Thomas Herndon is a student on that degree and chose to examine a well-known and highly influential paper, Growth in a Time of Debt by Carmen Reinhart then of the University of Maryland and Kenneth Rogoff of Harvard University and former chief economist of the IMF. Professors Reinhart and Rogoff used new data on 44 countries spanning about 200 years.
A key finding of their paper, published in 2010 in the American Economic Review Papers and Proceedings, is that once a country’s government debt exceeds 90% of GDP, growth rates fall considerably: the median across countries by about 1% and the mean considerably more.
The paper has been hugely influential. It has been used to justify the austerity programmes being pursued in many countries, including the UK and the eurozone. Cutting the government deficit to GDP ratio, and ultimately the government debt to GDP ratio, has been seen as a way of achieving higher growth over the longer term, and justifies the adverse effect on short-term growth from the dampening of aggregate demand.
Well, this seemed an interesting paper for Thomas Herndon to examine, and he was keen to show just how Reinhart and Rogoff’s data led to their conclusions. But try as he might, he could not replicate their results. His initial reaction was to think he had made an error, but each time he checked he came back with the same conclusion: they must have made errors in their calculations.
His supervisor at Amherst, Professor Michael Ash, after Thomas had checked and checked again, realised that something was wrong. He encouraged Thomas to write to Reinhart and Rogoff to request sight of their dataset. They duly obliged and it was then that Thomas spotted various errors. These are explained in the articles below, but the overall effect was to alter the conclusion. Although high debt may undermine growth to some extent, the effect is much less than Reinhart and Rogoff concluded, and there are several exceptions to this rule.
On 15 April 2013, Thomas, along with his supervisor, Michael Ash and his colleague, Robert Pollin, published a response to the Reinhart and Rogoff paper. In the abstract to their paper, Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff they state that:
… coding errors, selective exclusion of available data, and unconventional weighting of summary statistics lead to serious errors that inaccurately represent the relationship between public debt and GDP growth among 20 advanced economies in the post-war period. They find that when properly calculated, the average real GDP growth rate for countries carrying a public-debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not –0:1 percent as published in Reinhart and Rogoff. That is, contrary to RR, average GDP growth at public debt/GDP ratios over 90 percent is not dramatically different than when debt/GDP ratios are lower.
The authors also show how the relationship between public debt and GDP growth varies significantly by time period and country. Overall, the evidence we review contradicts Reinhart and Rogoff’s claim to have identified an important stylized fact, that public debt loads greater than 90 percent of GDP consistently reduce GDP growth.
So could this be you in the future? Will you take a famous paper and, by re-examining and reworking the data, find that its conclusions are wrong? Could you end up changing the world? Exciting stuff!
Austerity: A Spreadsheet Error? BBC, More or Less, Tim Harford (20/4/13)
Austerity justification study ‘inaccurate’ BBC Today Programme, Robert Pollin (18/4/13)
UMass Student Exposes Serious Flaws in Harvard Economists’ Influential Study The Atlantic Wire, J.K. Trotter (18/4/13)
Shocking Paper Claims That Microsoft Excel Coding Error Is Behind The Reinhart-Rogoff Study On Debt Business Insider, Mike Konczal (16/4/13)
How a student took on eminent economists on debt issue – and won Economic Times of India (19/4/13)
Meet the 28-Year-Old Grad Student Who Just Shook the Global Austerity Movement New York Magazine, Kevin Roose (19/4/13)
An economist’s mea culpa: I relied on Reinhart and Rogoff Confessions of a Supply-Side Liberal blog, Miles Kimball (22/4/13)
The Rogoff-Reinhart data scandal reminds us economists aren’t gods The Guardian, Heidi Moore (18/4/13)
Reinhart, Rogoff… and Herndon: The student who caught out the profs BBC News Magazine, Ruth Alexander (20/4/13)
George Osborne’s case for austerity has just started to wobble The Guardian, Polly Toynbee (18/4/13)
The error that could subvert George Osborne’s austerity programme The Guardian, Charles Arthur and Phillip Inman (18/4/13)
The Excel depression Sydney Morning Herald, Paul Krugman (19/4/13)
Europe: Retreat from austerity BBC News, Gavin Hewitt (23/4/13)
Guest post by Thomas Herndon
The Grad Student Who Took Down Reinhart And Rogoff Explains Why They’re Fundamentally Wrong Business Insider, Thomas Herndon (22/4/13)
Growth in a Time of Debt NBER working paper, Carmen M. Reinhart and Kenneth S. Rogoff (January 2010)
Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff PERI Working Paper 322, Thomas Herndon, Michael Ash and Robert Pollin (April 2013)
- What were the particular errors made by Reinhart and Rogoff?
- How has their paper been used as a basis for the design of macroeconomic policy?
- What are the limitations of using even accurate time-series data as the basis for policy measures?
- How might the work of Herndon change the direction of future macroeconomic policy?
- In his guest post in Business Insider (see link above), Herndon wrote: ‘The implication for policy is that, under particular circumstances, public debt can play a key role in overcoming a recession.’ What might this role be?
- Why might we have to be cautious in drawing policy conclusions from Herndon’s work?
Should economists have foreseen the credit crunch? A few were warning of an overheated world economy with excessive credit and risk taking. Most economists prior to 2007/8, however, were predicting a continuation of steady economic growth. Inflation targeting, fiscal rules and increasingly flexible markets were the ingredients of this continuing prosperity. And then the crash happened!
So why did so few people see the downturn coming? Were the models used by economists fundamentally flawed, or was it simply a question of poor assumptions or poor data? Do we need a new way of modelling the economy, or is it simply a question of updating theories from the past? Should, for example, models become much more Keynesian? Should we abandon the new classical approach of assuming that markets are essentially good at pricing in risk and that herd behaviour will not be seriously destabilising?
The following podcast looks at these issues. “Aditya Chakrabortty’s joined in the studio by the Guardian’s economics editor Larry Elliott, as well as Roger Bootle, the managing director of Capital Economics, and political economist and John Maynard Keynes biographer Robert Skidelsky. Also in the podcast, we hear from Nobel prize-winning economist, Elinor Ostrom, Freakonomics author Steven Levitt, and UN advisor and developmental economist Daniel Gay.”
The Business: A crisis of economics Guardian podcast (25/11/09)
See also the following news items from the Sloman Economics news site:
Keynes is dead; long live Keynes (3/10/09)
Learning from history (3/10/09)
Macroeconomics – Crisis or what? (6/8/09)
The changing battle grounds of economics (27/7/09)
Repeat of the Great Depression – or learning the lessons from the past? (23/6/09)
Animal spirits (30/4/09)
Keynes – do we need him more than ever? (26/10/08)
- Why did most economists fail to predict the credit crunch and subsequent recession? Was it a problem with the models that were used or the data that was put into these models, or both?
- What was the Washington consensus? To what extent did this consensus contribute to the current recession?
- What is meant by systemic risk? How does this influence the usefulness of ‘micro’ financial models?
- What particular market failures were responsible for the credit crunch?
- What is meant by ‘rational behaviour’? Is it reasonable to assume that people are rational?
- Is macroeconomics too theoretical or too mathematical (or both)? If you think it is, how can macroeconomics be reformed to improve its explanatory and predictive power?
- Does a ‘really good economist’ need to have a good grounding in a range of social sciences and in economic history?
According to Brad DeLong, professor of economics at the University of California at Berkeley, if we are to get a full understanding of the financial crisis and recession of the past two years, we need to take a historical perspective. In the following article from The Economic Times of India, he argues that modern macroeconomists need to learn from history if their assumptions and models are to be relevant and predictive.
The anti-history boys The Economic Times (India) (1/10/09)
A fuller version of the above article, along with comments from readers, can be found on Brad deLong’s blog site, a Semi-Daily Journal of an Economist at:
Economic History and Modern Macro: What Happened? (30/9/09)
- According to Narayana Kocherlakota, most macroeconomic models “rely on some form of large quarterly movements in the technological frontier. Some have collective shocks to the marginal utility of leisure. Other models have large quarterly shocks to the depreciation rate in the capital stock (in order to generate high asset price volatilities)…”. How could these models explain business cycles? Would you classify them as ‘real business cycle theories’: i.e. as ‘supply-side’ explanations?
- How does Brad deLong explain recessions?
- Why does a change in the velocity of circulation of money contribute to a crash?
- What are the strengths and limitation of using economic history to understand the current crisis?
Whilst some economists predicted the banking crisis of 2007/8 and the subsequent global recession, many did not. Was this a failure of macroeconomics, or at least of certain macroeconomic schools of thought, such as New Classical economics? Or was it a failure to apply the subject with sufficient wisdom? Should the subject be radically rethought, or can it simply be amended to take into account aspects of behavioural economics and a better understanding of systemic risk?
The four linked articles below from The Economist look at the debate and at the whole state of macroeconomics. The other articles pick up some of the issues.
Will the ‘crisis in macroeconomics’ lead to a stronger subject, more able to explain economies in crisis and not just when they are working well? Will a new consensus emerge or will economists remain divided, not only about the correct analysis of how economies work at a macro level, but also about how to tackle crises such as the present recession?
What went wrong with economics The Economist (16/7/09)
The other-worldly philosophers The Economist (16/7/09)
Efficiency and beyond The Economist (16/7/09)
In defence of the dismal science The Economist (6/8/09)
How to rebuild a shamed subject Financial Times (5/8/09)
What is the point of economists? Financial Times – Arena (28/7/09)
Macroeconomic Models Wall Street Pit (23/7/09)
Macroeconomics: Economics is in crisis – it is time for a profound revamp Business Day (27/7/09)
- Distinguish between ‘freshwater’, ‘saltwater’ and ‘brackish’ macroeconomics.
- Explain why economists differ over the efficacy of fiscal policy in times of recession. To what extent does the debate hinge on the size of the multiplier?
- Why is the potential for macroeconomics higher now than prior to the recession?
- What is meant by the ‘efficient market hypothesis’? How did inefficiencies in financial markets contribute to the banking crisis and recession?
- Should economists predict the future, or should they confine themselves to explaining the present and past?