Tag: evidence

Many of you reading this will be embarking on an economics degree. During your studies you’ll be developing the skills that economists bring to observing and analysing the world around us and considering the policy options to achieve various social and economic objectives. You’ll be learning how to become an ‘economic detective’ and to do ‘forensic economics’.

Identifying the nature of economic problems; collecting and examining the evidence; using the economist’s ‘toolkit’ of concepts and ideas to make sense of the evidence; looking for explanations; constructing hypotheses and theories; considering what can be done to tackle the problems and prevent them occurring in the future – these are the sorts of things you will be doing; and they involve detective work.

The podcast below looks at the methods of Sherlock Holmes. These are the sorts of methods successful economists use. John Gray identifies three types of reasoning. The first two are probably familiar to you, or soon will be.

1. Induction involves looking at evidence and then using it to construct general theories. So, for example, if you observe on many occasions that when the prices of various goods rise, the quantity demanded falls, you can then hypothesise that whenever the price of a good rises, the quantity demanded will fall; in other words, you induce that price and quantity demanded are inversely related – that demand curves are downward sloping. This is known as the ‘Law of demand’. Induction, of course, is only as good as the evidence. Nevertheless, inductive methods are logical and it can be demonstrated how the theories follow from the evidence.

2. Deduction involves using theories to draw conclusions about specific cases. So, for example, you could use the law of demand to deduce that when the price of a specific good rises, the quantity demanded of that good will fall. You would also assume that nothing else had changed that could influence the demand for the good. In other words, you assume ‘ceteris paribus‘ or ‘other things being equal’. As long as you have not made any logical errors, deduction is foolproof. As John Gray puts it:

Deduction is infallible as long as the premises are true, while induction yields probabilities that can always be falsified by events

But there is a third type of reasoning and this is where the true economic detective comes in. This is known as ‘abduction’. This is the type of logic that is used when evidence is thin or where there are lots of scraps of seemingly contradictory evidence. And this is the type of logic employed so successfully by Sherlock Holmes.

3. Abduction involves making informed guesses or estimates from limited evidence. It is using the scraps of evidence as clues as to what might be really going on. It is how many initial hypotheses are formed. Then the researcher (or detective) will use the clues to search for more evidence that can be used for induction that will yield a more robust theory. The clues may lead to a false trail, but sometimes they may allow the researcher to develop a new theory or amend an existing one. A good researcher will be alert to clues; to seeing patterns in details that might previously have been dismissed or gone unnoticed.

Before the banking crisis of 2007/8 and the subsequent credit crunch and recession in the developed world, many economists were picking up clues and trying to use them to develop a theory of systemic risk in financial markets. They were using the skills of an economic detective to try to discover not only what was currently going on but also what might be the consequences for the future. Some used abduction successfully to predict the impending crisis; most did not.

If you are embarking on an economics degree and will possibly go on to a career as an economist, then part of your training will be as a detective. With good detective skills – looking for clues, seeing connections, identifying what more evidence is required and where to find it, and then using it to provide explanations and policy prescriptions – you could make a very successful and sought-after economist. Being a good economist is not just about learning theories and techniques, although this is vitally important; it’s also about being imaginative and thinking ‘outside the box’. Good luck!

Podcast
Sherlock Holmes and the Romance of Reason BBC: A Point of View, John Gray (17/8/12) (Click here for a transcript.)

Articles and information
Detective work: forensic economics Business:Life, Tim Harford (2/5/12)
The Search for 100 Million Missing Women Slate, Stephen J. Dubner and Steven D. Levitt (24/5/05)
Abduction Stanford Encyclopedia of Philosophy, Igor Douven (9/2/11)
Abductive reasoning Wikipedia

Questions

  1. Explain the difference between induction and abduction.
  2. Identify the various ‘threshold concepts’ in economics. Does an understanding of these concepts help an economist do better detective work?
  3. How might forensic economics be used for crime fighting?
  4. Why might elegant and sophisticated economic theory be dangerous in the ‘messy’ and statistically ‘noisy’ real world?
  5. In trying to establish an explanation for “100 Million Missing Women”, what use was made of abduction, induction and deduction?

Economic assessment of real-world issues relies heavily on data. It is the same with economic policy recommendations. Both public- and private-sector organisations gather data, which are then used for analysis, often presented in a report. These reports are then often used as the basis for policy, whether by the government, local authorities or the private sector. Sometimes the data are those collected by national statistical agencies, such as the Office for National Statistics (ONS) in the UK; sometimes they are collected by private agencies; sometimes by individual researchers.

Clearly the analysis and the suitability of any policy recommendations depend on the quality of the data. But how much can we rely on the data? A problem is that people have an interest in gathering and/or selecting data that support their opinions. As a result, the data used for analysis and policy recommendations may be unreliable and incomplete.

This is not to say that the data collected by reputable agencies such as the ONS are wrong. Rather, it is the selective use of them that can be highly misleading. Sometimes, however, the data that some agencies produce may indeed be unreliable, with too small or unrepresentative samples. If they rely on surveys, the survey questions may be poorly framed or lead the respondent into giving a particular answer.

Newspapers make use of data and reports all the time to make a particular case – a case in line with the newspaper’s political stance. The lesson for economic students is that we need to be alert all the time as to just how reliable data are; and to whether the conclusions drawn from them are correct.

The following two articles by Ben Goldacre, from the Guardian’s Bad Science series, look at the misuse of data. The first looks at the case of the Health Service; the second at the possibility of savings by local government in their procurement activities.

Articles
How far should we trust health reporting? Guardian, Ben Goldacre (17/6/11)
Misleading money-saving claims help no one Guardian, Ben Goldacre (24/6/11)

Report
Realising Savings through Procurement Optimisation Opera Solutions

Questions

  1. According to the first article above, how much newspaper reporting based on the use of data is unreliable?
  2. What are the reasons for the unreliability of newspaper reporting?
  3. For what reasons might the ONS and other reputable agencies periodically have to amend time series data?
  4. “Council incompetence ‘costs every household £452 a year'”. Critically examine this claim by the Daily Mail.
  5. Why may Opera Solutions be seen as not wholly independent in reporting the possibilities of cost savings by local government?
  6. In the absence of reliable data, can any economic policy conclusions be drawn from economic models? Explain.