Economic assessment of real-world issues relies heavily on data. It is the same with economic policy recommendations. Both public- and private-sector organisations gather data, which are then used for analysis, often presented in a report. These reports are then often used as the basis for policy, whether by the government, local authorities or the private sector. Sometimes the data are those collected by national statistical agencies, such as the Office for National Statistics (ONS) in the UK; sometimes they are collected by private agencies; sometimes by individual researchers.
Clearly the analysis and the suitability of any policy recommendations depend on the quality of the data. But how much can we rely on the data? A problem is that people have an interest in gathering and/or selecting data that support their opinions. As a result, the data used for analysis and policy recommendations may be unreliable and incomplete.
This is not to say that the data collected by reputable agencies such as the ONS are wrong. Rather, it is the selective use of them that can be highly misleading. Sometimes, however, the data that some agencies produce may indeed be unreliable, with too small or unrepresentative samples. If they rely on surveys, the survey questions may be poorly framed or lead the respondent into giving a particular answer.
Newspapers make use of data and reports all the time to make a particular case – a case in line with the newspaper’s political stance. The lesson for economic students is that we need to be alert all the time as to just how reliable data are; and to whether the conclusions drawn from them are correct.
The following two articles by Ben Goldacre, from the Guardian’s Bad Science series, look at the misuse of data. The first looks at the case of the Health Service; the second at the possibility of savings by local government in their procurement activities.
Articles
How far should we trust health reporting? Guardian, Ben Goldacre (17/6/11)
Misleading money-saving claims help no one Guardian, Ben Goldacre (24/6/11)
Report
Realising Savings through Procurement Optimisation Opera Solutions
Questions
- According to the first article above, how much newspaper reporting based on the use of data is unreliable?
- What are the reasons for the unreliability of newspaper reporting?
- For what reasons might the ONS and other reputable agencies periodically have to amend time series data?
- “Council incompetence ‘costs every household £452 a year'”. Critically examine this claim by the Daily Mail.
- Why may Opera Solutions be seen as not wholly independent in reporting the possibilities of cost savings by local government?
- In the absence of reliable data, can any economic policy conclusions be drawn from economic models? Explain.