Putting evidence at the heart of policy debate

Putting evidence at the heart of policy debate
This is one of a series of policy notes to support our policy positions as summarised in the RSS Data Manifesto, and was published in April 2016. You can download the PDF version at this link. To discuss policy advocated here, please contact our Press and Policy Officer, Magdalen Wiley (m.wiley@rss.org.uk).

Summary

Evidence must be taken more seriously in policy formulation and evaluation, and official statistics must be at the heart of the policy debate"

The RSS’ Data Manifesto calls on the UK government to take evidence more seriously in policy formulation and evaluation, and highlights that official statistics should be at the heart of policy debate. Whilst evidence based policy making is an established concept, more must be done to ensure that decision makers are committed to incorporating it as an intrinsic part of the policy making process. The RSS recognises that policymaking is a complex process and that elected representatives have to balance a range of considerations including public opinion, political judgement, ideology etc. We are not arguing for a technocratic approach to policy which ignores this, but we do think that evidence should have a stronger role as policy is formed.  Even when resources are lacking, policy makers should be able to take into account the probable quantifiable consequences of all the options. Resources need to be strategically allocated to strengthen data infrastructure, to identify data gaps, and to share and supply data that society needs.

Independent publication of data and evidence that underpin public policies, separate from political commentary, helps make our decision-makers truly accountable. This is also crucial for building public trust. The evidence and data that informs decisions must be made clear, if government policies are to be properly assessed and for the political process to be truly democratic. Well-founded independent communication of evidence is necessary for trust in information, and forms the basis for wider public understanding of science, statistics and evidence.

We should also take pride in wider systems for evidence based scrutiny, and ensure that evidence gathered outside government is well accounted for. We believe links between Parliament, the UK’s science and research base, and other public bodies will lead to more effective policy making. We welcome the ongoing commitment to bodies such as the What Works Centres, and would like to see greater commitment by government to long term evaluation of policy. We would like to see all avenues of scrutiny further promoted including a strengthened regulatory function for the UK Statistics Authority, and continuation of parliamentary scrutiny as a key way in which policy makers can be held to account.

Strengthen data infrastructure and identify data gaps to share and supply data that society needs

We would like to see more confidence across government to share the data that it already holds, and to do so by establishing safeguarded approaches

The UK needs to strengthen its data infrastructure, and should assess and address data gaps for statistics and research that will inform decisions across our society and economy. We would like to see a strong policy focus on this issue with a concerted campaign to share and supply the data that is needed nationally and locally. This should engage the UK’s third sector and private sector research base in collaboration with the statistical service across government. Such collaboration is sorely needed. For example, the Bean Review of Economic Statistics has found that the UK lacks data that is needed to measure the modern economy, and needs to update methods of production, so that we can have more accurate statistics that draw on multiple data sources.1 The Equalities and Human Rights Commission has reported on Britain’s progress on equality since 2010, and found that worryingly, there are significant data gaps on sections of the population, in particular, marginalised groups.2 This is a major concern as these are groups who are in need of public services, but currently the government and its services lack sufficient information on these groups. We would like to see more confidence across government to share the data that it already holds, to support statistics and research in the public interest, and to do so by establishing safeguarded approaches, with no unnecessary disruption of access for researchers and statisticians.

The establishment of the UK Statistics Authority in 2008 provided a professional centralised body for the independent scrutiny of all official UK statistics. It should take a leading role in addressing data gaps and data sharing, through its ability to scrutinise the scope and use of official statistics throughout the public sector. To produce statistics that are comprehensive for the UK, the Office for National Statistics must also be able to exercise powers in legislation to access the data it requires. Across Europe, National Statistical Institutes are expected to exercise their legal mandate for data collection, to access all the data they need to produce comprehensive statistics.3 We support that the UK’s mandate for this must be fully applied in the UK. Further investment must also be made in an infrastructure that allows, as is the case in Finland, the Office for National Statistics access to all major public and private datasets for statistical purposes (which are regulated and controlled to focus not on us as individuals, but on how society is changing as a whole).

Greater access rights to data should be accompanied by more a significant role for the users of statistics. Users need to be continuously involved, by ONS and the Government Statistical Service, to help identify the key outstanding questions for statistics to answer, gaps in the data, and quality requirements. High level user groups should be established in all areas of official statistics to ensure user needs are kept in mind when statistical outputs are being produced. This will ensure our data infrastructure is strong enough for the needs of a changing economy and society.

Data and statistics must be used effectively for evidence based policy making

We would like to see a more authoritative approach taken with regards to new data collection and new methodologies to form evidence. Policy makers have begun to adopt big data and commission research making use of statistical methods, such as predictive analytics and machine learning, to inform their decisions. Some of these methods, such as randomised control trials, provide a powerful tool to evaluate policy options before long term decisions go ahead. Investment in the seven existing What Works Centres including the National Institute for Health Care Excellence has been hugely important for offering objective, evidence based guidance on which effective policy can be built. However, the issue remains that policy makers are not always aware of the distinction between more robustly tested evidence and more speculative or partially tested results, and there are times when the standard of research referenced by policy makers is unclear.

As more sources of data become open and more big data methods are utilised it is important that the government calls on appropriate expertise to assess statistics. The RSS also welcomes the increased authority and reputation of Select Committees in scrutinising the evidence base for decisions. Several Select Committees have published “evidence checks” on particular policies, we would like to see frameworks like this introduced on a more regular basis. The House of Lords plays a further, crucial role, in part due to its ability to evaluate and scrutinise in an often more detailed way than the House of Commons is able to.

The Government must publish the data and evidence behind its policies

Researchers must be prepared to link their research up with the policy domain and to translate complex workings. This must be accompanied by a greater willingness to open up government data for research.

Transparent communication of government data and evidence supports evidence gathering and open policy making to inform decisions. Leadership, clear guidelines and mechanisms are required for the publication of data and evidence. For example, we welcome the ministerial commitment at the Department for Environment, Food and Rural Affairs, to the progressive opening up of environment data in the UK, including the commitment to release 8000 new open data sets within one year.4 Cultural shifts are also needed to allow more proactive working within departments on opening and evaluating their own data. Progress has been made under the Government’s open data initiative, as more data has been made available for scrutiny by citizens. However, this does not necessarily link data back to specific policies, and it remains the case that the decision on whether data is open and publicly available remains within government.

The RSS wishes to see a commitment from all government departments to publish their evidence in parallel with policy announcements and consultations. There is a need to approach this in a more systematic, open, clear, and accessible way, with a coherent strategy in place. Currently, government departments vary greatly in terms of how much of their data is publicly available and also the process and timeline for publication. Access to externally-commissioned research is affected by similar issues. An independent review has found that only four departments provide a public, searchable database of all their commissioned research, and called for a central register of this information across all the departments to bring those that are lagging up to speed.5 Freedom of Information remains an important mechanism for members of the public and the media to request 'hidden' data and information from the government on issues that are of importance to them; such requests should also lead to systematic improvements.

Policy making is also improved by effective engagement with relevant researchers, experts and citizens providing scrutiny and evidence. The “Evidence Transparency Framework” published by the Institute for Government in partnership with the Alliance for Useful Evidence and Sense About Science, offers an assessment tool for the public to assess the transparency of government departments and a checklist for the departments to follow.6 Such frameworks are a helpful step to facilitate communication and transparency in the policy making process. The more regular publication of data and evidence by government will allow parliamentarians to effectively question why some policies may have ‘evidence gaps’ and also recognise when government departments have used evidence effectively in policy making.

The RSS seeks to play its part to inform use of data for effective policy making and engage statistical experts inside and outside government. Researchers must be prepared to link their research up with the policy domain and to translate complex workings so they can be more easily understood by civil servants. However, this must be accompanied by a greater willingness from the government to open up their data for research purposes. With our reach to data analysts, data scientists and statisticians inside and outside government, the RSS is glad to help to inform policy with evidence.

Endnotes
1. Bean, C. (2016). Independent Review of Economic Statistics. London: HM Treasury. Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/507081/2904936
_Bean_Review_Web_Accessible.pdf
[Accessed: March 2016]
2. Equality and Human Rights Commission. (2015). Is Britain Fairer? The state of ofequality and human rights 2015. London: Equality and Human Rights Commission. Avaliable at: http://www.equalityhumanrights.com/sites/default/files/uploads/IBF/Final-reports/revised/EHRC_IBF_MainReport_acc.pdf  [Accessed: March 2016]
3. Eurostat, & ESS. (2011). European Statistics Code of Practice. Avaliable at: http://ec.europa.eu/eurostat/web/products-manuals-and-guidelines/-/KS-32-11-955 [Accessed: March 2016]
4. Elizabeth Truss MP. (2015, June 25). Environment Secretary unveils vision for open data to transform food and farming. Avaliable at: https://www.gov.uk/government/news/environment-secretary-unveils-vision-for-open-data-to-transform-food-and-farming [Accessed: March 2016]
5. Sedley, S. (2016). Missing evidence: An inquiry into the delayed publication of government-commissioned research (PDF). Sense about Science, JRRST Charitable Trust. Available from: http://www.instituteforgovernment.org.uk/blog/wp-content/uploads/2016/06/MissingEvidence-DigitalPDF.pdf [Accessed: June 2016]
6. Institute for Government. (2015 ). Evidence Transparency Framework. London : Institute for Government. Avaliable at: http://www.instituteforgovernment.org.uk/sites/default/files/publications/4545%20IFG%20-%20Evidence%20Trans%20framework%20v6.pdf [Accessed: March 2016]