Annex 5 - Why Share Microdata? – A View from ODW#
Open Data Watch (ODW) is an international, non-profit organization working at the intersection of open data and official statistics. Open Data Watch is working to support NSOs to release their microdata by evaluating dissemination practices and the obstacles to the beneficial use of microdata.
Without microdata, we cannot keep the promise of the Sustainable Development Goals that no one will be left behind. However, to harness the full potential of microdata, they must be open and accessible.
Out of 3348 censuses and surveys recorded by the International Household Survey Network (IHSN) between 2009 and 2018, only 1320 – 40 per cent – make their data available online. The datasets that are available are often out of date. The reasons for withholding access are numerous: lack of capacity to prepare data for publication; a desire to obtain royalties or other payments for their use; and concern about the release of confidential information that could be used to identify individuals or establishments. Even where microdata is nominally available, unstated restrictions and cumbersome procedures may effectively limit access (Woolfrey, 2012) (🔗).
Like other data, microdata is a public good: virtually costless to disseminate once they have been collected and not diminished by use. As a public good, they can be used and reused many times, each time increasing the social and economic benefits from new products and services created or, more indirectly, from efficiency gains and the reduction of transaction costs (European Commission, 2015) (🔗). To maximize the efficient use of a public good, data should be available to all users at little or no cost and with few, if any, restrictions on their use. This is particularly true of data that have been created at public expense, by governments through taxpayers or organizations supported through public funds. Disseminating microdata can benefit national statistical offices (NSOs) by enhancing their credibility; improving the reliability and relevance of data; reducing the cost of data dissemination; and leveraging funding for statistics (Dupriez and Boyko, 2010) (🔗).
Considering the high value of microdata for solving some of the world’s most pressing challenges, there are good reasons for NSOs to expand their efforts to make their microdata available. With the agreement of country partners, the DHS Program has published public use files from their Demographic and Health Surveys. These files, which can be downloaded by registered uses, have been widely used by academic researchers and bilateral and multilateral development agencies to measure progress on important development goals. Along with censuses and administrative data, they provide the basis for the Multidimensional Poverty Index developed by Alkire and Foster (Alkire, Foster, et al., 2015) (🔗).
Administrative data systems also contain important information about the status and welfare of people, especially when they can be linked. Working with de-identified, linked individual-level information from the Norwegian Population Register, the National Registry for Personal Taxpayers, the Cause of Death Registry, and the National Education Database, Kinge, Modalsi, and Overland (2019) (🔗) find a strong association between wealth and life expectancy in Norway. And when surveys are well documented, it is possible to undertake large scale, cross-country analyses, as demonstrated by Boudet, Buitrago, and others (2018) (🔗). Utilizing household surveys from 89 countries, they examined gender differences in poverty and household composition.
The case for open microdata has been widely supported by many international agencies, including the OECD, citing open microdata as a crucial element in open government and democratic societies and as a driver of economic and social benefits through innovation and new uses of data (OECD, 2014). The International Household Survey Network (IHSN) echoes these sentiments, adding enhanced credibility of national statistical offices and increased funding opportunities as other potential outcomes of open microdata (IHSN, n.d.; Durpriez and Boyko, 2010). Although there are privacy, funding, and legal challenges to opening microdata, there are also resources and frameworks to ease the burden on countries wishing to make their microdata available. (See links in the guidelines below).
Open Data Watch work with NSOs started with the Bridging the Gap: Mapping Gender Data Availability in Africa report with Data2X (ODW, 2019) (🔗) and now with the Access to Microdata (ATM) project. The ATM project will request downloads of microdata to test consistency with published rules and to examine the characteristics of datasets in different access categories. The results of this study will provide insight into how countries are disseminating microdata, implementing standards, and protecting the privacy of individuals in their datasets. This project is expected to deliver results to the international community by spring of 2020.
Bridging the Gap: Mapping Gender Data Availability in Africa (2019) (🔗);
Dissemination of Microdata Files - Principles, Procedures and Practices (2010) (🔗);
Introduction to Statistical Disclosure Control (SDC) (2014) (🔗);
Leveraging data in African countries: Curating government microdata for research (2013) (🔗);
Managing Statistical Confidentiality and Microdata Access (2007) (🔗).