2.6 Statistical confidentiality

NSOs have a long tradition of keeping their data confidential. In developed statistical systems of democratic countries, it has been standard practice for a long time to respect the confidentiality of individual data and ensure that information that is collected or acquired for statistical purposes is used solely for those purposes and no other. These cardinal rules of statistical confidentiality and privacy are set out in clear terms in Principle 6 of the UNFPOS: “Individual data collected by statistical agencies for statistical compilation, whether they refer to natural or legal persons, are to be strictly confidential and used exclusively for statistical purposes.”

This has several implications. One is that the main focus is on publication of aggregate statistics and that individual data is rarely disseminated and then only after being anonymised, i.e. cleaned of individual identifiers. Another consequence is that the individual data is kept confidential within the NSO and not handed out to other authorities. Furthermore, care is taken in tabulation that microdata on persons, households and businesses cannot be traced, directly or indirectly, back to specific persons, households, and businesses.

Observing confidentiality of individual data can be said to be first and foremost a mindset of the management and staff of the NSO and other producers of official statistics. This is also the first rule that all new statistical staff are taught – that the individuals and businesses supplying the data have a right to confidentiality and that their data is to be handled as confidential and with respect. Therefore, NSOs in organising data collections of individual data promise and publicly make it clear that they will ensure confidentiality and will not release or provide access to the individual data that can be identified. It follows from this that the statistical producers must take great care when analysing and storing their microdata. It is recommended that microdata is stored either anonymised or without individual identifiers. Some NSOs carry out the anonymisation of all microdata immediately after processing. However, it is considered important to conserve anonymisation keys in some instances to link data over time or between datasets at a later date.

If confidentiality is not observed, the trust that data providers place in the NSO is eroded, which in turn undermines the willingness of people and businesses to provide data. Many statistical agencies have a procedure for making this clear to staff by having all new staff members agreeing to and signing a confidentiality statement whereby they pledge to keep all individual data confidential and not to release or hand over confidential microdata they work with, or become aware of, in performing their work. It is recommended that such pledges are renewed at a few years’ intervals.

The implication of the rule of confidentiality that individual data may not be handed over to other authorities is not difficult to explain and teach but may be stressful to uphold in some exceptional circumstances. The 6th Principle of the UNFPOS is particularly helpful in such situations, particularly after the Principles were unanimously adopted by the UN General Assembly in 2014.

Ensuring that information published in statistical tables cannot be directly or indirectly traced to individual persons or businesses can be somewhat difficult. Such difficulties have in recent years been solved by statistical methods and techniques which involve suppressing in the published tables any values that might be traced back to the individual subjects to which the data refers.