13.2 Assessment and Evaluation of Adherence

13.2 Assessment and Evaluation of Adherence#

[Source: Adapted from Background Document item 3(b) 50th session on Supplementing the United Nations Fundamental Principles of Official Statistics: Implementation Guidelines. Prepared by the United Nations Statistics Division]

Criteria

Evaluation criteria generally mean benchmarks or yardsticks against which results can be compared. For each principle, evaluation criteria could frame a set of outcomes based on non-adherence risks. Table 13.1 presents an example of such criteria.

Table 13.1. Evaluation Criteria Based on the risks of non-adherence

Criteria based on the risks from the implementation guide	Sources of evidence for evaluation
The production of official statistics is supported through legal mandate and/or policy settings.	• Enacted legislation • Published policy settings • Reviews of the legal and regulatory framework
Official statistics are recognised nationally and internationally as being produced on an impartial basis	• Trust and use type survey results • Media reports - national and international which recognise impartiality • Self and/or Independent assessments • Country reviews e.g. OECD, IMF
Official statistics policies and practices are acknowledged as being guided by professional principles and guidelines.	• Adherence with code of practices • Media reports - national and international which acknowledge professionalism • Self and/or Independent assessments
Official statistics policies and practices are acknowledged as being transparent.	• Published material which outlines official statistics policies and practices • Reports on the monitoring of official statistics policies and practices • Adherence with code of practices • Media reports - national and international which acknowledge transparency • Independent assessment • Country reviews e.g. by OECD, IMF
Official statistics policies and practices are recognised as being guided by ethical standards.	• Published material which outlines ethical standards and their application • Records of staff capability, training and monitoring • Adherence with code of practices • Records of breaches and the consequences • Media reports - national and international which recognise ethical behaviour • Independent assessment • Country reviews e.g. by OECD, IMF

Application

Table 13.2 illustrates the application of the evaluation criteria, which involves identifying the evidence base and assessing whether, based on the evidence, adherence is strong, weak, or absent.

A strong evidence base would support adherence.
A weak evidence base would suggest a risk of non-adherence.
The absence of evidence would mean non-adherence.

This approach would enable countries with weak or absent evidence bases to continue engaging in relevant activities aimed at future adherence, thereby helping to mitigate the consequences of non-cadherence. Additionally, it recognizes areas where countries show progress towards full adherence.

Table 13.2. Areas of adherence and types of evidence-base

Areas of Adherence / Evidence Base	Strong	Weak	None
The production of official statistics is protected through legal mandate and/or policy settings.	Full Adherence	At risk of non-adherence	Non-adherence
Official statistics are recognised nationally and internationally as being produced on an impartial basis	Full Adherence	At risk of non-adherence	Non-adherence
Official statistics policies and practices are acknowledged as being guided by professional principles and guidelines.	Full Adherence	At risk of non-adherence	Non-adherence
Official statistics policies and practices are acknowledged as being transparent.	Full Adherence	At risk of non-adherence	Non-adherence
Official statistics policies and practices are recognised as being guided by ethical standards.	Full Adherence	At risk of non-adherencee	Non-adherence

The above evaluation and assessment guide is a work in progress. As part of the recommendation, the results for each region and/or country would be published and subject to public scrutiny and debate. The reasons for this include:

It would enable countries to assess their adherence to the Fundamental Principles and identify both strengths and weaknesses.
It would help identify common themes and shared issues within the international statistical community, which could be addressed collectively through the UNSC and the Consultative Advisory Board.
It would showcase effective practices that could improve adherence.