8.4 National quality assurance frameworks, guidelines, and tools#
Several NSOs have developed and implemented their own quality assurance frameworks more or less from scratch. The benefit of starting with a clean sheet is the feeling of ownership. Others have taken advantage of the UN NQAF, the ES CoP, or the quality assurance frameworks of NSOs. This section outlines some national quality assurance frameworks, guidelines and tools that are well established and readily available on the Internet.
8.4.1 Statistics Canada’s Quality Assurance Framework and Quality Guidelines#
Statistics Canada’s Quality Assurance Framework (🔗) was one of the first such frameworks introduced (2002). The most recent version (2017) serves as the highest-level governance tool for quality management at Statistics Canada. It gives an overview of the quality management and risk mitigation strategies used by the organization’s program areas. It is used in conjunction with the organization’s management practices, as described in its Quality Guidelines.
Statistics Canada’s Quality Assurance Guidelines (🔗) (first published 1985, most recent version 2009) bring together guidelines and checklists on the many issues that need to be considered in the pursuit of quality objectives. The focus is on how to assure quality through effective and appropriate design or redesign of a statistical project or program from inception through to data evaluation, dissemination and documentation. The guidelines are useful to staff engaged in the planning and design of surveys and other statistical projects, as well as to those who evaluate and analyse the outputs of these projects.
Statistics Canada also publishes a Compendium of Management Practices for Statistical Organizations from Statistics Canada’s International Statistical Fellowship Program (🔗). Chapter 7.5 focuses on quality management and is an excellent source of advice for developing NSOs.
8.4.2 Statistics Finland Quality Guidelines#
Statistics Finland’s Quality Guidelines for Official Statistics (2nd edition, 2007) (🔗) is intended for all who are interested in the functioning of statistical systems, as well as for the users and producers of statistics. As suggested by its title, it presents quality guidelines. In addition, it outlines the framework within which the field of statistics operates in Finland and describes the relevant legislation, as well as current best methods and recommendations. The aim of the document is to improve the usability of the skills and competence required in the designing and implementing of statistical systems by gathering the existing principles into common knowledge capital.
8.4.3 Australian Bureau of Statistics Data Quality Framework#
The ABS Data Quality Framework (ABS DQF) provides the standards for assessing and reporting on the quality of statistical information. It is designed for use by a range of data users and providers in different settings, including government agencies, NSOs and independent research agencies. It improves a user’s ability to determine whether a statistical product is fit for purpose and to interpret data. It can also assist in the development of statistical processes.
The ABS DQF is based on Statistics Canada’s quality assurance framework and the ES CoP. It defines seven dimensions of quality, namely: institutional environment, relevance, timeliness, accuracy, coherence, interpretability and accessibility.
8.4.4 UK Office for National Statistics Quality Management Strategy and Framework#
The ONS Quality Management Strategy (last revised 2015) (🔗) sets out the organizational commitment and approach to quality and quality goals. It helps to ensure that ONS meets its obligations under the UK Code of Practice for Official Statistics. It commits the organization to further develop a culture of quality to ensure that it:
produces statistical outputs that meet user needs for quality;
explains the quality of outputs to users by providing up to date metadata;
improves the quality of outputs and processes through standardisation, continuous improvement and quality reviews.
The ONS Statistical Quality Framework (🔗) supports the Quality Management Strategy by setting out the initiatives and activities that support, improve and assure the quality of outputs. It describes the day-to-day activities that are in place at an organizational level for:
quality assurance - anticipating and avoiding problems by walkthroughs of statistical outputs, providing guidance and training in quality assurance practices;
quality control - responding to observed problems, using policies for describing how corrections and revisions are handled;
quality improvement – undertaking improvements identified during methodological and quality reviews on a rotating basis; and
quality reporting - informing users of the quality of our outputs.
8.4.5 South African Statistical Quality Assurance Framework#
Statistics South Africa first developed its South African Statistical Quality Assurance Framework (SASQAF) in 2008 and revised it in 2010. It serves as a toolkit for the certification of statistical products as official, and the maintenance of the quality of the statistical products. It is primarily geared towards serving the needs of data producers and data assessors. However, it may be a useful tool for any agency concerned with data quality. Its formulation allows for easy integration with other national and international quality reporting tools.
It presents background information on the quality of statistics and the certification process applicable in South Africa. The eight quality dimensions are defined (nine including Prerequisites for Quality). They are based on the IMF’s DQAF quality dimensions and the ES CoP output principles. Each quality dimension is supported by quality indicators, each of which is broken down into standards that need to be implemented to ensure conformance with the indicator. Associated with each standard is an assessment level expressed in the form of four mutually exclusive categories. Guidelines to meet standards are provided for each dimension.
8.4.6 Italian National Institute of Statistics Quality Guidelines#
In 2012 Istat introduced its Quality Guidelines for Statistical Processes (🔗), which build on the ES CoP, also taking into account the IMF’s Data Quality Assessment Framework, especially in relation to economic statistics and the National Accounts. Following the ES CoP output principles, the requirements for statistical outputs are:
to be relevant with regard to users’ information needs;
to be accurate, that is to provide estimates or indicators that are as reliable as possible;
to be timely in measuring the phenomena being observed;
to be easily accessible and supported by metadata allowing for a full understanding of data; and
to enable comparisons over time or among different sources.
The Guidelines aim at describing the principles to be followed when planning, running and assessing a statistical process, as well as at illustrating quality requirements of statistics. They are in two parts.
The first part is dedicated to process quality and follows the phases of the statistical production process. For each phase, the principle or target to be achieved is stated, and it is accompanied by summary instructions or guidelines to be followed in order to accomplish it.
The second part concerns product quality. It describes and explains the quality requirements but does not contain guidelines for measuring quality, which are found in the first part.
The Guidelines are addressed to survey managers responsible for statistical production. They provide benchmarks for assessing process and product quality (as well as the degree of compliance with other European and national standards) using self-assessment and internal statistical audit. In each case, quality assessment is based on ascertaining the degree of compliance of statistical processes and products with the Guidelines’ principles and requirements.
8.4.7 Statistical Institute of Jamaica Quality Assurance Framework#
The Quality Assurance Framework of the Statistical Institute of Jamaica (SQAF) is structured in accordance with the first four sections of the UN NQAF, comprising:
Quality context – the circumstances and key issues driving the need for quality management, benefits and challenges, and relationships to other quality frameworks and code of practice.
Quality concepts – comprising methodological soundness, integrity and eight data quality dimensions.
Quality assurance guidelines – comprising 18 SQAF lines, generally following the UN NQAF lines but without coordination of the NSS or metadata management.
Quality assessment and reporting – as in the UN NQAF.
8.4.8 Palestinian Central Bureau of Statistics Code of Practice#
Through the Code of Practice for Palestine’s Official Statistics (🔗) the Palestinian Central Bureau of Statistics (PCBS) seeks to develop nationwide statistical practices and strengthen confidence in the Palestinian Statistical System. The Code draws on the experiences of statistically developed countries. It touches upon legal framework, the areas covered by official statistics, the importance of utilizing data compiled by PCBS, the importance of the media in statistics, and the role of statistical units in ministries and government agencies in addition to the PCBS.
The Code discusses best practices for statistical work based on the Fundamental Principles of Official Statistics, including the relationship between the PCBS President and the statistical units at ministries and government agencies, the role of the Advisory Council for Official Statistics, and interpretation and implementation of the Code.
In line with the Code, the PCBS:
applies the European Self-Assessment Checklist for Survey Managers (DESAP);
received ISO-9001:2008 certification in 2010;
received a Committed to Excellent certificate from EFQM in 2017; and
prepared guidelines for a Palestinian NQAF based on the UN NQAF in 2018.