8.1 Introduction#
8.1.1 Overview#
This chapter discusses quality management for a national statistical office (NSO), including general quality management principles, the development and administration of a statistical quality assurance framework, definition and implementation of quality monitoring and evaluation, user surveys, the labelling of official statistics, the quality certification of the NSO, and the relationship of quality management to other strategic initiatives, such as risk management.
Quality management is essential in building the value of official statistics and a key element in increasing user confidence. NSOs should have institutional arrangements that ensure compilation of high quality, objective and independent statistics that are not influenced by any interest. As stated in the ECE’s Recommendations for Promoting, Measuring and Communicating the Value of Official Statistics (🔗), the unique value of official statistics lies in the rigorous quality criteria and professional ethics of statistical production in accordance with the Fundamental Principles of Official Statistics (🔗). The benefits of official statistics compared to data from other sources flow from the application of these principles.
A quality management framework takes, as its initial starting point, the mission, vision, values and strategic objectives of an NSO. Its basic objective is to build quality into all processes within the organization that are pertinent to its core business of producing statistics. As this requires a significant investment of resources, such a framework has to be justified in terms of benefits it brings, which may be articulated as follows.
It provides a basis for creating and maintaining a culture of quality within the organization and, more generally, within the national statistical system (NSS).
It is focused on users and their needs and thus calls for regular review of changes in society and among stakeholders that may affect the work of the organization.
It provides a systematic mechanism for ongoing identification of quality problems and possible actions for their resolution, whether by incremental improvement or re-engineering.
It supports a continuous review of processes and quality improvement actions.
It stimulates staff participation, engagement and interaction throughout the organization and the NSS.
It documents guidelines, processes and tools for assuring quality and for training statisticians.
It draws attention to synergies and interconnections within the organization and helps to integrate processes, systems and tools.
It gives greater transparency to the processes by which quality is assured and reinforces the image of the organization as a credible provider of good quality statistics.
It is the mechanism for the promotion of quality assurance across the NSS.
It is the mechanism for the exchange of ideas on quality management with international statistical organizations.
More generally, a quality management framework is a mechanism by which an NSO can ensure that, given the resources at its disposal, the value of the statistics it produces is optimised. As further stated in the Recommendations for Promoting, Measuring and Communicating the Value of Official Statistics (🔗) referenced above:
“Producers of official statistics are by no means the sole supplier of information. The world is awash with data. There is a growing confusion between official statistics and less reliable data. This may give more weight to opinions and impressions. Official statistics need to stand out as a trustworthy source of information. This is also crucial for persuading respondents about the importance of replying to statistical surveys…”
“Demand for statistics is rapidly growing. An increasingly globalized and interconnected world creates new needs for accurate information about economies and societies…”
A quality management framework is the mechanism by which an NSO can ensure that it recognises such opportunities and challenges as they arise and can respond to them.
8.1.2 Terminology#
It is important to establish a common understanding of the terminology as quality-related terms tend to be used with a variety of meanings. For example, consider quality management framework, quality management system, quality assurance framework, and data quality assessment framework. Are these all the same thing? If they differ, how do they differ? The following paragraphs define the key terms used in the chapter. The starting point is the ISO 9000:2015 Quality Management System (🔗) standard (further described in Chapter 8.2 — Generic quality management systems and other relevant standards). The definitions are then specialised to the specific context of NSOs, drawing heavily on the United Nations National Quality Assurance Frameworks Manual for Official Statistics (UN NQAF Manual) (🔗), which is elaborated in Chapter 8.3 — Quality assurance frameworks, guidelines, and tools.
Quality: degree to which a set of inherent characteristics of an object fulfils requirements.
This rather succinct, even opaque, definition from ISO 9000:2015 is widely used. Inherent means existing in the object, as opposed to assigned (like a price). The object may be anything that is perceivable or conceivable. Specifically, in the context of an NSO, the object may be a statistical output, the statistical process that produced it, the institutional environment housing the process, or the whole statistical system.
A simpler characterisation of quality is fit for use or fit for purpose.
It is users’ needs that define output quality. Different users may have different needs that must be balanced against each other to provide the quality concept with concrete content. Over the past 20 years, NSOs have arrived at the consensus that quality is multidimensional and that there is no single measure of quality. Thus, the definition of output quality is operationalized by specifying a set of dimensions that characterize it, typically relevance, accuracy and reliability, timeliness and punctuality; accessibility and clarity; and coherence and comparability, as further discussed in Chapter 8.3 — Quality assurance frameworks, guidelines, and tools.
Quality management: coordinated activities to direct and control an organization with regard to quality. Quality management includes establishing quality policies and objectives, and processes to achieve these objectives through quality planning, quality assurance, quality control, and quality improvement.
Quality management system: a set of interrelated or interacting elements of an organization to establish quality policies and quality objectives, and processes to achieve those objectives. In the context of an NSO, a quality management system is more commonly referred to as a quality management framework, or simply a quality framework.
Generic quality management system: a quality management system model or standard that can be applied to any type of organization. Examples are the ISO 9001:2015 Quality Management System - Requirements (🔗) and the European Foundation for Quality Management (EFQM) Excellence Model (🔗), both of which are further described in Chapter 8.2 — Generic quality management systems and other relevant standards.
Quality management principles: a comprehensive set of principles on which a quality management system is based. The most widely used articulation of quality management principles is included in the ISO 9000:2015 and accompanying documents In the context of NSOs, and in this chapter, a distinction is made between general quality management principles, which are associated with an organization as a whole, and statistical quality principles, which are associated with the core statistical infrastructure and processes.
Quality assurance: the part of quality management focused on providing confidence that needs or expectations regarding quality will be met. Quality assurance provides an organization’s guarantee that the product and service it offers meet accepted quality standards. In the context of an NSO, it comprises a planned and systematic pattern of actions necessary to provide confidence that a product, and the process that produces it, conform to established requirements. It is achieved by identifying statistical quality principles, by applying them to the core statistical infrastructure, processes, and outputs, and to the institutional environment and whole statistical system within which they are embedded, and by measuring the extent of their achievement.
Quality assurance framework: the procedures and systems that support quality assurance within an organization. The term quality assurance framework is used in the context of NSOs to mean the part of the quality management framework that provides confidence that the stated needs or expectations of users are being met. It is based on the definition of quality, the statistical quality principles, and the methods and tools that are used to ensure the principles are implemented. A quality assurance framework together with the procedures for application of general quality management principles constitute a quality management framework.
Quality assessment: the part of quality assurance that focuses on assessing the extent to which quality requirements have been fulfilled. In the context of NSOs, quality assessment, quality evaluation, and quality review are regarded as synonyms and for brevity are referred to simply as evaluation, the term used in the Generic Statistical Business Process Model (GSBPM).
8.1.3 Developing a quality management framework#
In line with the definitions above, development of a quality management framework involves consideration of two elements.
The first and most significant element is a quality assurance framework that specifically addresses the core business of the NSO. Many countries have developed their own individual national quality assurance framework (NQAF), often making use of an already existing generic quality assurance framework or taking advantage of NQAFs of other countries.
The second element is focused on implementing general quality management principles for the organization as a whole. It concentrates on promoting a culture that ensures quality. It can be based on a generic quality management system, which may then be used as the basis for quality certification of the organization. Alternatively, it can be achieved simply by supplementing the NQAF with general quality management principles. Most NSOs choose this latter approach and define a single quality management framework embracing both elements, typically still referring to it as a quality assurance framework. A few NSOs choose to separate the elements and introduce two parallel systems:
a quality assurance framework, which focuses on individual statistical production processes and the statistical infrastructure that supports them; and
a quality management system, based on a generic system, which deals with the organization as a whole.
If an NSO makes the latter choice, it is typically because it wants the discipline of seeking certification for the entire organization based on an international or supranational quality management system standard such as ISO 9001 or the EFQM Excellence Model.