2.13 The statistical production processes#
The process of producing official statistics can, in simple terms, be described as involving four logical steps – identifying user needs, collecting data, processing and analysis of the statistics, and reporting and disseminating the findings. Traditionally, these processes have, as a rule, been undertaken within the different divisions or units of NSOs. Thus, as an example, the agricultural division of the NSO has taken care of all the individual steps of producing agricultural statistics. The division’s statistical staff have designed the necessary surveys, collected the data, checked for errors, cleaned and edited the data, processed and tabulated, analysed the statistics, and reported on the findings. The price statistics have been compiled similarly within the department of price statistics, and the same goes for the various other statistics of the NSO.
This traditional system is referred to as a stovepipe or silo system. The reason for and the strength of this system is that it has ensured that there is systematic knowledge of the different subjects for which the statistics are to be compiled. The ensuing weakness of the system is that it does not focus on the statistical functions that are common to all statistical production processes and does not allow or encourage internal cooperation across the boundaries of different subject matter departments. Many NSOs have sought to alleviate these shortcomings by organizing centralised support for different subject matter departments in areas like questionnaire design, methodology, data collection, IT services, data editing and dissemination, all of which have helped increase the efficiency of the statistical production. However, this has been considered insufficient and efforts have been made to create a more functional system for the statistical production processes.
Significant support in these efforts was the creation of the Generic Statistical Business Process Model (GSBPM). This model has been developed under the coordination of the Statistical Division of the UNECE in Geneva, based on innovative practices in a few leading NSOs. The GSBPM seeks to describe and guide the overall process of the statistical production as well as the individual production processes. The idea behind the GSBPM is that the statistical production is better organized around functions than subject matters and that the same procedures can be utilised for the generation of several subject matter statistics. Thus, as an example, the same procedures for collecting data apply to several subject matter areas. Also, instead of designing and building specific methods and IT tools for each subject matter area, the idea is to build methods and tools for the different functions that can be utilised in many subject matter areas.
It is useful to describe the overall statistical production process in terms of the GSBPM. The model identifies and describes eight phases of the overall statistical production process (specify needs, design, build, collect, process, analyse, disseminate, and evaluate), divided into sub-processes; 44 sub-process in all.
The production process starts by identifying the needs of the particular statistics that are being considered. Here, the recommended procedures apply equally to all types of statistics; it is necessary to determine what statistics are needed, who needs them and for what purposes, if there are similar statistics available, and what are the pros and cons, gains and costs of producing new statistics. This phase ends with deciding whether to proceed and plan for a new or modified statistical product and, if so, what this product should look like.
The second phase (design) involves determining how the new product should be produced and designing the methods and procedures for creating it.
The third phase (build) involves building the tools for producing new or amended products. Both this and the design phase make heavy demand for the IT and methodological services of the NSO. Here, the basic assumption is that the same methods and IT applications can be used in the production of several different products. This requires that the software and applications be designed and built as modules that can be used in many production streams and interchanged. This is one of the keys to enhancing the efficiency of the production processes.
The fourth phase (collect) involves collecting data needed for the new or amended statistical product. The data collection procedures are based on outputs of the previous phases. The collection methods have been determined and designed so all that is needed is to organize, prepare, and implement the data collection. This phase includes hiring or selecting and training staff involved in data capture, both in surveys and in other data collection modes. It also includes a possible trial run of the data collection, usually referred to as conducting a pilot survey or data collection.
The fifth phase (process) involves checking and editing the collected data and preparing it for analysis, as well as carrying out the necessary tabulation.
The sixth phase (analyse) involves analysing the new statistics, laying the foundation for analytic reports of the new or amended statistics.
The seventh phase (disseminate) involves writing and editing the analytic reports, preparing press releases based on the new statistics, including producing such graphs and other visual means that may enhance the message brought out by the new statistics. This phase involves the actual release of the statistics and subsequent press releases and reports in accordance with the release calendar of the NSO, editing of the website on which the statistical products are posted, and communicating with users, seeking and capturing their feedback.
The eighth phase (evaluate) involves evaluating the new product and the production processes that were applied. This evaluation is carried out for each sub-process applied in the production of the new or amended statistics. The basic idea is to assess the quality and efficiency of each step of production as well as the overall quality of the end product.
The evaluation of a product and the process by which it was produced requires that all decisions and actions taken in each sub-process of each phase be thoroughly documented in such a way that the documentation at each stage forms the basis and is used for subsequent stages. This documentation is referred to as process or structural metadata.
The GSBPM is said to have several over-arching processes, i.e. processes that apply to the whole production process. One of these is metadata management and involves both the creation of metadata at each stage and its transfer and utilisation at subsequent stages of the overall production process. Metadata may be grouped into two types, process metadata and product metadata. The process metadata informs in detail on the methods and procedures applied in the statistical production, as described above. The process metadata is for specialised use in the statistical processes and for use by experts for enabling them to evaluate in detail the quality and the robustness of the statistics. The product metadata is compiled to inform the users about the specifications of the statistics, their strengths, weaknesses, applicability, comparability, and delimitations. Most NSOs strive to compile product metadata, at least that pertaining to the statistics most used, and publish on the web.
Another main over-arching process concerns quality management. To improve quality, quality management should be present throughout the business process model, based on the evaluation and quality control at each stage, each sub-process. If done in accordance with the suggestions of the GSBPM, quality failures can be detected and analysed at every stage of the process, traced to failures at previous stages, corrected or amended, thus raising the quality of each sub-process and the final product.
The awareness and use of the GSBPM have grown substantially in the last few years. The main importance can be the impact it has had on replacing the traditional stovepipe thinking and subject-oriented approach to producing statistics and encouraging planning based on functions that are common to all statistical production. In this way, the GSBPM has increased communication across subject matter boundaries as well as cooperation between methodologists and subject matter experts. It has encouraged IT experts to design their applications as interchangeable modules between production processes and can be reused in several processes and for several products. The GSBPM has also led to increased focus on documentation of the production processes of different products, thereby greatly facilitating amendments of processes and products leading to increased quality of the statistics.