Volume 29 (2013): Edition 1 (March 2013)

With tighter federal budgets on the horizon, the National Agricultural Statistics Service decided in 2009 to pursue three architectural transformations, primarily to provide savings in staff resource costs by enabling the centralization or regionalization of survey operations. The transformational initiatives involved: (1) centralizing and consolidating network services from 48 locations; (2) standardizing survey metadata and integrating survey data into easily accessible databases across all surveys; and (3) consolidating and generalizing survey applications for the agency’s diverse survey program. The three architectural transformations will be described as well as initial efforts to consolidate and standardize survey operations across the agency.

There is a risk of introducing survey error at every stage of any study involving a survey: design, data collection, processing, and analysis. Effectively managing the survey sample from the time of sample selection through the survey lifecycle is essential to producing highquality data on schedule and within budget. Managing the survey lifecycle using software systems that are not fully integrated can result in error and cost inefficiencies. The development of an integrated data collection and management system that supports monitoring of survey error has the potential to reduce errors and improve operational efficiencies. This system, referred to as Nirvana, uses a standardized database, protocol, and terminology. It integrates case status and history information across modes of data collection and tracing as well as sample and contact information. Nirvana also centralizes questionnaire development, quality monitoring, and support for real-time survey management decisions.

In response to a changing environment, Statistics Netherlands has embarked on a large-scale redesign of the way statistics are produced. The aim is to increase the capability to respond to changing information demand, to lower the response burden for surveys, especially for businesses, and to improve efficiency, while preserving the overall quality level. The redesign is carried out within the framework of a so-called enterprise architecture, which gives overall guidance when structuring the processes of the organisation, including statistical methods and IT tools used. The article describes the redesign approach and explains the main features of the architecture. The emphasis is on experiences that may be relevant to other national statistical institutes operating in a similar environment.

In 2011, Statistics NZ embarked on an ambitious ten-year programme of change called Statistics 2020 Te Kāpehu Whetū: Achieving the statistics system of the future. This article outlines a key component of this approach to transforming how Statistics NZ delivers its statistics. This involves modernising all aspects of statistical production - from identifying need, right through to dissemination - to ensure the organisation has the functionality and capacity to deliver now and into the future. Standardisation of processes, methods, tools, and systems to increase flexibility and efficiency is a key component of this transformation.

This article focuses on methods for enhancing access to survey data produced by government agencies. In particular, the National Center for Health Statistics (NCHS) is developing methods that could be used in an interactive, integrated, real-time online analytic system (OAS) to facilitate analysis by the public of both restricted and public use survey data. Data from NCHS’ National Health Interview Survey (NHIS) are being used to investigate, develop, and evaluate such methods. We assume the existence of public use microdata files, as is the case for the NHIS, so disclosure avoidance methods for such an OAS must account for that critical constraint. Of special interest is the analysis of state-level data because health care is largely administered at the state level in the U.S., and state identifiers are not on the NHIS public use files. This article describes our investigations of various possible choices of methods for statistical disclosure control and the challenges of providing such protection in a real-time OAS that uses restricted data. Full details about the specific disclosure control methods used by a working OAS could never be publicly released for confidentiality reasons. NCHS is still evaluating whether to implement an OAS that uses NHIS restricted data, and this article provides a snapshot of a research and developmental project in progress.

This article outlines a framework for formal description, justification and evaluation in development of architectures for large-scale statistical production systems. Following an introduction of the main components of the framework, we consider four related issues: (1) Use of some simple schematic models for survey quality, cost, risk, and stakeholder utility to outline several groups of questions that may inform decisions on system design and architecture. (2) Integration of system architecture with models for total survey quality (TSQ) and adaptive total design (ATD). (3) Possible use of concepts from the Generic Statistical Business Process Model (GSBPM) and the Generic Statistical Information Model (GSIM). (4) The role of governance processes in the practical implementation of these ideas.

These five articles and the framework proposed by Eltinge, Biemer, and Holmberg illuminate an exciting view of the future of official statistics. The articles are innovative, forward-looking, and remarkably complementary to one another. Below, for brevity, I refer to the five articles as the NASS, StatsNL, StatsNZ, RTI and Westat/NCHS articles. I will comment on each article individually, on them collectively in the context of the framework, and on the entire concept of “systems and architectures for high-quality statistics production.” First, however, I present two complementary perspectives on official statistics.

This special issue on “Systems and Architectures for High-Quality Statistics Production” is a stimulating resource for statistical agencies and private sector data collectors in a challenging time characterized by massive amounts of data, from a variety of sources, available in varying intervals, and with varying quality.

Traditionally, statistical products were created from a single source, most often through surveys or administrative data. However, neither surveys nor administrative data alone can match the data needs of today’s society. In addition, the need to reduce the costs of data production necessitates that multiple sources are used in combination. The need to reduce costs also necessitates the streamlining of production cycles, and the increasing difficulties in data collection itself require such systems to be much more flexible than they have been in the past. Increasingly, these reasons are driving statistical agencies and private data collectors to redesign their entire data production cycle. The examples in this special issue from Statistics Netherlands and Statistics New Zealand demonstrate such developments in government agencies; the example from RTI reflects efforts visible among private sector data collectors. This commentary will highlight some issues of general interest related to organizational challenges, and some that create the basis for reproducible research and are therefore of general interest to the research community.