Quality Measures

BLS employs many methods to ensure the quality of the data it publishes. These quality procedures exist throughout every stage of the survey lifecycle — from planning to collection to publication, as described in Statistical Policy Directive No. 2. This page describes the methods BLS uses to ensure data quality so users may assess whether the data are suitable for their own purposes.

Planning and Design

Most BLS measures are derived from surveys. New surveys and new processes undergo rigorous testing. These tests include sample design and selection, questionnaire design, estimation methodology, and other steps to ensure the measures collected from the survey accurately represent the population. These steps are critical to confirm that the survey instruments and procedures yield reliable and valid data, minimize bias, and ensure representativeness.

Some BLS data are from nonsurvey sources. These sources include administrative records from other government agencies and free or purchased datasets on commercial transactions. These nonsurvey sources undergo rigorous review to ensure they meet BLS standards for quality, reliability, timeliness, transparency, and other criteria. BLS also considers potential cost savings when evaluating whether to use alternative data sources.

Sample Selection

Most BLS data come from a sample or subset of the population we are trying to measure. BLS uses samples because collecting all the employment data or all the prices or wages from every firm in the nation would be time-consuming and expensive. Samples save time and money and are efficient at providing timely data.

BLS selects samples carefully to ensure they represent a population. One objective is to ensure samples are not biased to overcount or undercount any industry, geographic area, demographic group, or other characteristic of a population. At the same time, many samples are also designed to collect enough information to provide a greater level of precision about those groups.

Response Rates

Response rates are often viewed as a quality measure but are really a measure of productivity. Response rates show the proportion of the intended sample we successfully collected. Response rates are an indicator of the risk of bias, but not necessarily the presence of bias. Learn more on our page about household and establishment survey response rates.

Rigorously Trained Data Collectors

Much of the data BLS collects are self-reported by respondents through web questionnaires. However, some BLS data are collected through interviews by BLS or U.S. Census Bureau staff. For these data, interviewers are rigorously trained to ensure they follow established procedures to help respondents understand the questions. Interviewers can provide prompts if necessary. Learn more about resources for survey respondents.

Data Validation

BLS staff reviews collected data to validate respondents' reports by checking for completeness and ensuring interviewers correctly recorded their responses. The initial review often includes comparing the reported values with the respondent’s prior reporting history. These reviews may be automated and prompt a follow-up question to explain a change in reporting.

Example

Company A typically reported employment of 60 employees. In the most recent month, however, Company A reported employment of 600 employees, a 10-fold increase. This increase would prompt a follow-up question from BLS asking to explain the large increase. The follow-up question may result in an internal note that there was a large hiring event, or it may result in the respondent correcting the data.

Outlier Detection

After data are collected and compiled, BLS economists and statisticians look at the grouped observations (by industry, occupation, geography, and so forth) of the collected data and identify if an observation is behaving differently from the group. This may prompt a follow-up to the respondent.

Example

The average wage for widget polishers in the widget manufacturing industry is $22.45 per hour, and the 90th percentile wage is $29.35. (The 90th percentile means 90 percent of workers earn that wage or less.) A company in that industry reports that it pays its widget polishers $45.00 per hour. This response may result in a follow-up with the respondent and a possible correction of the reported wage.

Imputation Counts or Rates

Imputation is a statistical process of filling in missing survey data, usually with information from other survey respondents that have characteristics similar to the respondent with missing information. Imputation counts or rates measure the frequency with which BLS fills in missing observations using trusted techniques.

To learn more, watch the Consumer Price Index imputation video.

Estimates of Variation and Standard Error

Because most BLS data come from a sample or subset of the population, we publish estimates of sample variance, standard errors, and coefficients of variation. Although these estimates each have slightly different nuances, smaller error estimates generally mean an economic measure derived from the sample is closer to the true measure from the population.

Publication Criteria

BLS balances the desire to publish as much data as possible with the need to protect the privacy and confidentiality of the people and businesses who willingly respond to our surveys. Each BLS program assesses the data it produces to ensure it meets all the criteria required for publication, which include confidentiality protection, statistical quality, and timeliness.

BLS will often publish preliminary estimates, which are subject to revision. See our revisions section to learn more about the types of revisions made.

The Handbook of Methods provides more information on sample design, data validation, outlier detection, revisions, variance and standard errors, imputation, publication criteria, and other quality methods in specific BLS programs.