Benchmarking the Current Employment Statistics state and area estimates

The Current Employment Statistics (CES) survey, conducted by the U.S. Bureau of Labor Statistics (BLS), is a monthly survey of approximately 147,000 business and government agencies, representing 634,000 individual worksites. The CES program provides detailed estimates on employment, hours, and earnings for the nation, states, Puerto Rico, the U.S. Virgin Islands, and more than 450 metropolitan statistical areas (MSAs). The CES survey is widely considered one of the most timely and accurate economic indicators published by the federal government.

Annually, the CES program benchmarks, or reanchors, the sample-based employment estimates for March of each year to the universe counts derived principally from the Quarterly Census of Employment and Wages (QCEW) program. This benchmark process is done to account for survey and other errors that accumulate over the year. The QCEW counts are much less timely than sample-based estimates and provide an annual point-in-time census for employment.

The current research agenda for the CES program includes examining alternative benchmarking approaches for national, state, and metropolitan area data. As background for that examination, this article discusses the current methods used to benchmark state and metropolitan area data and is a companion piece to three other articles on benchmarking in the CES program. In the first of the three articles, Christopher Manning and John Stewart discuss the current benchmark methods and results for national CES data. In the second article, Kenneth Robertson presents the conceptual underpinnings of benchmarking and a high-level overview of recent research. Finally, in the third article, Mark Loewenstein and Matthew Dey author a more technical article in which they discuss the alternative method that has shown the most promise.

Establishing benchmark levels and the postbenchmark period

With each annual benchmark, the standard practice for state and area series is to revise 20 months of not seasonally adjusted data before the normal monthly estimation processes begin on the new levels. For example, with the development of the March 2016 benchmark, levels were reestablished for the April 2015 through November 2016 reference months. December 2016 final and January 2017 preliminary estimates were developed based on those newly established levels.

A snapshot of the QCEW is the starting point for building the CES March benchmark level. Added to that level is an accounting of employment that is covered under the CES definition, but not by state unemployment insurance (UI) tax laws. This second segment of employment is called noncovered employment and is only present in select industries.¹

The total of the QCEW and noncovered employment (referred to as the population) replaces March sample counts for all national, state, and metropolitan series. The benchmark method differs between the state and metropolitan series and the national series. For the state and metropolitan area series, the employment levels for all months, from the April before the March benchmark through the September after the March benchmark, are also replaced by these population counts. For the national series, only the March benchmark month has its employment replaced by the population counts. The CES program staff adjusts the monthly levels before the benchmark month by applying a linear wedge that is based on the revision to the March level. Months subsequent to March are sample-based estimates from the new March benchmark level.²

CES staff generates October and November estimates as sample-based estimates linking from the newly established September level. Note that the original postbenchmark population values of April through September are updated again with newer versions of the QCEW and noncovered employment counts are updated with the next benchmark cycle. As given in the example at the beginning of this section, April 2016 through September 2016 employment data are replaced once as part of the March 2016 benchmark process. The April 2016 through September 2016 employment data are replaced again a year later as part of the March 2017 benchmark process.

In addition to replacing the estimates, BLS also addresses some administrative issues in the QCEW. The presence of noneconomic code changes (NECCs) is one such issue. As an example, each year approximately one-third of all establishments in the QCEW are contacted as part of the Annual Refiling Survey. State Labor Market Information agencies contact companies and ask them to verify their NAICS (North American Industry Classification System) industry classification, ownership, and location. These NECC corrections are implemented in the QCEW with first-quarter data. The effect of changes that represent less than 6 percent of the employment in a series is distributed across 12 months. For changes of 6 percent or more, CES staff will lengthen the number of months that the employment change is distributed across.

Reasons behind the current method

In general, small geographic areas and industries have fewer sample units, which leads to more variability in the estimates. Although error is associated with the administrative data that are based on the QCEW, BLS decided more than 30 years ago that the error in the administrative data was a better alternative to preserving the variability of the estimates as the final benchmarked time series.

Although the sample sizes from when the current benchmarking method was established are not readily available, the current sample sizes in cells illustrate the frequency of series with small sample sizes. CES estimation occurs at the most detailed levels, referred to as basic cells, and higher level industry totals are the sum of these basic cells. The CES program models more than 55 percent of the slightly more than 6,000 basic series, rather than using only the sample to estimate the data. It also models series when the sample is deemed too small,³ which is generally when the sample has fewer than 30 companies (as defined by a distinct UI account). The CES models combine a sample-based estimate with a trend component to smooth the data.⁴

Complications affecting the current method

Research by the Dallas Federal Reserve has shown that CES benchmarked population data display a seasonal pattern different from the sample-based estimates.⁵ The CES program accounts for the differences in the patterns by using a two-step seasonal adjustment process to develop the final seasonally adjusted series. However, BLS seasonally adjusts only about 2,000 state and metropolitan series compared with the 17,000 not seasonally adjusted series. Analyzing employment changes can be complicated, particularly over-the-year comparisons, without seasonally adjusted data. These changes cross over the splice point between population- and sample-based data.⁶

The seasonal differences also complicate interpreting revisions associated with the benchmark process. Tracking the population and sample differences nationally shows that typically the QCEW data grow by approximately a quarter million more than the CES sample-based data from March through September each year. The difference grows greater through the fourth quarter, and then the population data have a much larger seasonal decrease in the first quarter.⁷ As a result, QCEW data grow by approximately a quarter of a million less than CES sample-based data from September through March each year. These differences may represent administrative error or reporting error rather than true economic differences.⁸

As mentioned, state and metropolitan data are replaced with population data through September with each benchmark. Historically, BLS has highlighted the March revision as the primary indicator of the accuracy of the CES survey. However, given the seasonal differences described previously, in aggregate under the current method, the expected differences in March should average a downward revision of approximately 250,000.

Results

Table 1 provides a summary of March benchmark revisions at the state nonfarm level from 2003 to 2016. Table 2 provides comparable revisions for MSAs.

At the state nonfarm level, the average absolute March revisions were 0.5 percent between 2003 and 2016. Over the same period, the largest average absolute percent revisions at the supersector level were in mining and logging (4.3 percent) and construction (3.0 percent). The smallest average absolute percent revisions were in government (0.7 percent); trade, transportation, and utilities (0.9 percent); and education and health services (1.0 percent).

The average mean percent revision over the last 14 years is 0.05 percent. However, one should be cautious when interpreting that number. Because the number does not reflect the size of the individual states, it should not be interpreted as an average revision of the sum of states over the period.

Revisions since the 2005 benchmark for MSAs are included in table 2.⁹ For comparisons, breakdowns are by size of the MSAs. Across the most recent 13 years, the average absolute revisions have averaged 1.2 percent across all MSAs. As is expected, the larger the MSA, the smaller the absolute percent revisions are. MSAs more than 1 million in employment have averaged an average absolute percent revision of more than 0.6 percent, while MSAs with less than 100,000 employment have averaged slightly more than 1.4 percent.

Conclusions

The current benchmarking method used for state and metropolitan area estimates addresses the concern of excessive variability in the final benchmarked data that might result from estimates produced with small samples. In addition, replacing estimates from April through September following the March benchmark updates the series with the most current population information available. However, replacing the estimates with population values does not directly address many of the issues associated with the differing seasonal patterns between the population and sample data. Current research into alternative benchmarking approaches might yield a method that addresses some of the limits of the present-day benchmark method, while continuing to address concerns related to the volatility of small domain estimates.