Starting with the 2002–2012 projections evaluation, BLS standardized a set of methods for evaluating future projections sets. Those methods are described here.
Comparable projections are not available for most publication series, including the detailed sectors of both employment and the labor force. As an alternative evaluation measure, BLS uses a series of naïve models for comparison purposes.
A naïve model is any model that is simple to estimate and does not require sophisticated systems or processes. The more elaborate BLS model should outperform a naïve model for a majority of the variables presented.
For the macro naïve model, the average growth rate for CBO's potential GDP (as calculated in the base year) is calculated for the previous 10 years, and projected forward 10 years.
BLS projects the labor force by analyzing trends in the labor force participation rates (LFPR) of granular demographic groups, which are then applied to population projections from the U.S. Census Bureau. Our naïve model is a least squares linear regression based on the growth rates observed in the ten years preceding the base year.
The naïve model used for evaluating industry projections is a least squares linear regression based on the growth rates observed in the ten years preceding the base year.
The BLS projections method speculates about small changes to the share of industry employment for each occupation. These research–based changes represent BLS expectations of the next ten years of gradual changes to the mix of occupations which industries use to produce their output. For example, BLS analysts look for trends which are changing how hospitals use nurses or how construction companies use plumbers and adjust the projections to match those trends. The occupational–share naïve model shows how the projections would have looked if the overall proportion of each occupation in the economy were projected not to change.
A dissimilarity index is a way of measuring the amount of difference between two distributions. In our application, the value of the dissimilarity index is the amount that one distribution would have to change in order to match the other. Calculation details are available in Evaluating the 1996–2006 employment projections.
The mean absolute percent error (MAPE) provides a measure for comparing the accuracy of projections for the components of the labor force. The weighted MAPE weights each component in accordance with its share of the total, placing less emphasis on smaller components. Calculation details are available in Evaluating the 1996–2006 employment projections.
For macro variables—GDP and the Unemployment Rate—comparisons are made to potential values. Potential is the estimate of a value that would have occurred had the economy's resources been fully utilized in a sustainable manner. It is roughly the peak of a business cycle. Potential values are estimated by CBO. The comparison between rounds potential values are updated every time a new projection round is evaluated.
The economy changes over time and with it the data collected to describe it.
Changes in the race categories collected by the Census can limit the extent to which we can evaluate our labor force projections by race.
The Standard Occupational Classification (SOC) system used by Employment Projections is routinely re–evaluated so that it can capture new and emerging occupations. When these changes occur it can result in some occupations not being comparable between when the projection was made and when the projection can be evaluated.
More information on the SOC is available here:
In 2012 the North American Industry Classification System (NAICS) was updated. Several industries from 2012 NAICS and 2002 NAICS were combined, broken apart, and sometimes recombined in ways which precluded analyzing data at the most detailed level for affected industries. Greater variability exists among individual industries compared with sectors and major sectors. Therefore, we analyze only the major sectors of detail, rather than the roughly 200 industries published within the projections publications.
Internal, unrounded and unsuppressed data were used for some calculations. Use of publically available data will result in figures with compounded rounding issues and provide answers which may be different in significant ways.
Return to Projections Evaluation Homepage
Last Modified Date: August 1, 2018