Handbook of Methods Office of Productivity and Technology Data Sources
Office of Productivity and Technology: Data Sources
The U.S. Bureau of Labor Statistics (BLS) productivity program uses data from a variety of different sources for its estimates of productivity growth. Data sources for each component needed to measure labor productivity and total factor productivity (TFP) are described briefly in this section.
A comprehensive list of data sources that BLS uses to estimate productivity measures can be found in the Data Sources section of the productivity website. The Office of Productivity and Technology (OPT) adheres to the BLS confidentiality pledge and follows all applicable data confidentiality laws.
Output
For business, nonfarm business, private business, private nonfarm business, and nonfinancial corporate business sectors of the U.S. economy, an output index is constructed using real value-added output measures published by the U.S. Bureau of Economic Analysis (BEA). These output measures are based on and are consistent with the National Income and Product Accounts (NIPA), including the gross domestic product (GDP) measure also constructed by BEA.1 BEA calculates quarterly and annual measures of business sector output by removing the value added of general government, households, and nonprofit institutions serving households from GDP. These measures, as well as nonfarm business sector output, are the real output series used to calculate BLS measures of labor productivity in the U.S. business and nonfarm business sectors. For the nonfinancial corporate business sector, BEA further excludes unincorporated business and those corporations classified as offices of bank holding companies, offices of other holding companies, or offices in the finance and insurance sector. To measure TFP, BLS further restricts output to the U.S. private business sector, excluding the output of government enterprises. Value-added output data for federal, state, and local government enterprises are obtained from BEA. BLS TFP statistics for the private business and private nonfarm business sectors are constructed using annual BEA output data.
For the major industries, as well as the manufacturing durable and nondurable goods sectors, the annual output index is calculated using a sectoral output concept based on data obtained from the U.S. Census Bureau. Output indexes for industries within the manufacturing sector use the current dollar industry value of production with intra-industry transactions removed (to avoid double counting production) and are deflated using prices from the BLS Producer Price Index program. For some nonmanufacturing industries, physical quantities of output are measured using data from the U.S. Department of the Interior and the U.S. Department of Transportation. For the remainder of nonmanufacturing industries, output measures are constructed using data from BEA’s Industry Economic Accounts.
Quarterly indexes of manufacturing output underlying the quarterly labor productivity data are estimated using annual manufacturing sectoral output data from the Census Bureau and the monthly industrial production indexes from the Federal Reserve Board. Because of a lag in the availability of the annual sectoral output data, recent quarterly and annual manufacturing output measures are extrapolated based on changes in the industrial production indexes.
BLS uses BEA’s real value-added GDP by state- and industry-level detail to publish annual state-level measures of output for the private nonfarm sector. BLS adjusts BEA’s real GDP by state for the private sector by removing the output of the farm sector, the output of households, and owner-occupied housing to derive output for the private nonfarm sector. While BEA publishes real GDP by state for the farm sector, they do not for households and owner-occupied housing. BLS uses BEA’s state employee compensation and national chain-type price indexes data to estimate output for households. To estimate output for owner-occupied housing, BLS uses unpublished data on state-level imputed rent and applies each state’s share of national imputed rent to BEA’s estimates of national owner-occupied housing GDP. BLS then uses BEA’s national chain-type price index for owner-occupied housing to deflate this estimate.
For detailed industry-level productivity output data, output indexes are prepared using data published by various public and private agencies, at the most granular level possible. The economic censuses and annual surveys of the Census Bureau are the primary sources of revenue data used in developing measures of sectoral output. Price data used to deflate current dollar values into constant dollars include the BLS Producer Price Index (PPI), BLS Consumer Price Index (CPI), and BEA NIPA Merchant Wholesale deflators. Data for industry-level output measures come from several data sources, including the U.S. Departments of Energy, Interior, and Transportation; the Federal Reserve Board; the Federal Deposit Insurance Corporation; and the U.S. Postal Service. Data from trade associations are also used for some industries.
The source of industry revenue data varies by sector. For manufacturing industries, the Census Bureau’s Annual Survey of Manufactures (ASM) and Census of Manufactures are the primary data sources. For the mining sector, data from the U.S. Geological Survey (USGS) and U.S. Energy Information Administration (EIA) are used. For retail trade industries, the primary data sources are the Census Bureau’s Annual Retail Trade Survey (ARTS) and the Census of Retail Trade. Wholesale trade industry data comes from the Census Bureau’s Annual Wholesale Trade Surveys (AWTS) and the Census of Wholesale Trade. Most of the services-providing industry data comes from the Census Bureau’s Service Annual Survey (SAS) and Census of Service Industries.
To develop measures of sectoral output, BLS estimates intrasectoral transactions, which are transactions between businesses operating in the same industry or sector. These are excluded from the output of industries or the sector. For the wholesale trade industries, total revenues are adjusted by distribution figures from the Economic Census release in the “Miscellaneous Subjects-Sales by Class of Customer” table. In certain service-providing industries, revenues are adjusted by within-industry access charges from the Census SAS table on “Estimated Selected Expenses for Employer Firms.” For intrasectoral transactions in the manufacturing industries, BLS uses the Census Bureau’s “Materials Consumed by Kind of Industry” table. The table measures transactions only from the consumer side, which results in a few limitations that BLS adjusts for. Consumption figures must be adjusted to remove imports and extraneous margin costs, which are not part of domestic production. To remove these costs, the “Import Matrices/Before Redefinitions” and “Use Tables” from BEA’s Input-Output Accounts are used. Consumption figures must also be adjusted to account for production of secondary products. The “Materials Consumed by Kind of Industry” table does not specify whether the listed products are from primary or secondary producers. BLS uses primary and secondary product data from the Census Bureau’s detailed industry statistics to distribute production of intermediate materials across both primary and secondary industries.
Hours worked
The primary source of hours and employment data is the BLS Current Employment Statistics (CES) survey, which provides monthly data on weekly hours and employment in nonfarm establishments. These data are adjusted from an hours-paid to an hours-worked basis using data from the BLS National Compensation Survey (NCS) to account for paid time off and the Current Population Survey (CPS) to account for off-the-clock hours worked by salaried workers. In addition, CPS data are used to estimate the hours worked by the unincorporated self-employed and unpaid family workers.
For some industries, estimates of hours worked are derived from other sources:
- Estimates of hours worked in government enterprises are calculated as the product of employment from BEA and average hours worked from the CPS.
- Nonprofit and corporate factor ratios to remove hours for these portions of the economy for major sectors and industries come from BEA and the Census Bureau.
- For TFP measures for major industries, farm hours are based on U.S. Department of Agriculture farm survey data. Hours worked measures for agriculture-related industries are created using employment data from the BLS Quarterly Census of Employment and Wages (QCEW) and average weekly hours from the CES.
- For mining industries, estimates of nonproduction worker hours are derived from data collected by the U.S. Mine Safety and Health Administration.
- For the air transportation industry, hours-worked measures are calculated with data from the U.S. Bureau of Transportation Statistics (BTS).
- For line-haul railroads, hours-worked measures are derived with data from the U.S. Surface Transportation Board (STB) and supplemented with data from the Association of American Railroads (AAR).
- CES provides employment data for postal service employees and the U.S. Postal Service provides their hours worked.
- Hours worked in the armed forces are calculated as the product of employment from the U.S. Department of Defense (DOD) and the average hours worked from the CPS.
Data source limitations for output and hours worked
Various data sources are used as the starting point for output measures but may have limitations that need to be addressed. Limitations are unique to each dataset, but common ones include the following:
- Time lags between measurement and publication
- Incomplete coverage of more detailed industries
- Missing or undisclosed values
- Limited or no data on nonemployer output
BLS uses supplemental data sources to overcome many of these limitations. To address time lags, BLS uses data from higher-frequency datasets, such as the Monthly Retail Trade Survey (MRTS), Monthly Wholesale Trade Survey (MWTS), and Quarterly Services Survey (QSS). To estimate data for more detailed industries, BLS uses data from other datasets, such as the Economic Census, to apportion the output from a published aggregate industry. Some businesses have no paid employees; thus, to account for nonemployer output, BLS uses data from the Census Nonemployer Statistics (NES).
Industry and hours data for second jobs (for multiple jobholders) in the Current Population Survey are available starting in 1994. To estimate hours worked by unincorporated self-employed and unpaid family workers for more detailed industries, BLS uses employment and average weekly hours worked data from the CPS. However, the CPS does not have full coverage of the detailed industries that CES covers. To match industry coverage with CES, BLS uses multiple data sources depending on the industry to apportion the CPS employment and hours worked. These data sources include Census Nonemployer Statistics (NES), Internal Revenue Service (IRS) partners and partnership data, County Business Patterns (CBP), Service Annual Survey (SAS), Annual Retail Trade Survey (ARTS), Economic Census, and CES. Measures from these data sources vary and include the number of proprietorships and partnerships, number of employees, nonemployer firm revenue, and number of establishments with 1 to 4 employees.
Total factor productivity estimation uses a measure of labor input that accounts for the skills and experience of different types of workers. For TFP measurement, BLS cost-share weights the hours of different groups of workers based on their characteristics of age, sex, education, and class of worker (wage and salary or self-employed). The difference between the growth in labor input and the growth in labor hours worked is the growth in labor composition.
The primary data source for the number of workers in each group, their hours, their compensation, and demographic characteristics is the 1-Year American Community Survey (ACS). Data are first tabulated for each 4-digit North American Industry Classification System (NAICS) industry and then aggregated up to the major industries that span the economy.
BLS supplements the 1-Year ACS data with other data sources to address its limitations. To address the unreliability of measures at the detailed level, BLS uses a small area estimation model that uses data from both the 5-Year ACS and the CPS. To account for multiple jobholders, data from the CPS is used to adjust employment and hours across industries. Data from the CPS is also used to control the industry-level estimates to an aggregate level. Finally, industry hours are scaled to the BLS productivity program’s published hours.
Labor compensation
Current dollar labor compensation measures are prepared using employee compensation data from BEA. Compensation data include wage and salary accruals (including executive compensation), commissions, tips, bonuses, payments in kind (representing income to the recipients), and employer-paid supplements. Supplements include employer contributions to funds for government social insurance, private pension and health insurance plans, compensation for injuries, etc. For labor productivity measurement, compensation per hour for the unincorporated self-employed is assumed to be equal to employee compensation per hour.
For TFP, compensation for the unincorporated self-employed is derived from BEA’s proprietors’ income. Proprietors’ income includes both capital and labor income. An initial value of labor compensation per hour for the unincorporated self-employed is assumed to be the same for an average payroll employee in that sector. Initial estimates of capital income and labor compensation are then adjusted to be consistent with the value-added estimates from the Industry Accounts at BEA.
Measures of real compensation per hour are derived by adjusting hourly compensation for changes in consumer prices. The price changes for recent quarters are based on the BLS Consumer Price Index for All Urban Consumers (CPI-U). For earlier periods, consumer prices are based on the BLS Consumer Price Index retroactive series (R-CPI-U-RS). These earlier periods do not represent the quarters of the current year but do include the previous calendar year of price data, which is obtained by OPT in April.
For detailed industries in the service-providing, trade, and mining sectors, current dollar labor compensation is derived using annual wage data from the BLS QCEW, along with data on employer costs for supplemental benefits from the Census Bureau and BEA. For detailed industries in manufacturing, annual payroll and supplemental benefit data from the Census Bureau are used. BLS does not produce real compensation per hour for major and detailed industries.
At the major sector and 2- and 3-digit industry levels of aggregation, for depreciable assets, like structures and equipment, capital inputs measures are based on BEA’s fixed asset accounts by detailed asset and BEA’s GDP by industry. For more detailed manufacturing industries, investment amounts for broad categories of capital assets are derived using annual capital expenditures from the economic censuses and annual surveys of the Census Bureau. Additional detailed asset investment data comes from the fixed asset accounts from BEA. Annual investment data are supplemented with more detailed benchmark data from BEA’s Capital Flow table and the Census Bureau’s Annual Capital Expenditures Survey. Price deflators for each asset category are constructed by combining detailed price indexes (mostly BLS producer price indexes) with weights that reflect each industry’s relative use of individual asset commodities.
The U.S. Bureau of Transportation Statistics (BTS) provides annual quantities of airframes and engines that compose a substantial part of the capital stock in the air transportation industry. For air transportation assets other than airframes and engines, detailed annual expenditures on equipment and structures from BEA is used. Inventories of parts and supplies from BTS are also included; the current dollar series is deflated with a weighted cost index based on data from Airlines for America and BTS. The Surface Transportation Board (STB) and Amtrak provide current dollar investment data for capital equipment and structures within line-haul railroads. Capital investment is deflated with either BLS PPIs or deflators based on BEA data. Estimates of investments in land from the STB and Amtrak are deflated with price indexes from BEA.
For nondepreciable assets, such as inventories and land, stocks are developed using data from BEA and the Internal Revenue Service. Farmland input is based on data from the Economic Research Service of the U.S. Department of Agriculture.
Data on intermediate inputs—energy, materials, and services—are calculated from measures based on data from BEA Industry Accounts. For detailed manufacturing industries, nominal values of these inputs, along with quantities of electricity consumed, are obtained from economic censuses and annual surveys conducted by the Census Bureau. To avoid double counting, an adjustment is made to the materials estimates to exclude the value of intra-industry commodity transfers. Services are estimated using annual industry data and benchmark Input-Output tables from BEA. Constant dollar materials consumed are calculated by dividing annual current dollar industry purchases by a weighted price deflator for each industry. Aggregate materials deflators are constructed for each industry by combining producer price indexes (PPIs) and import price indexes from BLS for detailed commodities. The deflators are combined using weights based on detailed commodity data from the BEA benchmark Input-Output tables. Aggregate price indexes to deflate services are constructed in a similar manner using consumer price indexes (CPIs), PPIs, and deflators developed by BEA. The value of fuels consumed by each industry is deflated with a weighted price deflator based on PPIs for individual fuel categories; the weights reflect fuel expenditures by industry from the Energy Information Administration (EIA), U.S. Department of Energy.
For air transportation, the Bureau of Transportation Statistics data on cost of materials, services, fuels, and electricity are deflated using cost indexes from Airlines for America. For line-haul railroads, estimates of intermediate purchases from the Surface Transportation Board are supplemented with data from other sources, including the Association of American Railroads, Amtrak, U.S. Energy Information Administration, and the Edison Electric Institute. The nominal values are deflated with PPIs from BLS and implicit price deflators from BEA.
Last Modified Date: September 30, 2025
United States Department of Labor
