BLS handling of missing data to produce 2025 consumer spending estimates and use in price indexes products

• Overview
  • Overview of the Consumer Expenditure Surveys
  • Overview of Consumer Price Index
• Impact of missing consumer expenditure data
  • Interview Survey impact
  • Diary Survey impact
  • Consumer Price Index impact
  • Published Estimates impact
• Criteria for options
  • Mitigating the impact on estimate accuracy
  • Maintain integrity of calculations
  • Ensure timeliness of publications
• Options considered
  • Weight adjustments
    • Effect of weight adjustments
  • Unit-level modeling
    • Effect of unit-level modeling
  • Domain Estimation
    • Effect of Domain Estimation
• Conclusion

Overview

To understand the impact of the missing data, it is helpful to begin with an overview of the surveys, methods, and products. This section includes an overview of the Consumer Expenditure Surveys (CE) and the Consumer Price Index (CPI).

Overview of the Consumer Expenditure Surveys

The CE are nationwide household surveys designed to collect expenditure data that measure how U.S. households spend their money. The CE consist of two separate surveys, a quarterly Interview Survey and a weekly Diary Survey. The U.S. Census Bureau collects the data for both surveys under contract with BLS and has a target population of the U.S. civilian noninstitutional population, which we refer to as the “U.S. population.”

The Interview Survey collects data on major or recurring expenditures, such as rent, mortgage, and car payments. Its design is a rotating panel survey, where a consumer unit (CU) completes one interview every 3 months, for 4 quarters.[1] Each quarterly interview captures expenditures in a 3-month recall, so expenditure data are recorded directly after the quarterly expenditures occur.

The Diary Survey is designed to collect data on smaller or frequently purchased items, such as groceries and apparel, that would be difficult to recall for a 3-month period. Its design is a 2-week reporting period, where a CU completes one diary per week reporting all their expenditures for that week. The expenditures occurring in each of these weekly diaries are assigned the month that the diary-keeping period began.

Each responding CU is weighted to represent a portion of the U.S. population. The weight for each responding CU comprises a base weight (which is the inverse of its probability of selection), a nonresponse adjustment factor, and a calibration adjustment.[2] Weights are assigned to CUs on a quarterly basis, and the weights sum to the total U.S. population each quarter. Using the weights, the BLS estimates average annual expenditure per CU at varying levels of detail (e.g., the highest level of aggregation is total expenditures and the lowest level of aggregation is a category such as “eggs”).

BLS calculates annual expenditure estimates using a standard weighted average formula, adjusting for the use of 4 quarters of data (i.e., in the formula, quarterly population weights are divided by 4). To create calendar-year estimates based on the date of expenditure from the 3-month recall in the Interview Survey, further adjustments are made to factor in the number of months that are in scope for the calendar year. This adjustment is not necessary for the Diary Survey because all expenditures are assigned to the months within the calendar year. However, because expenditures are based on weekly data, they are multiplied by 52 to create annual estimates.

For expenditures captured in both surveys, the best source (Interview or Diary Survey) for each expenditure category is selected to create integrated annual total estimates for the CE’s average annual expenditure tables[3]. In 2024, there were 189 Universal Classification Codes (UCCs, the lowest level of expenditure category detail estimated in the CE) sourced from the Diary Survey, making up 19.8 percent of total expenditures in 2024 ($15,565 of $78,535). 392 UCCs sourced from the Interview Survey, making up the remaining 80.2 percent.[4]

BLS calculates aggregated measures of spending and income for publication in tables and databases. Data are produced at various levels of item and geographic detail, demographic groups, and across multiple time spans.

A feature of the CE data is the rich set of demographic characteristics collected with expenditure and income information. BLS publishes statistics for 17 demographic groups (like housing tenure), resulting in estimates for over 100 demographic characteristics (like homeowners with and without mortgages and renters). Providing greater demographic detail, BLS produces tables for eight combinations of demographic groups (like housing tenure by region) which results in additional detail for 240 demographic characteristics (like renters in the Midwest). The extensiveness of demographic characteristics complicates the missing data problem.

The public-use microdata files (PUMD) are also published annually with the CE data release. While the tables provide detail, the PUMD allows flexibility to generate data for demographic characteristics of interest. The PUMD are a subset of microdata files with additional perturbations applied for disclosure avoidance.

Overview of Consumer Price Index

The CPI is a measure of the average change over time in the prices paid by urban consumers for a market basket of consumer goods and services. Household spending weights are used to average the price changes in component goods and services across the market basket. BLS calculates spending weights using CE microdata.

BLS publishes many measures of consumer price change, for different items, geographic areas, and populations. BLS calculates indexes for 7,776 item-area combinations (basic indexes). BLS annually updates population-specific market baskets using two demographic groups: All Urban Consumers (CPI-U) and Wage Earners and Clerical Workers (CPI-W).

BLS produces another measure of inflation, the Chained CPI-U (C-CPI-U). In contrast to the CPI-U, the C-CPI-U is designed to capture consumer substitution that occurs across items, due to changing relative prices of the items. Historically, the average annual increase in the C-CPI-U is about 0.2 percentage points lower than the CPI-U.[5] The C-CPI-U uses the Törnqvist price index formula to aggregate the basic indexes into the all-items, U.S. City average estimate. The Törnqvist formula requires an expenditure estimate for both months over which price change is measured. The use of monthly spending weights is a key feature of the C-CPI-U, in contrast to the CPI-U and CPI-W, products which are estimated with annual spending weights.

The CE microdata required to produce monthly spending weights are not available to BLS in real-time.[6] Hence, BLS publishes a preliminary version of the C-CPI-U in real-time using the constant elasticity of substitution price index formula and annual expenditure weights that are available at the time of publication.[7] BLS subsequently revises this preliminary version with a final version 10-12 months later when the requisite monthly expenditure data become available. For example, the January 2025 C-CPI-U was released as a preliminary estimate with January 2025 CPI-U indexes in February 2025. Later, BLS released the final version of the January 2025 C-CPI-U in February 2026.

BLS uses CE microdata to produce monthly and annual weights. Index aggregation requires 7,776 spending weights to calculate the CPI-U, CPI-W, and the C-CPI-U. BLS uses smoothing processes to reduce volatility in those weights. To smooth annual weights, BLS uses 4 quarters of data to composite estimate the weight for an item in a basic area with a broader geographic area definition.[8]

BLS uses a different process to smooth monthly weights. The monthly weights that enter the Törnqvist in calculation of the Chained CPI derive from area allocations of national item month expenditure estimates. Sample sizes in CE are not sufficiently large to produce reliable estimates of the monthly expenditure for all 7,776 basic index cells without use of a variance reduction technique. BLS uses CE data from 11 months prior to the reference month to define smoothed geographic allocation fractions for each item (namely, the average share each area contributed to national expenditures on a given item in the 1-year period ending in the reference month).[9]

While BLS also uses CE microdata to select a sample of establishments and item categories for many categories, the focus of this report is the CPI weights.

Impact of missing consumer expenditure data

The 2025 lapse in appropriations had different impacts on the two Consumer Expenditure Surveys. In turn, the missing CE data have downstream impacts on the CPI annual and monthly weights. This section will describe the effects of the missing data for each survey and the resulting publishable estimates.

Interview Survey impact

As described earlier, the Interview Survey relies upon CUs to recall purchases for the previous 3 months. Because interviewing did not occur in October and November 2025, the amount of collected expenditure data for July, August, September, and October 2025 was affected. Furthermore, the effect was not evenly distributed between the months. Relative to the survey design, 33 percent fewer expenditures are expected in July and October 2025, whereas 66 percent fewer expenditures are expected in August and September (table 1).

Diary Survey impact

The Diary Survey is a 2-week survey where two weekly diaries are placed with respondents, then picked up 2 weeks later. With the lapse in appropriations, diaries could not be placed in October and November and as a result, there are no expenditure data for October or November collected in the Diary Survey. Because the Diary Survey is a one-time collection, rather than the panel design of the Interview Survey, the sampled addresses for these months were never interviewed.

An additional impact of the lapse in appropriations was that some September diaries that were placed later in the month were not collected in October. Because not all September diaries were completed, the response rate was lower in September than usual (30 percent in September 2025 compared with 43 percent in September 2024). BLS could address lower response rates using existing methods of nonresponse weighting for this month.

The impact of the lapse in appropriations on the CE Diary Survey data collection is illustrated below (table 2). Months January through August and December are unaffected by the missing data, there were reduced response rates in September, and there were no responses in October and November.

Calibration weighting for CE is a quarterly process that occurs separately for each survey. This process calibrates the nonresponse-adjusted base weight, which produces CU full-sample weights and additional replicate weights. The calibration weighting process ensures that estimates produced with the full-sample weights are consistent with current U.S. population totals, which are updated quarterly. The current U.S. population groupings are for age, race, household tenure (owner or renter), division of the country, urbanicity (urban or rural), and ethnicity (Hispanic or non-Hispanic). Although this process occurs separately for each survey, both surveys use the same quarterly population counts to ensure each survey’s weights are consistent with population total estimates.

During a typical quarter, calibration weighting calculates CE population weights that sum to a quarterly U.S. population total. As the fourth quarter 2025 was an atypical quarter due to the missing October and November data, the calibration process required adjustments. BLS is adjusting the calibration processes so the responding households in December 2025 would be calibrated to itself only, by using one-third of the quarterly population totals, and not to the full quarterly population totals. In sum, BLS procedures to adjust for missing data were insufficient to address the extent of missing data due to the lapse in appropriations and adjustments were needed.

Consumer Price Index impact

Missing CE data in October and November 2025 affects spending weights for the CPI. As an annual practice, each January, BLS updates spending weights used to calculate the CPI-U and CPI-W to reflect consumer spending. These updated spending weights help ensure that the CPI weights reflect the latest data available on consumer spending behavior. With the release of January 2027 indexes, BLS plans to update spending weights used to calculate the CPI-U and CPI-W to reflect consumer spending in 2025. Since consumer expenditure data from July through November 2025 are affected, adjustments are needed to calculate annual spending weights for the CPI.

With the release of July 2026 indexes, BLS plans to finalize the C-CPI-U for the third quarter 2025 (July, August, and September). With the release of October 2026 indexes, BLS plans to finalize the C-CPI-U for the fourth quarter of 2025 (November, December). BLS did not publish October 2025 preliminary C-CPI-U values due to the lapse in appropriations.[10] This was because there was missing price data. Therefore, the October 2025 final C-CPI-U index will not be published, due to the lack of both price and expenditure data.

Published Estimates impact

If BLS did not adjust for the missing data, annual consumer spending estimates would not be reflective of a full year of spending, and the CPI spending weights would be incomplete. As described above, there are no October and November data to calibrate CE weights to known totals. Similarly, there are no CUs interviewed whose weights can be adjusted using a non-interview adjustment factor. The result is biased estimates of spending.

Measures of price change would also be biased. Annualized spending shares for the CPI-U and CPI-W would reflect less than a full year of spending. Monthly spending shares for July, August, and September would reflect lower spending on items sourced from the Interview Survey. Monthly spending shares for October and November would reflect no spending on items sourced from the Diary Survey.

BLS simulated the effects of missing data by replicating the 2025 conditions in datasets for 2022 through 2024. Portions of the Interview Survey data were removed for the unit-level modeling and weight adjustment methods. All Interview Survey data for June through October were removed for the domain estimation method. Data for items sourced from the Diary Survey were removed in the months of October and November.

To demonstrate the impacts of missing data, chart 1 shows the monthly relative importance of owner’s equivalent rent in the CPI market basket using simulated data in 2024 compared with published data. The simulated weight of owners’ equivalent rent is lower than the published value in July, August, and September. Conversely, the simulated weight is higher than the published value in October and November.

Based on simulated data from 2022-2024, the impact on spending results in upwardly biased measures of inflation. Table 3 displays the impact on the annualized change to the final Chained CPI-U simulating the missing data in the last three years. On average, inflation would have measured 0.58 percentage points higher than the published measure.

The PUMD will also reflect missing data. Data users will need to develop a method to adjust for missing data to suit their purposes.

Criteria for options

BLS developed criteria against which options can be evaluated, to provide accurate, objective, relevant, timely, and accessible statistics. BLS applies statistical data quality principles provided in guidance from the Office of Management and Budget (for example, OMB Statistical Policy Directives) and the National Research Council’s Principles and Practices for a Federal Statistical Agency[11]. In addition, BLS established a common foundation of data quality considerations using the Federal Committee on Statistical Methodology’s (FCSM) data quality framework[12]. From these principles, BLS determined criteria to evaluate options for addressing the missing data. The selected option should mitigate the impact on estimate accuracy in the face of missing data, while maintaining the integrity of calculations. There can be a trade-off between the accuracy and integrity of an option and the BLS ability to ensure the timeliness of publications. These three criteria (accuracy, integrity, and timeliness) must be weighed together.

Mitigating the impact on estimate accuracy

Accuracy is defined as the closeness of an estimate from a statistical product to its true value. While the true value is not known, data simulations can approximate the impact. Comparisons with other sources can also be helpful to estimate bias relative to the true value. The measures BLS are considering when assessing accuracy is average annual expenditure mean (all consumer units) and the 12-month change in the final C-CPI-U (All items, U.S. city average). When comparing options to address missing data, BLS aims to minimize bias and maximize precision of these target estimates. For context of typical measures of bias, a 2003 study of bias in the CPI estimated the effects of quality change and introduction of new goods had the largest effect, around 0.3 percentage points per year.[13] While BLS has made methodological improvements since then, this figure gives a sense of scale.

Furthermore, to maintain the accuracy of the CPI-U and CPI-W, the annual weight update should not be delayed. There is generally an upward bias in Laspeyres price indexes as the age of spending weights increases relative to the price index reference period[14].

Maintain integrity of calculations

The FCSM data quality framework includes integrity as a dimension of quality change, and is composed of scientific integrity, credibility, computer and physical security, and confidentiality.[15] To maintain scientific integrity and credibility, it is imperative BLS produce estimates without error given a defined methodology. BLS rigorously tests all software systems, methods, and procedures used to produce official statistics. All presented options are interventions in the existing production systems, since the extensiveness of the missing October and November 2025 CE data is unprecedented. As such, BLS must extensively test the selected option to ensure quality assurance standards are met. The further a method deviates from existing production systems, the more testing and quality assurance is required, increasing the resource cost and time to implement. Fewer intervention points require relatively less testing.

Ensure timeliness of publications

Timeliness is defined as the length of time between the reference period of the consumer experience and the publication date of the statistical product. Table 4 lists the scheduled publication dates for statistical products affected by the missing CE data. BLS strives to maintain all publication dates since statistical policy directive three requires timely release of data.[16] Maintaining the publication dates in table 4 constrains the options available to BLS. BLS could continue researching options so that a broader set is available should missing data occur in the future. 

Options considered

BLS is considering three broad options to account for the missing data: weight adjustments, unit-level modeling, and domain estimation. Weight adjustments for complete observations can compensate for the missing months of data collection. Alternatively, it is possible to consider model-based estimation of missing expenditures either at the unit-level (such as eggs for a CU) or domain level (such as eggs for total consumers). Models require predictor variables, and so model-based estimation is only as successful as the usefulness of known predictor variables.[17] Ultimately, as the complexity of a model increases to better mitigate impacts to estimate accuracy, BLS must balance against criteria to maintain integrity of calculations (i.e., to get it right) and ensure timeliness of publications.

Weight adjustments

Since data were collected for all expenditure reference months in the Interview Survey, a weight adjustment can compensate for the missing data. Weight adjustments would be applied to the estimates after the calibration weights are constructed. Furthermore, weight adjustments would be applied to microdata to produce BLS outputs, and not to PUMD.

For the Interview Survey, the weight of CUs can be adjusted by multiplying by 3 or 3/2 to adjust for 2/3 or 1/3 data missing, respectively. This option assumes that the responding CUs interviewed in August, September, December, and January report the month of expenditures accurately. If respondents do not report the months of expenditures accurately, this option could exacerbate the effect of that error. For example, if respondents are more likely to assign an expenditure to the most recent month in the recall period, using only interviews conducted in December to estimate September expenditures could lead to underreporting.

For the Diary Survey, weights for responding CUs can be adjusted to compensate for the missing data in October and November. Under the premise of temporal, 1-month autocorrelation, a reasonable weight adjustment for Diary Survey is to double the weights of CUs that completed diaries in September and December to adjust for missing October and November data. This option assumes that the September and December expenditures reported by CUs are the most like expenditures that would have been reported in October and November. There is an increased risk that seasonal patterns that would have occurred for some expenditures in October and November will not be captured in the mean estimate using the Diary Survey weight adjustments. Due to the lower sample size for the Diary Survey, the proposed adjustments to the data will likely increase the variances and standard errors of expenditures.

These weight adjustments for the Interview and Diary Surveys require additional assumptions. The first assumption is that the observed and missing data are identically distributed for all consumer expenditures of interest in completed and missing months. The groups that receive weight adjustments are meaningfully defined based on socio-demographic population characteristics and/or the spatial adjacency of the geographic areas for which data are collected. Also, the rotating panels of respondents assigned for each month of data collection must represent the same population within each domain.

While weight adjustments could incorporate modeled information based on predictors, BLS proposes one set of weights for all expenditure estimates of interest. While this addresses integrity and timeliness criteria (expediency in producing all required estimates using a single set of weights), it comes at the potential cost of accuracy by neglecting possible temporal differences of consumer expenditures due to seasonality and trends. This may introduce estimation bias.

Another outcome of the weight adjustment approach is the potential for increased variance. Direct sampling variance estimation will depend on the inflated weights for the observed data. If the initial survey weights are variable for the observed data, the inflated weights will be even more variable making the inflated estimates more inefficient (i.e., higher variance). This is mitigated by weighting calibration.

Effect of weight adjustments

At the time of this report (May 2026), processed 2025 calendar year data were not fully available to use in validating methods. To best approximate the effect of the missing data and subsequently the effectiveness of the options, data were simulated. Weight adjustments were applied to the simulated 2024 data to compare differences in means, standard errors (SEs), and relative standard errors (RSEs) to the published 2024 calendar year.

Overall, the simulated 2024 data did not have much, if any, effect on demographics, as the changes between these datasets were negligible. For example, the published data population count (in number of consumer units) was about 135.8 million. In the simulated data, after employing the adjustments, this same population count rises from about 135.8 million to 136.0 million. Similarly, the percentages for which various demographic groups account (e.g., housing tenure, race, and ethnicity) ranged between a difference of -0.2 and 0.2 percentage points. This is not surprising since the weights are calibrated to select population targets, including housing tenure, race, and ethnicity.

Unlike demographics, there are more meaningful differences in the expenditures between the simulated and published data. Chart 2 shows the percent differences between means at the UCC level calculated with the simulated 2024 data compared with the published 2024 data, grouped by level of difference. The change in means resemble a symmetric and unimodal distribution, with the most noticeable decline in means at 100.0 percent for digital video players and recorders, and the most noticeable increase in means of 133.8 percent for new motorcycles – two expenditures with a relatively low percentage of CUs reporting them each year (in production data, 0.01 and 0.04 percent of consumer units reported these expenditures, respectively).

Chart 3 shows the percent differences between standard errors (SE) calculated for the production and simulated datasets. [18] Changes in SEs are more pronounced, which is expected because of lower sample size resulting from the missing data. However, an increase to the SE was not consistently observed for all expenditures, nor are all the decreases inconsequential. The observed SE changes ranged from 221.3 percent, for septic tank cleaning (renter) to -90.0 percent, for men’s costumes.[19] SEs that were lower in the simulated data suggest that these expenditures were more variable in October and November 2024 than in the rest of 2024. If this peculiarity of the data is common for these months, then we could expect a similar pattern for the 2025 data; however, by definition, there will not be any way to measure the true effect of the missing data in 2025 without known expenditures available for comparison.

Chart 4 shows the percent differences between the relative standard errors (RSE) calculated for the production and simulated datasets. Changes in RSEs are similar in magnitude to those of SEs, not surprising given that RSEs are the ratios of SEs to means, and means changed little compared to SEs. Men’s Accessories saw the most noticeable decline in RSEs, at -50.5 percent. Conversely, material for patio, walk, fence, drive, masonry, brick, and stucco had the most noticeable increase in RSE, at 200.4 percent.

BLS considers mean estimates with RSEs of 25 percent or more to be unreliable and flags those estimates in published consumer expenditure tables to inform data users of the high RSE estimates. Therefore, one notable comparison to make between the production and simulated datasets is the number of expenditures with RSEs that exceed this threshold. In the production dataset, there were 144 expenditures with RSEs greater than 25 percent at the most granular publication level. In the simulated dataset, 157 expenditures have RSEs greater than 25 percent. Notably, some expenditures had RSEs greater than 25 percent in the production dataset that were less than 25 percent in the simulated dataset.

The above charts and discussion summarize the differences at the most granular level of detail available in the CE data products (estimates at the UCC level), which are more variable than the estimates at higher levels of aggregation the CE produces for the news releases. The higher level of aggregation mitigates some of the effects seen at the lower level of detail.

Table 5 shows the same information as Charts 2, 3, and 4, but at aggregated estimates to simulate higher level results. For example, instead of looking at the statistics for Fresh milk, all types; Butter and margarine; Cheese; Ice cream and related products; and Other dairy and related products each individually, the total expenditure estimates are summed as “Dairy products” and the calculations for SEs and RSEs are based on the summed calculations. Data at higher levels of aggregation have more reports and are less variable. As a result, the comparisons between the production dataset and simulated dataset at a higher level of aggregation have a narrower range of differences than the comparisons at the lowest (UCC) level.

Comparing the results of the simulated 2024 data with the published 2024 data is a useful tool to anticipate the likely nature of the impacts to the 2025 data. However, it is not possible to know the full effect for the 2025 calendar year data until they are received and processed.

Given the simulation results for annual consumer expenditure data, BLS expects the weight adjustments to mitigate the impacts of missing data on the annual spending weights for the CPI-U and CPI-W. The simulated results for annual mean expenditures indicate a low risk of bias in the fixed quantity weights used in the Laspeyres formula.

The weight adjustments can also be applied to mitigate the impact of missing data on the monthly spending weights for the final C-CPI-U. At a monthly level, the Interview Survey adjustment results in a complete estimate of spending for every month. The mechanics of adjusting September and December weights will not achieve complete estimates of spending in October and November for items sourced from the Diary Survey. To achieve the spirit of the weight adjustment given the mechanics of the Törnqvist formula, BLS would instead copy the September expenditure data as October and copy the December expenditure data as November.

Based on simulated data from 2022-2024, the weight-adjustment method mitigates the impact of the missing data on measures of inflation. Table 6 displays the impact on the annualized change to the final C-CPI-U if similar data were missing in the last 3 years. On average, there would have been no change in inflation compared to the published measure (average difference is 0 percentage points).

Although there is little overall change on an annual basis, the results of the C-CPI-U data simulation vary by month. Chart 5 displays differences in the 1-month percent change of the published C-CPI-U and the simulated index results using the weight adjustment approach.

Unit-level modeling

An alternative to weight adjustments is estimation. Model-based estimation of missing respondent expenditures could help mitigate the impact on estimate accuracy and reduce variance. The extent of improvement depends on the usefulness of the predictors available, and appropriateness of the model. Custom-built models require an extensive investment and are not feasible without significant delay to publications (a year or more). BLS could leverage readily available software packages for large-scale, mixed-type data imputation. Extensive testing and quality assurance would still be required to maintain the integrity of estimates.

Unit-level is defined as monthly spending for every CU. This is the finest level of aggregation – monthly spending at the CU level, aggregated into roughly 700 UCCs. An advantage to unit-level modeling, is once monthly estimates of spending at the CU/UCC level is complete, all other estimates can be generated without further intervention.

Estimation at the CU level requires that there are some predictors known about the sampled unit. For the interview survey, approximately 75 percent of the sampled units were in the sample in a previous panel with some portion having a complete interview. These sampled units do have information associated with their consumer unit. For the sampled units that did not complete at least one previous panel or were new to the sample in October or November, BLS is limited to information associated with the survey frame – specifically, only information related to the geographic location of the unit (zip code, census block).

Unlike the Interview Survey, the Diary Survey is not a panel, so the only known predictors for the sampled unit are those variables available from the survey frame.

BLS has worked with Census in the past to link sample addresses to administrative records (e.g., federal tax information, property tax data); however, these linking projects take time and often have some limitations in their applications making them infeasible for the 2025 publication timelines. This is an area that could be explored further under a longer timeframe to increase the available predictors for model-based estimation.

Modeling CU-level expenditures could leverage temporal correlations. A simple approach that could be accomplished in less time than more sophisticated approaches is to fully replace impacted 2025 data with data from those months in 2024 with an inflation factor to account for price changes. While this might be a reasonable estimation approach at aggregate levels, this becomes questionable at the household level for several reasons. Household level purchasing decisions are heavily influenced by demographics and current economic conditions. Modeling shifts in demographic distribution shifts and changing economic conditions introduce complexities.

Effect of unit-level modeling

To demonstrate unit-level modeling, BLS simulated a simple method. As with the earlier simulations, data were removed to mimic the missing data. Expenditures from the same months in the prior year adjusted for inflation serve as the estimated expenditures in the current year. Based on simulated data from 2022-2024, the unit-level estimation produces higher levels of inflation, on average an increase of 0.07 percentage point per year (table 7).

The effect of unit-level modeling on C-CPI-U indexes varies by month. Chart 6 displays the difference in 1-month percent changes between the published C-CPI-U and the index calculated using monthly weight data simulated using unit-level modeling.

While this approach can produce estimates of inflation that are reasonably accurate (though typically not as accurate as the weight adjustment approach, and with an apparent positive bias), its use could produce undesirable effects when applied to the separate task of producing the array of expenditure estimates BLS publishes across a variety of demographic groups.

Domain Estimation

In contrast to modeling at the CU level, BLS also considered statistical modeling at aggregate levels. That is, conducting domain estimation for CE publication cells (such as renters in the Midwest) or CPI basic index cells (such as eggs in Boston). The average expenditure for a defined domain in October and November could be modeled based on any number of predictors. This approach is beneficial because it could better address any seasonality concerns by leveraging temporal correlations of estimates for consecutive months (provided a long time series is available) and socio-demographic covariates to produce smoothed estimates for the missing months. This approach could better mitigate the impact of missing data on estimate accuracy, and improve the estimate efficiency (i.e., lower variance) for the collected months.

Although this approach is promising, due to the large number of demographic characteristics for which BLS produces CE publications, it quickly becomes massive in scale. Additionally, the estimates are not independent (for example, demographic cross-tabulations must all sum to the same totals). Modeling multiple series of domain estimates together has the advantage of within-model calibration. Modeling the series separately is easier to scale by all the cross-classification of socio-demographic characteristics, but after-model calibration would be needed, and would add variance (limiting the usefulness of the model). To reasonably estimate aggregate-level defined domains, BLS would have to reduce the scope of its current publications and limit the level of detail published and subsequent output. Like unit-level modeling, this is an area that could be explored further under a longer timeframe.

To demonstrate domain estimation, BLS simulated two univariate methods: linear interpolation and Auto Regressive Integrated Moving Average (ARIMA). In each case, missing data were simulated as described earlier.

Linear interpolation uses the two boundary months with data surrounding the missing months and creates a straight line connecting the boundary points to compute values for the missing data. Linear interpolation was applied at the item-U.S. level by CE survey source. Interview Survey data for the months July through October were replaced by interpolation using boundary data from June and November. For Diary Survey data, interpolation was applied to the months of October and November using data from September and December. Data for items sourced from both Diary and Interview Surveys data were interpolated from July through November based on June and December data.

The ARIMA method is a simple standardized time-series model. For the ARIMA simulation, BLS utilized the X-13ARIMA-SEATS application with an (011) (011) model and automatic outlier detection selected to mitigate additive outliers and level shifts with a critical threshold of 5.0.  

BLS also considered a multivariate forecast model, which was in development for the purpose of improving the accuracy of preliminary estimates of the C-CPI-U. The model predicts the C-CPI-U by forecasting monthly expenditures at the item-U.S. level and allocating them to the 32 areas in the geographic structure by fixing all allocation fractions to their last known values.

More specifically, national item expenditures, E_i,t^US , are modeled together while using known national item price indices, P_i,t^US. This basic structure allows expenditures on different items to be interdependent and to be influenced by the real-time state of item prices, both natural expectations. The vector time series model used in this approach is called Echo State Networks (ESNs), a type of Recurrent Neural Network model. The ESNs have been modified and augmented to account for features of the problem:

1. The goal is a weighted aggregate

2. The relative scale of the errors varies considerably across items

3. The model incorporates both exogenous and endogenous inputs

4. Seasonality varies among items. The suite of ESN fitting and forecasting functions were built in-house to include these structural features.

Recurrent Neural Networks are nonlinear time series models that learn relationships between inputs and outputs (a.k.a. responses) through the states of a large set of intermediary variables (“neurons''). ESNs are a type of Recurrent Neural Network in which the internal network couplings are sparse and randomly drawn. Couplings between inputs and neurons are also randomly drawn, leaving only couplings between neurons and outputs to be estimated. This represents a dramatic computational simplification, as once internal and input-to-neuron couplings are fixed, output couplings may be estimated through (fast) regularized linear regression.

ESNs effectively trade the problem of fixing the individual entries in the three coupling matrices simultaneously with that of choosing reasonable values for a comparatively small set of scalar parameters that control general features of the network.

Let t=1, 2, …. T  label time points for which there are observations of an N-vector response of interest, y_t, believed to be related to p-vector inputs, x_t.

Equation 1: Basic form for an Echo State Network

where h_t is an m-vector containing the state of the neurons at time t, U, V, and W are matrices of sizes m by p, N by m, and m by m, respectively, and g(·) is the “neuron activation function" applied element-wise. The BLS model uses g(s) = tanh(s), a common choice. ρ_W is the spectral radius of the random matrix W.α and ν are tuneable scalars that lie between 0 and 1. Their values are set through cross-validation, along with scalars that control probability distributions for random matrices W and U, and those related to the regularization. (Ultimately predictions for the responses are averaged over an ensemble of ESNs generated by repeatedly drawing W and U pairs from their respective distributions, and learning the matrix V for each draw).

As noted, this method has been in development for the purpose of improving preliminary estimation of the C-CPI-U (preliminary estimates are necessary due to the 4-quarter time-lag monthly expenditure data are subject to). During preliminary estimation of the C-CPI-U for month t, one is missing all monthly expenditure data for the prior 10, 11, or 12 months depending on which month of the quarter the reference month t is but is in possession of all requisite item-area price indices. In the context of the 2025 lapse in appropriations, there is missing or incomplete expenditure data for 5 months. The forecast method may be applied to this problem by simply shortening the horizon to 5 months.

ESNs are flexible models that can learn complex temporal dynamics after a relatively simple training process. Certain choices related to this application to the C-CPI-U, such as the particular partition of items and relative penalization across items in the estimation of output weights are important to performance. The prototype models incorporate an item’s typical share of expenditures in the training period and its sampling error (which vary considerably across items) for this task. This aspect of the method is in development. Example results using several 6-month time periods in 2022, 2023, and 2024 as forecast intervals are given for prototypes to give a sense of performance.

The echo-state network research is also helpful to evaluate the appropriate level at which to define the domain. In this research it has been observed that although the allocation fractions are not static, empirically the dynamics at the level of the chained index are predominantly driven by the evolution in national-item expenditure estimates (in the study period analyzed). BLS is taking a time series modeling approach that aims to forecast monthly item expenditures at the national level and subsequently reconstruct a prediction of the C-CPI-U by distributing the forecast national item expenditures to the 32 areas using the last known set of item-area allocation fractions (i.e., those defined at the last time-point in the training period).

Effect of Domain Estimation

As discussed above, modeling all consumer spending estimates published on the BLS website would create challenges maintaining the integrity and timeliness of the estimates. However, to demonstrate the approach, BLS modeled some estimates that are sourced from the Diary Survey where there is a greater risk of seasonal patterns in the data not being reflected with the weight adjustment. Table 8 shows the effect of different domain estimation techniques on mean annual expenditures for a few selected items.

Based on simulated data from 2022-2024, the domain estimation method mitigates the impact of the missing data on measures of inflation. Table 9 displays the impact on the annualized change to the final C-CPI-U if similar data were missing in the last 4 years. Results are shown for three techniques to estimate item-U.S. monthly expenditures, allocated to the area level for computation in the Törnqvist index. On average, table 9 shows that ESN performed closest to the published data, while ARIMA and interpolation had  larger differences. It is important to note that interpolation had 3 years of comparisons while ARIMA and ESN only had 2 years. As with other simulations, the effect varies by month (chart 7).

A weakness of using forecasting methods (whether ESN, ARIMA or otherwise) to fill the missing estimates item-level U.S. estimates is that they do not make use of data ‘on the other side of the gap,’ nor any of the partial household data collected during the affected months. At least one path to improvement is apparent: item-U.S. level forecasts can be benchmarked to the levels on the right side of the gap, once the expenditure estimates derived from data collected under normal operations become available.

That is, the forecast horizon can be taken to be 1 month longer than the actual interval over which item expenditure estimates were affected by the shutdown (in this case one month longer than the 5-month period, July through November 2025). A simple way to benchmark each item’s forecast to their actual December 2025 estimates once they become available would be to allocate the difference in level between the forecast and actual December values linearly over the preceding 5 months. For example, if the discrepancy between the forecast December expenditure for item i is ΔE_i^US, one-sixth of the difference can be added to the expenditure estimate forecast for the first month, two-sixths to the value forecast for the second month, and so forth. By construction, the forecast values for December will adjust to exactly coincide with the actual known levels.

Conclusion

The lapse in government  appropriations resulted in missing consumer expenditure data in October and November 2025 which affects the quality of CE and CPI estimates. BLS is considering three broad intervention options to mitigate the impact of missing data on the quality of CE and CPI estimates. Each option represents a host of possible methods. The extent to which the integrity of calculations and timeliness of publications can be maintained depends on the specific method adopted. The greater the method complexity, the greater the risk to both integrity and timeliness.

BLS is providing this summary of options so that data users can be informed about the tradeoffs of each option. BLS research continues while obtaining feedback. Once a method is selected, BLS will inform users of the methods implemented on the BLS public website.

Last Modified Date: May 1, 2026