How Have We Improved the Consumer Price Index? Let Me Count the Ways

Soon after I became Commissioner, the top-notch BLS staff was briefing me on the many programs and details that make up BLS. I asked the staff working on the Consumer Price Index (CPI) if they could list for me some of the improvements that have occurred over the past few years. It has been nearly a quarter century since the Boskin Commission studied the CPI and recommended enhancements. I knew many of these enhancements had been implemented, along with other improvements. But I was shocked to see my staff come back with an 8-page, detailed listing of 77 substantive improvements that have been implemented since 1996.

You may have thought a price index that has been around since 1913 is happy to rest on its laurels. Well, you’d be wrong. There are improvements to the CPI going on all the time. As I reviewed the list, I noticed a number of improvements involve the introduction of new or changed goods and services, such as cell phones or streaming services. I also noticed improvements in how we handle product changes, as I wrote recently in a blog about quality adjustment. But these topics only scratch the surface.

I’m not going to describe every CPI enhancement that has taken place over the past 24 years; you can find much more detail on the CPI webpage. But I will whet your appetite by highlighting a few categories of improvements.

Keeping the CPI market basket up to date

The goal of the CPI is to track the change in the prices consumers pay for a representative market basket of goods and services. Let’s look at the what and where of that market basket.

• What we collect — goods and services. We collect prices for a market basket of goods and services, designed to represent what consumers are buying. In January 2002, we switched from updating that market basket every 10 years to every 2 years, providing a better representation of current spending patterns.
• What we collect — housing. We track the cost of housing by a separate sample of housing units. In 2010, we increased that sample to improve accuracy. In 2016 we began rotating that sample every 6 years. Previously, the housing sample was only rotated when new geographic areas were introduced after the U.S. census.
• Where we collect — outlets. We collect prices from stores and businesses that are chosen based on where consumers shop and buy goods and services. In January 1998, we switched from updating this sample of “outlets” every 5 years to every 4 years. And in January 2020, we switched the source used to determine those outlets to the Consumer Expenditure Survey, which is also the source of our spending information for the market basket of goods and services people buy. Previously we used a separate survey of households to identify outlets.
• Where we collect — geography. We collect prices for goods and services in selected geographic areas, designed to represent all urban areas of the United States. In January 2018, we updated the geographic areas, designed to represent current population trends. We last updated these areas in 1998.

Collecting CPI information

According to folklore, CPI data collection was accomplished by staff who dressed up in high fashion, the ladies in fancy hats and white gloves and the gentlemen in the finest haberdashery, who then went shopping to determine the latest prices. That’s not how it happens. CPI staff are not paid to “shop” to collect prices. We use trained experts who are skilled at gaining cooperation from many different types of businesses; ensuring they are obtaining price information for goods and services that are consistent from one month to the next or making appropriate adjustments; and gathering information from thousands of outlets about hundreds of thousands of goods and services over a short data-collection period. Let’s look at how the data-collection process has improved:

• When we collect — In June 2005, the CPI switched from a collection period that spanned the first 15–18 days of the month to collection across the entire month. This provides more representative data, especially for items that frequently vary in price within the same month.
• How we collect — In January 1998, the CPI began using computerized data-collection tools, which automate certain math functions and screen for errors or inconsistencies. We continue to upgrade our processes; CPI data-collection staff recently began using a new generation of tablet computers.
• Alternative collection — Not all price information comes from traditional collection with stores. Some information comes from websites, corporate data files, third parties that combine data from different sources, and more. In fact, the CPI and other BLS programs are focused on identifying even more alternative collection methods in the coming years.

Calculating the CPI

Once we collect the prices on all these goods and services, we need to calculate an index. In simple terms, we find the difference between the price in month 1 and the price in month 2, and express that difference as a rate of change from month 1. We publish rates of change and also express current prices as an index, which is equal to 100 in a base period.

Many factors and decisions go into combining data for an item and then combining data for all items into the published CPI. We’ve improved those calculations in several ways over the past few years.

• Geometric mean — In January 1999, the CPI switched the formula for calculating price changes at the component item level from an arithmetic mean to a geometric mean. This allows the overall index to capture substitutions consumers make across specific products within a component item category when the prices of those products change relative to one another. With the geometric mean formula, BLS does not assume consumers substitute hamburgers for steak, which are in different component categories. The formula only captures substitution within a component category, such as among types of steak.
• More decimal places — In January 2007, the CPI began publishing index numbers to 3 decimal places, which improved consistency between published index numbers and rates of change.

New information available to the public

While the CPI has been around for over a century, we have added a number of new indexes over time, to provide a variety of inflation figures. Here are some of the newest products in the CPI family:

• Research (Retroactive) series — The CPI Research Series incorporates many of the improvements that came out of the Boskin Commission. The series provides a pretty consistent way to measure price changes from 1978 up to the most recent full year.
• Chained CPI series – The Chained CPI uses an alternative formula that applies spending data in consecutive months to reflect any substitution that consumers make across component item categories in response to changes in relative prices. For example, this index would capture consumer substitution of hamburger for steak. This measure is designed to come closer to a “cost-of-living” index than other BLS measures. The series was first produced in 2002.
• Elderly research series – The CPI for the Elderly reweights the component CPI data based on the spending patterns of elderly households. This series, mandated by Congress, began in 1988. In 2008, we extended the series retrospectively back to 1982.

As I have mentioned in the past, we are always working to improve the CPI. We recently contracted with the Committee on National Statistics, part of the National Academy of Sciences, to provide guidance on a variety of issues. I’ll use this space to report on the Committee’s work, as well as other improvements underway in the CPI.