Changing the item structure of the Consumer Price Index

The changes in the 1998 CPI market basket will reflect and anticipate shifts in the consumer marketplace; what factors are involved?

The Bureau of Labor Statistics calculates the Consumer Price Index (CPI) to measure the monthly average change in prices of the goods and services U.S. consumers buy to satisfy their daily needs. The CPI is the average change in the prices paid for a fixed set or "market basket" of goods and services that consumers bought during a selected period. The market basket needs to be modified occasionally, because, for example, consumers change their preferences or new products and services emerge. During these occasions, the Bureau reexamines the CPI item structure, which is the classification scheme of the CPI market basket. The item structure is a central feature of the CPI program and many CPI processes depend on it.

As part of the 1998 CPI revision, the Bureau of Labor Statistics will introduce fairly sweeping changes in the CPI item structure. The objective of the classification revision project was to improve the index by making the structure more effective for the various ways that the CPI is used. The primary goal is to produce the most accurate all items CPI. A secondary goal is to produce more detailed indexes that are meaningful and useful to our customers and valuable for analyzing the all items index.

Major changes to the CPI item structure are intentionally infrequent. BLS is well aware that it must not undertake changes to the item structure casually or capriciously. The item structure is so central to the CPI that changes to it inevitability impose some costs to the CPI program and some burden on CPI users. For these reasons, BLS is conservative in its consideration of structural changes. However, having an item structure that is too rigid or out of date can weaken or distort the index.

For two reasons, it is best to change the item structure at the time of a CPI revision. First, the core activity of a CPI revision program is to update the CPI market basket to reflect more recent consumer spending patterns; this provides a logical opportunity to rework the item classification scheme so that it will recognize the emergence of new items, the changes to existing items and their relative importance, and the ways that people view consumer goods and services. Second, major CPI revision programs generally entail many additional activities designed to achieve major improvements to CPI processes such as its sampling and estimation techniques. The CPI sampling and estimation processes use the item structure, and changes to the item structure support these methodology improvements.

In every CPI revision since the one in 1940, there have been structural changes—some major, some minor—that yield a more accurate representation of the consumer marketplace. In the 1953 revision, the CPI added strata for frozen foods, restaurant meals, and televisions. The 1978 revision began the use of top-to-bottom sampling methods to give all consumer goods and services an opportunity for inclusion in the CPI. Accommodating the new sampling techniques required item structure changes. The 1983 introduction of the rental equivalence method of measuring the cost of owner-occupied housing¹  also required some changes in the CPI item structure. The 1983 changes—in the estimation method as well as in the item structure—were unusual, in that BLS incorporated them outside of a major CPI revision. The 1987 changes to the structure consisted of adding new components for information processing and some other high tech items and collapsing a few components that were very small. The 1998 CPI revision will make more substantial changes to the structure. Not only will there be new components for new goods and a reduction of the structure for less important ones, but there will be a reorganization of sectors of the market basket such as ‘food away from home,’ ‘medical care,’ and ‘communications’ to account for the modern views of these consumption sectors.

Uses of the item structure

The CPI item structure is the classification system used to partition the set of all consumer goods and services into a hierarchy of increasingly detailed categories. The levels of the CPI classification are:

• All items
• Major groups
• Intermediate aggregates
• Expenditure classes
• Item strata
• Entry level items

Users of CPI data may choose the level of detail needed for their analysis. The CPI program uses the levels in a number of ways. The following processes use one or more levels of the CPI item structure hierarchy:

• Index estimation.  The item stratum-index area combination is the basic building block of the CPI. For example, ‘breakfast cereal’ is an item stratum and the Philadelphia-Wilmington metropolitan area is an index area; ‘breakfast cereal’ in Philadelphia-Wilmington is an item stratum-index area combination. CPI samples are drawn for each of these building blocks, which BLS calls basic indexes (or elementary aggregates). The item strata level is the lowest level in the item structure for which a comprehensive set of indexes are computed. Expenditure weights for the basic indexes reflect the consumption pattern of the CPI expenditure base period (currently, 1982–84; 1993–95, as of January 1998) and remain constant between CPI revisions. With these expenditure-period weights, the CPI aggregates the basic indexes to create indexes for the higher levels.

• Imputation.  Expenditure classes are the first level of index aggregation. For example, men’s footwear, boys’ and girls’ footwear, and women’s footwear are item strata within the expenditure class for ‘footwear.’ An expenditure class serves two functions. It is a natural grouping of products from the user’s point of view (footwear, in this case) and it is used during index estimation to impute price change when actual price change data are not available. The expenditure classes are used to impute the index values for unsampled (unpriced) item strata, and for priced strata in some cases of temporary nonresponse. Because the expenditure classes are used for index imputation, the strata within an expenditure class are generally a group of products which are expected to have similar price movements.

• Publication.  BLS publishes national CPI indexes for all levels of the item structure down to and including the item strata, that is, all items, major groups, intermediate aggregates, expenditure classes, and item strata. For the geographic regions and CPI local areas, BLS publishes a reduced set of CPI indexes that include all items, all major groups and selected intermediate groups, and expenditure classes.

• Analysis.  The CPI press release announcing each month’s CPI includes the major groups to explain the sources of price change that month. In their analysis, economists and the media use major groups and intermediate aggregates to dissect and explain the price change story. Special indexes—such as energy and all items less food and energy—cut across the structure to provide a different view of price change and price change trends. For example, many analysts believe that the all items less food and energy index, which is often called the underlying or core rate of inflation, is especially useful in forecasting longer term price movements because it excludes the portion of the consumer market that is most subject to sudden, dramatic, and often temporary, price changes due to demand and supply shocks such as bad weather or political crises.

• Sampling.  BLS statisticians determine how many price quotations and observations should be collected for each basic index at the expenditure class level within groups of index areas. This process is called sample allocation.² The primary objective of sample allocation is to produce the most accurate national all items index as possible with the funds available. In each index area, each item stratum is allocated a minimum number of observations. If its expenditure class represents a large weight, or if the stratum is subject to high variability, or if it is relatively inexpensive to collect, then BLS statisticians allocate additional sample to the stratum. BLS statisticians select entry level items within item strata and match them with the sampled retail outlets for price collection. The BLS field staff who collect CPI prices, use the entry level items as the starting point for the selection of the unique product or service—within the outlet—whose prices they will follow.

Changing the structure

The CPI item classification scheme is a structured approach to the consumer marketplace, with consumer goods defined and classified according to the consumer’s view of the market. To make improvements to the CPI item structure, we have to change the way the CPI market basket defines and groups items. To accomplish this, we changed the CPI’s classification of its market basket to correspond to recent changes in the marketplace. This task includes providing a place in the market basket for new consumer products, increasing the prominence of those whose importance has grown, and diminishing the prominence of (or phasing out altogether) those things that have become obsolete or whose importance has diminished. Additional tasks include designing ways to define or view the items themselves.

New goods.  A new consumer good or service has no clear predecessor in the marketplace and, therefore, has no obvious place in the CPI classification. In some cases, this inhibits the introduction of the new good into the CPI sample at times of sample rotation or item substitution. For example, the ‘information processing equipment’ strata in the current CPI item structure provides for the inclusion of computer software as well as personal computers in the index, but there are just a few software products that are actually included in the price sample. When such an item enters the consumer marketplace and becomes an important component of consumer spending, the CPI must incorporate it into the item classification scheme so that the CPI will continue to be an index of all consumer goods and services. The new item structure recognizes the growing importance of these products by including an item stratum for ‘computer software and accessories.’ The new classification scheme also provides new item strata or entry level items for ‘cellular telephones,’ ‘leased cars and trucks,’ and ‘delivery services.’

Blends.  Unlike a new item, which has no clear predecessor, a "blend" has multiple predecessors. When a blend straddles two strata, it can cause difficulties for the CPI. The arrival of blends is often a sign that it is time for the CPI to combine item strata. A simple example is the butter-margarine combination products; these forced the CPI to combine the strata and put the new stratum into ‘fats and oils.’ A related problem occurs when two different strata contain items that are close substitutes. The biggest example of this is the strata for motor vehicles. The old structure has separate strata for automobiles and for trucks, making a distinction between passenger vehicles and cargo-carrying vehicles. In recent years, consumers have shifted their spending from automobiles to trucks—in particular, from station wagons to mini-vans and sport utility vehicles. These new kinds of vehicles may be thought of as blends of automobiles and trucks as the line between automobiles and trucks has become blurred to the point of virtual meaninglessness to consumers. Consequently, the new structure has just one stratum for new vehicles.

Increasingly important items.  The new structure adds new entry level items or item strata to give greater prominence to ‘nonalcoholic beverages,’ ‘baby food,’ ‘salad dressing,’ ‘gardening and lawncare services,’ ‘motor vehicle property taxes,’ ‘parking and tolls,’ ‘nursing homes and adult day care services,’ and many other elements of consumer spending. The revised structure also will create a new major group: Education and Communication (discussed later in the article). As a result, all of these items will get a greater share of the CPI pricing samples and, in some cases, will permit us to begin publishing indexes for them.

Obsolete and less important items.  When consumer items diminish in importance, the item structure needs to give them less prominence so that they will not absorb an excessive share of the program’s resources. Often, the easiest way to accomplish this is to combine strata or entry level items. For example, the five entry level items for various kinds of dishes and drinking glasses will collapse into a single entry level item called ‘dishes.’ In addition, the new structure will no longer have separate strata or entry level items for ‘white bread,’ some cuts of meat, or types of automobile repair. Portable dishwashers will no longer be an entry level item. All of these items are still represented in the CPI, but they will receive less sample and, in some cases, we will discontinue publishing index series for them.

New views of consumer items.  In some cases, consumers and sellers significantly change in the way they perceive consumer items, such that the CPI must replace the methods used to define and price these items. The best example of this is medical care. The current CPI has distinct item strata for various medical inputs, such as the cost of a day in a hospital room. This structure makes it difficult for the CPI to accommodate the many changes in the medical care industry (such as the shift to performing outpatient procedures). The new item structure combines the three hospital item strata (‘hospital room, in patient’; ‘other inpatient services’; and ‘hospital outpatient services’) into a single item stratum, ‘hospital services.’ This will permit the CPI to take a "treatment approach" to hospital services, and will position the CPI to eventually take an "outcomes approach" to pricing medical care.³

The strata for ‘food away from home’ were radically changed for this reason. The traditional divisions in the sector provide strata for lunch, for dinner, and for other meals and snacks. The chief distinction among these strata is time of day. This has proven to be an unsatisfactory way to break down this fairly large component of consumer spending. Heeding the suggestions from users of these data, we created strata corresponding to types of establishments: full service restaurants, limited-service meals and snacks, food at employee sites and schools, and food from vending machines. The change in the organization of the items within ‘food away from home’ will not affect higher level indexes, but these new strata may exhibit some differential movement that will help analysts understand price change within this component. For example, the full service restaurants stratum may be more sensitive to changes in labor costs, the limited service meals and snacks stratum may be driven by advertising costs and competitive pressures, and the food at employee sites and schools stratum may be highly responsive to changes in public subsidies. Taken in their entirety, the changes in the 1998 CPI item structure will reflect and anticipate the dramatic shifts that are taking place in the consumer marketplace.

Out of scope items. While there has always been some debate concerning whether or not a few spending elements actually belong in the CPI market basket, a detailed account of these controversies is beyond the purpose of this article. However, it is noteworthy to acknowledge a few items. One of these elements is ‘finance charges’: the current CPI has an item stratum for ‘automobile finance charges,’ but we now consider them out of scope because it is not possible to account for intertemporal purchases in an index that measures price change between periods. Therefore, interest charges are not included in the new structure. Another out-of-scope item is ‘residential insurance.’ The adoption of the rental equivalence approach to measuring the change in the cost of owner-occupied shelter made insurance solely based for residential structures, along with most spending on home maintenance and repairs, out of scope. Also, ‘household insurance,’ which excludes insurance on residential structures, because that is subsumed under ‘owners’ equivalent rent,’ will now be combined with ‘tenants insurance,’ which is the only part of this stratum that is and will be sampled. The maintenance and repair items are scaled back to such a degree that they now are just part of ‘tools, hardware, and supplies’ or ‘household operations.’

The steps toward change

As mentioned earlier, the item strata are the basic building blocks of the CPI. To revise the strata, our first task is to define each one. The item strata definitions are used to partition the complete set of consumer goods and services in the CPI market basket. After partitioning the goods and services, we determine if the products and services within the stratum will be sampled (priced) or unsampled (unpriced). Unsampled strata typically have a tiny share of the expenditure in the market basket and are generally difficult to define in terms of product specifications and price characteristics so that there is no reliable technique to track their price changes. These item strata have an explicit expenditure weight assigned to them, but the products and services represented by these expenditures are not included in the CPI price collection program; thus, they are "unpriced." One reason for leaving a stratum unpriced is the difficulty of finding stable outlets for it; this is true of services such as babysitting and carpools. Another reason is that the items in the stratum are so complex and changeable that we cannot accurately separate their pure price changes from changes associated with new features; examples include yachts and recreational vehicles.

Define item strata. The task of defining each stratum is important to the CPI. These strata will have weights that are constant until the next CPI revision. As we defined each stratum, other considerations had to be made. The following summarizes these points.

Determine the number of strata.  The number of priced strata increased from 183 to 186, remaining basically unchanged from the old to the new classification. However, within this process, we created some new strata for new products and collapsed some existing ones. The number of "old line" strata diminished (for example, six beef strata became four because consumers changed their preferences for beef) and the number in strata of growing importance rose. (For example, nonalcoholic beverages went from three to five strata.)

Decide the optimal number of item strata. The CPI weights are fixed at the item strata index area level. In contrast, weights within the strata change as samples rotate, as sample items are replaced by substitute items, and as sample outlets that disappear are replaced by augmentation outlets. On the one hand, if there are very few strata, the CPI’s conceptual framework of holding the expenditure weights for the reference period market basket constant will be weak. On the other hand, if there are many strata, the structure will be too rigid to accommodate changes in the market place and the samples will be too small to yield accurate measures of price change. Beginning in 1998, the item strata’s fixed expenditure weights will come from the Consumer Expenditure Surveys for 1993, 1994, and 1995. Defining a large number of strata will strain the ability of these surveys to yield accurate weights. Most strata must be sampled—selecting retail outlets for them and then selecting unique items within those outlets to represent them. In addition, the strata must be priced, tracking the prices of the selected items over time. Consequently, it is important not to define an excessive number of priced strata. Too many strata will mean very small sample sizes for each basic index. Small samples are subject to high variability and other types of inaccuracies.

Establish the optimal size of an item stratum. We measured stratum size from preliminary estimates of their relative importances (the strata expenditure weight relative to the total expenditure weight of the market basket) which we derived from preliminary Consumer Expenditure Survey data. Final data on the relative importances for the new structure will be available with the release of the January 1998 CPI.

We agreed that there was no reason to set a maximum size for a stratum. However, we set a goal for minimum stratum size for priced strata of 0.2 percent ‘nonfood items’ and 0.1 percent for ‘food items.’ ‘Food items’ display considerable variability and are inexpensive to collect; moreover, the CPI has traditionally had greater detail in ‘food items’ and users such as the U.S. Department of Agriculture depend on this detail; therefore, we set a lower minimum strata size for ‘food’ strata. We did not create priced strata below these limits without a special reason for doing so.

Some CPI customers are primarily interested in the price movement of particular sectors of the consumer economy and request indexes for very specific products and services within these sectors. Making these specific items into item strata (and allocating sufficient sample to each of them so as to produce publishable indexes) would take sample away from other more important items and reduce the accuracy of the national index. To determine efficient sample allocation, BLS employs "sample optimization," a technical objective which takes priority over the customer requests for specialized data series. Using empirical data from recent Consumer Expenditure Surveys, the 1998 item structure was defined to avoid strata that represent a very small share of consumer spending.

In some cases, we did create small strata. One reason for this is that we expect the strata to grow sharply and we want to prepare for this growth. A second reason is that including these items with the others to make a stratum would create an index for a rather meaningless aggregate. For example, we created a stratum for ‘chicken’ and left a small stratum— ‘other poultry including turkey.’ However, we refrained from combining ‘miscellaneous household products’ with either ‘household cleaning products’ or ‘household paper products’ for this reason. Another example is that we let ‘cigarettes’ stand as an item stratum even though this "orphaned" ‘tobacco products other than cigarettes.’

Other considerations.  Besides assuring that strata were not too small, we also tried to see that they formed natural groups, as consumers would view them. In the past, some items were in industry groups. We formed item strata by putting "like items" together. "Like items" were defined according to the consumer view of products and services, not in the way that the producing industries see them. For example, using the consumer view, items within the same stratum should have some affinity, such as substitutes (butter and margarine), or complements (washers and dryers).

There were some cases, however, where putting close substitutes together in the same item strata would have made the task of drawing the outlet samples and item samples within the selected ‘retail outlets’ very difficult. For example, we did not combine ‘housing at school’ with the far larger stratum for ‘other lodging away from home’ because these services are provided by distinctly different providers. For similar reasons, we did not combine ‘fuel oil’ with ‘other household fuels.’

Summary of actual changes

About half of the sampled and priced item strata will undergo a definition change. These changes include two or more strata collapsing together, a stratum splitting into two or more strata, and more complex realignments in which a stratum may shed part of its product/service domain while picking up other products and services. In some cases, the definitional changes do not cause a break in the index continuity. In other cases, the definitional change in the item strata is so significant that the index series is considered a new series. No index history will be available for these series. Appendixes 3, 3a, and 3b provide information about index continuity. Table 1 shows the number of priced item strata that will undergo a definition change in the 1998 item structure as a percentage of the total number of priced strata by major group.

Priced versus unsampled (formerly called unpriced) strata. The item strata that are unsampled are not assigned any sample and their price movement is imputed from the movement of priced strata. The following tabulation shows that the number of unsampled strata grew from 24 to 25. (The count of priced strata includes four strata representing ‘health insurance’ retained earnings. Movements in the CPI indexes for these strata are measured as the product of changes in medical care price indexes and changes in estimated retained earnings ratios for categories of insurers.) Regardless, the share of the weight for all unpriced strata is expected to be less than 1 percent of the total expenditure weight:

Grouping item strata.  The decisions on forming sampling strata such as our item strata are based on the needs of the CPI customers and the professional judgment of the item specialists responsible for sectors of consumer spending. Table 2 shows the change in the distribution of priced strata from the current item structure to the new one, by major group.

The total number of priced strata remained virtually unchanged; but there is considerable movement between the groups. The most obvious change is the movement from the ‘housing’ group to the new groups, ‘recreation’ and ‘education and communication.’ Exhibit 1 summarizes the changes within the major groups. Appendix 1, provides a complete listing of the new and old item structures.

Perhaps the most obvious change to the structure is the addition of a new major group—‘Education and communication.’ It contains items that are of growing importance and it is expected to expand in the future. For consumers, education is expanding to become a life-long undertaking and one that they increasingly pay for out of their own pockets. In the past, ‘education’ has been part of the ‘other goods and services’ group. Communications is another growing area, which, in many ways is associated with ‘education’ and may be increasingly intertwined with it. In fact, we gave serious consideration to combining ‘education’ and ‘communication’ with ‘recreation’ because computers and some other electronic items may be used for recreation, education, or communication.

We moved items out of ‘housing’ because we wished to limit this major group to shelter items and items associated with shelter. Televisions and other electronic equipment may be used within a housing unit, but they are no more required for shelter than are food-at-home items. Furthermore, the new battery-powered, highly portable electronic devices, designed for use while walking, jogging, or swimming, made it difficult for us to maintain that these items belong in ‘housing’ in any sense. Similar changes have occurred in telephones and telephone services. Telephones are now commonly found in vehicles and even in pockets and purses.

We attempted to remove items from categories called "other" or "miscellaneous." As mentioned, ‘education’ expenses moved out of ‘other goods and services.’ Another example is the movement of ‘nonalcoholic beverages,’ a category that excludes milk and milk-based drinks, out of the ‘other foods’ group. Over the years, it has grown so much in importance that we let it stand alone as a separate group.

‘Apparel and upkeep’ changed to ‘apparel’ because apparel services, luggage, and sewing materials moved to other major groups, leaving just apparel commodity items. Apparel services consist largely of laundry and dry cleaning services. Over the years, we have made valiant efforts to distinguish apparel laundry from household textile laundry (which has been in ‘housing’), but it is quite difficult for data collectors to maintain the distinction in the field. Based on the fact that much sewing in the home is now recreational, ‘sewing materials’ along with ‘sewing machines’ was moved from ‘housing’ to ‘recreation.’

Comparisons with other classification systems

The item classification scheme presented in this article is unique to the CPI. There are other statistical series that also classify the set of all consumer goods and services which the CPI calls the market basket. These series sometimes refer to this set as "consumption space." The BLS Consumer Expenditure Survey, for example, also provides data on personal consumption by category.⁴  Another organization of consumption space is found in Personal Consumption Expenditures, which the Bureau of Economic Analysis provides as part of the National Income and Product Accounts.

The CPI staff worked with the Consumer Expenditure Survey staff during the item classification project, in part because the CPI must derive its weights from that program. Similarly, the Bureau of Economic Affairs uses CPI indexes to deflate most of the components of the Personal Consumption Expenditure series, so CPI staff tried to meet their needs to the greatest extent possible.

One point of difference among these programs is the scope of coverage. The CPI is limited to consumer items that people buy out of pocket. (One exception is the rental equivalence component for measuring owner-occupied shelter costs, which is included because it is the only way to isolate the pure consumption costs from the investment part of shelter costs.⁵) The Consumer Expenditure Survey reports other expenditures, such as mortgage costs, that are not within the CPI concept. The Personal Consumption Expenditure employs a broader definition of consumption. It includes, for example, employer-provided health insurance and the expenditures of nonprofit organizations.

Outside the United States, the European Economic Community recently developed the "Classification of Individual Consumption by Purpose (COICP)" for use in programs such as consumer price indexes within the European community. COICP also has some differences from the CPI in scope and definition. However, the European Community’s classification scheme, like the CPI’s, classifies items according to how consumers view them rather than by industry; as a result, the systems are fairly similar.

Footnotes

¹ See "Changing the CPI Homeownership Method to Rental Equivalence," CPI Detailed Report (Bureau of Labor Statistics, January 1983), pp. 3–17.

² Sylvia G. Leaver, William H. Johnson, Robert Baskin, Samuel Scarlett, and Robert Morse, "Commodities and Services Sample Redesign for the 1998 Consumer Price Index Revision," Proceedings of the Survey Methods Section (American Statistical Association, 1996), forthcoming.

 ³ See Elaine Cardenas, "Revision of the CPI hospital services component," in this issue, pages 40–48, for a complete description of the CPI medical care methodological changes.

 ⁴ Consumer Expenditure Survey, 1992–93, Bulletin 2462 (Bureau of Labor Statistics, September 1995) is the most recent Consumer Expenditure Survey publication.

⁵ See "Changing the CPI Homeownership Method."

Walter Lane is a branch chief in the Division of Consumer Prices and Price Indexes, Bureau of Labor Statistics.

Last Modified Date: October 31, 2018