February 2023

Improving response rates and representativity in the CPI medical care index

Respondents’ low response rates that affect not only sample size but also sample composition have emerged as a central challenge in measuring consumer price change for medical care. In the case of medical care, insufficient responses from respondents have made getting an accurate mix of prices by payer type (self-pay, insurance, Medicare Part B) more difficult. This article summarizes recent methodological improvements to the medical care index that address a decline in response rates over the last two decades and an overrepresentation of self-pay price quotes.

Since 2002, the medical care component of the Consumer Price Index (CPI) has seen response rates decline from approximately 90 percent to approximately 50 percent. Then in 2021, they dropped to below 40 percent, because of coronavirus disease 2019 (COVID-19) pandemic-related difficulties. (See chart 1.) The decline began after a 2002 decision to replace total hospital charges with just reimbursed rates and then continued for reasons regarding the difficulty of collecting medical care prices (outlined later). Low response rates may undermine the accuracy of these indexes.

Meanwhile, a January 2019 article by Stephen B. Reed and John Bieler notes that “in recent years, self-pay quotes have been overrepresented in the [physicians’ services] sample, partly because physicians find these prices relatively easy to provide,” whereas “private insurance quotes, in contrast, have been severely underrepresented.”¹ Self-pay prices have also been overrepresented in the prescription drugs index sample.² This problem also highlights how response rates can overstate respondent cooperation, because some respondents do not provide the data that BLS staff request.

The decline in medical care response rates and the overrepresentation of self-pay quotes for physicians’ services and prescription drugs are two challenges that threaten the accuracy of the CPI medical care index. This index constitutes 1 of 8 major expenditure groups in the CPI aggregation structure. As of December 2020, consumers allocated approximately 9 percent of their total budget to medical care commodities and services, with medical care services constituting most of these expenditures at 7 percent. As a result of these issues, the U.S. Bureau of Labor Statistics (BLS) has undertaken several projects to address these challenges. This article looks at two of these challenges: an overrepresentation of self-pay price quotes and the decline in response rates over the last two decades.

Addressing the overrepresentation of self-pay price quotes

The CPI medical care index is potentially biased because of an overrepresentation of self-pay prices in the samples for the both the physicians’ services and prescription drugs indexes. BLS has taken measures to address this problem in both indexes. In the case of the physicians’ services index, the CPI uses data from the Medical Expenditure Panel Survey (MEPS) to weight the sample by payer type, at the metropolitan area level, collected as part of the physicians’ services sampling process. MEPS data are also used to determine sales percentages used in selecting a payer type during the disaggregation process that identifies a unique item for price collection.³ In the case of prescription drugs, BLS receives a large dataset from one respondent consisting of average prices of each prescription drug for all sales to consumers using insurance. The use of actual transaction data from this one respondent helps mitigate an overrepresentation of self-pay prices in the in-store portion of the CPI sample for the prescription drugs index.

Improving payer weights for the physicians’ services index

As explained in more detail in a January 2019 BLS article titled “Improving the CPI physicians’ services index,”⁴ insurance prices have proved especially difficult to collect from physicians’ offices. Because of concerns about confidentiality and the costly efforts involved in looking up prices from private insurers (e.g., from a third-party agency), respondents have become increasingly reluctant to provide prices paid by insurers for the provision of services, either at the initiation stage or during collection periods after initiation. As a result, self-pay prices have become a larger portion of the CPI sample. This would not be a problem if self-pay prices and insurer prices changed at similar rates, but they do not. As a result, if self-pay prices are overrepresented in the sample relative to their actual market share, the CPI physicians’ services index risks being skewed toward self-pay prices.

In recent years, two methodological changes have been implemented in the CPI that specifically address the overrepresentation of self-pay prices in the sample of quotes for the physicians’ services index. The first change occurred in October 2017, for the physicians’ services index, when BLS ceased to allow substitution to a self-pay price if an insurance price was unavailable in a collection period. Then in April 2018, BLS reweighted the prices associated with payer type by using data from the MEPS—the second change. The weights are applied at the metropolitan area level and then aggregated to the national level during index construction. National-level data, however, are used to determine the probabilities of payer-type selection during the disaggregation process.

Since substitution to self-pay prices is no longer allowed when insurance prices are unavailable, price collectors are instructed to obtain prices for a service only for the payer type selected as one of the price-determining characteristics for the particular service selected during disaggregation. Unfortunately, as just noted, many respondents are unwilling to provide insurance prices when the insurance payer type is selected during disaggregation. With substitution no longer an option, price collectors record the insurance price as temporarily unavailable pending the respondent’s future willingness to provide a price. If a price collector is still unable to obtain a price after a year, the quote may be considered for deletion from the sample.

Although MEPS national expenditures data are used during disaggregation to determine the probabilities of selecting a payer type, respondents are reluctant to provide insurance prices when the insurance payer type is selected. As a result, self-pay prices remain overrepresented in the physicians’ services index example, although the degree of overrepresentation has declined in recent years. BLS continues to conduct research on how to obtain a sample of quotes that better represents the actual proportions of payer types in the market. For example, the CPI is researching medical claims data and other alternative sources of data that contain prices for appropriate physicians’ services.

In the meantime, BLS implemented a change in April 2018 that applies weights to prices by payer type at the regional level. Since MEPS data more accurately represent market proportions of payer type, the prices by payer type receive a weight that better represents their position in the marketplace. Although self-pay prices remain oversampled in the CPI, they now receive a smaller weight than insurance prices, which, conversely, receive a higher weight. This change in the relative weights helps to correct the bias that results from being underrepresented in the CPI sample.

Using retail pharmacy data for prescription drugs index

In addition to physicians’ offices, pharmacies are also reluctant to share insurance prices or are often simply cannot access insurance prices because they do not have the necessary patient insurance card available. They are also often unable to share prices because of how their databases are organized. As a result, under traditional in-store collection methods, self-pay prices have been overrepresented in the CPI sample of prescription drug prices. However, the CPI no longer receives prescription drugs prices exclusively from in-store collection. In March 2015, BLS began receiving a bimonthly dataset from a pharmaceutical retailer (anonymized as “Corp Y”), consisting of average prices from a sample of the company’s in-store prescription drug transactions. Since this dataset averages the prices of drugs purchased by consumers using insurance, it helps mitigate any overrepresentation of self-pay prices in the portion of the CPI sample collected in the store.

Using retail pharmacy data to improve efficiency of incorporating generic drugs into prescription drugs index

Another challenge for the prescription drugs index is the handling of transitions from branded to generic drugs. Approximately 6 months after patent expiration, the price collector follows standard guidelines for disaggregating among branded and generic versions of drugs. Rather than using revenue as a measure or size, the price collector uses the number of prescriptions filled over the previous 3 months. The 6-month lag is assumed to permit enough time for the generic versions to achieve market penetration. At this point, the generic drug could replace the branded drug if selected (it is also possible to continue with the branded drug if that version is selected). The price difference is treated as a pure price change.

Corp Y provides average prices for a particular drug molecule—identified by a generic code number—which averages across both branded and generic versions of the drug. Average prices change as consumers substitute between the brands and generics. Since branded and generic versions of drugs are similar in quality, the change in price reflects an accurate estimation of the average change in prices of prescription drugs. The Corp Y dataset effectively automates the process of incorporating generic prices into the index as generic versions of drugs penetrate the market. This automation saves the CPI program time and effort necessary to track each branded drug in the sample to determine the date of patent expiration.⁵

Improving response rates

Response rates have been declining in the medical care index for the last two decades. This section addresses several projects that BLS has undertaken to address this problem.

Reduced rotation for hospital services

Low response rates are especially acute for the hospital services index because of unusual difficulties that arise with price collection. Hospital staff are often too busy to run through a lengthy list of specifications to identify a unique service for pricing. The frequent unavailability of the staff extends the time to initiate a price quote beyond a standard 6-month rotation period, so initiation can take as long as 2 years. In the 4-year rotation cycle used by the CPI, this means the quote would remain in the sample for only 2 years before rotation begins anew.

In February 2019, BLS paused the 4-year rotation period until February 2023, when an 8-year rotation cycle will begin for hospital services. In February 2023, BLS plans to renew the sample with a new round of rotation, after which the sample will not rotate for another 8 years. The impact of the COVID-19 pandemic may push the sampling date to August 2023 or later.

A key assumption underlying the decision to extend the rotation period is that the number, composition, and size of hospitals remain stable over time. An added benefit is that hospitals maintain similar types and levels of services such as cardiac or neurological units over long periods. Heavy regulation, as well as economies of scale and scope, presents high barriers to entry that help preserve industry stability.

As explained in more detail in a July 2019 Monthly Labor Review article, BLS conducted a study to test the key assumption that the industry remains stable over time in terms of the number, composition, and size of hospitals.⁶ Using a third-party source for data on hospitals, the study compared data on the number, composition, and size of hospitals in 2006 and 2015. The 2015 dataset consisted of 6,227 hospitals, only 4 percent more than the number of hospitals in the 2006 dataset (5,972). Results revealed that the number of hospitals in the industry does not change dramatically over a lengthy period. The study also found that 85 percent of the hospitals in the 2006 dataset were also in the 2015 dataset, demonstrating that the composition of hospitals did not change a great deal over a long period. Finally, the study revealed that 82 percent of the hospitals in the third-party aggregator dataset (217 hospitals) were also in the CPI sample (265 hospitals), suggesting that hospitals in the CPI sample are representative of the industry.

In addition to comparing the number and composition of the CPI sample, the study compared the number of patient visits (as opposed to hospital revenue) with the size of hospitals. This analysis was done to examine whether the size of hospitals changed over time. If so, the sample might be biased even if the number and composition of hospitals in the CPI sample remained stable because the size of hospitals affects the probability of selection during sampling. Between 2006 and 2015, in terms of number of patient visits, the size of an average hospital in the sample increased 0.1 percent. Despite a few instances of large changes in size, the average hospital size remained stable.

Since the data showed that the number, composition, and size of hospitals remained stable from 2006 to 2015, BLS concluded it was justified in extending the sample rotation period from 4 to 8 years, reducing respondent burden and preserving the sample of quotes.

Mixed medical samples

In August 2018, BLS implemented a “mixed-samples” method to further address lower response rates in the hospital services and physicians’ services indexes. Price collection for physicians’ offices and hospitals can be time consuming. Initiating a price quote requires cooperation from respondents who often include high-level executives. These respondents require the expertise to go through a set of complex pricing specifications to obtain a price quote for a unique service provided by a physician or hospital. Many potential respondents refuse to participate because of the respondent burden.

To preserve an adequate sample size, price collectors make additional efforts to secure cooperation from respondents. For example, if a physician’s office initially declines to participate, the price collector does not immediately record a refusal. To allow time for the office to reconsider, the price collector asks if sending a form via email or fax is permissible, which accommodates the office’s need for additional flexibility. The price collector may also send a letter from the BLS regional commissioner stressing the importance of the office’s cooperation. BLS staff may even send another price collector for a second attempt at price initiation. Perhaps a staff turnover at the outlet may result in hiring a more cooperative respondent. If an office of either the hospital or the physician declines to participate, the price collector follows up with similar questions about whether price collection by email, fax, or phone would be more convenient. Given the complexities of price collection for the hospital services and physicians’ services indexes, web collection is not feasible.

As a result of these difficulties, initiating a price can take months or years, reducing the time in which prices are collected during a rotation period. The mixed-samples method addresses this problem by continuing to rely on old samples when new samples generate zero or insufficient numbers of quotes. In the past, the physician and hospital indexes relied on one sample at a time. When a rotation period was set to begin, the old sample was replaced with the new sample. Unfortunately, productive old samples of price quotes from cooperative outlets would be eliminated, often replaced by less productive samples because price collectors again were forced to deal with the reluctance of newly selected respondents to participate.

Under the new procedure, the CPI uses six rules at the end of an initiation period to determine whether to mix old and new samples during the rotation period for the physicians’ services and hospital services indexes. If they do not mix the samples, the old sample will rotate out to make way for the new sample. If they decide to mix the samples, the new procedure allows for productive old samples to continue to be used for index construction. Old samples can only be mixed with new samples once. For example, if the physicians’ services sample for an urban area (e.g., Chicago–Naperville–Elgin) in February 2014 was mixed with a physicians’ services sample for a primary sampling unit (PSU) in February 2018, then the February 2014 sample would have been dropped in 2022, regardless of whether the February 2018 sample is mixed with the February 2022 sample.

Preselection of pricing specifications for hospital services index

In August 2015, BLS began selecting a nationally representative subset of specifications before visits to hospitals to initiate new quotes during sample rotation. “Preselection” streamlines the item selection process so that busy respondents who work in demanding work environments are less burdened. The goal is to raise the number of “usable” quotes (quotes for which viable prices are available) as a fraction of “live” quotes (i.e., total number of quotes in the sample). With the selection of price quotes streamlined, preselection helps address the problem that most live but unused quotes in the hospital services index sample are unused because they have never been initiated. Preselection is an additional methodological improvement designed to address low response rates in the medical care index.

The three specifications for which preselection is available are payer type, delivery setting, and the type of service. To determine the specific payer types, delivery settings, and types of services without respondents’ input on the revenues associated with these specifications (the revenues are used to determine probabilities during disaggregation), BLS staff rely on alternative sources of data. Revenue data on inpatient and outpatient delivery are obtained at the national level from the American Hospital Association, and data on expenditures per payer type are obtained from the MEPS. National expenditures on the type of service for insured and noninsured patients are obtained from the Agency for Healthcare Research and Quality.

Once BLS staff obtain the data, they generate joint probabilities for all the combinations of delivery setting and payer type. Depending on which combination of delivery setting and payer type, BLS staff then use diagnostic classification to select the appropriate shares for various inpatient services.⁷

The carry forward method

Low response rates traditionally increase the use of nonresponse imputation.⁸ In June 2017, BLS began using the carry forward method to collect quotes in months in which prices do not change. The carry forward method effectively “collects” a quote by using the price from the previous period for the price of the current period, resulting in a price change of zero percent. Thus, the index changes only in months in which prices are directly collected from the outlet. This process makes sense for quotes whose prices typically change during periodic contract renegotiations or change only periodically for some other reason. In these situations, BLS would collect the same quote each month directly from the outlet because contracts designate price changes only in months when the prices are renegotiated. The carry forward method allows BLS to collect quotes that do not change without burdening the respondent with monthly visits.

During designated collection months, data collectors attempt to obtain a price. If successful, the price is carried forward until the next designated collection month. The carry forward method works well in CPI categories for which respondent burden is high and prices change in specific months each year. The reduced burden on respondents helps stabilize response rates. It also provides the additional benefit of avoiding the use of nonresponse imputation for quotes in which the prices remain the same. Note, however, nonresponse imputation is still used if price collectors are not successful in obtaining a price during the designated collection month. If they are unsuccessful, CPI uses nonresponse imputation for each successive month until the price collector successfully obtains a price.

Nonresponse imputation can provide inaccurate estimates of price change when successful collection would have showed no change in price. Though prices are eventually collected for the quote, frequent self-correction may increase index volatility. Imputation, however, can help smooth out this instability. Carry forward imputation avoids accuracy and precision mismeasurement for indexes in which prices change only in specific months.

Internal BLS research, although preliminary, shows that the carry forward method has expanded since its introduction in June 2017. Of all the quotes for which prices have been collected, approximately one-third of the quotes for physicians’ services and for hospital services are approved for collection using the carry forward method. The approval of the quotes does not, however, imply that the method is necessarily used for two reasons. First, the carry forward method is not used if the collection period is a month in which prices are collected (often in January or February or in June or July). Second, the carry forward method is not used if price collectors were unable to obtain a quote during the month when its price was scheduled to be collected or the quote was dropped from the sample. In the second case, BLS resorts to nonresponse imputation until a price can be collected. Internal research shows that nonresponse imputation for quotes approved for the carry forward method affects less than a quarter of the carry forward quotes.

Because the carry forward method generates a price relative of 1, it decreases the rate of growth in the indexes in the short term, which likely improves the accuracy of the indexes because carry forward is used for quotes whose prices typically do not change except in specific months. This method also reduces nonresponse imputation-induced volatility in the indexes.

Moving home healthcare

In 2020, BLS moved home healthcare services from the index for services by other medical professionals to the index for the care of invalids and elderly at home. Home healthcare includes the services of nurses, therapists, and other medical professionals who provide in-home medical care, such as dressing wounds, injecting medication, physical therapy, and drawing blood. Home healthcare is separate from home care services provided by nonmedical professionals who visit a customers’ home to help with laundry, bathing, feeding, dressing, and cooking.

Before 2020, prices collected for nonmedical home care services were included in the sample for the care of invalids and elderly at home index, whereas prices collected for home healthcare services provided by nonphysician medical professionals in the home were included in the sample for the services by other medical professionals index. Since 2020 the CPI includes prices collected for both medical and nonmedical home care services in the index for the care of invalids and elderly at home.

Several reasons underlie this decision to include prices for both services in the same index. First, the BLS Producer Price Index program, U.S. Bureau of Economic Analysis, Economic Census, and Centers for Medicare and Medicaid Services use the North American Industry Classification code 6216, home healthcare services, which combines medical home care and nonmedical home care.⁹ Thus, the decision to include both medical and nonmedical home care services in the index for care of invalids and the elderly at home synchronizes CPI practice with that of other major government organizations.

Second, the impact on the sample for services by other medical professionals is minimal. Before 2020, the “place of services” was one of the price-determining specifications used to identify a unique price for a quote in this index. In 2020, however, the sample contained no quoted prices for home as a place of service. In other words, the index for services by other medical professionals was not capturing any price change related to home healthcare services.¹⁰

Third, including home healthcare services in the index for care of invalids and the elderly at home would improve the sample of outlets used for this index. Previously, many potential outlets were dropped because they provided medical services but not nonmedical services, making them eligible only for the sample of outlets used to construct the index for services by other medical professionals. Internal analysis revealed that, over a roughly 5-year timespan, 20 percent of outlets were dropped from the sample because they only provided medical home care. Adding home healthcare to the sample of outlets for the care of invalids and elderly at home index helps to increase the number of outlets in the sample in which prices can be collected.

Future work

This article has focused on improvements to methods used in the CPI medical care index that have already been implemented. This final section discusses ongoing research that examines the use of claims data obtained from insurance companies. The discussion addresses both declining respondent cooperation among medical care providers and declining responses from households that participate in surveys designed to select the provider outlets.

As just explained, manual price collection for the CPI medical care indexes faces unique challenges. The services provided by hospitals and physicians’ offices are complex and require much time and effort to gain the cooperation of respondents, to review a set of specifications to identify a unique price for a unique service, and to collect the price of the same service monthly. Given a busy work environment and complex pricing specifications, hospital respondents have become increasingly reluctant to cooperate in price collection. Moreover, physicians’ offices have become increasingly reluctant to share data on the prices of services covered by private insurance companies lest they risk the privacy and safety of proprietary data. In addition, household surveys designed to select providers have also met challenges to sample integrity because households are increasingly reluctant to participate. The result is a sample of prices that is too small and may not be representative.

One of the primary ways BLS has attempted to address low response rates during manual price collection is to pursue alternative sources of data. As explained by Crystal G. Konny, Brendan K. Williams, and David M. Friedman, alternative data sources may “increase sample sizes, reflect consumer substitution patterns more quickly, reduce or eliminate respondent burden, help address non-response problems in the CPI’s surveys, and reduce collection costs.”¹¹ One potential source of alternative data comes from insurance claims aggregators. These databases include the transaction prices of healthcare services for which clients seek coverage.

The challenge for BLS in using insurance claims data is to develop a method for incorporating such data into the sample without introducing bias or unduly increasing the variability of the sample. Earlier internal studies determined that the costs of using insurance claims data outweighed the benefits.¹² The challenges are many, including

·       the time lag in receipt of data from the insurance company because of the time insurance companies take to settle claims from their beneficiaries

·       choice of an appropriate weighting scheme

·       added costs to CPI price collection because insurance company data are not representative of market prices and thus can only supplement, not replace, manual price collection

·       missing prices because of services not being billed in a particular month.

However, because of deteriorating sample integrity, BLS has renewed research efforts to devise an appropriate method for integrating insurance claims data in the CPI physicians’ services and hospital services indexes. This research has gone through two phases.

The first phase is described in a paper by John Bieler, Caleb Cho, Brett Matsumoto, Brian Parker, and Daniel Wang.¹³ They constructed price indexes using prices obtained from an insurance company for services provided in a large city. BLS obtained a 2-month-lagged dataset from the insurance company consisting of an expenditure-weighted (probability proportional to size) random sample of insurance claims for unique procedures—current procedure terminology codes for physicians and hospital outpatient and diagnostic-related group codes for hospital inpatient—provided in one city during 2009 and 2010. From each of the top 10 hospital outlets, 10 services were selected as ranked by expenditure, whereas 5 services were selected from each of the top 25 physician outlets as also ranked by total expenditure.

The sampling frame was set up to select only claims for services that were provided by an outlet at least a certain number of times in a month, to control for the possibility of no claims data for a particular service in a particular outlet in a particular month. According to John Bieler, Caleb Cho, JD Gayer, Brett Matsumoto, Brian Parker, and Daniel Wang, the data consisted “of the average allowed amount or total payment to the provider, standard deviation of the average allowed amount, and the number of claims for every selected provider and procedure unit each month from January 2009 through December of 2010.”¹⁴ They then constructed “price indexes for physicians’ services and hospital services using different weighting structures for aggregating and then compare[d] the indexes to the corresponding CPI indexes for the area.”¹⁵ The article also uses data on average prices and quantities, for particular medical service-provider combinations, from IBM Watson Health’s MarketScan data to calculate indexes considered a comparison dataset because the data include a broad set of private insurance providers.¹⁶

Researchers compared indexes by using sum of squared errors between indexes as a measure of goodness of fit. When outliers were removed as deemed appropriate and a variety of weighting schemes were used, the experimental indexes performed well compared with the MarketScan and CPI medical care benchmark indexes. The research demonstrated that alternative data adequately supplemented the CPI. However, because index accuracy can decrease as the lag time increases, indexes that use alternative data from a single insurer tend to be less accurate than indexes that use alternative data from a more exhaustive and representative source, and performance discrepancies arise between PSU-level data and national data that determine how long of a lag is acceptable.

The second phase of research is described in an upcoming article.¹⁷ The researchers conducted a similar analysis using purchased medical claims data that cover all the geographic areas of the CPI over 4 years.

The data also had a 3-month lag instead of a 2-month lag and included multiple health insurance providers instead of a single insurance provider. Unlike the first phase, however, the researchers analyzed data only for physician’s services and outpatient hospital services. They decided not to use the inpatient hospital data without additional research on how it would affect the variance of CPI index calculation. Because of diverse patient characteristics, inpatient hospital prices show higher volatility, which is difficult to control.

The analysis showed that medical claims data can help address reduced response rates by supplementing the CPI monthly sample with additional observations that improve accuracy and reduce volatility. Moreover, the choice of index formula did not greatly change the results. The researchers used three different index formulas and arrived at similar results in each case. For consistency, however, they do recommend that CPI calculation continue to rely on current production methods rather than switch to use of a superlative formula.

Additional details are beyond the scope of this article. In general, considering insurance claims as an alternative data source for use in CPI medical care indexes is an important part of ongoing research addressing the problems of low response rates and overrepresentation of self-pay quotes. BLS has approved the use of medical claims data to supplement the physicians’ services and outpatient hospital services indexes. These data will be incorporated into CPI calculations once BLS information technology systems are modified to accommodate the data. BLS plans to continue research into the use of medical claims data for improving the inpatient hospital services component of the basket.

Summary

BLS has implemented several methodological improvements to address low response rates and the overrepresentation of self-pay quotes in the medical care sample. BLS now uses MEPS data to obtain a more accurate sample of quotes by payer type for the physicians’ services index. BLS uses data from a retail pharmacy to obtain a more accurate sample of quotes by payer type for the prescription drugs index, which also improves the efficiency of incorporating generic drugs into the prescription drugs index.

Additional improvements that have helped stabilize response rates in recent years include longer rotation cycles for hospital services, mixed medical samples for the physicians’ services and hospital services indexes, preselection of pricing specifications for the hospital services index, the carry forward method, and the movement of home healthcare from the index for services by other medical professionals index to the index for the care of invalids and elderly at home.

Finally, a major implementation effort involves the use of insurance claims data to address low response rates. An analysis by Bieler, Cho, Gayer, Matsumoto, Parker, and Wang shows that claims data can improve accuracy and reduce volatility.¹⁸

top Back to Top