• Overview
• Concepts
• Data sources
  • Survey notification and collection method
  • Interview Survey details
  • Diary Survey details
  • Integrated survey data
  • Quality control and confidentiality
  • Data collection and processing
• Design
• Calculation
• Presentation
• History
• More Info

• Methods Home
• About
• Programs
• Glossary
search

Handbook of Methods Consumer Expenditures and Income Collections & Data Sources

Consumer Expenditures and Income: Collections & Data Sources

The concepts section gave some information on the two types of surveys the Consumer Expenditure (CE) program administers: the Interview Survey and the Diary Survey. This section will provide more detailed information on how these two surveys are administered.

Survey notification and collection method

CE data are collected for the Bureau of Labor Statistics (BLS) by the U.S. Census Bureau.  A selected sample housing unit is notified in advance by a letter from the Census Bureau informing the occupants about the purpose of the survey and the upcoming visit by the interviewer. Both the Interview Survey and the Diary Survey are conducted primarily by personal visit with some telephone interviewing. The interviewer uses a structured questionnaire to collect both the demographic and expenditure data in the Interview Survey. The demographic data for the Diary Survey are collected by the interviewer, whereas the expenditure data are entered on the diary form by the respondent. Any eligible household member who is at least 16 years old can serve as the respondent in either survey.

Interview Survey details

The Interview Survey collects detailed data on an estimated 60 to 70 percent of total family expenditures. In addition, global estimates are obtained for food and other selected items such as alcoholic beverages and tobacco products. These global estimates account for an additional 20 to 25 percent of total expenditures. Global estimates are based on responses to global questions in the survey. A global question is designed to elicit comprehensive information about multiple subcategories using a single question. Global questions can be contrasted with detailed questions, in which a series of questions about specific types within a larger category are asked. For example, rather than asking individual detailed questions about purchases of pants, dresses, and footwear, and then aggregating to a total expenditure amount, a global question might ask for the total amount spent on clothing. On average, it takes 60 minutes to complete the interview.

In the Interview Survey, each family in the sample is interviewed every 3 months over four calendar quarters. The sample for each quarter is divided into three monthly panels, with consumer units (CUs) being interviewed every 3 months in the same panel of every quarter.

After the fourth interview, the sample unit is dropped from the survey and replaced by a new sample unit. For the survey as a whole, 25 percent of the sample in each quarter are new families introduced into the sample, to replace families that have completed their participation. Data collected in each quarter are treated independently, so that published 12-month estimates are not dependent upon a particular family participating in the survey for a full four quarters.

Exhibit 1 shows how Interview Survey consumer units rotate in and out of the sample.  In this example, the first interviews start in April, May, and June 2015.  Three months later, the second interviews begin. Consumer units first interviewed in April 2015 are re-interviewed in July, October, and January 2016.  And while the second set of interviews begins in July 2015 for the units first interviewed in April, a new set of addresses is starting their set of four interviews.

During the initial interview (interview set 1 on exhibit 1) information is collected on demographic and family characteristics and on the inventory of major durable goods of each consumer unit. Expenditure information that is based on 3 months of respondents’ recall is also collected during this interview.

Data collected in these questionnaires are arranged by major expenditure component (for example, housing, transportation, medical, and education), and form the basis of the expenditure estimates derived from the Interview Survey. Wage, salary, and other information on the employment of each member of a consumer unit are also collected in the first interview and updated in the fourth interview. Expenditure data are collected via two major types of questions. The first set of questions asks the respondent for the month of purchase of each reported expenditure. The second asks for a quarterly amount of expenditures. The use of these two questions varies, depending on the types of expenditures collected. Most of the data are collected using the direct monthly method. A portion of the data are collected by asking for quarterly expenses, but this also includes asking for the amount that was spent in the current month so as to only have the expenses that occur in the 3-month reference period. In the final interview, an annual supplement is used to obtain a financial profile of the consumer unit. This profile consists of information on the income of the consumer unit as a whole, including unemployment compensation; income from royalties, dividends, and estates; alimony and child support. A 12-month recall period is used in the collection of income- and asset-type data.

Diary Survey details

The separate Diary Survey is completed by the respondent family for two consecutive 1-week periods.

Two separate questionnaires are used by the Census Bureau to collect Diary Survey data: a Household Characteristics Questionnaire and a Record of Daily Expenses. In the Household Characteristics Questionnaire, the interviewer records information pertaining to age, sex, race, marital status, and family composition, as well as information on the work experience and earnings of each member of the consumer unit. This socioeconomic information is used by the Bureau to classify the consumer unit for publication of statistical tables, as well as for economic analyses. Data on household characteristics also provide the link in the integration of Diary Survey expenditure data with Interview Survey expenditure data that permit the publication of a full profile of consumer expenditures by demographic characteristics.

The daily expense record is designed as a self-reporting, product-oriented diary, in which respondents record a detailed description of all expenses for two consecutive 1-week periods. Diarykeeping can start on any day of the week. Data collected each week are treated as statistically independent – each week’s diary is separately weighted to be representative of the sample. The diary is divided by day of purchase and by four classifications of goods and services—food away from home, food at home, clothing, and all other goods and services—a breakdown designed to aid the respondent in recording the entire consumer unit’s daily purchases. The items reported are subsequently coded by the Census Bureau, so that BLS can aggregate individual purchases for representation in the CPI and for presentation in statistical tables.

The Diary Survey asks for almost all expenses that the consumer unit incurs during the survey week. Expenses incurred by family members while away from home overnight or on vacation, and for credit and installment plan payments are excluded.

It takes approximately 25 minutes over three visits for the interviewer to collect the demographic data and to instruct the respondent on how to keep the diary. It is estimated that it takes the respondent 15 minutes each day to complete the diary.

Integrated survey data

Integrated data from the BLS Diary and Interview Surveys provide a complete accounting of consumer expenditures and income, which neither survey component alone is designed to do. Most apparel and food expenditures in the integrated data tables come from the Diary, while rent, mortgage interest, utilities, and major appliances come from the Interview. Some expenditure items are collected only by the Diary or Interview Survey. For example, the Diary Survey collects data on detailed food expenditures and items, such as postage and nonprescription drugs, which are not collected in the Interview Survey. The Interview Survey collects data on expenditures for overnight travel and information on insurance reimbursements for medical-care costs and automobile repairs, which are not collected in the Diary Survey. Data on average annual expenditures that come exclusively from the Interview Survey, including global estimates, such as those for food and alcoholic beverages, average about 95 percent of the total estimated spending, based on integrated Diary and Interview Survey data. For items unique to one or the other survey, the choice of which survey to use as the source of data is obvious. However, there is considerable overlap in coverage between the surveys. Because of the overlap, the integration of the data presents the problem of determining the appropriate survey component from which to select the expenditure items. When data are available from both survey sources, the more reliable of the two is selected, as determined by statistical methods. The selection of the survey source is evaluated every 2 years. https://www.bls.gov/cex/anthology11/csxanth3.pdf.

Quality control and confidentiality

Quality control and data integrity are provided by a re-interview program, which evaluates the performance of the individual interviewer, to determine how well the procedures are being carried out in the field. The re-interview is conducted by a Census Bureau supervisor or an interviewer at a National Processing Center Data Contact Center (a centralized telephone call center for conducting interviews). Subsamples of approximately 9 percent of households in both the Interview and Diary surveys are re-interviewed on an ongoing basis.

All data collected in both surveys are subject to Census Bureau and BLS confidentiality requirements that prevent the disclosure of the respondents' identities. The information that respondents provide is used solely for statistical purposes. All Census Bureau and BLS employees who work with the CE data take an oath of confidentiality and are subject to fines and imprisonment for improperly disclosing information provided by respondents. Confidentiality certification training is required annually.

Names and addresses are removed from all forms and datasets prior to transmission from the Census Bureau to BLS and are not included in any statistical releases. At BLS, the data are processed and stored on secure servers, with access limited to employees having security clearances. As a further precaution, BLS applies certain restrictions to the microdata available on the public-use files. These include geographical and value restrictions that prevent identification of respondents.

Data collection and processing

Due to differences in format and design, the Interview Survey and the Diary Survey are collected and processed separately. The Census Bureau, under contract with BLS, carries out data collection for both surveys. In addition to its collection duties, the Census Bureau does field editing and coding, checks consistency, ensures quality control, and transmits the data to BLS. In preparing the data for analysis and publication, BLS performs additional review and editing procedures.

Interview Survey. Beginning April 2003, census field representatives started collecting the Interview Survey data using a Computer Assisted Personal Interview (CAPI) instrument. This was a major improvement from the paper-and-pencil data collection that had been in place since 1980. The CAPI instrument enforces question skip patterns, allows for data confirmation of high expenditure values, and reduces processing time. The CAPI instrument allows the field representative to see and select from a list the category that an expense would fit into as well as any classifying information about the expense. Categories of expenses would be pants, oil changes, or trips. Examples of classifying information consist of age and sex of clothing purchase, make and model of vehicles owned, and trip destination. See Exhibit 2 for the diagram of the process flow from collection by the field representative (FR) to delivering the data to the CPI program, and then to publishing the information for the public.

Exhibit 2.

Data are electronically transferred from a field representative’s laptop at completion of the interview to the Census Master Control System. The Census Bureau does post processing by reformatting the data into datasets and performing special processing, including converting missing values to special characters and merging data records into the required BLS output structure. Some inventory data, such as those for vehicle and mortgage records, are copied into an input file that is loaded onto the laptops for subsequent interviews during the next quarter. This way, a few fields are updated each quarter, rather than an entire data record. As mentioned earlier, names and addresses of respondents are not transmitted to BLS.

At BLS, a series of automated edits are applied to monthly data. These edits check for inconsistencies; identify missing expenditure amounts for later imputation; impute values for missing demographic variables; calculate weights; and adjust data to include sales tax, exclude business expenses, and net out reimbursed expenditures.

Monthly data files then are combined into quarterly databases, and a more extensive data review is carried out. During this data review, BLS conducts the following steps: verifies counts and means by region, checks family relationship coding inconsistencies, and inspects selected extreme values for expenditure and income categories. Outlier values are confirmed or corrected. Other adjustments convert mortgage and vehicle loan payments into principal and interest (using associated data on the interest rate and term of the loan). In addition, BLS verifies the various data transformations it performs. As with outlier values, other cases of questionable data values or relationships are investigated, and errors are corrected before release of the data for public use.

Three major types of data adjustment routines—imputation, allocation, and time adjustment—improve estimates derived from the Interview Survey. Data imputation routines account for missing or invalid entries and address all fields in the database, except assets. Allocation routines are applied, when respondents provide insufficient detail to meet tabulation requirements. For example, combined expenditures for the fuels and utilities group are allocated among the components of that group, such as natural gas and electricity. Time adjustment routines are used to classify expenditures by month, prior to aggregation of the data to calendar-year expenditures. Tabulations are made before and after data adjustment routines, to analyze the results.

The CE implemented multiple imputations of income data starting with the publication of 2004 data. Before that, only income data collected from complete income reporters (those CUs whose reference person provided answers to most major sources of income) were published. However, even complete income reporters may not have provided information on all sources of income for which they reported receipt. With the collection of bracketed income data starting in 2001, this problem was reduced but not eliminated. One limitation was that bracketed data only provided a range in which income falls, rather than a precise value for that income. In contrast, imputation allows income values to be estimated when they are not reported. In multiple imputations, several estimates are made for the same consumer unit, and the average of these estimates is published. See https://www.bls.gov/cex/csximpute.htm and https://www.bls.gov/opub/mlr/2009/08/art3full.pdf.

Beginning with the 2014 published data, the CE imputes all state and federal income taxes for all consumer units. The CE uses an internal BLS version of the National Bureau of Economic Research’s TAXSIM software in estimating tax liabilities. Starting in 2015, respondents are no longer asked questions about their income taxes, which reduced respondent cognitive burden.

The CE Interview questionnaire is revised every 2 years to incorporate new products and services, to clarify instructions, to improve navigation through the instrument, to incorporate changes required for the CPI, and to streamline the interview by deleting outdated items. Whereas changes to the questionnaire are made biennially, CE staff continuously monitors the emergence of new goods and services available in the marketplace, as well as changes in the relative importance of existing items in consumers’ budgets.

Updated information on how to report new goods and services is provided to the field representatives on a regular basis. Also, new items are incorporated in a product index that enables Census field representatives to classify these new items by the appropriate item codes. The product index is a file that contains a list of items that could be found under a particular item code. Given the space limitation in the information book shown to respondents, the Census Bureau is not able to include a lot of items under each category. The product index contains more descriptions or examples of items. For example, camping equipment has “tent” as an example in the information booklet, but not “tent poles.” So the field representative could look up tent poles in the product index and see which item code it belongs under. This is a file that is accessible through the CE Quarterly Interview Survey API instrument. It is maintained by the BLS CE Production and Control staff as part of normal revisions to the survey.

Diary Survey. At the beginning of the 2-week collection period, the Census Bureau interviewer, using the Household Characteristics Questionnaire (a CAPI instrument), records demographic information on members of each sampled consumer unit.

At the end of the first week, the interviewer collects the diary, reviews the entries, answers any questions that the respondent may have, and leaves a second diary. At the end of the second week, the interviewer picks up the second diary and reviews the entries. During this time, the interviewer again uses the Household Characteristics Questionnaire to collect previous-year information on work experience and income.

Diaries for both week 1 and week 2 may be placed with the consumer unit at the first week interview under certain circumstances depending on distance and special circumstances for the respondent. When the interviewer places both diaries at the first week dropoff, the interviewer conducts all first week dropoff procedures and informs the respondent of the need to keep 2-weeks of diaries. At the end of the 2-week collection period, the interviewer conducts all other necessary interview activities that occur at the end of the 2-week period. Each week of a consumer unit's participation in the survey is treated as a separate entity. See Exhibit 3 for a description of the process starting from data collection by the field representative (FR) to the regional office (RO) to the National Processing Center (NPC) or directly to the Census headquarters, all the way to delivering the data to BLS for the CPI, and making the information available to the public.

Exhibit 3.

The Census Bureau performs preliminary processing activities, including a number of data edits and adjustments. Data in the diaries are reviewed during a field edit for completeness and consistency. After the diaries are sent to the Census Bureau’s National Processing Center, expenditure data captured in the diaries are key-entered into electronic formats assisted by auto-coding to fill in typical descriptions; and a computer file of the database containing these data is produced and transmitted monthly to Census headquarters, along with image files of the diaries. Census headquarters merge the expenditure data with the data collected in the Household Characteristics Questionnaire, removes personal identifying information, and transmits the merged file monthly to BLS. At BLS, data are processed by computer to calculate population weights, to impute demographic characteristics for missing or inconsistent demographic data, to impute values for weeks worked when nonresponse is encountered, and to apply appropriate sales taxes to the expenditure items.

The Diary Survey combines three monthly diary data files into quarterly databases; and BLS screens the data for invalid coding and inconsistent relationships, as well as for extreme values recorded or keyed erroneously. BLS then corrects any coding and extreme-value errors found.

Two types of data adjustment routines—allocation and imputation—improve the Diary Survey estimates. Allocation routines transform reports of nonspecific items into specific ones. For example, when respondents report expenditures for meat rather than beef or pork, allocations are made, using proportions derived from item-specific reports in other completed diaries. BLS imputes missing attributes, such as age or sex, or a product’s type of packaging needed for mapping Diary Survey expenditures. Income data from the Diary Survey are processed in the same way as in the Interview Survey. Income tax data are not estimated for the Diary Survey because the published integrated information uses Interview Survey after-tax income information, and because of the limited amount of background detail collected in the Diary Survey.