Individual Diaries Feasibility Test
The Individual Diaries Feasibility Test was designed to test the feasibility and impact of using both personal (individual) diaries for each household member and online diaries with multiple modes (mobile and desktop) to collect Consumer Expenditure diary expenditures.
The current Consumer Expenditure Diary Survey (CED) at the Bureau of Labor Statistics (BLS) uses a pencil-and-paper instrument (PAPI) to collect expenditure information from a household using a main diary respondent to record expenditures for all household members. The Gemini Project to redesign the CE Survey has been exploring online diaries as an alternative to PAPI data collection since PAPI diaries have some inherent limitations because the paper diary has to be carried around by the diary keeper throughout the day for contemporaneous reporting. Additionally, having one person in the household maintain the diary for all members of the household has drawbacks, such as potential underreporting of expenditures because the diary keeper may be unaware of expenditures made by other household members.
The research questions of interest were:
1. Feasibility: What are the operational issues that arise in fielding individual online diaries?
2. Participation: Do individual online diaries improve survey cooperation compared to household-level paper diaries?
3. Contemporaneous reporting: To what extent do online diaries, which can be accessed from any internet-enabled location, lead to contemporaneous reporting of expenses incurred?
4. Reporting in Individual Diaries: What effect did diary-keeping at the individual household member level, compared to allowing one respondent report for the entire household, have on the overall number of household entries?
5. Individual online mode data quality: Do online diaries result in improvements in data quality over paper diaries? Within the online mode, were there differences in data quality between mobile and desktop diaries?
6. Ideal household characteristics for the online diaries: What household characteristics were associated with online diaries having a larger number of entries?
7. Respondent feedback: What can we learn from respondents about online diaries and their association with reporting behavior?
Household members in the research sample were first offered a mobile diary if they had access to the internet through a mobile device, and then a desktop diary if they only had home access to the internet. This choice of mode was offered to each age-eligible member of the household to complete their online diary individually.
This page includes a description of the test design, links to materials used during the test, and a summary of test results.
Below are links to a brief overview of the Individual Diaries design and related material. For a detailed overview of the design, click on the above PDF link for the Individual Diaries Feasibility Test Detailed Overview.
An advance letter was sent via first class mail, one week prior to each month of interviews.
The control group consisted of production sample households from August 2014 through December 2014 and was administered the current diary using standard procedures. After the field period of the control group concluded, socio-demographic matching to the test group was completed for analysis.
The test group consisted of research sample households, who filled out online diaries for each eligible member using procedures that were modified from the standard production procedures. The contact schedule for the Test Group was as follows:
All sampled households, randomly assigned to the test group were asked the internet access screening question and follow-up questions on how they access the internet. The desktop or mobile-optimized online diaries were assigned to each participant in person. If eligible household members were not available at placement, the main respondent determined which type of diary the unavailable household members received. Household members were given materials describing the desktop online diary or mobile-optimized online diary and instructions on how to log in and complete the diary. If the household did not have an internet connection, then the household was deemed ineligible to participate. Questions on general housing and member characteristics were asked as is done in the production survey, using a Computer Assisted Personal Interview (CAPI) instrument as follows:
Since a Spanish version of the online diary has not been developed, non-English speaking households in the test group were not eligible to participate and were counted as noninterviews.
Prior to the second visit, households received a reminder telephone call about keeping the diary.
The households were visited in-person at the end of the diary period to thank them for their participation. At that time, additional questions were asked regarding work experience and income:
Interviewers also collected receipts, if provided by respondents. The CAPI instrument was modified to include a tab that was designed to replicate the online survey instrument. Interviewers used this tab to enter any additional expenditures that the respondent did not enter into the online diary, including any receipts and recalled expenditures. Interviewers did not have access to respondents' online diaries and were therefore unable to verify if expenditures entered this way were not already expenditures entered normally through the online instrument.
At the end of the last visit, respondents were asked a set of debriefing questions regarding their experiences with the diaries. Following the interview, the interviewers were also asked a set of debriefing questions.
Results indicate that cooperation rates were not higher for online diaries compared to PAPI diaries. The mean and median expenditure values reported were not higher than the PAPI diaries and the number of expenditure were actually lower than the number from household-level paper diaries. However, a number of characteristics, such as increased contemporaneous reporting as well as limited proxy reporting, which are associated with increased expenditure entries and more accurate expenditure amounts were identified and respondent feedback was also largely positive. Overall, providing households with individual online diaries did not represent an improvement over a single household PAPI diary. This finding may in part be attributable to the obstacles in implementing the test protocol as designed including a low occurrence of all eligible household members cooperating, complex password requirements, a long website address, and inconsistent implementation of the online diary protocol by interviewers. These lessons learned became inputs into the Proof of Concept Test.
The detailed results of the Individual Diaries Feasibility Test are available in the final report.
Last Modified Date: July 26, 2018