Department of Labor Logo United States Department of Labor
Dot gov

The .gov means it's official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Bureau of Labor Statistics > Office of Survey Methods and Research > Publications > Browse Research Papers

How Much Can you Trust the Answers You Get Using Cognitive Interviews?

Christine Rho and Roberta L. Sangster

Abstract

Cognitive interviews has been widely used and studied as a research tool to reduce sources of response error and improve survey questions. Cognitive interviews are based on the assumption that respondents are reasonably reliable informants about how they comprehend and answer questions. We recently conducted a study using a greater than average number of cognitive interviews to test question comprehension (n=40) in a housing survey. The study was conducted in two parts: 1) survey completion and 2) retrospective probing. In the first part, respondents answered questions from the housing survey with the option of indicating when they did not know the answer or understand the questions. In the second part, respondents were interviewed and probed about the clarity of questions, their subjective experience of answering the survey questions and how they arrived at the answers. This study yielded both quantitative and qualitative data, which allowed a comparison of actual survey responses and information respondents reported about clarity and comprehensibility of questions. This comparison showed that while respondents would say a question was clear and that they understood it, their actual responses to survey indicated a lack of comprehension. This study will show data from the housing study that indicate this discrepancy and will discuss the limits of relying on respondents as informants of their thought processes.