The Use of Paradata for Evaluating Interviewer Training and Performance

William Mockovak and Randall K. Powers


Audit files produced by CAI software present a record of the actions and entries made by interviewers as an interview is completed. These data provide objective feedback on the interviewing process, which can lead to changes in training or supervision. In this study, a software tool was developed that searched for and provided frequency counts of text that described selected interviewer actions. Survey production files were then analyzed to answer a series of questions including how interviewers chose to enter data (keyboard vs. mouse), when and how they navigated in a complex instrument, how interviewers handled edit messages when they were triggered, and how more complex sections of the instrument (e.g., tables) were handled. The advantages and disadvantages of using paradata for these purposes are discussed.