U.S. flag

An official website of the United States government

Department of Labor Logo United States Department of Labor
Dot gov

The .gov means it's official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Bureau of Labor Statistics > Office of Survey Methods Research > Publications > Browse Research Papers

Horizontal vs. Vertical Scales vs. Use of a Grid in Online Data Collection: Which Is Better?

William Mockovak

Abstract

When designing online questionnaires, survey designers often have the option of choosing between the use of horizontal or vertical rating scales, or possibly a grid, if several survey questions use a common response scale. Previous research has investigated use of these alternative scales, but with inconsistent findings. As a result, some researchers advocate use of horizontal scales, others recommend the use of vertical scales,1 whereas the use of grids has been generally discouraged. Although the use of alternative scales has been heavily researched, the familiarity and expertise of online respondents with different question formats have continued to increase, so studies that were done years ago may no longer be relevant. Therefore, this study revisited the topic of question format and its impact on data quality. Using online instruments that presented the exact same question order, content, and scale direction (from positive to negative), unipolar horizontal and vertical scales were compared across two questions, and with the use of a grid in four additional questions. All individual questions and the grid appeared on separate web pages. Participants were recruited through Amazons Mechanical Turk and saw only one version of the questions. Across the six questions that compared horizontal or vertical scales (N=193 and 229, respectively), no significant differences were found on five of the questions. On one question, the use of a horizontal scale led to significantly higher ratings. In the comparison of the 4-question grid (N=279) to the same questions presented using horizontal or vertical scales, three of the questions showed no significant differences among any of the contrasts (vertical vs. horizontal vs. grid). The only significant difference occurred on one question where the mean of the grid question was significantly higher than the mean for the horizontal scale, but not significantly higher than the vertical scale. Data quality comparisons among the grid, horizontal, and vertical scales were also explored, but no significant differences were found in scale reliability (measured with Chronbachs alpha), in the amount of straight-lining that occurred, in item non-response, or in the resulting factor structure. In summary, the question formats studied did not consistently affect the results or associated measures of data quality for the questions in this study.e