<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>Grid questions are frequently employed in web surveys due to their assumed response efficiency. In line with this, many previous studies have found shorter response times for grid questions compared to item-by-item formats. Our contribution to this literature is to investigate how altering the question format affects response behavior and the depth of cognitive processing when answering both grid question and item-by-item formats. To answer these questions, we implemented an experiment with three questions in an eye-tracking study. Each question consisted of a set of ten items which respondents answered either on a single page (large grid), on two pages with five items each (small grid), or on ten separate pages (item-by-item). We did not find substantial differences in cognitive processing overall, while the processing of the question stem and the response scale labels was significantly higher for the item-by-item design than for the large grid in all three questions. We, however, found that when answering an item in a grid question, respondents often refer to surrounding items when making a judgement. We discuss the findings and limitations of our study and provide suggestions for practical design decisions.</p>
</abstract>

Grid questions are frequently employed in web surveys due to their assumed response efficiency. In line with this, many previous studies have found shorter response times for grid questions compared to item-by-item formats. Our contribution to this literature is to investigate how altering the question format affects response behavior and the depth of cognitive processing when answering both grid question and item-by-item formats. To answer these questions, we implemented an experiment with three questions in an eye-tracking study. Each question consisted of a set of ten items which respondents answered either on a single page (large grid), on two pages with five items each (small grid), or on ten separate pages (item-by-item). We did not find substantial differences in cognitive processing overall, while the processing of the question stem and the response scale labels was significantly higher for the item-by-item design than for the large grid in all three questions. We, however, found that when answering an item in a grid question, respondents often refer to surrounding items when making a judgement. We discuss the findings and limitations of our study and provide suggestions for practical design decisions.

Using Eye-Tracking Methodology to Study Grid Question Designs in Web Surveys

Journal of Official Statistics

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

{"article-title":"Using Eye-Tracking Methodology to Study Grid Question Designs in Web Surveys"}

Grid questions are frequently employed in web surveys due to their assumed response efficiency. In line with this, many previous studies have found shorter...

Using Eye-Tracking Methodology to Study Grid Question Designs in Web Surveys

Published Online: Mar 16, 2023

Page range: 79 - 101

Received: Jan 01, 2021

Accepted: Aug 01, 2022

DOI: https://doi.org/10.2478/jos-2023-0004

Keywords
Web surveys, response behavior, cognitive processing, question design, eye-tracking methodology

© 2023 Cornelia E. Neuert et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Using Eye-Tracking Methodology to Study Grid Question Designs in Web Surveys

Published Online: Mar 16, 2023

Page range: 79 - 101

Received: Jan 01, 2021

Accepted: Aug 01, 2022

DOI: https://doi.org/10.2478/jos-2023-0004

KeywordsWeb surveys, response behavior, cognitive processing, question design, eye-tracking methodology

© 2023 Cornelia E. Neuert et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Keywords
Web surveys, response behavior, cognitive processing, question design, eye-tracking methodology