A Permutation Test for Correlated Errors in Adjacent Questionnaire Items

Abstract

Response patterns are of importance to survey researchers because of the insight they provide into the thought processes respondents use to answer survey questions. In this article we propose the use of structural equation modeling to examine response patterns and develop a permutation test to quantify the likelihood of observing a specific response pattern. Of interest is a response pattern where the response to the current item is conditioned on the respondent's answer to the immediately preceding item. This pattern manifests itself in the error structure of the survey items by resulting in larger correlations of the errors for adjacent items than for nonadjacent items. We illustrate the proposed method using data from the 2002 Oregon Survey of Roads and Highways and report SAS code that can be easily modified to examine other response patterns of interest.

Keywords:

• correlated errors
• general-specific questions
• permutation test
• response patterns in surveys

Notes

¹Although it would potentially be computationally feasible in this example to compute all 3,003 models, this need not be the case, especially when there are more than six indicators associated with a latent variable. For example, suppose there are 10 indicators, implying nine first-order correlations and 45 possible pairs of correlated errors. This would lead to 886,163,135 different models, which is not computationally feasible.