Validating Translations of Rating Scale Questionnaires Using Rasch Analysis

It is often important to demonstrate that a translated version of an existing questionnaire consists of items that are comparable to the original and measure the same underlying construct. In this issue, Kashkouli and colleagues present a Persian version of the Graves Orbitopathy Quality of Life Questionnaire (GO-QOL).¹ To test whether their Persian version is properly adjusted for cultural differences, they employ many standard techniques, from back-translation, committee review and pretesting, factor analysis to determine the dimensionality of the questionnaire, and commonly used statistics like Cronbach’s alpha to measure internal consistency. The innovative aspect of their study is that they employ Rasch analysis. The purpose of this editorial is to explain in general terms what Rasch analysis is, why it should be used, and why it makes certain statistics like Cronbach’s alpha unnecessary to report.

Rasch analysis was first developed by Georg Rasch in 1961 to estimate the relative difficulty of items (item measures) and the relative abilities of people (person measures) from a set of responses people assign to items on a questionnaire.² In his original formulation the only acceptable responses to items were 0 or 1. This was extended to the more general case of any number of rating categories by others, notably Andrich.³ The important difference between Rasch analysis and other methods for estimating item difficulty and person ability is that Rasch analysis estimates item measures and person measures on an invariant scale where the units of measurement remain the same along the entire axis. This allows differences between item and person measures to mean the same thing no matter where they lie on the scale.

The reason Rasch analysis is well suited for demonstrating a translated version of an existing questionnaire has comparable items to the original is its sensitivity to differences in item difficulty. Cronbach’s alpha, for example, is insensitive to this, and can have the same value for two questionnaires whose items differ in levels of difficulty. To give an example of why item difficulty is important, suppose an item from one questionnaire asks a person to rate how difficult walking to the nearest store is. Even if the question is properly translated into different languages, the difficulty of the task may depend on the distance to the nearest store, differences in terrain, and how dangerous walking the path is. Naturally, we would want any translated version of a questionnaire to account for this.

Item difficulty also depends on the abilities of the people who rate the items. People in one group may for whatever reason be better at a given task, possibly due to differences in age, health, education or work experience. Thus, any method that estimates item difficulty must assume item measures and person measures are independent of each other. Rasch analysis goes further and assumes every item, person, and threshold (boundaries between neighboring rating categories) used by any person are independent of one another. This allows item measures and person measures to lie on an invariant scale where the units of measurement do not depend on the particular set of items chosen for the questionnaire.

With Rasch analysis, direct comparison of the item measures for corresponding items is possible. Estimated item measures are also insensitive to additional items added, or items subtracted, because of assumptions of independence. This is a crucial difference with a statistic like Cronbach’s alpha that is sensitive to the number of items in the questionnaire; add more items and alpha naturally increases. To test whether a set of items is measuring the same underlying construct, Rasch analysis computes what are called “Infit” and “Outfit” mean square statistics that compare the observed variance in responses to expected variance assuming all items measure the same construct.⁴ Ratios of observed to expected variance ideally are distributed around 1 for both statistics when the estimated measures are valid. Thus, Cronbach’s alpha is essentially unnecessary to report when the more principled approach of Rasch analysis is performed.

Kashkouli and colleagues have applied Rasch analysis to their Persian version of the GO-QOL. Since this is the first translation of the GO-QOL where Rasch analysis has been applied, there is no way at the moment to compare estimated item measures in different versions. Validation of a translated version simply requires demonstrating a linear relationship between estimated item measures from different versions. We suggest that future studies apply Rasch analysis not just to other versions of the GO-QOL, but also to different versions of any questionnaire, in order to demonstrate different versions of the same questionnaire contain comparable sets of items.