Measuring subjective social stratification: how does the graphical layout of rating scales affect response distributions, response effort, and criterion validity in web surveys?

Figures & data

Figure 1. Screenshot of the question on subjective social stratification with a rating scale in the form of a ladder. Note. We used an optimized survey layout preventing horizontal scrolling and facilitating survey navigation. The screenshot shows the presentation on a PC.

Figure 2. Screenshot of the question on subjective social stratification with a rating scale in the form of a pyramid. Note. We used an optimized survey layout preventing horizontal scrolling and facilitating survey navigation. The screenshot shows the presentation on a PC.

Table 1. Response distributions (in percentage) and median response times (in seconds) of the target question on subjective social stratification by experimental condition (ladder or pyramid)

Response distribution	Ladder	Pyramid	Significance level
1 Top	1.3	0.6	χ^2(10) = 496.58, p < 0.001
2	3.2	1.2
3	14.5	4.9
4	25.9	12.1
5	28.6	23.7
6	12.5	27.7
7	7.9	15.9
8	4.6	10.1
9	1.0	2.9
10 Bottom	0.5	0.9
Median response times (sec)	20	21	χ^2(1) = 2.30, p = 0.129

Table 2. OLS regressions predicting personal income, school-leaving qualification, and professional qualification with subjective social stratification (unstandardized coefficients)

Dependent variables	Ladder Coefficients (SE)	R^2	Pyramid Coefficients (SE)	R^2	Differences between coefficients
Personal income	0.52*** (0.04)	0.09	0.50*** (0.04)	0.09	0.02 (0.05)
School-leaving qualification	0.18*** (0.02)	0.06	0.11*** (0.02)	0.02	0.07** (0.02)
Professional qualification	0.49*** (0.04)	0.08	0.36 (0.04)	0.04	0.13* (0.05)

Note. *p < 0.05, **p < 0.01, ***p < 0.001. Standard errors (SE) in parentheses. Independent variable: subjective social stratification.