Computerized adaptive testing with decision regression trees: an alternative to item response theory for quality of life measurement in multiple sclerosis

Figures & data

Figure 1 Structure of tree obtained using CART with minsplit = 100.

Notes: At each node, the split is written as x._j < a (top), and the mean score predicted at this node is given (bottom). Observations that satisfy the splitting rule are sent to the left child node.
Abbreviation: CART, classification and regression tree.

Figure 2 IRT-based computerized adaptive test algorithm.

Abbreviation: IRT, item response theory.

Table 1 Comparisons of DRT- and IRT-based approaches

Indicators	DRT-based CAT				IRT-based CAT
	minsplit = 100	minsplit = 50	minsplit = 10	minsplit = 5	SEM < 0.33	SEM < 0.44	SEM < 0.55
Mean score	50.09	50.09	50.09	50.09	50.00	49.77	49.68
SD score	8.26	8.52	8.94	9.05	8.64	8.37	7.86
Minimum score	32.97	32.97	23.4	18.46	15.77	15.77	27.19
Maximum score	67.93	73.23	78.38	78.83	79.56	79.56	73.62
Number of items	5 (1)	6 (1)	9 (2)	14 (11)	9 (3)	5 (2)	2 (1)
Accuracy (R)	0.91	0.93	0.98	0.99	0.96	0.90	0.83
RMSE	0.33	0.28	0.16	0.09	0.22	0.35	0.45

Notes: Bold values represent the best models for both DRT- and IRT-based approaches. Data are presented as mean (standard deviation).

Abbreviations: CAT, computerized adaptive testing; DRT, decision regression tree; IRT, item response theory; RMSE, root mean square error; SEM, standard error of measurement.

Table 2 Comparison of DRT-based CAT score with sociodemographic, clinical and QoL data

Characteristics	P	Mean (SD)	p-value
Sociodemographic data
Age	−0.08		0.002
Gender
Male		51.03 (8.74)	0.011
Female		49.75 (8.97)
Educational level
High		52.01 (8.87)	<0.001
Low		48.84 (8.80)
Marital status
Not alone		50.61 (9.53)	0.022
Single		49.85 (8.81)
Occupational status
Active		51.31 (8.97)	<0.001
Unemployed		48.04 (8.80)
Geographical area
South Africa		46.91 (7.92)
Europe*		49.92 (8.88)
Argentina		47.94 (9.11)
North America**		50.29 (8.91)
Eastern Middle***		51.99 (9.82)
Russia		48.77 (6.99)	0.08
Clinical data
EDSS score****	−0.19		<0.001
SPT sum scale	−0.43		<0.001
Disease duration	−0.03		0.313
MS subtype
RR		50.61 (9.08)	<0.001
PP		50.31 (8.28)
SP		48.05 (8.08)
CIS		55.09 (10.33)
QoL data
MusiQoL
ADL	0.55		<0.001
PWB	0.80		<0.001
RFr	0.23		<0.001
SPT	0.49		<0.001
RFa	0.26		<0.001
RHCS	0.23		<0.001
SSL	0.36		<0.001
COP	0.54		<0.001
REJ	0.51		<0.001
Index	0.78		<0.001
SF-36
PF	0.36		<0.001
SF	0.65		<0.001
RP	0.52		<0.001
RE	0.59		<0.001
MH	0.87		<0.001
VT	0.79		<0.001
BP	0.45		<0.001
GH	0.57		<0.001
PCS	0.33		<0.001
MCS	0.82		<0.001

Notes:

* Europe includes Germany, France, Greece, Italy, Norway, Spain and UK.

** North America includes Canada and USA.

*** Eastern Middle includes Israel, Lebanon and Turkey.

**** The EDSS is an ordinal clinical rating scale ranging from 0 (normal neurological examination) to 10 (death due to MS).

Abbreviations: ADL, activity of daily living; BP, bodily pain; CAT, computerized adaptive testing; CIS, clinically isolated syndrome; COP, coping; GH, general health; DRT, decision regression tree; EDSS, Expanded Disability Status Scale; MCS, mental composite score; MH, mental health; MS, multiple sclerosis; MusiQoL, Multiple Sclerosis International Quality of Life; PCS, physical composite score; PF, physical function; PP, primary progressive; PWB, psychological well-being; QoL, quality of life; RE, role emotional; REJ, rejection; RFa, relationships with family; RFr, relationships with friends; RHCS, relationships with health care system; RP, role physical; RR, relapsing–remitting; SF, social function; SF-36, Short-Form 36; SP, secondary progressive; SPT, symptom; SSL, sentimental and sexual life; VT, vitality.

Figure S1 Cross-validation results (1,000 replicates, 200 values of minsplit).

Note: (A) The number of items in function of minsplit, (B) the correlation in function of minsplit, (C) RMSE in function of minsplit.

Abbreviation: RMSE, root mean square error.

Table S1 Cross-validation results for the DRT-based approach

minsplit	Mean score	SD score	Minimum score	Maximum score	Number of items	Accuracy (R)	PMSE
2	50.24	9.19	21.69	80.06	4.43	0.91	0.38
10	50.16	8.84	26.58	76.88	4.60	0.91	0.37
20	50.15	8.64	29.51	74.06	4.53	0.90	0.39
30	50.15	8.52	30.82	73.00	4.46	0.90	0.41
40	50.13	8.42	31.87	71.74	4.52	0.89	0.42
50	50.13	8.34	32.66	70.93	4.56	0.88	0.43
60	50.15	8.27	33.20	69.09	4.50	0.88	0.44
70	50.15	8.21	33.57	67.91	4.42	0.87	0.45
80	50.15	8.16	34.15	67.81	4.32	0.87	0.45
90	50.16	8.11	34.33	67.80	4.20	0.86	0.46
100	50.16	8.06	34.92	67.43	4.06	0.86	0.47
110	50.17	7.99	35.26	66.16	3.90	0.85	0.48
120	50.17	7.94	35.39	65.68	3.77	0.85	0.48
130	50.17	7.91	35.57	65.56	3.64	0.84	0.49
140	50.17	7.87	35.72	65.42	3.53	0.84	0.49
150	50.15	7.81	35.84	65.05	3.43	0.84	0.50
160	50.14	7.77	35.97	64.33	3.33	0.83	0.51
170	50.14	7.74	36.10	64.01	3.24	0.83	0.51
180	50.14	7.70	36.36	63.78	3.13	0.82	0.52
190	50.13	7.67	36.78	63.77	3.05	0.82	0.52
200	50.13	7.63	37.40	63.72	2.94	0.82	0.53

Note: Bold values represent the best model.

Abbreviations: DRT, decision regression tree; RMSE, root mean square error.