Medical chart validation of an algorithm for identifying multiple sclerosis relapse in healthcare claims

Figures & data

Table 1.  Demographic characteristics of the sample.

Characteristic	Total MS population (N = 11,326)	Medical record not reviewed			Medical record reviewed			p-value
		Total (N = 11,026)	No relapse* (N = 7,588)	Relapse* (N = 3,438)	Total (N = 300)	No relapse† (N = 150)	Relapse† (N = 150)	No relapse‡	Relapse‡
Age, years; mean (SD)	45.6 (10.6)	45.6 (10.6)	46.4 (10.5)	43.9 (10.8)	45.1 (9.8)	47.0 (9.8)	43.2 (9.6)	0.467	0.428
Sex, n (%)
Female	8,767 (77.4)	8,525 (77.3)	5,889 (77.6)	2,636 (76.7)	242 (80.7)	125 (83.3)	117 (78.0)	0.095	0.706
Male	2,559 (22.6)	2,501 (22.7)	1,699 (22.4)	802 (23.3)	58 (19.3)	25 (16.7)	33 (22.0)
Geographic location, n (%)
Northeast	1,595 (14.1)	1,570 (14.2)	1,107 (14.6)	463 (13.5)	25 (8.3)	12 (8.0)	13 (8.7)	0.023	0.090
Midwest	3,047 (26.9)	2,979 (27.0)	2,064 (27.2)	915 (26.6)	68 (22.7)	35 (23.3)	33 (22.0)	0.291	0.210
South	4,455 (39.3)	4,321 (39.2)	2,845 (37.5)	1,476 (42.9)	134 (44.7)	62 (41.3)	72 (48.0)	0.336	0.220
West	2,229 (19.7)	2,156 (19.6)	1,572 (20.7)	584 (17.0)	73 (24.3)	41 (27.3)	32 (21.3)	0.048	0.167

*Relapse according to operational claims definition.

†According to the operational claims definition; sample was, by definition, selected with half designated ‘relapse’ and half with ‘no relapse’.

‡Comparison of patients who had their charts reviewed and those who did not.

Table 2.  MS-related medication in the follow-up period (claims).

	Total MS population (N = 11,326)	Medical record not reviewed			Medical record reviewed			p-value
		Total (N = 11,026)	No relapse* (N = 7,588)	Relapse* (N = 3,438)	Total (N = 300)	No relapse† (N = 150)	Relapse† (N = 150)	No relapse‡	Relapse‡
Corticosteroids, n (%)
ACTH	20 (0.2)	16 (0.2)	1 (0.0)	15 (0.4)	4 (1.3)	0 (0.0)	4 (2.7)	0.888	<0.001
Dexamethasone	538 (4.8)	520 (4.7)	144 (1.9)	376 (10.9)	18 (6.0)	2 (1.3)	16 (10.7)	0.615	0.917
Methylprednisolone	2,828 (25.0)	2,714 (24.6)	530 (7.0)	2,184 (63.5)	114 (38.0)	6 (4.0)	108 (72.0)	0.154	0.034
Prednisolone	7 (0.1)	7 (0.1)	3 (0.0)	4 (0.1)	0 (0.0)	0 (0.0)	0 (0.0)	0.808	0.676
Prednisone	2,245 (19.8)	2,182 (19.8)	488 (6.4)	1,694 (49.3)	63 (21.0)	5 (3.3)	58 (38.7)	0.124	0.011
No corticosteroid	7,026 (62.0)	6,878 (62.4)	6,559 (86.4)	319 (9.3)	148 (49.3)	137 (91.3)	11 (7.3)	0.082	0.420
Disease modifying therapy, n (%)
Glatiramer acetate	2,881 (25.4)	2,813 (25.5)	1,894 (25.0)	919 (26.7)	68 (22.7)	28 (18.7)	40 (26.7)	0.077	0.986
IFN-β1a/β1b	5,675 (50.1)	5,499 (49.9)	3,585 (47.3)	1,914 (55.7)	176 (58.7)	87 (58.0)	89 (59.3)	0.009	0.377
Natalizumab	75 (0.7)	75 (0.7)	21 (0.3)	54 (1.6)	0 (0.0)	0 (0.0)	0 (0.0)	0.519	0.122
No disease-  modifying  therapy$	3,128 (27.6)	3,057 (27.7)	2,266 (29.9)	791 (23.0)	71 (23.7)	40 (26.7)	31 (20.7)	0.397	0.504

*According to operational claims definition.

†According to operational claims definition; sample was, by definition, selected with half designated ‘relapse’ and half with ‘no relapse.’

‡Comparison of patients who had their charts reviewed and those who did not.

$Includes untreated patients.

ACTH, adrenocorticotropic hormone; IFN, interferon.

Table 3.  Positive and negative predictive value of the operational claims-based definition of relapse.

	Relapse based on medical record
Claims-based	No relapse (N = 154)		Relapse (N = 146)		Chi-square	Kappa	Kappa
relapse definition*	n	%	n	%	p-value		p-value
Relapse	49	32.7	101	67.3	<0.001	0.373	<0.001
No relapse	105	70.0	45	30.0

*Multiple sclerosis (MS) inpatient hospitalization at any time during follow-up or MS outpatient visit with a corticosteroid within 7 days.

Table 4.  Positive and negative predictive value of alternative definitions of relapse.

Claims-based relapse definition		Relapse based on medical record
		No relapse (N = 154)		Relapse (N = 146)
		n	%	n	%	Chi-square p-value	Kappa	Kappa p-value
1	Relapse	44	31.9	94	68.1	<0.001	0.359	<0.001
	No relapse	110	67.9	52	32.1
2	Relapse	8	32.0	17	68.0	0.043	0.066	0.022
	No relapse	146	53.1	129	46.9
3	Relapse	47	33.3	94	66.7	<0.001	0.339	<0.001
	No relapse	107	67.3	52	32.7
4	Relapse	47	33.3	94	66.7	<0.001	0.339	<0.001
	No relapse	107	67.3	52	32.7
5	Relapse	53	34.9	99	65.1	<0.001	0.334	<0.001
	No relapse	101	68.2	47	31.8
6	Relapse	58	35.6	105	64.4	<0.001	0.342	<0.001
	No relapse	96	70.1	41	29.9
7	Relapse	78	39.6	119	60.4	<0.001	0.306	<0.001
	No relapse	76	73.8	27	26.2
8	Relapse	63	39.9	95	60.1	<0.001	0.241	<0.001
	No relapse	91	64.1	51	35.9
9	Relapse	69	39.9	104	60.1	<0.001	0.263	<0.001
	No relapse	85	66.9	42	33.1
10	Relapse	74	40.9	107	59.1	<0.001	0.251	<0.001
	No relapse	80	67.2	39	32.8
11	Relapse	88	41.1	126	58.9	<0.001	0.288	<0.001
	No relapse	66	76.7	20	23.3
12	Relapse	91	41.4	129	58.6	<0.001	0.289	<0.001
	No relapse	63	78.8	17	21.3
13	Relapse	111	45.3	134	54.7	<0.001	0.194	<0.001
	No relapse	43	78.2	12	21.8

Claims-based relapse identification algorithms: 1. Multiple sclerosis (MS) outpatient visit with a corticosteroid within 7 days; 2. MS inpatient hospitalization at any time during follow-up; 3. Methylprednisolone or prednisone at any time during follow-up; 4. MS inpatient hospitalization at any time during follow-up or MS outpatient visit at any time during follow-up and use of methylprednisolone or prednisone at any time during follow-up; 5. Use of corticosteroid at any time during follow-up and an MS outpatient visit at any time during follow-up; 6. MS inpatient hospitalization at any time during follow-up or use of corticosteroid at any time during follow-up and an MS outpatient visit at any time during follow-up; 7. MS inpatient hospitalization at any time during follow-up or MS outpatient visit with a corticosteroid within 7 days or not adherent to at least one of the disease-modifying therapies; 8. Lumbar puncture, magnetic resonance imaging (MRI) of brain or spinal column, or evoked potential testing on the same date as an MS outpatient visit; 9. Lumbar puncture, MRI of brain or spinal column, or evoked potential testing ±7 days of an MS outpatient visit; 10. Lumbar puncture, MRI of brain or spinal column, or evoked potential testing at any time during follow-up; 11. Use of corticosteroid within 7 days of an MS outpatient visit or lumbar puncture, MRI of brain or spinal column, or evoked potential testing ±7 days of an MS outpatient visit; 12. MS inpatient hospitalization at any time during follow-up or MS outpatient visit with a corticosteroid within 7 days or lumbar puncture, MRI of brain or spinal column, or evoked potential testing ±7 days of an MS outpatient visit; 13. MS inpatient hospitalization at any time during follow-up or MS outpatient visit with a corticosteroid within 7 days or lumbar puncture, MRI of brain or spinal column, or evoked potential testing ±7 days of an MS outpatient visit or not adherent to at least one of the disease-modifying therapies.