A phenotyping algorithm to identify acute ischemic stroke accurately from a national biobank: the Million Veteran Program

Figures & data

Figure 1 Stroke-classification flowchart for chart reviews.

Abbreviations: EHR, electronic health record; P(stroke), probability of stroke.

Figure 2 Development of structured acute ischemic stroke algorithm.

Abbreviations: CMS, Centers for Medicare and Medicaid Services; CVA, cerebrovascular accident; NDI, National Death Index; TIA, transient ischemic attack; VA, Veterans Affairs.

Figure 3 Predicted probabilities of stroke based on charts reviewed.

Notes: Thresholds optimized for largest n (excluding P_control<P<P_case) with Cohen’s k>0.9 between algorithm labels and review labels.
Abbreviation: AIS, acute ischemic stroke.

Table 1 List of predictors and variable importance in the acute ischemic stroke algorithm

Predictors	Description	β (log odds)
Intercept	Baseline risk (without predictors)	−3.288
Tirschwell and Longstreth classifier	AIS if 433.x1, 434, or 436*	2.286
Tirschwell and Longstreth ICD9	436	1.496
	Log (number of 434.91)	1
	433.x1	0
Other AIS ICD9 codes	Log (number of 433, 434, 436, 437.0, and 437.6)	0.586
ICD9, ICD10 codes	All CVD-related,# excluding above	0+
CPT codes$	MRI or CT brain/neck angiography	0+
CMS codes‡	All CVD and stroke-related	0+
Medication	All stroke-related	0+
Vitals	Systolic blood pressure	0+
Cause of death	Stroke, CVD-related, other	0+

Notes:

* Excluding 434.x0, 430 (subarachnoid hemorrhage), and 431 (intracerebral hemorrhage). Counts of 433.x1 were unimportant after inclusion of the Tirschwell and Longstreth⁷ classifier.

+ Not included in the optimal predictor set or estimated 0 by adaptive LASSO.

# Diabetes (250), CHD-related (410–415, 427), cerebrovascular disease (430-438), hypotension (458), syncope (780.2), TIA Hx (V12.54), fall Hx (V15.88), aortocoronary bypass (V45.81), coronary angioplasty (V45.82).

$ Current procedural terminology (CPT) codes are used in the VHA for reporting medical services and procedures.

‡ Procedure codes used by the Centers for Medicare and Medicaid Services (CMS).

Abbreviations: AIS, acute ischemic stroke; CVD, cardiovascular disease; MRI, magnetic resonance imaging; CT, computed tomography; LASSO, least absolute shrinkage and selection operator.

Table 2 Classification performance in the validation set (n=134)

Algorithm	Stroke*	No stroke	Sensitivity	Specificity	PPV	AUC
Tirschwell and Longstreth#			0.889 (0.81–0.96)§	0.83 (0.76–0.90)	0.727 (0.62–0.84)
Longitudinal cohort	P‡>0.5	P<0.5	0.844 (0.75–0.94)	0.875 (0.82–0.93)	0.776 (0.68–0.87)	0.926 (0.89–0.96)
Case–control	P≥0.75	P≤0.15	0.906 (0.81–0.97)	0.946 (0.90–0.99)	0.879 (0.78–0.97)	0.948 (0.90–0.98)

Notes:

* Decision rule for classifying acute ischemic stroke;

# from Tirschwell and Longstreth;⁷

‡ predicted from classification model;

§ performance measure (bootstrapped 95% CI).

Abbreviations: PPV, positive predictive value; AUC, area under the curve.

Table 3 Baseline characteristics of populations with predicted acute ischemic stroke (strict algorithm)

	Chart reviews	Million Veteran Program	CVD-risk cohort
Predicted strokes (n)/population size (n)	60/199	3,423/323,122	80,508/2,114,458
Age (years), mean ± SD	59.0±11.4	56.4±9.7	64.8±11
<30 years	0	25 (0.7%)	147 (0.2%)
30–49.99 years	5 (9.8%)	694 (20.5%)	5,741 (8.2%)
50–59.99 years	23 (45.1%)	1,570 (46.5%)	19,305 (27.4%)
60–69.99 years	7 (13.7%)	776 (23%)	19,453 (27.7%)
>70 years	16 (31.4%)	313 (9.3%)	25,687 (36.5%)
Men	51 (100%)	3,241 (97.5%)	67,583 (98.1%)
Race
White	30 (63.8%)	2,433 (75.4%)	53,902 (81.3%)
American Indian/Alaska native	0	17 (0.5%)	370 (0.6%)
Asian	0	21 (0.7%)	444 (0.7%)
Black/African-American	17 (36.2%)	735 (22.8%)	10,785 (16.3%)
Native Hawaiian or other Pacific Islander	0	21 (0.7%)	824 (1.2%)
Cardiovascular risk factors
Smoking, current or past	38 (90.5%)	1,329 (41.4%)	66,227 (82.3%)
Body-mass index (kg/m^2)	28.8±6.4	30±5.5	28.8±5.4
Hypertension (%)	49 (81.7%)	2,667 (77.9%)	61,084 (75.9%)
SBP (mmHg)	143±27.2	138.9±23	139.8±22.8
DBP (mmHg)	79.7±15.1	79.6±13.6	76.1±13
Hyperlipidemia (%)	31 (51.7%)	2,391 (69.9%)	48,837 (60.7%)
Total cholesterol (mg/dL)	168.8±44.2	170.7±42.8	173.9±43.4
HDL cholesterol (mg/dL)	43±14.8	41.7±12.2	42.4±12.9
LDL cholesterol (mg/dL)	98.8±33.9	99.4±37.4	101.4±36
Triglycerides (mg/dL)	146.2±131.5	160.1±129.2	157.9±124.2
Diabetes mellitus (%)	25 (41.7%)	1,233 (36%)	27,628 (34.3%)
HbA1c (mmol/mol)	6.8±1.7	6.7±1.6	6.8±1.5
eGFR (mL/min/1.73 m^2)	64.9±27	74.4±20.5	67.8±20.7
Medication
*Aspirin	42 (70%)	1,289 (37.7%)	26,035 (32.3%)
Clopidogrel	28 (46.7%)	843 (24.6%)	20,230 (25.1%)
tPA: alteplase or reteplase	2 (3.3%)	6 (0.2%)	75 (0.1%)
Warfarin	12 (20%)	558 (16.3%)	12,186 (15.1%)
Statins	42 (70%)	2,458 (71.8%)	50,712 (63%)
β-Blockers	33 (55%)	1,546 (45.2%)	35,017 (43.5%)
ACE inhibitors/ARBs	34 (56.7%)	1,600 (46.7%)	37,514 (46.6%)
Comorbid conditions
Atrial fibrillation	8 (13.3%)	406 (11.9%)	10,336 (12.8%)
COPD	15 (25%)	420 (12.3%)	12,599 (15.6%)
Coronary heart disease	30 (50%)	1,101 (32.2%)	29,604 (36.8%)
Peripheral vascular disease	10 (16.7%)	263 (7.7%)	7,146 (8.9%)
Congestive heart failure	15 (25%)	224 (6.5%)	6,620 (8.2%)
Chronic kidney disease	7 (11.7%)	299 (8.7%)	6,164 (7.7%)
Chronic liver disease	0	51 (1.5%)	678 (0.8%)
Deep-vein thrombosis	0	19 (0.6%)	209 (0.3%)
Pulmonary embolism	1 (1.7%)	31 (0.9%)	536 (0.7%)

Notes: Ages computed at a baseline year of 2002. Descriptive statistics for continuous variables computed using first lab values within a year following the first stroke event. Dichotomous variables, such as medications, are positive if any records found within a year of first stroke event.

* Aspirin is taken by many patients as an over the counter and/or non-VA medication instead of a prescription, and thus the reported percentage in this table is an underestimation of aspirin use.

Abbreviations: ACE, angiotensin converting enzyme; ARBs, angiotensin-receptor blockers; CVD, cardiovascular disease; DBP, diastolic blood pressure; eGFR, estimated glomerular filtration rate; HDL, high-density lipoprotein; LDL, low-density lipoprotein; SBP, systolic blood pressure; tPA, tissue plasminogen activator.

Figure S1 Database platform for the chart review process.

Note: In addition to what is presented in the above figure, the platform also included information on patient demographics, ICD-9 codes, and data from Centers for Medicare and Medicaid Services and the National Death Index.

Table S1 Process of case selection for review (n=300 charts)

Case selection process*
Selection criteria: Random selection of medical charts from the CVD Merit cohort with ICD-9 codes for acute ischemic stroke (AIS) (433.x, 434.x, 436)
<g>Criteria	Count of charts meeting criteria in the national Veterans Affairs cohort
10 charts with only one inpatient ICD-9	2,710
10 charts with only one outpatient ICD-9	10,809
10 charts with one inpatient ICD-9 AND multiple outpatient ICD-9s	37,199
10 charts with no inpatient ICD-9 AND multiple outpatient ICD-9s	62,773
10 charts with multiple inpatient ICD-9s AND multiple outpatient ICD-9s	25,648

Note:

* This process was replicated for each chart reviewer.

Table S2 Cumulative incidence of acute ischemic stroke (derived from rules based and statistical algorithm) from 2000 to 2015 in the national Veterans Cardiovascular Disease risk cohort

Year	Rules-based algorithm (Tirschwell)			Longitudinal cohort statistical algorithm
	Count	Crude incidence	Incidence per 10,000 persons	Count	Crude incidence	Incidence per 10,000 persons
2000	13515	0.006407864	64.07864486	12359	0.005858918	58.58917605
2001	14792	0.007058559	70.5855855	13620	0.006494761	64.94760559
2002	16717	0.008033852	80.33852039	15599	0.007487083	74.87082754
2003	17637	0.008544632	85.44631736	16659	0.00805617	80.56170256
2004	17408	0.008506371	85.06371472	16594	0.00808991	80.89910428
2005	15232	0.007506932	75.06931788	14423	0.007088852	70.88852223
2006	14039	0.006971307	69.71307352	13318	0.006592482	65.92481858
2007	12189	0.006095149	60.95149133	11536	0.005748278	57.48277659
2008	8710	0.004382174	43.8217386	8164	0.004091562	40.9156198
2009	7817	0.003950198	39.50198293	7372	0.003709813	37.09813292
2010	7262	0.003684291	36.84291433	6839	0.003454407	34.54406781
2011	6876	0.003501359	35.01358839	6509	0.003299119	32.99118934
2012	6798	0.003473803	34.7380314	6367	0.003237828	32.37827508
2013	6295	0.003227982	32.27981653	5755	0.002936112	29.36112139
2014	5554	0.002857231	28.57231048	5183	0.002652073	26.52073355
2015	4571	0.002358269	23.5826911	4312	0.002212261	22.12261119

Table S3 Cumulative incidence of acute ischemic stroke (derived from statistical algorithm) from 2000 to 2015 in the Million Veteran Program

Year	Rules-based algorithm (Tirschwell)			Longitudinal cohort statistical algorithm
	Count	Crude incidence	Incidence per 10,000 persons	Count	Crude incidence	Incidence per 10,000 persons
2000	274	0.000848	8.482394	234	0.000724	7.243977
2001	294	0.000911	9.109274	253	0.000784	7.83784
2002	315	0.000977	9.768835	275	0.000853	8.526074
2003	346	0.001074	10.74071	323	0.001002	10.02281
2004	363	0.001128	11.28054	325	0.001009	10.09499
2005	399	0.001241	12.41328	380	0.001182	11.8153
2006	399	0.001243	12.42871	364	0.001133	11.3312
2007	443	0.001382	13.81646	407	0.001268	12.68415
2008	487	0.001521	15.20977	434	0.001354	13.54278
2009	603	0.001886	18.86131	536	0.001675	16.74833
2010	729	0.002285	22.84557	666	0.002085	20.84533
2011	794	0.002494	24.93954	616	0.001932	19.32064
2012	930	0.002928	29.28433	641	0.002014	20.14368
2013	993	0.003136	31.35994	672	0.002116	21.16049
2014	923	0.002924	29.24097	692	0.002184	21.83647
2015	853	0.00271	27.1026	607	0.00192	19.19616

Table S4 Sensitivity and PPV of code-groups in chart-reviewed VA sample (N=300)

Code Group	ICD-9	Total	AIS	Possible AIS	ICH/SAH	TIA	No Stroke	Sensitivity	PPV
Tirschwell	433.x1, 434, 436	144	85	25	5	2	27	0.934	0.599
AHA/ASA	433.01, 433.11, 433.21, 433.31, 433.81, 433.91, 434.01, 434.11, 434.91	72	50	6	2	1	13	0.549	0.704
ICH	431	8	4	0	4	0	0	0.800	0.571
SAH	430	3	2	0	1	0	0	0.200	0.500

Abbreviations: PPV, positive predictive value; VA, Veterans Affairs; AIS, acute ischemic stroke; ICH, intracranial hemorrhage; SAH, subarachnoid hemorrhage; TIA, transient ischemic attack; AHA/ASA, American Heart Association/American Stroke Association

TirschwellDLLongstrethWTValidating administrative data in stroke researchStroke200233102465247012364739

 PubMed Web of Science ®Google Scholar