Home-Time as a Surrogate Measure for Functional Outcome After Stroke: A Validation Study

Abstract

Purpose

Home-time has been found to correlate well with modified Rankin Scale (mRS) scores in patients with stroke. This study aimed to determine its correlations in patients with different types of stroke at various time points after stroke in a non-Western population.

Methods

This study used linked data from multi-center stroke registry databases and a nationwide claims database of health insurance. Functional outcomes as measured with the modified Rankin Scale were obtained from the registry databases and home-time was derived from the claims database. Spearman correlation coefficients were used to assess the correlation between home-time and mRS scores. The area under the receiver operating characteristic curve (AUC) was used to evaluate the performance of home-time in predicting good functional outcome.

Results

This study included 7959 patients hospitalized for stroke or transient ischemic attack (TIA), for whom mRS scores were available in 6809, 6694, and 4330 patients at 90, 180, and 365 days, respectively. Home-time was highly correlated with mRS scores at the three time-points in patients with ischemic (Spearman’s rho −0.69 to −0.83) or hemorrhagic (Spearman’s rho −0.86 to −0.88) stroke, but the correlation was only weak to moderate in those with TIA (Spearman’s rho −0.32 to −0.58). Home-time predicted good functional outcome with excellent discrimination in patients with ischemic (AUCs >0.8) or hemorrhagic (AUCs >0.9) stroke but less so in those with TIA (AUCs >0.7).

Conclusion

Home-time was highly correlated with mRS scores and showed excellent discrimination in predicting good functional outcome in patients with ischemic or hemorrhagic stroke. Home-time could serve as a valid surrogate measure for functional outcome after stroke.

Keywords:

• home-time
• stroke
• disability

Introduction

Despite the decrease in age-standardized stroke mortality, the global burden of stroke, as measured in terms of the loss of disability-adjusted life years, keeps increasing.¹ In most countries, stroke is among the leading causes of acquired adult disability.¹ Consequently, functional recovery should be taken into account in the evaluation of stroke therapies. Although administrative claims data, derived from claims for healthcare services, are commonly used in epidemiological research and outcome studies of stroke patients,^2,3 variables regarding post-stroke functional outcomes, such as the modified Rankin Scale (mRS) score, are often lacking.⁴ Therefore, post-stroke mortality or readmission is used as the study endpoint in most claims-based outcome studies. However, such events are far from an ideal proxy for functional outcomes.

Post-stroke home-time, defined as the number of days alive and spent outside an acute care setting after the index acute stroke, is a highly valued patient-centered outcome.^5,6 Recently, home-time has been found to correlate well with mRS scores in patients after ischemic stroke or other types of stroke.^7–9 Besides, it can be easily retrieved from administrative claims databases and may potentially save the costs and efforts in obtaining mRS scores for large-scale registry-based studies. Nevertheless, the correlation between home-time and mRS scores has rarely been examined in non-Western countries.

A previous study found that home-time in stroke survivors varied across hospitals in different locations (urban/rural) and geographic regions under the Medicare system in the US.¹⁰ In addition, home-time after stroke is largely determined by the availability of post-discharge care and support systems, which definitely vary in different healthcare systems and countries.⁶ Therefore, the purpose of this study was to examine the correlation between home-time and mRS scores in Taiwanese stroke patients using data linkage between a population-based administrative claims database and hospital-based stroke registries.

Methods

Data Source and Linkage

Three data sources were linked for this study: the National Health Insurance Database (NHID), Cause of Death Registry and multi-center stroke registry databases. The NHID and Cause of Death Registry are provided by the Ministry of Health and Welfare and maintained by the Health and Welfare Data Science Center (HWDC) in Taipei City, Taiwan.¹¹ The NHID from 2001 to 2015 and the Cause of Death Registry were used in this study. The NHID is derived from the National Health Insurance program, which covers virtually all of Taiwan’s population. It includes registry for beneficiaries, ambulatory care claims, inpatient claims, and prescriptions dispensed at pharmacies. Each claim contains International Classification of Diseases, Ninth Edition, Clinical Modification (ICD-9-CM) diagnosis and procedure codes, expenditures per visit or admission, and details of drug prescriptions. The Cause of Death Registry contains a national identification number, date of birth, sex, date of death, and cause of death, which is recorded as ICD-9-CM codes up to the end of 2007 and as ICD-10-CM codes thereafter. For details of NHID and Cause of Death Registry, refer to Hsieh et al.²

The Taiwan Stroke Registry (TSR), launched in August 2006, is the first nationwide stroke registry to collect real-world data on the quality of stroke care and provide evidence for healthcare policymaking.^4,12 The ongoing TSR enrolls patients who are hospitalized for acute stroke or transient ischemic attack (TIA) within 10 days of symptom onset. More than 100,000 stroke events have been registered in the TSR up to 2015.¹³ The functional status of each patient is measured using the mRS at 90 days, 180 days, and 365 days after stroke by in-person assessment or telephone interview. The multi-center stroke registry databases in this study were obtained from four TSR participating hospitals, including National Cheng Kung University Hospital, Chi Mei Medical Center, Ditmanson Medical Foundation Chiayi Christian Hospital, and Landseed Hospital.

The databases were linked by the national identification number in the HWDC to ensure security of personal information and the researchers could only access the linked databases in the HWDC. Because only aggregate data without any individual-level information could be released from the HWDC, informed consents from individual patients were not required. The research protocol was approved by the Institutional Review Board of National Cheng Kung University Hospital (IRB No. A-EX-106-053).

Study Population

Patients hospitalized for acute stroke between 2006 and 2010 were identified from the stroke registry databases (Figure 1). After data linkage with the NHID, claims records of successfully linked patients were retrieved. Patients without data in the NHID were eliminated. Also excluded were those with a principal diagnosis other than ischemic stroke (ICD-9-CM 433 or 434), hemorrhagic stroke (ICD-9-CM 430, 431, or 432), and TIA (ICD-9-CM 435).

Figure 1 The assembly of the study population.

Abbreviations: mRS, modified Rankin Scale; NHID, National Health Insurance Research Database; TIA, transient ischemic attack.

Variables

All the diagnosis codes (Supplemental Table 1) from the stroke hospitalization, as well as ambulatory and inpatient claims within the one-year look-back period prior to the stroke hospitalization were used to ascertain comorbidities.¹⁴ The main outcome of interest was the follow-up mRS scores at 90 days, 180 days, and 365 days, which were obtained from the stroke registry databases. The mRS is a global measure of functional status. It is frequently used to assess functional outcome after stroke. The scale ranges from 0 to 6: 0 indicates no symptoms, 1–5 represent escalating degrees of disability and functional dependency, and 6 means death.¹⁵ The mRS scores were dichotomized into good (mRS 0–2) and poor (mRS 3–6) outcomes.¹⁶

The main independent variable was home-time evaluated at three time points, ie, 90 days, 180 days, and 365 days. Home-time was defined as the total number of days alive and out of acute care hospitals from the admission date of the index stroke hospitalization to the time point of evaluation or death, whichever came first (Figure 2).

Figure 2 The scheme illustrating the calculation of home-time (days).

Statistical Analysis

Baseline characteristics were described with mean and standard deviations for continuous variables and counts and proportions for categorical variables. Stroke severity was assessed with the National Institutes of Health Stroke Scale (NIHSS). The distribution of home-time and mRS scores at the three time points was represented by box-and-whisker plots. The correlation of home-time with mRS scores was assessed by Spearman’s rank correlation analysis. Logistic regression analysis was used to model the association between home-time and good functional outcome. Model discrimination was assessed using the area under the receiver operating characteristic curve (AUC). The optimal cut-off value for home-time to predict good functional outcome was determined by the Youden index. In subgroup analyses, the study population was divided according to the types of stroke, ie, ischemic stroke, hemorrhagic stroke, and TIA and similar analysis procedures were repeated.

To further elaborate on the analogy between home-time and mRS scores, logistic regression analysis, using the baseline characteristics as independent variables, was performed to identify the predictors of home-time and mRS scores. In the analysis, home-time was dichotomized according to the optimal cut-off value mentioned above. All statistical analysis was performed using SAS version 9.4 (SAS Institute, Cary, NC).

Results

A total of 7959 patients with acute stroke or TIA were included in this study. Among them, 1150 (14.4%), 1265 (15.9%), and 3629 (45.6%) patients were lost to follow-up at 90, 180, and 365 days, respectively. Therefore, mRS scores at 90, 180, and 365 days were available for 6809, 6694 and 4330 patients, respectively. Table 1 lists the baseline characteristics of the study population. The mean age was 71.8 years and 43.7% were female. Approximately three-fourths of patients were hospitalized for ischemic stroke. Supplemental Table 2 gives the comparisons of baseline characteristics between participants and those lost to follow-up. In general, patients lost to follow-up were younger, more likely to be male, and differed in the distribution of stroke types than participants. The prevalence of comorbidities and stroke severity were essentially similar between groups at 90 and 180 days. However, patients lost to follow-up at 365 days had milder stroke severity and less comorbidities including diabetes, ischemic heart disease, atrial fibrillation, heart failure, chronic obstructive pulmonary disease, and chronic kidney disease.

Table 1 Characteristics of the Study Population

	All, N = 7959	Patients with 90-Day mRS, N = 6809	Patients with 180-Day mRS, N = 6694	Patients with 365-Day mRS, N = 4330
Age, mean (SD)	71.8 (12)	72.3 (11.2)	72.3 (11.2)	73.2 (11.1)
Female	3475 (43.7)	3006 (44.1)	2959 (44.2)	1893 (43.7)
Stroke type
Ischemic stroke	6025 (75.7)	5203 (76.4)	5108 (76.3)	3208 (74.1)
Hemorrhagic stroke	1480 (18.6)	1216 (17.9)	1201 (17.9)	865 (20.0)
TIA	454 (5.7)	390 (5.7)	385 (5.8)	257 (5.9)
Comorbidity
Hypertension	6419 (80.7)	5518 (81.0)	5419 (81.0)	3508 (81.0)
Diabetes mellitus	3256 (40.9)	2798 (41.1)	2748 (41.1)	1827 (42.2)
Hyperlipidemia	2862 (36)	2465 (36.2)	2409 (36.0)	1520 (35.1)
Ischemic heart disease	1785 (22.4)	1550 (22.8)	1527 (22.8)	1022 (23.6)
Atrial fibrillation	991 (12.5)	855 (12.6)	844 (12.6)	571 (13.2)
Heart failure	837 (10.5)	719 (10.6)	713 (10.7)	505 (11.7)
COPD	802 (10.1)	696 (10.2)	682 (10.2)	475 (11.0)
Chronic kidney disease	378 (4.7)	315 (4.6)	317 (4.7)	232 (5.4)
Liver diseases	701 (8.8)	601 (8.8)	590 (8.8)	389 (9.0)
Cancer	1012 (12.7)	848 (12.5)	844 (12.6)	547 (12.6)
Depression	342 (4.3)	290 (4.3)	288 (4.3)	181 (4.2)
NIHSS, median (IQR)	5 (2–14)	5 (2–14)	5 (2–15)	6 (3–19)

Note: Data are numbers (percentage) unless specified otherwise.

Abbreviations: COPD, chronic obstructive pulmonary disease; IQR, interquartile range; mRS, modified Rankin Scale; NIHSS, National Institutes of Health Stroke Scale; SD, standard deviation; TIA, transient ischemic attack.

Figure 3 and Supplemental Table 3 show the distribution of home-time across mRS scores at 90 days, 180 days, and 365 days, respectively, in patients with different types of stroke. Regardless of stroke types and time points of evaluation, home-time was inversely correlated with mRS scores (Table 2). In patients with ischemic stroke, home-time had a strong to very strong correlation with mRS scores with Spearman correlation coefficients ranging from −0.69 at 90 days to −0.83 at 365 days. In patients with hemorrhagic stroke, home-time had a very strong correlation with mRS scores (Spearman’s coefficients from −0.86 to −0.88). However, home-time was only weakly to moderately correlated with mRS scores in patients with TIA (Spearman’s coefficients from −0.32 to −0.58).

Figure 3 Box-and-whisker plots showing the distribution of home-times across mRS scores at 90, 180, and 365 days in all patients and subgroups of patients.

Abbreviations: mRS, modified Rankin Scale; TIA, transient ischemic attack.

Table 2 The Correlation Coefficients Between Home-Time and the mRS Score by Spearman’s Rank Correlation Analysis

	All	Ischemic Stroke	Hemorrhagic Stroke	TIA
	Spearman’s Rho (95% CI)	Spearman’s Rho (95% CI)	Spearman’s Rho (95% CI)	Spearman’s Rho (95% CI)
90 days	−0.74 (−0.75 to −0.73)	−0.69 (−0.71 to −0.68)	−0.86 (−0.87 to −0.84)	−0.32 (−0.41 to −0.23)
180 days	−0.76 (−0.77 to −0.75)	−0.71 (−0.73 to −0.70)	−0.88 (−0.89 to −0.86)	−0.42 (−0.50 to −0.33)
365 days	−0.85 (−0.86 to −0.84)	−0.83 (−0.84 to −0.81)	−0.88 (−0.90 to −0.87)	−0.58 (−0.66 to −0.50)

Abbreviations: CI, confidence interval; mRS, modified Rankin Scale; TIA, transient ischemic attack.

Table 3 lists the AUCs and optimal cut-off values of home-time in predicting good functional outcome. Among stroke types, the AUCs were highest for hemorrhagic stroke with values of >0.9. Home-time also had excellent discrimination with AUCs >0.8 for ischemic stroke. Finally, home-time achieved an acceptable discrimination ability (AUCs >0.7) in predicting good functional outcome in patients with TIA. In all stroke patients, the optimal cut-off values of home-time to differentiate good from bad functional outcome were 79 days, 166 days, and 345 days at 90 days, 180 days, and 365 days, respectively. The optimal cut-off values of home-time in patients with hemorrhage stroke were lower than those in patients with ischemic stroke or TIA.

Table 3 Area Under Curve and Optimal Cut-off Values for Home-Time to Predict Good Functional Outcome

	All				Ischemic Stroke				Hemorrhagic Stroke				TIA
	AUC (95% CI)	Cut-off (days)	Sen (95% CI)	Spe (95% CI)	AUC (95% CI)	Cut-off (days)	Sen (95% CI)	Spe (95% CI)	AUC (95% CI)	Cut-off (days)	Sen (95% CI)	Spe (95% CI)	AUC (95% CI)	Cut-off (days)	Sen (95% CI)	Spe (95% CI)
90 days	0.874 (0.865–0.882)	79	0.750 (0.740–0.760)	0.845 (0.836–0.854)	0.853 (0.843–0.863)	80	0.736 (0.725–0.747)	0.824 (0.815–0.833)	0.917 (0.902–0.932)	65	0.750 (0.740–0.760)	0.928 (0.922–0.934)	0.784 (0.710–0.858)	79	0.542 (0.530–0.554)	0.934 (0.928–0.94)
180 days	0.886 (0.878–0.894)	166	0.749 (0.738–0.760)	0.867 (0.859–0.875)	0.868 (0.858–0.878)	169	0.771 (0.761–0.781)	0.810 (0.800–0.820)	0.924 (0.910–0.939)	156	0.812 (0.802–0.822)	0.885 (0.877–0.893)	0.833 (0.770–0.897)	170	0.698 (0.687–0.709)	0.888 (0.880–0.896)
365 days	0.917 (0.909–0.925)	345	0.771 (0.758–0.784)	0.910 (0.901–0.919)	0.902 (0.892–0.913)	345	0.729 (0.715–0.743)	0.919 (0.911–0.927)	0.953 (0.940–0.966)	338	0.867 (0.857–0.877)	0.921 (0.913–0.929)	0.884 (0.828–0.941)	354	0.790 (0.778–0.802)	0.851 (0.840–0.862)

Abbreviations: AUC, area under the receiver operating characteristic curves; CI, confidence interval; Sen, sensitivity; Spe, specificity; TIA, transient ischemic attack.

Supplemental Table 4 lists the odds ratios and confidence intervals of the baseline characteristics in predicting functional outcome at 90 days represented by home-time or mRS score in all study patients. Among the baseline characteristics, older age, a higher NIHSS score, diabetes, and chronic kidney disease were predictors of poor outcome when outcome was represented by either home-time or mRS, whereas hyperlipidemia protected against poor outcome.

Discussion

The main finding of this study was that home-time was highly associated with the degree of global disability as determined by the mRS. Home-time assessed at 90, 180, and 365 days showed excellent discrimination with AUCs >0.8 in predicting good functional outcome. In particular, home-time for patients with hemorrhage stroke exhibited outstanding discrimination with AUCs >0.9 in predicting good functional outcome at the three time points. These results were in line with previous studies in other countries,^7,8 and suggested that home-time can serve as a surrogate measure of functional outcome after stroke for studies using Taiwan’s healthcare administrative databases.

As seen in a previous study,⁸ patients with hemorrhagic stroke had lower home-time and patients with TIA had higher home-time than those with ischemic stroke. It probably reflected the different functional prognosis among stroke types in that hemorrhagic stroke is the most severe type of stroke followed by ischemic stroke and TIA. Of note, the AUCs of home-time also differed among stroke types, with the highest value in hemorrhagic stroke and the lowest in TIA (Table 3). Therefore, the application of home-time in future stroke research should take into account the variation in home-time across stroke types. We speculate that the ability of home-time to discriminate good from bad functional outcome after stroke may depend on the distribution of functional prognosis in each type of stroke. For example, because patients with TIA generally do not have functional disability after the event, home-time is more likely affected by factors other than functional disability. Therefore, home-time was less correlated with mRS scores and had a lower discriminatory ability in predicting good functional outcome in patients with TIA.

The mRS has been the preferred outcome measure for most of the acute stroke trials in the past decades.¹⁷ Typically, mRS rating is assessed in an unstructured patient interview by clinicians.¹⁸ Although the mRS ranges from 0 to 6, it is generally dichotomized in stroke trials based on the initial severity of stroke.¹⁹ This study did not intend to propose a new outcome measure for stroke trials by demonstrating the close correlation between home-time and the mRS score. Instead, it aimed to establish a valid proxy for functional outcome in future claims-based stroke research, in particular in those based on the NHID. Home-time can be treated as a continuous outcome variable on the one hand; on the other, it can be dichotomized to represent either good (mRS 0–2) or poor outcome (mRS 3–6) based on the optimal cut-off values. Therefore, researchers can use home-time as a proxy to evaluate the effect of a specific treatment on functional outcome in claims-based stroke studies, rather than simply using mortality as an endpoint.²⁰

Moreover, in large-scale registry-based studies where long-term follow-up assessment of mRS is costly and laborious, or even infeasible, home-time can be treated as an alternative and reliable patient-centered measure to assess stroke outcome. Record linkage between clinical registry databases and administrative claims databases may be an effective strategy to complement the advantages of each type of databases.⁴ The combination of clinical data from registry databases and long-term outcome data from claims databases, such as home-time, can provide opportunities to investigate early predictors of functional recovery following stroke, such as blood pressure variability²¹ and C reactive protein.²² The linked databases can also be used to validate various stroke outcome prediction scores.²³

In addition to clinical investigations, home-time may potentially be used in other applications such as quality measurement and policymaking in healthcare considering that administrative claims databases have been commonly used in these fields.^24–26 For example, healthcare providers and consumers can compare the quality of stroke care across hospitals using home-time as a proxy indicator. Policymakers can also refine resource allocation decisions based on the population-level trend of home-time after stroke. Being a composite measure of staying alive and out of acute care hospitals, home-time may be a preferred outcome variable over readmission, which is theoretically affected by the competing risk of death. In the past, stroke research based on administrative claims data has been limited by the lack of adequate clinical information.²⁷ In the future, researchers may have a chance to mitigate this shortcoming when conducting stroke outcome studies using the NHID through the use of home-time as a functional endpoint, combined with the use of a claims-based proxy for stroke severity.^28,29

This study has the following limitations. First, home-time was defined as the time spent outside the acute care setting. We were unable to differentiate whether patients lived in their own home or in a skilled nursing facility, nursing home, or other chronic care facility. Second, the sample size of TIA patients was relatively small. The robustness of the study results for TIA patients is open to question and needs further examination with more samples. Third, the study results may only apply to claims-based research using the NHID as the data source. Further validation is needed before they can be used in studies based on administrative claims databases from other countries.

Conclusions

This validation study showed that home-time was closely correlated with the mRS score and predicted good functional outcome with excellent discrimination at 90, 180, and 365 days after stroke. Because it can be readily and reliably assessed from claims data, home-time may serve as a valid proxy for functional outcome after stroke in studies using administrative claims databases.

Acknowledgments

This research was supported in part by the Ministry of Science and Technology, Taiwan (grant number MOST 107-2320-B-006-035). We are grateful to the Health Data Science Center, National Cheng Kung University Hospital for providing administrative and technical support. We would like to thank Ms. Li-Ying Sung for English language editing.

Disclosure

The authors declare they have no conflicts of interest with respect to this research study.