https://orcid.org/0000-0003-0889-0151
Abstract
Introduction
Objective markers for asthma, that can be measured without extra patient effort, could mitigate current shortcomings in asthma monitoring. We investigated whether smartphone-recorded nocturnal cough and sleep quality can be utilized for the detection of periods with uncontrolled asthma or meaningful changes in asthma control and for the prediction of asthma attacks.
Methods
We analyzed questionnaire and sensor data of 79 adults with asthma. Data were collected in situ for 29 days by means of a smartphone. Sleep quality and nocturnal cough frequencies were measured every night with the Pittsburgh Sleep Quality Index and by manually annotating coughs from smartphone audio recordings. Primary endpoint was asthma control assessed with a weekly version of the Asthma Control Test. Secondary endpoint was self-reported asthma attacks.
Results
Mixed-effects regression analyses showed that nocturnal cough and sleep quality were statistically significantly associated with asthma control on a between- and within-patient level (p < 0.05). Decision trees indicated that sleep quality was more useful for detecting weeks with uncontrolled asthma (balanced accuracy (BAC) 68% vs 61%; Δ sensitivity −12%; Δ specificity −2%), while nocturnal cough better detected weeks with asthma control deteriorations (BAC 71% vs 56%; Δ sensitivity 3%; Δ specificity −34%). Cut-offs using both markers predicted asthma attacks up to five days ahead with BACs between 70% and 75% (sensitivities 75 - 88% and specificities 57 - 72%).
Conclusion
Nocturnal cough and sleep quality have useful properties as markers for asthma control and seem to have prognostic value for the early detection of asthma attacks. Due to the limited study duration per patient and the pragmatic nature of the study, future research is needed to comprehensively evaluate and externally validate the performance of both biomarkers and their utility for asthma self-management.
Introduction
Asthma is one of the most prevalent chronic respiratory diseases. It affects an estimated 300 million people globally.Citation1 Despite the availability of effective treatment options, suboptimal asthma control is still common.Citation2,Citation3
To prevent potentially life-threatening asthma attacks, patients need to be aware of their actual asthma control and early warning signs of deterioration.Citation2 However, prior research has found that patients’ subjective self-assessments are biased towards an overestimation of asthma control.Citation4 For patients with a longer duration of illness, in particular, subjective assessments are of limited reliability.Citation5 Furthermore, objective methods of asthma monitoring (eg, self-administered asthma control test (ACTCitation6) or peak flow monitoring) are considered impractical by patientsCitation7 and are still poorly adopted in practice.Citation4
Novel objective markers, that can be measured without extra patient effort, could mitigate these shortcomings.Citation8 Nocturnal cough and sleep quality, for example, have been reported to be statistically associated with asthma control in cross-sectional studies.Citation9–Citation11 Research in the domain of mobile sensing demonstrated their measurability by means of a smartphone’s built-in sensors.Citation12,Citation13
We investigated whether nocturnal cough and sleep quality may constitute useful markers for asthma. Expanding the findings of prior research, we first examined whether both markers are associated with asthma control in a longitudinal (ie, within-patient) setting. We then investigated whether both markers can enable the detection of periods with uncontrolled asthma or clinically meaningful deteriorations of asthma control, and prediction of asthma attacks.
Methods
The study protocol of this two-center, longitudinal, observational study has already been publishedCitation14 (Clinicaltrials No: NCT03635710). Here, we report results on the association of nocturnal cough, sleep quality, asthma control, and asthma attacks (2nd study stage). Details about nocturnal cough characteristics (1st study stage) will be published in a separate paper.Citation15
The study has been reviewed and approved by the responsible ethics commission (“Ethikkommission Ostschweiz”, ID: 2017–01872). All patients provided written informed consent. The study was conducted in accordance with the Declaration of Helsinki.
Study Subjects
Adult patients with a history of asthma () were included between February 2018 and June 2019 in two study centers in Switzerland (the Lung Center of the Cantonal Hospital St. Gallen and the mediX Group practice Zurich in cooperation with the University of Zurich).
Figure 1 Study flowchart.
No treatments were administered or prescribed by the study personnel. All patients were treated during the study period according to current medical knowledge and practice by their usual health-care partner.
Study Design
Study duration per patient was 29 days. On d1 and d29, medical assessments of patients were performed at the study centers. In the time between study center appointments, questionnaire and sensor data were collected in situ with a smartphone (Samsung Galaxy A3 2017, SM-A320FL) application based on MobileCoach.Citation16,Citation17
This work has four objectives: (1) to examine the statistical associations of both markers (ie, nocturnal cough and sleep quality) with asthma control, in particular with regard to within-patient associations; to investigate whether they enable detection of weeks, in which (2) patients had uncontrolled asthma or (3) clinically significant deteriorations in asthma control occurred; and (4) to explore whether they can be used to predict asthma attacks in advance.
The study was powered to examine statistical associations between both markers and asthma control.Citation14 Data from 79 patients could be analyzed after accounting for dropouts, so that the targeted sample size was missed by 6 patients (). Since the power analysis was based on conservative assumptions (ie, disregard of longitudinal measurements), the slightly smaller sample size likely sufficed for statistical analysis.
Measures
The study protocolCitation14 contains a description of all measures we have obtained in the four study weeks per patient. Additional information on predictors and endpoints is provided in the online supplement (Supplementary Text S1).
Predictors
A patient’s sleep quality was assessed daily via an adapted version of the Pittsburgh Sleep Quality Index (PSQICitation18). Although devices for assessing sleep quality passively and with a higher temporal resolution are available,Citation19 we decided to use a questionnaire-based assessment to reduce the burden for patients during the study (ie, patients had to fill out questions on the smartphone every morning either way). PSQI score was standardized on a scale from 0 (worst possible sleep quality) to 1 (best possible sleep quality). In 2291 nights, the 79 patients slept rather well with an average sleep quality of 77.5% (SD = 15.2%; cf. Figure s1 in the online supplement for the distribution of average sleep quality scores per patient). Patients have completed the PSQI for 2250 nights (98.2%). For missing values, the patient mean was imputed (cf. Table s1 in the online supplement for the benchmarking of different imputation methods).
Nocturnal cough frequencies of patients were obtained for every night by manually labelling audio recordings from the smartphone’s built-in microphone. On average, patients coughed 9.9 times per night (SD = 27.1) with a median of 1 c/n (IQR 0–8). Figure s2 in the online supplement provides an illustration of the distribution of the average nocturnal cough count per patient. Data were available for 2145 out of 2212 nights (97.0%). Missing values were imputed by means of a Kalman Smoother.
Endpoints
Primary endpoint was asthma control. A weekly version of the ACTCitation6 assessed a patient’s level of asthma control retrospectively for a given week. Thus, four ACT measurements per patient were obtained. We observed statistically significant differences in ACT scores between seasons (Kruskal–Wallis’ H(3) = 8.76, p = 0.03): The lowest (worst) average scores occurred between February and April (M = 18.9, SD = 1.83), while the highest (best) scores were reported between November and January (M = 21.5, SD = 1.29). Please refer to the online supplement for more information on seasonality, also with regard to the predictors (cf. Figures s3 – s5).
For the analysis of statistical associations (ie, objective (1) of the study), the raw ACT score was used. For objective (2), asthma was considered uncontrolled in weeks in which patients had an ACT score below 20 points.Citation20 The target outcome of objective (3) was defined according to the minimally important difference of the ACT: a decrease of >2 points in the ACT was considered a clinically meaningful deterioration in asthma control.Citation21
Secondary endpoint was asthma attack days indicated through a “yes/no” questions for each day. Prior research has used “yes/no” questions for the self-reports of both asthma attacksCitation22 and symptomsCitation23 before. Both endpoints were reported by patients at the end of each study week, ie, on d8, d15, d22, and d29.
Data for the primary and secondary endpoint were available for 308 out of 316 study weeks (97.5%). No imputations were performed with regard to the endpoints.
Analysis
All data analyses were performed in R 3.5.1.Citation24–Citation28 Additional information, including a table summarizing the analysis plan, is available in the online supplement (cf. Tables s2 and s3).
For the analysis of objective (1) to (3), predictors were transformed and aggregated according to the ACT recall week, implying that predictors and outcome stem from the same time frame. For objective (4), days with asthma attacks were predicted with measurements from preceding days. Thus, predictions in a strictly temporal sense were only made in the context of objective (4).
Mixed-effects regressionsCitation27 were employed in the analysis of objective (1).
For the analysis of objective (2) and (3), decision trees based on recursive partitioning analysisCitation29,Citation30 were applied. Detections were internally validated with a 5-fold cross-validation scheme.Citation31
For objective (4), optimal cut-offs of predictor variable values were identified for attack prediction in the train folds and applied to each test fold, which represented an 8-fold cross-validation approach.
For objectives (2) to (4), averaged balanced accuracy (BAC; ie, mean of sensitivity and specificity) across all test folds (consisting of patients previously not seen by the model) was defined as the main evaluation metric. We further report sensitivity (ie, true positives/all positives) and specificity (ie, true negatives/all negatives).Citation32
Data Sharing Statement
It is not intended to share individual participant data.
Results
summarizes the baseline characteristics of the 94 included patients.
Table 1 Baseline Characteristics
Asthma Control and Asthma Attacks
For the analysis of statistical associations, raw ACT scores were considered. The 79 analyzed patients had a median ACT score of 21 points (IQR = 18–23) in the 308 available weeks (Figure s6 in the online supplement depicts the distribution of weekly ACT scores).
As the second objective, weeks with uncontrolled asthma were detected. Asthma was controlled in 192 weeks and uncontrolled in 116 weeks.
As the third objective, weeks with deteriorations in asthma control were detected. Changes in weekly asthma control could be calculated for 227 weeks. Clinically significant deteriorations occurred in 29 weeks in 25 patients.
Finally, the fourth objective was to investigate the prediction of asthma attacks. Twelve patients registered 35 attack days (mean = 2.9 days, SD = 2.5). Eight asthma attacks remained in a total of 2004 study days after keeping only the first attack day per patient and excluding all patients with attacks in the first study week.
Objective 1: Statistical Associations of Nocturnal Cough and Sleep Quality with Asthma Control Using Mixed Effect Regression Modelling
All regression coefficients, regardless of variable centering, were statistically significantly associated with asthma control (p < 0.05; cf. ). Thus, patients, who coughed more/had worse sleep quality, tended to have worse asthma control (between-patient) and increases in nocturnal cough/decreases in sleep quality were statistically associated with decreases in asthma control in the same time and patient (within-patient). Interestingly, we have observed no statistical interaction between nocturnal cough and sleep quality when modelling their associations with asthma control (B = −.14, t(271.34) = −1.06, ns). Additional correlational analyses (Figure s7 for within-patient correlations between nocturnal cough and sleep quality and Figure s8 for correlations between cough based on subsegments of the night and asthma control; Supplementary Text S2) and the complete regression results table (Table s4) are available in the online supplement (Supplementary Text S3).
Table 2 Statistical Associations of Nocturnal Cough and Sleep Quality with Asthma Control Using Mixed Effect Regression Modelling
Objectives 2–4: Cross-Validated Models for the Detection or Prediction of Asthma Endpoints
contains the results of all cross-validated models (additional result metrics can be found in Table s5 of the online supplement; also Supplementary Text S3). We considered cough of the first half hour of bedtime as a separate predictor set, since most coughs occurred in this time frame (a second paper contains a detailed discussion on the prevalence of nocturnal cough including possible explanations for the increased incidence of cough in the beginning of the night).Citation15 We also included coughs of the remaining night (ie, after the first half hour of bedtime) as an additional predictor set. Since multiple variables have been used for growing the decision trees in objective (2) and (3), we facilitate model interpretation by providing estimates for the importance of each variable in the online supplement (Table s6).
Table 3 Cross-Validated Models for the Detection or Prediction of Asthma Endpoints – Results
For the second objective (ie, detection of weeks with uncontrolled asthma), we found that none of the five predictor sets achieved balanced accuracies of 70%. Sleep quality (BAC = 68%), however, was more useful for this objective than nocturnal cough (BAC = 60–62%). Combining predictor sets has not yielded an increase in prediction performance.
For the third objective (ie, detection of weeks with clinically meaningful deteriorations in asthma control), the pattern was inverted: Nocturnal cough (BAC = 71%) outperformed sleep quality (BAC = 56%) by a considerable margin. Interestingly, coughs within the first half hour of bedtime performed substantially worse than the entire night of cough (ΔBAC = −17%), while coughs after the first half hour of bedtime performed only slightly worse than the entire night (ΔBAC = −6%). Furthermore, combining predictors resulted in a decreased monitoring performance in comparison to nocturnal cough of the entire night.
The fourth objective was prediction of asthma attacks. BACs of 70% were achieved for subsegments of nocturnal cough three and four days ahead of the attack. Sleep quality was close to a BAC of 70% three to four days ahead of the attack. Combining predictor sets increased prognostic performance, regardless whether cough of the entire night (BACs of ≥70% for d0 – d−4; cf. ) or cough within (BACs of ≥70% for d−3 – d−5; cf. ) or after (BACs of ≥70% for d0 and d−2 – d−4; cf. ) the first half hour of bedtime was considered.
Figure 2 Asthma attack prediction results of cut-offs based on the entire night (A), cough of first 30 minutes of bedtime (B), and cough after 30 minutes of bedtime (C). No subsegments of the night could be extracted for sleep quality. Trend lines were estimated through local polynomial regression. Black lines at 0.50 indicate the balanced accuracy of a model that always predicts the same class.
Discussion
In this study, smartphone-recorded nocturnal cough and sleep quality were systematically investigated as markers for asthma control and asthma attacks. We found that both markers could be potentially useful for monitoring asthma control. Combining both markers enhanced prognostic performance up to five days ahead of an asthma attack.
We replicated the between-patient statistical associations of nocturnal cough and sleep quality with asthma control reported by prior research. Marsden et al described weak to moderate correlations between nocturnal cough of a single night and asthma control.Citation11 Luyster and colleagues have found similar cross-sectional correlation strengths with regard to sleep quality.Citation10 With longitudinal data, however, we found stronger associations and were able to show that they hold for the within-patient level.
Our declining asthma attack prediction performance over time converges with prior descriptions of symptom deteriorations in developing asthma attacks. Tattersfield and colleagues descriptively investigated 425 severe attacks in the two weeks prior to the attack.Citation35 A decline in symptoms and peak expiratory flow (PEF) “occurred gradually between day −10 and day −3 and then more rapidly”.
From a clinical point of view, long-term goals of asthma management are symptom control and risk reduction of asthma exacerbations and asthma-related deaths.Citation2 Asthma attacks can be fatal and are more common and more severe when asthma is not well controlled. Regular, objective assessment of asthma control is therefore a key element of successful asthma management. By recording markers of asthma control effortless and patient-friendly, for example, via a smartphone as employed in this study, access to objective assessments of asthma control can be facilitated. For example, we were able to show that deep learning models trained on data from this study can detect and count nocturnal cough automatically with a comparable accuracy to human annotators.Citation36
However, the accuracies presented in this paper for detecting and predicting relevant asthma outcomes do not warrant the use of nocturnal cough and sleep quality as self-contained biomarkers for asthma management. The limiting factors for marker performance remain unclear. Possible reasons are the pragmatic study design with a limited study duration per patient and a smartphone-based data collection that is prone to operating errors, a sample characterized by rather well-controlled and intermittent-to-mild asthma, and disease-related reasons such as asthma’s heterogeneity,Citation3 which could limit marker applicability for certain asthma phenotypes and would result in an overall decreased marker performance if applied to the entire patient population.
Despite the moderate accuracies of our detection and prediction models based on nocturnal cough and sleep quality, we argue that both markers may be useful for asthma self-management. As part of its overall objective of increasing asthma control and preventing attacks, the GINA guideline recommends patients to monitor their symptoms.Citation2 The associations with asthma outcomes reported in this paper suggest that nocturnal cough and sleep quality may indeed be relevant parameters for asthma symptom self-monitoring. Furthermore, their moderate accuracies as self-contained biomarkers could be offset if assessments are subsequently confirmed by more accurate methods. Honkoop and colleagues found that action points based on PEF and symptom questionnaires enabled highly accurate early detection of asthma attacks.Citation37 Thus, passively measured nocturnal cough and sleep quality could serve as additional cues for active monitoring based on action points. More specifically, objective cues for initiating active monitoring could enrich patients’ subjective self-assessments, which have been reported to be biased.Citation5 Future research should investigate the efficacy and effectiveness of such a two-stage monitoring strategy empirically. In the long-term application of both markers in practice, we suspect that nocturnal cough might emerge as the more robust and useful marker due to its inherently higher specificity to the respiratory system and due to a possible confounding treatment effect of oral corticosteroids (improvement in asthma outcomesCitation2 accompanied by a simultaneous impairment of sleep quality).Citation38
Strengths and Limitations
This is the first longitudinal study to examine the utility of nocturnal cough and sleep quality as markers for asthma. By defining lenient eligibility criteria and collecting patient data in situ, the presented results are characterized by high external validity. In a more controlled setting, we would expect higher performance figures due to more reliable measurements. Furthermore, by taking advantage of prior research on ACT cut-offs for different classification purposes,Citation20,Citation21 we were able to investigate the utility of both candidate markers from multiple perspectives.
However, high external validity comes at the expense of compromised internal validity: we included patients with self-reported physician-diagnosed asthma and modelled patients’ subjective assessments, which likely introduced noise into our analysis, for example, due to varying definitions of asthma attacks between patients. Finally, the prognostic performance results have to be interpreted cautiously due to the low number of observed asthma attacks.
Acknowledgments
We thank all patients who participated in this study. Further, we thank the study teams of all participating institutions for their support.
Parts of this paper have been presented as an oral presentation and e-poster at the International Congress of the European Respiratory Society (05. – 09.09.2020) and as an oral presentation at the annual congress of the Swiss Society for Pulmonology (23.09.2020).
Furthermore, MB and TK should be considered joint senior author.
Disclosure
PT, FB, EF, and TK are affiliated with the Center for Digital Health Interventions (www. c4dhi. org), a joint initiative of the Department of Management, Technology and Economics at ETH Zurich and the Institute of Technology Management at the University of St. Gallen, which is funded in part by the Swiss health insurer CSS. CSS insurance supported the recruitment of participants but had no role in study design, app design, data management plans, or in reviewing and approving the manuscript for publication. PT further reports the foundation of a company, Resmonics AG, which is partly based on the results of this paper, during the conduct of the study. CS-S reports personal fees from AstraZeneca, GlaxoSmithKline, Novartis, and Boehringer Ingelheim, outside the submitted work. EF and TK further report, to be cofounders of Pathmate Technologies AG, a university spin-off company that creates and delivers digital clinical pathways and has used the open-source MobileCoach platform for that purpose, too (however, Pathmate Technologies is not involved in the study app described in this paper), outside the submitted work. The authors report no other potential conflicts of interest for this work.
Additional information
Funding
References
- James SL, Abate D, Abate KH, et al. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2018;392(10159):1789–1858.30496104
- Global Initiative for Asthma. Global strategy for asthma management and prevention; 2018 Available from: https://ginasthma.org/wp-content/uploads/2018/04/wms-GINA-2018-report-V1.3-002.pdf. Accessed 1110, 2020.
- Pavord ID, Beasley R, Agusti A, et al. After asthma: redefining airways diseases. Lancet. 2018;391(10118):350–400. doi:10.1016/S0140-6736(17)30879-628911920
- Murphy K, Meltzer EO, Blaiss MS, et al. Asthma management and control in the United States: results of the 2009 Asthma Insight and Management survey. Allerg Asthma Proc. 2012;33(1):54–64. doi:10.2500/aap.2011.32.3518
- Von Leupoldt A, Brassen S, Baumann HJ, et al. Structural brain changes related to disease duration in patients with asthma. PLoS One. 2011;6(8):e23739. doi:10.1371/journal.pone.002373921886820
- Nathan RA, Sorkness CA, Kosinski M, et al. Development of the asthma control test: a survey for assessing asthma control. J Allerg Clin Immunol. 2004;113(1):59–65. doi:10.1016/j.jaci.2003.09.008
- Miles C, Arden-Close E, Thomas M, et al. Barriers and facilitators of effective self-management in asthma: systematic review and thematic synthesis of patient and healthcare professional views. NPJ Prim Care Respir Med. 2017;27(1):57. doi:10.1038/s41533-017-0056-428993623
- Schatz BR. Population measurement for health systems. NPJ Digital Med. 2018;1(1):20174. doi:10.1038/s41746-017-0004-2
- Fukuhara A, Saito J, Birring SS, et al., Clinical characteristics of cough frequency patterns in patients with and without asthma. J Allergy Clin Immunol. 2019.
- Luyster FS, Teodorescu M, Bleecker E, et al. Sleep quality and asthma control and quality of life in non-severe and severe asthma. Sleep Breath. 2012;16(4):1129–1137. doi:10.1007/s11325-011-0616-822102290
- Marsden PA, Satia I, Ibrahim B, et al. Objective cough frequency, airway inflammation, and disease control in asthma. Chest. 2016;149(6):1460–1466. doi:10.1016/j.chest.2016.02.67626973014
- Barata F, Kipfer K, Weber M, Tinschert P, Fleisch E, Kowatsch T. Towards device-agnostic mobile cough detection with convolutional neural networks. In: 7th IEEE International Conference on Healthcare Informatics (ICHI 2019); 2019; Xi’an, China.
- Min J-K, Doryab A, Wiese J, Amini S, Zimmerman J, Hong JI. Toss’n’turn: smartphone as sleep and sleep quality detector. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; 2014; Toronto, Canada: ACM.
- Tinschert P, Rassouli F, Barata F, et al. Prevalence of nocturnal cough in asthma and its potential as a marker for asthma control (MAC) in combination with sleep quality: protocol of a smartphone-based, multicentre, longitudinal observational study with two stages. BMJ Open. 2019;9(1):e026323. doi:10.1136/bmjopen-2018-026323
- Rassouli F, Tinschert P, Barata F et al. Characteristics of asthma-related nocturnal cough – a potential new digital biomarker. Journal of Asthma and Allergy. 2020 In Press.
- Filler A, Kowatsch T, Haug S, Wahle F, Staake T, Fleisch E. MobileCoach: a novel open source platform for the design of evidence-based, scalable and low-cost behavioral health interventions: overview and preliminary evaluation in the public health context. In: Wireless Telecommunications Symposium (WTS), 2015; 2015; New York, NY: IEEE.
- Kowatsch T, Volland D, Shih I, et al. Design and evaluation of a mobile chat app for the open source behavioral health intervention platform MobileCoach. In: International Conference on Design Science Research in Information Systems; 2017; Karlsruhe, Germany: Springer.
- Buysse DJ, Reynolds CF, Monk TH, et al. The Pittsburgh Sleep Quality Index: a new instrument for psychiatric practice and research. Psychiatry Res. 1989;28(2):193–213. doi:10.1016/0165-1781(89)90047-42748771
- Matthews KA, Patel SR, Pantesco EJ, et al. Similarities and differences in estimates of sleep duration by polysomnography, actigraphy, diary, and self-reported habitual sleep in a community sample. Sleep Health. 2018;4(1):96–103. doi:10.1016/j.sleh.2017.10.01129332687
- Schatz M, Sorkness CA, Li JT, et al. Asthma Control Test: reliability, validity, and responsiveness in patients not previously followed by asthma specialists. J Allerg Clin Immunol. 2006;117(3):549–556. doi:10.1016/j.jaci.2006.01.011
- Schatz M, Kosinski M, Yarlas AS, et al. The minimally important difference of the Asthma Control Test. J Allerg Clin Immunol. 2009;124(4):719–723. e1. doi:10.1016/j.jaci.2009.06.053
- Sullivan PW, Campbell JD, Globe G, et al. Measuring the effect of asthma control on exacerbations and health resource use. J Allerg Clin Immunol. 2015;136(5):1409–1411. e6. doi:10.1016/j.jaci.2015.04.046
- Sembajwe G, Cifuentes M, Tak SW, et al. National income, self-reported wheezing and asthma diagnosis from the World Health Survey. Eur Respir J. 2010;35(2):279–286. doi:10.1183/09031936.0002750919741032
- Hyndman R, Kang Y, Montero-Manso P, et al. tsfeatures: time series feature extraction. 2019.
- Therneau T, Atkinson B, Ripley B, Ripley MB. Package ‘rpart’; 2015 Available from: cran.ma.ic.ac.uk/web/packages/rpart/rpart.pdf. Accessed 420, 2016.
- Kuhn M, Caret: classification and regression training, in Astrophysics Source Code Library. 2015.
- Bates D, Mächler M, Bolker B, et al. Fitting linear mixed-effects models using lme4. J Stat Softw. 2015;67(1). doi:10.18637/jss.v067.i01.
- R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2018.
- Breiman L. Classification and Regression Trees. Routledge; 2017.
- Fonarow GC, Adams KF, Abraham WT, Yancy CW, Boscardin WJ; ADHERE Scientific Advisory Committee. Risk stratification for in-hospital mortality in acutely decompensated heart failure: classification and regression tree analysis. JAMA. 2005;293(5):572–580. doi:10.1001/jama.293.5.57215687312
- Stone M. Cross‐validatory choice and assessment of statistical predictions. J Royal Statis Soc. 1974;36(2):111–133.
- Bossuyt PM. Clinical validity: defining biomarker performance. Scand J Clin Lab Invest. 2010;70(sup242):46–52. doi:10.3109/00365513.2010.493383
- Edwards LJ, Muller KE, Wolfinger RD, et al. An R 2 statistic for fixed effects in the linear mixed model. Stat Med. 2008;27(29):6137–6157. doi:10.1002/sim.342918816511
- Swets JA. Measuring the accuracy of diagnostic systems. Science. 1988;240(4857):1285–1293. doi:10.1126/science.32876153287615
- Tattersfield AE, Postma D, Barnes P, et al. Exacerbations of asthma: a descriptive study of 425 severe exacerbations. Am J Respir Crit Care Med. 1999;160(2):594–599. doi:10.1164/ajrccm.160.2.981110010430734
- Barata F, Tinschert P, Rassouli F, et al. Automatic recognition, segmentation and sex assignment of nocturnal asthmatic cough and cough epochs in smartphone-based audio recordings: results from an observational field study. J Med Internet Res. 2020;22(7):e18082. doi:10.2196/1808232459641
- Honkoop PJ, Taylor DR, Smith AD, et al. Early detection of asthma exacerbations by using action points in self-management plans. Eur Respir J. 2013;41(1):53–59. doi:10.1183/09031936.0020591122653768
- Idzikowski C, Shapiro CM. ABC of sleep disorders: non-psychotropic drugs and sleep. BMJ. 1993;306(6885):1118. doi:10.1136/bmj.306.6885.11188495162