https://orcid.org/0000-0002-7881-8430
Abstract
Background
Excessive daytime sleepiness (EDS) forms a prevalent symptom of obstructive sleep apnea (OSA) and narcolepsy type 1 (NT1), while the latter might always be overlooked. Machine learning (ML) models can enable the early detection of these conditions, which has never been applied for diagnosis of NT1.
Objective
The study aimed to develop ML prediction models to help non-sleep specialist clinicians identify high probability of comorbid NT1 in patients with OSA early.
Methods
Totally, clinical features of 246 patients with OSA in three sleep centers were collected and analyzed for the development of nine ML models. LASSO regression was used for feature selection. Various metrics such as the area under the receiver operating curve (AUC), calibration curve, and decision curve analysis (DCA) were employed to evaluate and compare the performance of these ML models. Model interpretability was demonstrated by Shapley Additive explanations (SHAP).
Results
Based on the analysis of AUC, DCA, and calibration curves, the Gradient Boosting Machine (GBM) model demonstrated superior performance compared to other machine learning (ML) models. The top five features used in the GBM model, ranked by feature importance, were age of onset, total limb movements index, sleep latency, non-REM (Rapid Eye Movement) sleep stage 2 and severity of OSA.
Conclusion
The study yielded a simple and feasible screening ML-based model for the early identification of NT1 in patients with OSA, which warrants further verification in more extensive clinical practices.
Introduction
Obstructive sleep apnea (OSA) is characterized by reduced (hypopnea) or absent (apnea) airflow during sleep, which excessive daytime sleepiness (EDS) commonly reported as one of the primary symptoms.Citation1,Citation2 Globally, there are approximately 936 million adults with mild to severe OSA and 425 million adults with moderate to severe OSA.Citation3 EDS constitutes one of the primary symptoms of narcolepsy type 1 (NT1) which acts as a main differential diagnosis of OSA.Citation4 It has been reported that approximately 20–30% of patients with narcolepsy coexist with OSA.Citation5–7 However, for the patients with OSA with EDS, the comorbid NT1 identification may be delayed. A study has showed that a mean delay in diagnosis of up to 15 years in these patients with NT1.Citation8 Cataplexy is seldom mentioned in patients with doubtful or atypical cataplexy, so is likely to be overlooked by clinicians other than sleep specialist in neurology. Sometimes, cataplexy in patients with NT1 may gradually improve and even disappear over time. Given the subjective nature of cataplexy, patients with OSA or depression may also claim the attacks. In those cases, the symptoms of EDS failed to be relieved completely even under continuous positive airway pressure (CPAP) therapy. Patients with undiagnosed with NT1 were even subject to unnecessary and expensive interventions such as surgery.Citation5,Citation7,Citation9
With the widespread application of home sleep apnea testing (HSAT) the last decade, OSA diagnosis shifted towards a more automated approach.Citation10 Machine learning (ML) based models have also been used for mass population screening among patients with OSA.Citation11–13 Compared to traditional logistic regression, ML algorithms can learn more complex patterns and relationships from data. By using different algorithms, ML can capture more nuanced information in non-linear features, providing higher prediction accuracy.Citation14,Citation15 In fact, ML-based model has showed significant advantages in the diagnosis and differential diagnosis of OSA.Citation11 Meanwhile, various kinds of data obtained in clinical practice are inevitably mixed with noise, and different ML-based models may have different prediction effects. Therefore, the objective of the present study was to develop and compare ML models for the prediction of comorbid NT1 in patients with OSA with EDS.
Methods
Experimental Design and Participants
All patients with OSA who underwent PSG were sourced from the Department of neurology, Xijing Hospital, Encephalopathy Department No.2, Baoji Hospital of Traditional Chinese Medicine and Encephalopathy Department No.10, Xi’an Hospital of Traditional Chinese Medicine from May 2020 to June 2023. The following clinical data were collected from electronic medical records system: (1) Demographic features: height, weight, body mass index (BMI), gender, and age; (2) Symptomatic features: snoring, witnessed apnea, frequent awakening, dry throat at waking, morning headache; (3) Clinical features: age of onset, course of disease, hypertension, hyperlipidemia, diabetes, coronary heart disease, cerebrovascular disease, epilepsy, psychiatric disorders; (4) PSG indicators; (5) Clinical diagnosis. Description and definition of OSA symptoms and PSG indicators were seen in Table S1.
Inclusion criteria included: (1) Diagnosis of OSA based on international classification of sleep disorders-3 (ICSD-3) diagnostic criteria; (2) Complaints of EDS or Epworth sleepiness scale (ESS) score ≥ 9. Exclusion criteria included: (1) Clinical data were incomplete or missing; (2) Any drug abuse; (3) Comorbid other severe sleep disorders (narcolepsy type 2, idiopathic hypersomnia, restless legs syndrome or severe periodic limb movement in sleep).
The eligible patients with OSA were assigned into group 1 (with comorbid NT1) or group 2 (without comorbid NT1). Diagnosis of NT1 was made by at least two experienced sleep specialist clinicians on the basis of ICSD-3. In addition, patients with NT1 included in our study had a history of no significant improvement in EDS after CPAP therapy and significant improvement after anti-cataplectic drugs or stimulants. The whole data acquisition path was summarized in a flowchart (). Written consent was obtained from all participants in the study, and all clinical data underwent irreversible anonymization. All procedures that contributed to this work are in accordance with the ethical standards set by the relevant national and institutional committees on human experimentation and comply with the principles outlined in the Helsinki Declaration. The study was approved by the Medical Ethical Committee of Xijing Hospital (KY20222053-C-1).
Figure 1 Flowchart.
Measurements
The PSG equipment and assessment process were consistent with our previous study.Citation16 Briefly, all patients were admitted to the separate sleeping ward at 4pm and underwent nocturnal PSG at habitual bedtimes based on their regular sleep–wake cycle on a XLTEK Natus system (Bio-Logic, USA). Video-PSG included electroencephalograms (EEG), electrocardiogram (ECG), electromyography (EMG, surface EMG of chin and limbs), electro-oculogram (EOG). Oro-nasal airflow was monitored by a nasal cannula connected to a pressure transducer and mouth thermistor. Chest and abdominal movements were assessed by piezoelectric straps, and arterial oxygen saturation (SaO2) was recorded continuously monitored by a pulse oximeter. Leg movements were measured using surface EMG-electrodes from bilateral tibial anterior muscles. Synchronous infrared video recording was conducted to assess the patient’s movements during sleep. Limb movements were documented as isolated limb movements (ILMs) and periodic limb movements (PLMs). Respiratory-related limb movements (RRLMs) were typically excluded according to the criteria for PLMs of international scoring guidelines.Citation17,Citation18 Beverages such as alcohol, tea and coffee were abstained 24 hours before PSG recording.
Data Cleaning and Standardization
The video-PSG recordings of all patients were evaluated by two experienced technicians to ensure the accuracy and consistency. A total of 49 clinical variables were collected, among which rapid eye movement (REM) latency from sleep onset was not on the variable selection list for the lack of REM sleep in some patients with severe OSA. Similarly, the course of disease was also not selected for the absence of Clinical application value.
Model Building
The prevalence of patients with OSA with comorbid NT1 was relatively low, so a one-to-two matching was performed to balance data according to propensity scores.Citation19 By using lasso regression, the study minimized the multicollinearity in clinical variables’ selection. The study population was randomly divided into training and testing sets (75% of the data as training set including 5-fold cross-validation and model tuning, the remaining 25% for model validation). The 5-fold cross-validation is a method that the dataset is divided into five equal-sized subsets, with four subsets used for training the model and the remaining subset for testing it. This process is repeated five times, each time with a different subset as the testing set and the others serving as the training set. Nine ML algorithms were utilized to build the prediction models for NT1 in patients with OSA, including decision tree (DT), rand forest (RF), eXtreme gradient boosting (XGBoost), lasso and ridge and elastic net, support vector machine (SVM), multilayer perceptron (MLP), gradient boosting machine (GBM), K-Nearest neighbor (KNN) and logistic regression (LR). To assess the prediction performance and clinical application value of the models, area under the receiver operating curve (AUC), accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive curve (NPV), f-measure, kappa, calibration curve, decision curve analysis (DCA) curve and precision-recall (PR) curve were performed. DeLong test was used to compare the differences in the receiver operating curve (ROC) between training and testing set. The feature importance of the models was investigated by SHapley Additive exPlanations (SHAP), and the interpretability of the models was demonstrated by SHAP feature importance ranking graph and SHAP summary plot.Citation20 The study utilized R packages tidymodels (1.1.0) to train and compare different ML models. The whole experimental design was summarized in .
Figure 2 Experimental design.
Statistics
Continuous variables were expressed as median (P25, P75) and categorical variables were expressed as counts and percentages. Mann–Whitney U-test and Fisher exact test were performed to compare the baseline characteristics between the two groups. A significance level of p < 0.05 was used to determine statistical significance. Statistical analyses were performed using R (4.3.0).
Results
In the present study, out of 2645 patients with OSA, 82 individuals were found to have comorbid NT1, resulting in a prevalence rate of 3.1%. For the 1076 patients with OSA with EDS, a total of 987 patients were eligible. After performing a one-to-two propensity score match based on age between the two groups, 246 patients were selected for model building (82 with and 164 without comorbid NT1 respectively). Data of pre- and post-treatment sleepiness of the patients had been provided in the Table S2.
Demographic, Symptomatic and Clinical Features
The study sample included 215 (87.4%) males and 31 females (12.6%), with the median age of 38 years (18 to 69 years). The difference in both age distribution and symptomatic features were not statistically significant between the two groups. The median age of onset was 26.5 and 32 years while the disease course was 8.5 and 3 years in patients with OSA with and without comorbid NT1. Comorbidities’ incidence except for hypertension were not statistically significant between the two groups. provided detailed information about demographic, symptomatic and clinical features.
Table 1 Comparison of the Clinical Characteristics and PSG Indicators Between Patients with OSA with Comorbidity of NT1 and Without Comorbidity of NT1
Feature Selection
Five demographic features, 5 symptomatic features, 9 clinical features and 30 PSG indicators were candidates for variables selection by lasso regression. Age, course of disease and REM latency from sleep onset were excluded. Eventually, 11 variables were retained and used as input for model design (), which was listed in .
Figure 3 Lasso regression. (A) LASSO coefficient path. (B) Cross-verification.
Multiple ML Models Comparison
Among the nine machine learning models used for predicting comorbid NT1, DT showed the lowest AUC value (AUC = 0.8293), while GBM had the highest AUC value (AUC = 0.8792) (). GBM also performed the best in the calibration curves, which shown fitted the ideal calibration curve (gray dotted line) better than other ML models (). The DCA curve revealed that the net benefit did not differ significantly between the nine ML models within the threshold probability of 0%–25%. However, from 26% to approximately 37% and 68% to 75%, GBM exhibited the greatest net benefit. From 75% to 100%, the net benefit of GBM was second only to logistic regression (). GBM has the second-highest PRAUC (0.8257), just behind RF (0.844) (). The performance of ML models was seen in . The difference in ROC was not statistically significant between training and testing set of GBM model (), and 5-fold cross validation was shown in .
Table 2 Comparison of the Prediction Performance of NT1 in Patients with OSA Between Nine Machine Learning Models
Figure 4 Predictive performance of nine machine learning models for NT1 in patients with OSA. (A) ROC curve. (B) Calibration curve. (C) DCA. (D) PR.
Figure 5 (A) ROCs for the training and testing set. (B) 5-fold cross validation in the training set.
Model Interpretability
The SHAP feature importance ranking graph for GBM revealed that the top four important features were age of onset, total limb movements index, sleep latency and non-REM sleep stage 2 (). The SHAP summary plot evinced that each feature contributed positive or negative effects to predicting comorbid NT1 (). Each dot represented the feature value (SHAP value) of one individual and colors indicated the level of feature value (red dot meant low and blue dot meant high feature value). The SHAP method calculated a total SHAP value for each individual. The higher the total SHAP value means the higher probability of comorbid NT1. Taking the feature of age of onset as an example, the blue dots were concentrated on the left side of the plot. This indicated that the total SHAP value increased with decreasing age of onset among patients with OSA ().
Figure 6 Feature importance. (A) The SHAP feature importance ranking graph. (B) The SHAP summary plot.
Model Application
A web‐based tool was developed for clinicians to use the proposed model (https://xijinghospital.shinyapps.io/NT1_among_patients_with_OSA). A patient with OSA and a patient with OSA with comorbid NT1 were respectively taken as examples. For example, age of onset: 30 years, total sleep time: 458 minutes, sleep latency: 11.1 minutes, sleep efficiency: 82.6%, non-REM sleep stage 2: 41.3%, ILM arousals index: 0.1, ILM index: 5.6, total limb movements index: 7.3, hypertension: Yes, severity of OSA: moderate and severity of hypoxemia: mild. The model predicted that the risk of comorbid NT1 in this patient was 17.89% below the cutoff value (0.3938) (), indicating that the patient was at low risk of comorbid NT1. Age of onset: 10 years, total sleep time: 509 minutes, sleep latency: 11.5 minutes, sleep efficiency: 88.8%, non-REM sleep stage 2: 40.8%, ILM arousals index: 0.6, ILM index: 4.7, total limb movements index: 5.8, hypertension: no, severity of OSA: mild and severity of hypoxemia: mild. The model predicted that the risk of comorbid NT1 in this patient was 93.25% above the cutoff value (0.9325) (). Clinicians should consider the potential risk of comorbid NT1 in this patient.
Figure 7 Cases of website usage. (A) Case 1 without comorbid NT1. (B) Case 2 with comorbid NT1.
Discussion
EDS appears ordinarily among individuals with sleep disorders, particularly those with OSA and NT1.Citation1,Citation2,Citation4 Although several studies have identified differences between OSA and NT1 on symptomatology and objective tests,Citation21,Citation22 there is still no reliable model to predict NT1 in patients with OSA.Citation11–13 Our study indicated the comorbid NT1 could be accurately predicted by certain ML models. After comprehensive evaluation, GBM is the optimal algorithm among the 9 ML models developed from clinical characteristics and PSG indicators. GBM refers to a supervised learning algorithm which combines boosting and regression trees, functional in exploring the complex relationships between variables.Citation23 Under a hyperparameter optimization, the GBM model is employed to optimize computational efficiency and enhance the robustness of detection outcomes,Citation24 which might underlie the satisfying performance of it in the present study.
Similar with the other study, the features reserved for the prediction of NT1 in our study included age of onset, total limb movements index, sleep latency, non-REM sleep stage 2, severity of OSA and 6 other features.Citation4 The risk for developing OSA increases with age, but that of NT1 typically manifests within the first two decades of life.Citation25 This could explain why the EDS appears earlier in patients with OSA with comorbid NT1 than without. A systematic review and meta-analysis reported a higher leg movement index in NT1 patients than in healthy controls (HC) or even patients with NT2,Citation26 consistent with our finding that patients with OSA with comorbid NT1 tended to have a higher total limb movements index. Several studies have reported differences in sleep architecture between patients with OSA and with NT1. A shorter mean sleep latency is generally indicative of a potential risk of NT1.Citation27 Patients with NT1 displayed a lower non-REM sleep stage 2 than HC due to increased REM of sleep time.Citation26 Conversely, patients with OSA, especially those with severe OSA, had a higher N2% due to decreased or absent REM sleep.Citation28,Citation29 In the present study, the decreased N2% also suggested an increased risk of comorbid NT1 in the GBM model. In summary, younger age of sleepiness onset, lower degree of OSA severity, shorter sleep latency and higher total limb movements and non-REM sleep stage 2 had the most predictive value for NT1. The correlation between clinical features and NT1 effectively demonstrated the reliability and feasibility of the GBM model. Moreover, SHAP overcome the “black box” nature of ML models, thus strengthening the reliability and interpretability aspect of our GBM model.Citation30
One of the major advantages of our model should be the utilization of clinical information of three specialized sleep centers, which ensured the integrity of case data, reduced the risk of selection bias, and increased the representativeness of the samples. In addition, features such as age of onset and PSG indicators, were routinely collected from patients with OSA. Low-cost, noninvasive, but objective features are more aligned with actual demand of clinical practice than the subjective indicators such as scale scores.
Our study was one of the first to use ML to predict comorbidities in patients with OSA, providing guidance and can be helpful for the diagnosis of NT1.Citation31 If ML of PSG features can be used to help diagnose comorbid NT1 then it may be a reason to perform PSG rather than HSAT if ESS is high and especially if the individual has experienced an early onset of EDS. In this way, the combination of HSAT and ML models may be more competitive in evaluation of EDS and identification of comorbid NT1, which would notify the necessity of the following MSLT or CSF orexin test by lumbar puncture. In many cases involving chronic diseases, clinicians place significant emphasis on secondary prevention. Using ML model will significantly save clinicians time and human resources by accurately identifying individuals at high risk for NT1 and raising awareness about hidden problems.Citation10,Citation32,Citation33
Limitation
There are several limitations for the present study. First, due to the low prevalence of NT1 in patients with OSA (3.1%), only internal validation was performed without external validation in order to ensure the required the study dataset for training the ML model. Our ML model should focus on validation in other sleep centers for clinical practice in the future. Second, although we reviewed three years of case data from three specialized sleep centers, the sample size of our study was limited and a higher input sample size would be required to further optimize the ML model. Third, all the patients in our study received PSG only once, so the influence of night-to-night variability was inevitable Fourth, in this retrospective study, only the patients with NT1 were included, but not patients with NT2 or idiopathic narcolepsy in order to avoid misdiagnosis. The future prospective cohort study needs to be carried out to verify the efficiency and accuracy of the ML models.
Conclusion
In conclusion, comorbid NT1 is rare in patients with OSA, so the potential risk of comorbid NT1 may be overlooked by non-sleep specialist clinicians. ML-based model would be a simple and feasible screening approach for early identification of NT1 in clinical practice, which warrants further verification and application.
Highlights
The first study utilizes ML to predict comorbid NT1 in patients with OSA.
GBM model is the best model for prediction.
Age of onset, total limb movements index, sleep latency and stage 2 of sleep time play important role in ML model.
Abbreviations
ML, machine learning; NT1, narcolepsy type 1; OSA, obstructive sleep apnea; EDS, Excessive daytime sleepiness; ROC, Receiver operator characteristic; AUC, Area under the receiver operating curve; DCA, Decision curve analysis; SHAP, SHapley Additive exPlanations; CPAP, Continuous positive airway pressure; MSLT, Multiple sleep latency test; CSF, Cerebrospinal fluid; PSG, Polysomnography; AHI, Apnea-hypopnea index; HSAT, Home sleep apnea testing; BMI, Body mass index; ICSD-3, International classification of sleep disorders-3; ESS, Epworth sleepiness scale; EEG, Electroencephalograms; ECG, Electrocardiogram; EMG, Electromyography; EOG, Electro-oculogram; ILMs, Isolated limb movements; PLMs, Periodic limb movements; RRLMs, Respiratory-related limb movements; REM, Rapid eye movement; DT, Decision tree; RF, Rand forest; XGBoost, eXtreme gradient boosting; SVM, Support vector machine; MLP, Multilayer perceptron; GBM, Gradient boosting machine; KNN, K-Nearest neighbor; PPV, Positive predictive value; NPV, Negative predictive curve.
Ethical Publication Statement
We confirm that we have read the Journal’s position on issues involved in ethical publication and affirm that this report is consistent with those guidelines.
Disclosure
The authors report no potential conflicts of interest in this work.
Acknowledgments
Yuanhang Pan, Di Zhao and Xinbo Zhang are co-first authors for this study. Anyone else who contributed to the manuscript but did not qualify for authorship had been acknowledged with their permission. All listed authors had made a significant scientific contribution to the research in the manuscript approved its claims and agreed to be an author.
Data Sharing Statement
The data that support the findings of this study are available from the corresponding author (Yonghong Liu) upon reasonable request.
Additional information
Funding
References
- Jordan AS, McSharry DG, Malhotra A. Adult obstructive sleep apnoea. Lancet. 2014;383:736–747. doi:10.1016/S0140-6736(13)60734-5
- Gottlieb DJ, Punjabi NM. Diagnosis and management of obstructive sleep apnea: a review. JAMA. 2020;323:1389–1400. doi:10.1001/jama.2020.3514
- Benjafield AV, Ayas NT, Eastwood PR, et al. Estimation of the global prevalence and burden of obstructive sleep apnoea: a literature-based analysis. Lancet Respir Med. 2019;7:687–698. doi:10.1016/S2213-2600(19)30198-5
- Barateau L, Pizza F, Plazzi G, et al. Narcolepsy. J Sleep Res. 2022;31:e13631. doi:10.1111/jsr.13631
- Sansa G, Iranzo A, Santamaria J. Obstructive sleep apnea in narcolepsy. Sleep Med. 2010;11:93–95. doi:10.1016/j.sleep.2009.02.009
- Frauscher B, Ehrmann L, Mitterling T, et al. Delayed diagnosis, range of severity, and multiple sleep comorbidities: a clinical and polysomnographic analysis of 100 patients of the Innsbruck Narcolepsy Cohort. J Clin Sleep Med. 2013;9:805–812. doi:10.5664/jcsm.2926
- Pataka AD, Frangulyan RR, Mackay TW, et al. Narcolepsy and sleep-disordered breathing. Eur J Neurol. 2012;19:696–702. doi:10.1111/j.1468-1331.2011.03610.x
- Thorpy MJ, Krieger AC. Delayed diagnosis of narcolepsy: characterization and impact. Sleep Med. 2014;15:502–507. doi:10.1016/j.sleep.2014.01.015
- Sureshbabu S, Asranna A, Peter S, et al. Secondary narcolepsy masquerading as obstructive sleep apnea. Ann Indian Acad Neurol. 2019;22:537–538. doi:10.4103/aian.AIAN_19_19
- Fietze I, Laharnar N, Bargiotas P, et al. Management of obstructive sleep apnea in Europe - A 10-year follow-up. Sleep Med. 2022;97:64–72. doi:10.1016/j.sleep.2022.06.001
- Ferreira-Santos D, Amorim P, Silva Martins T, et al. Enabling early obstructive sleep apnea diagnosis with machine learning: systematic review. J Med Internet Res. 2022;24:e39452. doi:10.2196/39452
- Ding Y, Sun Y, Li Y, et al. Selection of OSA-specific pronunciations and assessment of disease severity assisted by machine learning. J Clin Sleep Med. 2022;18:2663–2672. doi:10.5664/jcsm.9798
- Holfinger SJ, Lyons MM, Keenan BT, et al. Diagnostic Performance Of Machine Learning-Derived OSA prediction tools in large clinical and community-based samples. Chest. 2022;161:807–817. doi:10.1016/j.chest.2021.10.023
- Zhang Y, Wang S, Hermann A, et al. Development and validation of a machine learning algorithm for predicting the risk of postpartum depression among pregnant women. J Affect Disord. 2021;279:1–8. doi:10.1016/j.jad.2020.09.113
- Zhang X, Bellolio MF, Medrano-Gracia P, et al. Use of natural language processing to improve predictive models for imaging utilization in children presenting to the emergency department. BMC Med Inform Decis Mak. 2019;19:287. doi:10.1186/s12911-019-1006-6
- Hu G, Yuan N, Pan Y, et al. Electroclinical features of sleep-related head jerk. Nat Sci Sleep. 2021;13:2113–2123. doi:10.2147/NSS.S331893
- Berry RB, Budhiraja R, Gottlieb DJ, et al. Rules for scoring respiratory events in sleep: update of the 2007 AASM manual for the scoring of sleep and associated events. deliberations of the sleep apnea definitions task force of the American Academy of Sleep Medicine. J Clin Sleep Med. 2012;8:597–619. doi:10.5664/jcsm.2172
- Zucconi M, Ferri R, Allen R, et al. The official world association of sleep medicine (WASM) standards for recording and scoring periodic leg movements in sleep (PLMS) and wakefulness (PLMW) developed in collaboration with a task force from the International Restless Legs Syndrome Study Group (IRLSSG). Sleep Med. 2006;7:175–183. doi:10.1016/j.sleep.2006.01.001
- Chang TH, Stuart EA. Propensity score methods for observational studies with clustered data: a review. Stat Med. 2022;41:3612–3626. doi:10.1002/sim.9437
- Lundberg SM, Erion G, Chen H, et al. From local explanations to global understanding with explainable AI for Trees. Nat Mach Intell. 2020;2:56–67. doi:10.1038/s42256-019-0138-9
- Sahni AS, Carlucci M, Malik M, et al. Management of excessive sleepiness in patients with narcolepsy and OSA: current challenges and future prospects. Nat Sci Sleep. 2019;11:241–252. doi:10.2147/NSS.S218402
- Filardi M, Demir N, Pizza F, et al. Prevalence and neurophysiological correlates of sleep disordered breathing in pediatric type 1 narcolepsy. Sleep Med. 2020;65:8–12. doi:10.1016/j.sleep.2019.07.004
- Asadikia A, Rajabifard A, Kalantari M. Region-income-based prioritisation of sustainable development goals by gradient boosting machine. Sustain Sci. 2022;17:1939–1957. doi:10.1007/s11625-022-01120-3
- Hawkins DM, Basak SC, Mills D. Assessing model fit by cross-validation. J Chem Inf Comput Sci. 2003;43:579–586. doi:10.1021/ci025626i
- Dauvilliers Y, Montplaisir J, Molinari N, et al. Age at onset of narcolepsy in two large populations of patients in France and Quebec. Neurology. 2001;57:2029–2033. doi:10.1212/wnl.57.11.2029
- Zhang Y, Ren R, Yang L, et al. Polysomnographic nighttime features of narcolepsy: a systematic review and meta-analysis. Sleep Med Rev. 2021;58:101488. doi:10.1016/j.smrv.2021.101488
- Sateia MJ. International classification of sleep disorders-third edition: highlights and modifications. Chest. 2014;146:1387–1394. doi:10.1378/chest.14-0970
- Roche J, Gillet V, Perret F, et al. Obstructive sleep apnea and sleep architecture in adolescents with severe obesity: effects of a 9-month lifestyle modification program based on regular exercise and a balanced diet. J Clin Sleep Med. 2018;14:967–976. doi:10.5664/jcsm.7162
- Walter LM, Nixon GM, Davey MJ, et al. Sleep disturbance in pre-school children with obstructive sleep apnoea syndrome. Sleep Med. 2011;12:880–886. doi:10.1016/j.sleep.2011.07.007
- Linardatos P, Papastefanopoulos V, Kotsiantis S. Explainable AI: a review of machine learning interpretability methods. Entropy. 2020;23. doi:10.3390/e23010018
- Rosenberg R, Hirshkowitz M, Rapoport DM, et al. The role of home sleep testing for evaluation of patients with excessive daytime sleepiness: focus on obstructive sleep apnea and narcolepsy. Sleep Med. 2019;56:80–89. doi:10.1016/j.sleep.2019.01.014
- Caples SM, Anderson WM, Calero K, et al. Use of polysomnography and home sleep apnea tests for the longitudinal management of obstructive sleep apnea in adults: an American Academy of Sleep Medicine clinical guidance statement. J Clin Sleep Med. 2021;17:1287–1293. doi:10.5664/jcsm.9240
- Zhou ZR, Wang WW, Li Y, et al. In-depth mining of clinical data: the construction of clinical prediction model with R. Ann Transl Med. 2019;7:796. doi:10.21037/atm.2019.08.63