Personalized Sleep Parameters Estimation from Actigraphy: A Machine Learning Approach

Figures & data

Table 1 Number of Participants and the Number of Their Available Sleeping Periods in the Data

Number of Participants	Number of Available Sleeping Periods
8	1
19	2
29	3
9	8
2	10
14	11

Table 2 Estimates of Sleep Parameters Obtained with Personalized and Generalized Sleep-Wake States Predictors and PSG, Averaged Over all Participants Across All Studies Combined. In Each Column, the Boldface Number Indicates the Estimated Sleep Parameter Closest to That of PSG

Classifier	Personalized Approach					Generalized Approach
	TST	SOL	WASO	SE	NA	TST	SOL	WASO	SE	NA
NB	347.72	20.30	159.09	64.23	16.44	359.24	18.37	147.63	66.7	14.46
RLR	486.50	9.37	38.72	89.90	6.56	500.19	8.83	26.13	92.55	5.17
RF	457.67	10.00	66.83	84.57	25.59	474.54	9.35	50.87	87.83	29.20
AB	464.98	8.94	60.35	85.95	27.67	487.65	8.57	38.74	90.27	18.04
XGB	467.91	9.00	57.26	86.49	22.72	486.54	9.11	39.20	90.04	15.91
PSG	436.83	15.67	73.26	80.71	26.65	436.83	15.67	73.26	80.71	26.65

Abbreviations: NB, Naive Bayes; RLR, regularized logistic regression; RF, random forest; AB, adaptive boosting; XGB, extreme gradient boosting; PSG, Polysomnography; TST, total sleep time; SOL, sleep onset latency; WASO, wake after sleep onset; SE, sleep efficiency; NA, number of awakenings.

Table 3 Estimates of Sleep Parameters Obtained with Different Classification Thresholds via Personalized RF and XGB Sleep-Wake States Predictors, Averaged Over All Participants Across All Studies Combined. In Each Column, the Boldface Number Indicates the Estimated Sleep Parameter Closest to That of PSG

Threshold	TST		SOL		WASO		SE		NA
	RF	XGB	RF	XGB	RF	XGB	RF	XGB	RF	XGB
0.1	295.11	294.80	18.59	25.61	213.43	203.33	54.53	54.43	42.91	35.48
0.2	382.85	398.89	14.31	12.06	132.50	119.89	70.77	73.71	41.02	37.07
0.3	415.00	430.91	11.37	10.43	105.72	91.20	76.67	79.66	36.96	33.17
0.4	438.93	452.31	10.44	9.78	84.22	71.07	81.11	83.60	31.17	28.06
0.5	457.67	467.91	10.00	9.00	66.83	57.26	84.57	86.49	25.59	22.72

Abbreviations: RF, random forest; XGB, extreme gradient boosting; TST, total sleep time; SOL, sleep onset latency; WASO, wake after sleep onset; SE, sleep efficiency; NA, number of awakenings.

Table 4 P-Values of the T-Tests Between Estimates of Sleep Parameters Obtained with Personalized and Generalized Sleep-Wake States Predictors Compared to Those Obtained by PSG. Significance Level Is 0.05. Boldface Numbers Indicate Statistically Non-Significant Values

Classifier	Personalized Approach					Generalized Approach
	TST	SOL	WASO	SE	NA	TST	SOL	WASO	SE	NA
NB	<0.01	0.13	<0.01	<0.01	<0.01	<0.01	0.36	<0.01	<0.01	<0.01
RLR	<0.01	<0.01	<0.01	<0.01	<0.01	<0.01	<0.01	<0.01	<0.01	<0.01
RF	0.11	0.01	0.49	0.07	0.64	<0.01	<0.01	<0.01	<0.01	0.29
AB	0.02	<0.01	0.14	<0.01	0.65	<0.01	<0.01	<0.01	<0.01	<0.01
XGB	0.01	<0.01	0.07	<0.01	0.07	<0.01	<0.01	<0.01	<0.01	<0.01

Abbreviations: NB, Naive Bayes; RLR, regularized logistic regression; RF, random forest; AB, adaptive boosting; XGB, extreme gradient boosting; TST, total sleep time; SOL, sleep onset latency; WASO, wake after sleep onset; SE, sleep efficiency; NA, number of awakenings.

Figure 1 ROC curves of personalized and generalized XGB sleep-wake states predictors. The two darker ROC curves show the performance of the generalized predictor (blue) and personalized predictors (orange) averaged over all individuals across all studies. The lighter curves show performance of the generalized XGB predictor and personalized XGB predictors tested on each individual. © 2018 IEEE. Reprinted, with permission, from Khademi A, El-Manzalawy Y, Buxton OM, Honavar V. Toward personalized sleep-wake prediction from actigraphy. In 2018 IEEE EMBS International Conference on Biomedical and Health Informatics, BHI 2018 (Vol. 2018-March, pp. 414–417). Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/BHI.2018.8333456.⁴⁵

Table 5 Performance of Personalized and Generalized Sleep-Wake States Predictors Averaged Over All Participants Across All Studies Combined. In Each Column, tThe Boldface Number(s) Indicate(s) the Highest Value Obtained for the Corresponding Metric

Classifier	Personalized Approach					Generalized Approach
	ACC	SN	SP	MCC	AUC	ACC	SN	SP	MCC	AUC
NB	0.75	0.74	0.74	0.38	0.83	0.75	0.69	0.77	0.36	0.83
RLR	0.86	0.29	0.98	0.36	0.82	0.86	0.26	0.99	0.35	0.83
RF	0.85	0.45	0.93	0.40	0.81	0.86	0.41	0.96	0.41	0.80
AB	0.86	0.46	0.95	0.44	0.80	0.87	0.37	0.98	0.45	0.85
XGB	0.86	0.45	0.95	0.45	0.84	0.87	0.38	0.98	0.45	0.85

Notes: © 2018 IEEE. Reprinted, with permission, from Khademi A, El-Manzalawy Y, Buxton OM, Honavar V. Toward personalized sleep-wake prediction from actigraphy. In 2018 IEEE EMBS International Conference on Biomedical and Health Informatics, BHI 2018 (Vol. 2018-March, pp. 414-417). Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/BHI.2018.8333456.⁴⁵

Abbreviations: NB, Naive Bayes; RLR, regularized logistic regression; RF, random forest; AB, adaptive boosting; XGB, extreme gradient boosting; ACC, accuracy; SN, sensitivity; SP, specificity; MCC, Matthews correlation coefficient; AUC, area under curve.

Table 6 Performance of Personalized and Generalized Sleep-Wake States Predictors Averaged Separately Across Individuals Within Each Study. In Each Column of Each Study, the Boldface Number(s) Indicate(s) the Highest Value Obtained for the Corresponding Metric

Study	Classifier	Personalized Approach					Generalized Approach
		ACC	SN	SP	MCC	AUC	ACC	SN	SP	MCC	AUC
SR	NB	0.76	0.75	0.75	0.40	0.85	0.68	0.79	0.66	0.34	0.85
	RLR	0.88	0.32	0.98	0.42	0.86	0.88	0.33	0.98	0.41	0.86
	RF	0.87	0.47	0.94	0.44	0.85	0.87	0.50	0.95	0.47	0.85
	AB	0.88	0.49	0.95	0.49	0.85	0.88	0.46	0.97	0.50	0.89
	XGB	0.84	0.43	0.94	0.42	0.88	0.89	0.48	0.97	0.51	0.89
TI	NB	0.80	0.67	0.80	0.38	0.84	0.75	0.74	0.75	0.39	0.83
	RLR	0.86	0.23	1.00	0.35	0.85	0.85	0.20	0.99	0.27	0.84
	RF	0.89	0.52	0.95	0.49	0.84	0.87	0.40	0.96	0.41	0.83
	AB	0.90	0.48	0.98	0.53	0.86	0.87	0.37	0.98	0.45	0.87
	XGB	0.80	0.27	0.95	0.32	0.88	0.88	0.37	0.98	0.46	0.88
AC	NB	0.60	0.70	0.57	0.24	0.70	0.72	0.41	0.83	0.25	0.69
	RLR	0.76	0.13	0.98	0.18	0.67	0.77	0.10	1.00	0.23	0.68
	RF	0.72	0.26	0.88	0.17	0.64	0.76	0.17	0.96	0.21	0.63
	AB	0.75	0.26	0.92	0.22	0.62	0.77	0.15	0.98	0.25	0.69
	XGB	0.87	0.47	0.96	0.48	0.66	0.77	0.16	0.98	0.25	0.69
NW	NB	0.82	0.79	0.82	0.45	0.90	0.83	0.75	0.84	0.45	0.90
	RLR	0.91	0.40	0.98	0.44	0.89	0.91	0.33	0.99	0.42	0.90
	RF	0.91	0.53	0.95	0.49	0.88	0.91	0.49	0.96	0.48	0.86
	AB	0.91	0.55	0.95	0.50	0.84	0.92	0.44	0.99	0.54	0.92
	XGB	0.91	0.55	0.96	0.52	0.91	0.92	0.45	0.99	0.54	0.92

Notes: © 2018 IEEE. Reprinted, with permission, from Khademi A, El-Manzalawy Y, Buxton OM, Honavar V. Toward personalized sleep-wake prediction from actigraphy. In 2018 IEEE EMBS International Conference on Biomedical and Health Informatics, BHI 2018 (Vol. 2018-March pp. 414-417). Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/BHI.2018.8333456.⁴⁵

Abbreviations: SR, sleep restriction; TI, tiagabine; AC, acoustics; NW, night working, NB, Naive Bayes; RLR, regularized logistic regression; RF, random forest; AB, adaptive boosting; XGB, extreme gradient boosting; ACC, accuracy; SN, sensitivity; SP, specificity; MCC, Matthews correlation coefficient; AUC, area under curve.

Table 7 Performance of Personalized and Study-Specific Sleep-Wake States Predictors Averaged Separately Across Individuals Within Each Study. In Each Column of Each Study, the Boldface Number(s) Indicate(s) the Highest Value Obtained for the Corresponding Metric

Study	Classifier	Personalized Approach					Study-Specific Approach
		ACC	SN	SP	MCC	AUC	ACC	SN	SP	MCC	AUC
SR	NB	0.76	0.75	0.75	0.40	0.85	0.74	0.76	0.74	0.39	0.85
	RLR	0.88	0.32	0.98	0.42	0.86	0.87	0.38	0.97	0.44	0.86
	RF	0.87	0.47	0.94	0.44	0.85	0.86	0.59	0.91	0.48	0.86
	AB	0.88	0.49	0.95	0.49	0.85	0.88	0.53	0.95	0.51	0.89
	XGB	0.84	0.43	0.94	0.42	0.88	0.88	0.57	0.95	0.53	0.89
TI	NB	0.80	0.67	0.80	0.38	0.84	0.80	0.70	0.81	0.43	0.84
	RLR	0.86	0.23	1.00	0.35	0.85	0.86	0.22	0.99	0.29	0.84
	RF	0.89	0.52	0.95	0.49	0.84	0.87	0.49	0.95	0.48	0.84
	AB	0.90	0.48	0.98	0.53	0.86	0.88	0.44	0.98	0.49	0.88
	XGB	0.80	0.27	0.95	0.32	0.88	0.88	0.45	0.97	0.50	0.88
AC	NB	0.60	0.70	0.57	0.24	0.70	0.52	0.82	0.43	0.22	0.70
	RLR	0.76	0.13	0.98	0.18	0.67	0.83	0.52	0.89	0.44	0.85
	RF	0.72	0.26	0.88	0.17	0.64	0.73	0.25	0.89	0.17	0.62
	AB	0.75	0.26	0.92	0.22	0.62	0.76	0.19	0.96	0.24	0.69
	XGB	0.87	0.47	0.96	0.48	0.66	0.76	0.18	0.97	0.23	0.68
NW	NB	0.82	0.79	0.82	0.45	0.90	0.83	0.75	0.83	0.44	0.90
	RLR	0.91	0.40	0.98	0.44	0.89	0.91	0.39	0.98	0.43	0.89
	RF	0.91	0.53	0.95	0.49	0.88	0.91	0.57	0.95	0.52	0.88
	AB	0.91	0.55	0.95	0.50	0.84	0.92	0.55	0.97	0.56	0.91
	XGB	0.91	0.55	0.96	0.52	0.91	0.92	0.56	0.97	0.57	0.92

Abbreviations: SR, sleep restriction; TI, tiagabine; AC, acoustics; NW, night working, NB, Naive Bayes; RLR, regularized logistic regression; RF, random forest; AB, adaptive boosting; XGB, extreme gradient boosting; ACC, accuracy; SN, sensitivity; SP, specificity; MCC, Matthews correlation coefficient; AUC, area under curve.

Figure 2 Heatmap of the similarity between participants obtained according to the defined similarity measure. Axes group participants based on the study they were in. Each pixel corresponds to one participant. Dark blue means maximum similarity and yellow means minimum similarity. Participants from the Acoustics study are observed to be different than each other and those from other studies potentially explaining the difference in the performance of our predictive models on them.

Khademi A, El-Manzalawy Y, Buxton OM, Honavar V. Toward personalized sleep-wake prediction from actigraphy. In: 2018 IEEE EMBS International Conference on Biomedical and Health Informatics, BHI 2018 Las Vegas: IEEE; 2018:414–417.

 Google Scholar