Evaluating machine learning models to classify occupants’ perceptions of their indoor environment and sleep quality from indoor air quality

Figures & data

Table 1. Definitions and encoding of input features of the machine learning models.

Feature	Definition
FloorArea	Floor area (ft^2)
CeilingHgt	Ceiling height (ft)
N_Bedrm	Number of bedrooms
N_FullBath	Number of full bathrooms
N_HalfBath	Number of half bathrooms
Oven_Fuel	Oven fuel type (1 – gas, 0 – electric)
N_Occupant	Number of occupants
WHV_AsFound	Status of mechanical whole-house ventilation during first visit (1 – On, 0 – Off)
AF_Condition	Air filter condition (3 – very dirty, 2 – dirty, 1 – used, 0 – clean)
RHD_Grease_Screen	grease screen condition (2 – dirty, 1 – used, 0 – clean)
RHD_Low	Airflow of range hood at lowest fan speed (cfm)
RHD_High	Airflow of range hood at highest fan speed (cfm)
EXF_BA	Airflow of master bathroom exhaust (cfm)
Q50	Airflow at 50 Pa envelope pressure difference (cfm)
T	Temperature (^°C)
RH	Relative humidity (%)
PM	Indoor ^PM2.5 concentration ($\mu$ g/m^3)
PM_Out	Outdoor ^PM2.5 concentration ($\mu$ g/m^3)
CO2	CO2 concentration (ppm)
CO2_BR	CO2 concentration in master bedroom (ppm)
NO2	NO2 concentration (ppb)
NOX	NOx concentration (ppb)
FRM	HCHO concentration (ppb)

Table 2. IAQ satisfaction response encoding.

Likert Scale (Singer et al. 2020)	Text Answer (Zhao et al. 2021)	Encoding
1–2	I am unhappy with it	0
3–4	I feel it could be improved	0
5–6	I feel it’s fine	0
7–9	I am happy with it	1

Figure 1. Questions and possible responses regarding participant sleep quality included on morning EMAs sent four times during the week.

Figure 2. The BEVO Beacon and sensors included on the device.

Table 3. Recommendations used to encode the sleep metrics self-reported on EMAs.

Sleep Metric	Suggestion	Reference
Total Sleep Time	7 to 9 hrs	(Watson et al. 2015)
Sleep Onset Latency	$<$ 15 mins	(Ohayon et al. 2017)
Number of Awakenings	$<$ 2 times	(Ohayon et al. 2017)
Restful	$>$ 1	–

Table 4. Results from 5-fold cross validation and test scores of classifiers trained on varying combinations of features and targets.

Model	Dataset	Targets
		Original	Shuffled	Original
		Features
		All	All	Selected
LR	Train	$0.75\pm 0.02$	0.45	$0.69\pm 0.02$
	Test	$0.49\pm 0.04$	0.47	$0.60\pm 0.08$
NB	Train	$0.71\pm 0.03$	0.49	$0.68\pm 0.03$
	Test	$0.56\pm 0.15$	0.42	$0.57\pm 0.14$
DT	Train	$0.94\pm 0.06$	0.52	$0.99\pm 0.01$
	Test	$0.64\pm 0.06$	0.56	$0.64\pm 0.08$
RF	Train	$0.92\pm 0.02$	0.50	$0.99\pm 0.01$
	Test	$0.58\pm 0.10$	0.51	$0.69\pm 0.09$

Figure 3. Cumulative distribution plots for the feature values used to predict perception of IAQ. Each plot shares the same y-axis values. The first six features represent those related to IAQ while the remaining three, pictured at the bottom, are related to the mechanical or natural ventilation system.

Table 5. Performance metrics of ML models when predicting perception of IAQ.

Model	Accuracy			F1 (threshold)
	Train	Test	Dummy	Test	Dummy
LR	0.69	0.60	0.52	0.65 (0.29)	0.65
NB	0.68	0.57		0.66 (0.21)
DT	0.95	0.66		0.65 (0.34)
RF	0.94	0.69		0.76 (0.4)

Figure 4. Correlation matrix of the selected nine features used to predict IAQ satisfaction.

Figure 5. Relative feature importance when predicting perception of IAQ from the Decision Tree (DT) and Random Forest (RF) models.

Figure 6. Relationship between RHD_Low, Q50, RH and class of IAQ satisfaction (Class 0 – lower IAQ satisfaction, Class 1 – high IAQ satisfaction).

Figure 7. Cumulative distribution plots for the feature values used to predict sleep quality perceptions. Each plot shares the same y-axis values.

Table 6. Performance of ML models when predicting self-reported sleep metrics.

Target	Model	Accuracy			F1 (threshold)
		Train	Test	Dummy	Test	Dummy
TST	LR	0.65	0.64	0.64	0.79 (0.51)	0.78
	NB	0.65	0.64		0.78 (0.51)
	DT	0.65	0.61		0.78 (0.53)
	RF	0.64	0.64		0.78 (0.58)
SOL	LR	0.58	0.41	0.52	0.68 (0.38)	0.68
	NB	0.57	0.44		0.68 (0.36)
	DT	0.70	0.46		0.69 (0.04)
	RF	0.69	0.47		0.69 (0.10)
NAW	LR	0.59	0.46	0.53	0.7 (0.35)	0.69
	NB	0.60	0.46		0.69 (0.38)
	DT	0.72	0.49		0.69 (0.00)
	RF	0.71	0.50		0.69 (0.06)
Restful	LR	0.83	0.83	0.83	0.91 (0.65)	0.91
	NB	0.83	0.83		0.91 (0.66)
	DT	0.83	0.83		0.91 (0.71)
	RF	0.83	0.83		0.91 (0.75)

Table 7. Feature importances from DT and RF models.

Target	Model	Primary	Secondary
TST	DT	PM2.5 (0.38)	T (0.34)
	RF	PM2.5 (0.47)	T (0.15)
SOL	DT	CO (0.26)	TVOC (0.21)
	RF	CO (0.22)	T (0.19)
NAW	DT	PM2.5 (0.33)	CO2 (0.22)
	RF	RH (0.20)	TVOC (0.17)
Restful	DT	PM2.5 (0.30)	CO (0.29)
	RF	PM2.5 (0.45)	RH (0.24)

Singer, B. C., W. R. Chan, Y.-S. Kim, F. J. Offermann, and I. S. Walker. 2020. Indoor air quality in California homes with code-required mechanical ventilation. Indoor Air 30 (5):885–99. doi:10.1111/ina.12676.

 PubMed Web of Science ®Google Scholar

Zhao, H., W. R. Chan, S. Cohn, W. W. Delp, I. S. Walker, and B. C. Singer. 2021. Indoor air quality in new and renovated low-income apartments with mechanical ventilation and natural gas cooking in California. Indoor Air 31 (3):717–29. doi:10.1111/ina.12764.

 PubMed Web of Science ®Google Scholar

Watson, N. F., M. S. Badr, G. Belenky, D. L. Bliwise, O. M. Buxton, D. Buysse, D. F. Dinges, J. Gangwisch, M. A. Grandner, C. Kushida, et al. 2015. Recommended amount of sleep for a healthy adult: A joint consensus statement of the American academy of sleep medicine and sleep research society. J. Clin. Sleep Med. 11 (6):591–92. doi:10.5664/jcsm.4758.

 PubMed Web of Science ®Google Scholar

Ohayon, M., E. M. Wickwire, M. Hirshkowitz, S. M. Albert, A. Avidan, F. J. Daly, Y. Dauvilliers, R. Ferri, C. Fung, D. Gozal, et al. 2017. National sleep foundation’s sleep quality recommendations: First report. Sleep Health. 3 (1):6–19. doi:10.1016/j.sleh.2016.11.006.

 PubMed Web of Science ®Google Scholar

Data Availability Statement

The data that support the findings of this study are available from the corresponding author, HF, upon reasonable request. Data for the first analysis on IAQ and IAQ satisfaction has also been made publicly available by authors in [19] and [20].