IoT-Based Real-Time updating multi-layered learning system applied for a special care context during COVID-19

Abstract

In response to the COVID-19 pandemic and the need for increased research, this study aimed to develop a real-time learning system to provide infection control for residential special care contexts and in doing so, explored different crowdsourcing technologies, spatial usages, and data processing methods within the scope of smart health-care systems and environments. Experiments were conducted in the selected special care indoor environment, which was fitted with sensors and Internet of Things devices, from which generated data were used to train Convolutional Neural Networks, Long-Short Term Memory, and Binary Layered Long-Short Term Memory neural networks. Sequential neural networks were multi-layered and configured in tandem and from these, the real-time updating learning system was developed. The system monitors the user activity and environmental data and predicts critical cases to send alerts to caregivers. Findings showed that stacking neural networks over one another increases the efficiency in updating the training data of real-time learning system. Overall, the study concludes that the developed real-time learning system is lightweight, fast, and efficient for infection control and special care at the private scale and can be multiplied at multiple nodes of larger networks of smart health services and environments.

Keywords:

• COVID-19
• information and communication technology
• internet-of-things
• real-time learning models
• user behaviour
• health-care

PUBLIC INTEREST STATEMENT

In this research, a unique technological solution among the smart systems and sensors is organized together with the advanced artificial intelligence models via a real-time learning system, developed within the design of a special care context. The system is set to crowdsense the critical data about the indoor environment and the usage pattern of participants who needed intense care during the pandemic. The system learns and predicts the user activity to recognize critical circumstances for the infection control and wellbeing of users in the designed indoor space for palliative care during the COVID-19 pandemic. The system operates fast and efficiently and provides critical data from the indoor environment, including the activity patterns of users, and sends alerts to caregivers who take care of the participants.

1. Introduction

The COVID-19 pandemic has increased questions and thus, the number of studies about whether the impact and efficiency of technological and scientific investigations can be improved for health-care purposes. The capacities of public health-care facilities have also been challenged by the pandemic (D’Aeth et al., 2021). Thus, this study aims to develop efficient and practical learning and monitoring systems in indoor environments that help to fight against COVID-19 at the private scale. Accordingly, a real-time learning system is developed and operated in a special care context to support remote monitoring and smart healthcare to achieve greater infection control against COVID-19 and other ailments.

This study supports the idea that data from residences can generate big data for smart health-care monitoring, and regards the significance of multiplying infrastructural capacities at private scale by encouraging active participation and infection control in residential homes (Brookfield et al., 2015; Pham et al., 2018). Therefore, the present work concentrates on the scope of an active design of a private and special care context by investigating the user occupancy patterns in a selected experiment room at participants’ residence for remote health-care monitoring purposes (Luz et al., 2015). Participants were two family members over 65 years of age, one of whom is an asthma patient. The site for the experiment was designed in the private residence of the participants.

The primary objective of this research is to design and develop special care contexts for infection control, healthcare, and greater wellbeing during the COVID-19 pandemic using IoT technologies and learning models that predict users’ activities to provide big data for smart health-care monitoring (Chui et al., 2019). This research encourages the idea of users actively participating in infection control against COVID-19 at the residential scale, and generates ubiquitous big data and critical feedback that can be applied through remote monitoring of e-health and smart health-care systems (Brookfield et al., 2015; Chui et al., 2019; Dai et al., 2019; Hassanien et al., 2019; Mäkinen et al., 2020; Pham et al., 2018).

Thus, the motivation of this research can be briefly expressed as embedding IoT technologies into intelligent spaces (Erişen, 2021) designed for special care contexts in fighting against COVID-19. In this study, the application of intelligent systems in indoor environments for learning and monitoring user behaviours is investigated along with environmental data by surveying the related technological potential and spatial usage scenarios concerning infection control during the pandemic and the need for intensive care. IoT technologies with learning models (Chaumon et al., 2014) were embedded within the specific physical configuration of an indoor space, which was designed for the participants (Vasiliou et al., 2020). Specifically, IoT-based data on user occupancy from indoor spaces are evaluated within a time series using learning models toward a performance analysis and the physical configuration of the sensing layer of the system (Li et al., 2015). The IoT-based learning system identifies the data acquired from sensors located in the experimental environment and predicts user behaviours via the trained deep-learning models (Chung et al., 2021; Dai et al., 2019). The predicted user behaviour was also processed by the learning system to send alerts under critical circumstances.

This study is significant in that it integrates innovative research on real-time learning systems in indoor environments to monitor and predict the activity patterns of users with the environmental data. Similar studies have also surveyed the active design and construction of intelligent spaces for healthcare and infection control (Ahmed et al., 2020; Jens & Gregg, 2021; Pham et al., 2018). The originality of this research can be found in considering the physical setup of smart systems for generating context-based user activity patterns, as classified, and predicted by using real-time deep learning models in indoor spaces for special care. Moreover, real-time learning models that are stacked over and multi-layered, are regarded as innovative in terms of increasing the efficiency of updating the training data of the system, which predicts behaviour patterns. This real-time learning system is also practical for providing big data and sending alerts for critical conditions concerning healthcare and wellbeing (Chui et al., 2019; Hassanien et al., 2019).

This study is beneficial for infection control and the monitoring of activities of users who may need intense care and support at a private and individual scale. In terms of institutional and social benefits, the research encourages active participation of users and care contexts that can be multiplied at different nodes of smart system networks. The developed system is seen as practical and efficient and can be developed in public scale and for infrastructural purposes. The endeavours of this research for a scientific contribution can be highlighted as:

• Spatial and technological potentials are investigated at the private/residential scale for applicability of real-time learning system for smart health infrastructure

• IoT-based modules are designed in a special configuration pattern to generate the real-time data for special care

• Neural Network layers are configured in a tandem configuration, multi-layered for a real-time learning and monitoring system that is unique

• Bi-LSTM layers are tested together with Batch Normalization layer that work efficiently with the acquired IoT data

• The IoT-based modules provide lightweight data for the learning system

• A real-time learning system is developed for special care and monitoring critical activity in a special care context during COVID-19

• The real-time learning system quickly predicts the acquired data and provides feedback about the user activity that can be publicly monitored via the IoT cloud

• The real-time learning system ubiquitously updates training/validation data and trained neural networks in the system by the incoming online inputs

• The developed updating system maintains it accuracy without any need for further training or error regularisation session

This study can also be regarded as a foundation for configuring special care contexts with crowdsourcing techniques from which future works of extreme learning machines for energy-efficient systems, smart grids, e-health, and smart health systems and environments can be developed (Ergen et al., 2020; Erişen, 2021; Shah et al., 2020).

In terms of the remainder of the paper, Section 2 surveys the potential applications of sensorial IoT technologies and learning models for healthcare and wellbeing based on related studies. In Section 3, the design of smart systems together with their special configuration in a selected indoor space is considered for developing learning models of remote healthcare and wellbeing based on the methods and materials applied. In Section 4 , experiments assess the outcomes of IoT data by applying different learning models to develop a real-time learning system. The system predicts user occupancy patterns in real time and sends alerts indicating health conditions and critical circumstances for wellbeing during special care. In Section 5 , the outcomes are evaluated and the user occupancy patterns, configuration aspects, learning models, monitoring and alert system with crowdsourcing are discussed. Section 6 describes the limitations of the study and identifies future work with the potential to apply other technologies and big data. Section 7 is devoted to a brief set of concluding remarks.

2. Literature review

The COVID-19 pandemic forced this study to explore possible technological usages in environments for infection control and healthcare. The research was also inspired by previous works exploring different crowdsourcing techniques, data processing and monitoring technologies in different spatial scales and usage scenarios (Erişen, 2021, p. 115). Moreover, certain prominent researchers have made distinctions between personal and clinical cases in Healthcare IoT projects and have classified these cases in their physical environments (Habibzadeh et al., 2019). Primarily studying IoT technologies and smart systems, this research surveys potential crowdsourcing technologies and data processing and monitoring methods with different usage cases in environments, in the scope of healthcare and special care that can be integrated to smart health and e-health services (Habibzadeh et al., 2019); (Figure 1).

Figure 1. Comparative analyses of the surveyed literature and applied technologies in this research.

IoT and sensor-based technologies have been commonly used for healthcare and wellbeing in smart homes and smart grids (Abbas et al., 2020; Alsamhi & Lee, 2021; Choi et al., 2021; Maceli, 2021). Existing literature on smart systems employed in buildings has been advanced by significant studies on wellbeing and remote monitoring for smart health-care purposes and for special needs associated with self-care and elder care (Brookfield et al., 2015; Chalmers et al., 2019; Chui et al., 2019; Claypool et al., 2019; Deen, 2015; Erişen, 2021; Hassanien et al., 2019; Kawser & Ahmed, 2020; Lehman, 2017; Mäkinen et al., 2020; Shang et al., 2020). Besides familiar IoT technologies that have enabled improvements in the crowdsourcing of environmental data, the occupant behaviour and health in buildings are also vital aspects that should be considered for wellbeing (Abbas et al., 2020; Erişen, 2021; Guerra-Santin & Itard, 2010; Haghi et al., 2020; Nuhu et al., 2021; Roulet et al., 2006; Zhang et al., 2019).

The contexts of special care on user occupancy enable the generation of big data, and the systematisation of big data acquired from the special care contexts has considerable potential for new smart spaces and smart health-care services (Chui et al., 2019; Erişen, 2021; Hassanien et al., 2019; Pham et al., 2018; Sarirete et al., 2021). However, research looking at the prediction and updating of existing data regarding complex occupant behaviours and physical configuration constraints remains a developing area with considerable potential to explore different spatiotemporal usage scenarios (L. Chen et al., 2020; Shao et al., 2020). Hence, this study planned on developing IoT-based real-time updating learning models processing the physical configuration constraints and environmental data to predict user behaviour patterns in indoor spaces for the purposes of wellbeing during special care against the coronavirus pandemic.

The COVID-19 pandemic has compelled researchers to investigate the use of smart systems and learning models (Alsamhi & Lee, 2021; Chadaga et al., 2021; Shamsi et al., 2021). Smart systems and crowdsourcing technologies have been surveyed with greater emphasis on the recognition of user movements, locations, behavioural patterns; data for healthcare using sensors, imaging devices, and wearables, personal and mobile devices, contact tracing applications, GPS, and even EEG and diagnostic devices (Ahmed et al., 2020; Erişen, 2021; Jolfaei et al., 2020; Mäkinen et al., 2020; Mario, 2019; S.L. Wang & Lin, 2019; Y. Wang & Shao, 2018; D. Wu et al., 2020; T. Wu et al., 2020). The challenges associated with the pandemic have also directed researchers’ attention to the specific usages of sensors related to air quality and the activity patterns of users who may require intensive care (Guerra-Santin & Itard, 2010; Roulet et al., 2006). Temperature, humidity, and the air quality of indoor environments are vital for analysing user occupancy patterns and dealing with the challenging conditions required by the elderly, people with Alzheimer’s disease, or asthma patients (Bianchi et al., 2019; Brookfield et al., 2015; Guerra-Santin & Itard, 2010; Jaimini et al., 2017).

The use of machine learning and deep learning models has also been advanced for human activity recognition through the exploration of real-time IoT data and cloud computing (Liang et al., 2018; Pham et al., 2018; S.L. Wang & Lin, 2019; Weixian et al., 2018; Zhou et al., 2020). Thus, crowdsourced IoT data from indoor spaces gathered through the active participation of users (Abbas et al., 2020; Brookfield et al., 2015) can also be applied through machine learning, deep learning, and extreme learning machines (Ergen et al., 2020; D. Wu et al., 2020). It can also be integrated to large-scale transformations in workspaces and smart health-care buildings using similar remote monitoring systems for e-health, smart healthcare, and smart logistics (Chalmers et al., 2019; M. Chen et al., 2019; Chui et al., 2019; Habibzadeh et al., 2019; Haghi et al., 2020; Hassanien et al., 2019; Jens & Gregg, 2021; Ogbuabor et al., 2020; Zhou et al., 2020).

IoT technologies for healthcare and monitoring are also developed through blockchain technologies, remote monitoring tasks, swarms of robots for monitoring and alerting tasks, workspaces, and infrastructural transformation for smart cities (Al-Ali et al., 2017; Alsamhi & Lee, 2021; Hassanien et al., 2019; Jens & Gregg, 2021; Sarirete et al., 2021; Valks et al., 2021). A survey on the potential of learning models and algorithms like that described herein can be applied to different versions of “efficient real-time learning systems” for smart grids, automated IoT-based state-of-the-art systems, and smart infrastructure (Colmenares-Quintero et al., 2021; Ergen et al., 2020; Erişen, 2021; Han et al., 2021; D. Wu et al., 2020; Xu et al., 2014). Accordingly, the learning models developed through IoT-based real-time data from indoor spaces are considered in this study along with their spatial usage scenarios, thereby contributing to research on associated design and performance aspects, as well as healthcare in buildings and smart environments (Figure 1).

3. Methods and material

The distinction between personal and clinical cases in Healthcare IoT projects has been evaluated to make a difference between projects for private and public spaces for healthcare and special care. This distinction is primarily surveyed for the methodology of this study to develop appropriate technologies in the selected experiment space. The study supports the idea that the active participation of users and patients at the residential scale during COVID-19 has significant potential to be multiplied at many different nodes of smart health systems in addition to the existing health-care facilities (Brookfield et al., 2015; Habibzadeh et al., 2019). Thus, with the help of crowdsourcing technologies, data processing methods, and smart monitoring systems, the participation of users from private spaces are thought to generate big data and critical feedback for smart health-care monitoring. This study was conducted with the specific purpose of fighting against COVID-19 at the residential scale by developing a smart learning and monitoring system that predicts user activity and alerts people who provide care to users. Regarding the distinction between private and public care contexts, experiments were conducted in the private house of the two participants, who are over 65 years of age and one of whom is an asthma patient.

Thus, through multiple steps, an IoT-based real-time learning system was developed and applied to monitor the activities of users with special needs for infection control by actively learning and predicting user behaviours and environmental data. The infection control during COVID-19 pandemic at the participants’ homes has led to the consideration of the specific design of the physical configuration of the room enabling to monitor the critical distances for special care purposes but also the development of sensor systems and learning models therein, respectively. Regarding the inhabiting asthma patient, this study considers the air quality and associated data of the indoor environment. This research also considers monitoring and learning the user occupancy for infection control by tracing the activity pattern of the participants in the indoor space, inspired by the design and technology considerations of special care contexts for people with Alzheimer’s or dementia (Brookfield et al., 2015; Chaumon et al., 2014; Hung et al., 2021; Y. Wang & Shao, 2018). Accordingly, the critical distances between users requiring intensive care in specially designed spaces during the pandemic are considered, and user activity patterns are followed with the method of detecting moving agents (Hung et al., 2021).

Overall, this research correlates the data from indoor environments with the usage patterns of the occupants, defined in terms of the physical configuration of smart systems. This correlated behaviour pattern is further processed to classify 10 critical behaviour labels through IoT-based real-time learning models. IoT-based systems were applied to generate big data on user occupancy in the selected care context. This serves as a foundation for crowdsourcing and remote monitoring for smart healthcare, which have been identified as critical and classified by the learning systems to monitor wellbeing (Chalmers et al., 2019; Chui et al., 2019). Thus, the learning system was developed to monitor, learn, and predict behaviour patterns and environmental data as well as to send electronic mail alerts to people that can use IoT clouds such as doctors, caregivers, or family members.

In that regard, this study employed methods to (1) design the selected indoor environment with IoT-based sensor systems, which were then (2) configured through a physical setup to derive critical data to allow for (3) monitoring, learning and prediction by the real-time learning system developed for special care. Starting in November 2020, the experiments were conducted for one year during the COVID-19 pandemic to provide infection control and special care at the residential scale.

3.1. Designing the sensing and networking layer of the learning system

The sensing layer of the system, which was developed to generate real-time IoT data, includes three distinct modules, namely “Module 1”, “Module 2”, and “Module 3” (Figure 2). In designing the modules, ultrasonic sensors were used to track the movement of users and recognise critical distances and user occupancy patterns together with gas, humidity, and temperature sensors to learn data from the selected experiment area. Two modules are connected to the IoT cloud platform and the third one provides its WLAN-based web server to provide the sensing and networking layer of the system, thereby generating training and test data for the learning models (Li et al., 2015); (Figure 2). Learning models were configured in tandem for a real-time learning system which acquires data from the IoT cloud and monitors, learns, and predicts the behaviour as well as gives feedback to the IoT cloud, updating its training and validation data, and sending alerts (Figure 2).

Figure 2. Communication schema of the IoT-based learning system.

Module 1 is built on an Arduino microcontroller with one ultrasonic sensor (HC-SR04), used to track the distance of moving agents within the critical/defined range, in addition to a temperature and humidity sensor, DHT 11 (Figure 2). The MathWorks IoT platform ThingSpeak was accessed using an ESP8266-01 and then ESP32 devices, set on the microcontroller to send data through the networking layer (Figure 2). A microgrid has one LCD showing the data feed to the IoT channel, and experiments were conducted to read motion-tracking data from the IoT Cloud sent by Module 2 (Figure 2).

Module 2 was also designed with an identical ultrasonic sensor to have correlating motion-tracking data with Module 1. The challenging circumstances of the COVID-19 pandemic have illustrated the significance of gas sensors used in indoor environments, which are considered vital in smart health-care environments (Kou et al., 2017). Thus, this module includes a MQ-2 gas sensor to provide critical data regarding the air quality and the levels of particles in the interior environment, including methane (CH₄), butane (C₄H₁₀), liquefied petroleum gas (LPG), smoke, and carbon dioxide (CO₂). To provide crowdsourcing data on the health condition of the user, a remote-control device is also set for grading within the range of 1–5 using the same controller buttons; otherwise, the microgrid sends a defined default value of 888 (Table 1). For correlation with the data from Module 1, all the real-time data generated by Module 2 were connected to the different fields of the same IoT channel. Real-time learning system was also applied later to synchronise and send the serial data, acquired from Module 1 and 2 (Figure 2).

Table 1. Exemplar results of the classification of user behaviour

Six-input data from the IoT Cloud
x(cm)	L(cm)	User-defined health-check value*	Temperature (°C)	Humidity (%)	Gas sensor value	Data number	CASE(Figure 5)	Data Classification Label	Behaviour Label
72.3	54.6	888	25.9	50	566	62935–62936	Case 3	11	Caregiver sitting
57.2	41.1	888	24.6	58	502	65110–65111	Case 3
81	76.26	888	25.3	33	467	16579–16580	Case 2	12	Ventilatingthe room with guest
80	77	888	25.4	36	468	24478–24479	Case 2
81	76.46	888	20.6	32	469	86340–86341	Case 2	13	Temperature-humidity Minimum
81	76.26	888	20.9	32	488	86406–86407	Case 2
86	69.6	888	29.8	51	502	90219–90220	Case 3	14	Temperature-humidity Maximum
101.1	71.5	888	29.8	52	502	90477–90478	Case 3
149.5	102	5	24.6	48	542	62283–62284	Case 3	15	Leaving the room
149	129	5	24.7	53	502	68799–68780	Case 1
143.3	127	888	27.1	50	502	61167–61168	Case 1	16	Entering to the room
129.1	114.4	888	27.1	50	502	61171–61172	Case 1
59.2	48.5	5	24.8	48	517	62333–62334	Anomaly	17	Moving towards the bed
33.2	23.1	4	25.3	49	502	62499–62500	Anomaly
94.3	75.3	5	24.5	48	555	62257–62258	Case 3	18	Gas value decrease
107.1	85.5	5	24.9	48	502	62375–62376	Case 3
101	66.4	888	26.8	54	538	33387–33388	Anomaly	19	Two people in the room
103	66.4	888	26.6	54	534	33431–33432	Anomaly
42.2	34.1	5	24.5	48	552	62267–62268	Anomaly	20	Getting up from the bed
34.9	27.5	5	24.6	48	549	62269–62270	Anomaly

*: Health-check values are defined between 1–5. 888 indicates the default value that the user does not enter any input at that time.

Module 3 was designed discretely and can be controlled by smartphone applications through the HC-06 Bluetooth device (Figure 2). Module 3 is connected to its WLAN using Wi-Fi modules to collect relative user data by creating a networking layer (Ardito et al., 2020; Li et al., 2015). Thus, Module 3 works also as a secure access point (AP) and provides an additional rating and command system using the Wi-Fi module’s web server designed in HTML that can also be designed and used via the WLAN of sophisticated health-care environments (Figure 3).

Figure 3. Designed WLAN-based web server.

3.2. Developing the sensing layer in-place: Physical setup

The physical setup of the sensing layer of the system enables the correlation of the acquired real-time IoT data on movement pattern of users to be monitored for critical interactions and infection control. To derive a physical setup regarding the user behaviour, ultrasonic sensors were configured accordingly to track user movements. The critical distance from the patient’s furniture and distance between users for infection and interaction control was set at 150 cm, which has been recognised within the designed ranges of modules and categorised as critical data for infection and interaction control within the physical configuration of the furniture arrangements. The physical setup was also tested with different configuration angles of modules for a variety of usages and flexibilities of smart systems that can be developed for sophisticated smart health-care environments.

The physical setup is intended to correlate the data from Module 1 and Module 2 connected to the IoT cloud platform in deriving the user occupancy patterns to be learned by the real-time updating system. Accordingly, the physical setup is designed to obtain a certain location pattern by triangulation of the measured distances from modules, such that Module 2 is set on the diagonal axis of the room to intersect and correlate the motion-tracking data with the data from Module 1 (Figure 4). Module 1 is mounted on the wall, as an interactive area for the user, perpendicular to the entrance, allowing a direct view of the entry, whereas Module 2ʹs distance gauge is tilted at a configuration angle, theta (θ), to achieve the longest viewing distance to the entrance (Figure 4a). Module 2 was initially tested with different configuration angles of 23°, 30°, and 35° (Figure 4b). The measurement origins for Modules 1 and 2 are then fixed, based on the placement of the furniture used by the patient (Figure 4). Axial measurements from the modules are calculated accordingly to increase control and prevent infection within the users’ immediate surroundings.

Figure 4. (a) Physical setup: configuration of modules in the experiment room and (b) modules at different configuration locations, angles, and heights.

Figure 5. Distinguished cases of user behaviour: (a) Case 1, (b) Case 2, (c) Case 3, (d) Anomalies.

The angle, theta, was fixed during each experiment to derive correlating movement patterns in following and monitoring the critical user activities. In theory, the measured distances of moving agents from Modules 1 and 2, also illustrated as l and x, are supposed to fit into the Equation (1)(1) $l=x\ast cos\left(\theta \right)$(1) , in terms of the measurement origins (Figure 4).

(1) $l=x\ast cos\left(\theta \right)$(1)

3.3. Developing the real-time updating learning system

In the designated area, the experiments were to assess the influence of this spatial configuration by using the derived activity patterns and data from indoor spaces for an initial classification through machine learning algorithms to develop deep learning models (Chalmers et al., 2019). The processed IoT data were applied then to develop CNN, LSTM, Bi-LSTM, and Bi-LSTM with batch normalisation-based deep neural networks for monitoring and predicting behaviour (Chung et al., 2021; Dai et al., 2019). Learning models are further optimised through ceiling analyses and error regularisation algorithms to obtain an efficient prediction feedback, related to real-time user activity patterns derived from the physical configuration constraints.

A real-time learning system was developed using the trained neural networks to acquire and save online inputs with higher accuracy by predicting user behaviour and environmental data. The system serves to generate big data on labelled predictions of user occupancy that decision makers and caregivers can publicly monitor, and sends alerts to people through a user interface (Figure 2). The developed learning system was configured to respond as flexibly as possible to unknown online data. Thus, sequential neural networks were ordered in a multi-layer tandem configuration. The efficiency of the real-time learning system was accordingly analysed via different neural network prediction scores that are critically evaluated to send alerts.

4. Experiments and results

4.1. Assessing the user behaviour and indoor data

The experiments were first conducted to generate and acquire initial IoT data to classify related activity patterns as well as train learning models. All sensor information about the temperature, humidity, and gas sensor values and inputs for motion tracking in addition to the user ratings regarding their health states are fed to six distinct IoT channel fields as six-input data (Table 1). During the experiments, motion pattern recognition based on the physical configuration of the modules revealed three distinctive common cases, besides some anomalies and specific acts. Figure 5 illustrates these three distinguished cases and some anomalies, classified by the specific measurements from Modules 1 and 2, regarding the measurement origins; physical configuration angle, theta (θ); and the angles of the agents with respect to the modules, phi_n (ψ_n).

In Case 1, the distance of the moving agents from the points of origin perpendicular to the interactive wall fits into Equation (1)(1) $l=x\ast cos\left(\theta \right)$(1) ; Figure 5a). Nevertheless, different activities revealed that Case 2 fits into Equation (2)(2) $l\ast cos\left(\psi \right)=x\ast cos\left(\theta \right)$(2) ; Figure 5b), whereas Case 3 can be defined by Equation (3)(3) $l\ast cos\left({\psi }_1\right)=x\ast cos\left(\theta +{\psi }_2\right)$(3) ; Figure 5c).

(2) $l\ast cos\left(\psi \right)=x\ast cos\left(\theta \right)$(2)

(3) $l\ast cos\left({\psi }_1\right)=x\ast cos\left(\theta +{\psi }_2\right)$(3)

Moreover, anomalies (Case 0) are defined to classify measurements other than Cases 1–3. Anomalies are significant in terms of recognising critical actions, which are used as the specific user behaviours labelled in Table 1, for example, more than one person in the room, by tracking any person within the room, whether in need of care, wanting to visit, taking care of a patient, or interacting with elements in the room for certain treatments (Figure 5d).

All acquired initial IoT-data were processed as 2-input data by machine learning models. MATLAB Regression Learner and Classification Learner were applied to correlate and classify the derived cases on motion recognition. In regression tasks of the derived movement pattern, for instance, the best RMSE result for Case 1 was achieved by Exponential GPR with 0.81, and the best RMSE result for Case 2 was achieved by Matern 5/2 GPR with 0.19. Squared Exponential GPR gave the best RMSE result for Case 3 with 0.33, and Rational Quadratic GRP returned the best RMSE result for “Anomalies” with 15.96. In classification learning tasks, the Fine Gaussian SVM model returned the best accuracy of 98.6 in classifying the four labelled cases.

In addition to the experiments conducted on distance measurements, the correlating changes in room temperature, humidity, and gas sensor values were also separately studied through machine learning models. For instance, in the experiment conducted on 15 November 2020 natural ventilation was provided to a well-heated room in Ankara for 15 minutes at noon, between moments A and B, with an outside temperature of approximately 14°C (Figure 6a). A similar experiment was repeated on 26 April 2021 with an outside temperature of 14.7°C. This time the room was cross-ventilated, giving slightly different results (Figure 6b). Comprehensive experiments conducted on machine learning models revealed that Exponential GPR model returns the best RMSE values in predicting the correlating changes between the gas sensor and air index values. It can be reported that the RMSE result for the experiment conducted on 15 November was 0.39 (Figure 7a), and it was 0.13 for the experiment conducted on 26 April (Figure 7b). These outcomes also provide a substantial basis for an evaluation in energy-based modelling of indoor environments (Guerra-Santin & Itard, 2010) and have been studied further for the development of deep learning models.

Figure 6. Data from the experiments between moments A and B (a) Natural ventilation of the experiment room on November 15 and (b) cross-ventilation of the experiment room on April 26.

Figure 7. Exponential GPR learning outcome for gas sensor values on (a) 15 November, RMSE = 0.39 and (b) 26 April, RMSE = 0.13.

In this research, gas sensor and user-rated health check values were also correlated with the air index data, which are considered critical with regard to user occupancy. This correlation is also used for learning models that are trained through machine learning models and artificial neural networks. Accordingly, user behaviour patterns were classified by correlating motion-tracking and rated health-check values together with air, temperature, and humidity values as six-input real-time data sent to the six distinct channel fields of the IoT cloud¹ (Table 1). After exploring the cases and environmental data through machine learning models and algorithms, these data were further trained using deep learning models to identify and predict 10 different user activity cases as behaviour labels (Table 1). All computational experiments for developing learning models were conducted using a laptop with an Intel® Core™ i5-6200 U CPU at 2.30–2.40 GHz with 4.00 GB of RAM and MATLAB®.

4.2. Training deep neural networks for the real-time learning system

Six-input data from the IoT cloud, including the movement patterns, environmental, and user-defined crowdsensing data, were unfolded and pooled as a 1 × 6 × 1 image dataset for the CNN layers. The same dataset was folded/wrapped in cellular arrays of 6 × n sequential data to be trained using LSTM and Bi-LSTM neural networks. All training, validation, and test data in the datasets were arranged as 50%, 30%, and 20% of the data, respectively. Data on complex occupant behaviours with the environment, classified into 10 behaviour labels, were expressed in some exemplar sequences listed in Table 1.

All neural networks were generated from scratch without any pretraining. The CNN models were trained and tested using prepared image data using SGDM optimizers (Figure 8a). LSTM (Figure 8b) and Bi-LSTM neural networks (Figure 8c) were trained and tested using the Adam optimizer and the same real-time data, processed as sequential inputs. During the experiments, the dataset was explored in-depth for the ceiling analyses of neural networks; the slight confusion among the data classification labels of 12, 13, and 14 (Table 1) was overcome using a batch normalisation layer, particularly in Bi-LSTM neural networks introduced after Bi-LSTM layers. Although it is uncommon to use the batch normalisation layer in sequential neural networks, it enables filtering and normalisation of bi-directional computations and categorisation of classes with minimum and maximum values, particularly for this dataset.

Figure 8. Trained deep learning neural networks for the dataset: (a) CNN, (b) LSTM, and (c) Bi-LSTM.

The performance results of these neural networks are presented in Table 2. First, there is no error regularisation, followed by ceiling analyses. Thus, deep learning models are analysed for the exact precision of the training, validation, and test data until they are fully determined for use in a real-time updating learning system (Table 2). For this optimisation, all neural networks were perpetually re-trained, together with an error regularisation algorithm departing from the initial results. In the error regularisation steps, each error is indexed with the corresponding prediction score of the deep neural networks; erroneous inputs are either excluded from the dataset or swapped with consistent training or validation data. While excluding erroneous data from the datasets, the predicted classification results for behaviour labels and true categorical classes were compared with the unfit predictions to be regularised.

Table 2. Initial training outcomes and ceiling analyses with accuracy results on validation and test datasets

				2-input data			6-input data
Data type	Deep learning model	Learn rate	Number of iterations	Run time (s)	Val.Acc.	TestAcc.	Run time (s)	ValidationAccuracy	TestAcc.	Number of Hidden Units
	Initial results for networks trained from scratch
Image	CNN	0.001	1000	18	79.14	86.27	18	92.59	93.75	256
			2000	31	89.07	89.22	30	88.89	96.88	256
			3000	46	90.4	89.22	40	92.59	96.88	256
			15000	226	97.68	92.16	221	92.59	96.88	256
			30000	472	98.01	92.16	461	92.59	96.88	256
			60000	1225	98.01	91.18	1200	92.59	96.88	256
Sequence	Bi-LSTM		200	17	100	94.12	27	100	80	32
	Bi-LSTM+ Batch Norm.			18	100	75	31	100	90	32
	LSTM			14	100	97.63	21	100	90	32
	Ceiling analyses: optimized networks with error regularisation
Image	CNN	0.001	5000	76	100	100	66	100	100	144
Sequence	Bi-LSTM		1000	61	100	100	22	100	100	144
	Bi-LSTM+ Batch Normalization		500	-	-	-	15	100	100	144
	LSTM		1000	37	100	100	17	100	100	144

4.3. Experiments on the real-time updating learning system

After the ceiling analyses, the optimised neural networks were configured in the real-time learning system depending on their data types as image- and sequence-based neural networks, and according to their performance results regarding the run-time, practicality, and prediction accuracy (Table 2). The trained sequential neural networks were ordered in tandem as multi-layered compounds in this system to increase the real-time prediction efficiency in saving the registered online data (Figures 9 and 10). Overall, the updating neural network system was designed to classify and retain real-time data and exert the data with higher prediction accuracies into the training and validation datasets (Algorithm 1).

Figure 9. Real-time updating of smart learning system with different deep neural networks.

Figure 10. Experimental outcomes of the real-time learning system with deep learning models ordered in tandem with (a) Data Update numbers and (b) System Update numbers in total.

Real-time data pooling is achieved by defining a custom system layer with multiple inputs arraying the incoming data into both the image and sequence inputs (Figure 9). Image data were processed through a CNN, and sequential data were registered to the multi-layered sequential neural networks in order of LSTM, Bi-LSTM, and Bi-LSTM using batch normalisation (Figure 9). For each new input, real-time data prediction in image and sequential learning models are processed to have “data updates”. For such processing, each prediction score is evaluated using threshold values, such as 0.94, for higher precision (Algorithm 1, Figure 10). After conditional operations, real-time data are either registered for the data updates or saved as distinct data for further analyses (Figure 9, Algorithm 1). Data for an update are added to either the training or validation datasets, as decided by the data update number (Algorithm 1).

Therefore, the multilayer configuration applied in this real-time learning system is used to increase the number of data updates to back up the first system model, a CNN and/or an LSTM, to determine an update that the previous layer was unable to find (Figure 9, Algorithm 1). The number of changes in the values of the data updates is used to ensure periodic updates of the pre-existing neural networks in the system (Figure 9, Algorithm 1). The predicted user occupancy, applied as behaviour labels and their prediction scores in percentages, are also resent to different fields of the IoT cloud¹ (Figure 9).

Algorithm 1. Pseudo-code for real-time updating of learning system for data and system updates

n $\in \hspace{0.17em}N$

threshold > 0.94

[m, l] = size(trainingData)

[k, p] = size(validationData)

for i= 1:∞

[Prediction(i), score(i)] = classify(DNN_x,IoTData(i),…)

…

if (score(i) >threshold)

DataUpdate(n) = DataUpdate(n)+1

…

if x= 2n-1

trainingData(m + 1) = IoTData(i)

elseif x= 2n

validationData(k + 1) = IoTData(i)

end

…

else

E_x(i) = IoTData(i)

end

if DataUpdate(n) >SelectedValue

SystemUpdate(n) = SystemUpdate(n)+1

net_x1 = trainnetwork(CNN_x, trainingData,…)

net_x2 = trainnetwork(LSTM_x, trainingData,…)

net_x3 = trainnetwork(Bi-LSTM_x, trainingData,…)

net_x4 = trainnetwork(Bi-LSTM+BatchNorm_x,trainingData,…)

end

end

The efficiency of this learning system was tested using unknown real-time IoT data, initially with one instance for each behaviour label (Table 3, Figure 10a). These data are also saved to test different system options to assess the helping role of other sequential neural networks that are stacked over each other, as ordered in a tandem configuration (Figure 10b), to categorise data missed by the previous model. The outcomes are assessed through “data update numbers” (Figure 10a) as the response of each neural network during this experiment. The changes in the results are expressed in Figure 10 by decreasing the deciding threshold from 0.94 to 0.89 and then to 0.46 to compare different options for this associated learning system.

Table 3. Comparison of prediction results

	Count
Data Classification Label	Initial Experiment for Testing CNN that Updates	Experiment on 24.05.21	Experiment on 16.11.21
11	1	571	0
12	1	0	0
13	1	0	0
14	1	283	0
15	1	85	50
16	1	254	0
17	1	131	0
18	1	100	0
19	1	0	0
20	1	15	0
predicted by the first model	10	1439	50
out of	10	1440	50
Accuracy	100%	99.93%	100%

For a threshold value of 0.94, the LSTM neural network itself could only save three real-time data out of 10 inputs, with a 30% efficiency in data updates, whereas the CNN completely registered the same data as the image inputs. By contrast, three sequential neural networks in the system achieve an 80% success rate in data updates by saving eight inputs out of 10 incoming data, and thus a total success rate of 90% was achieved for the system updates (Figure 10b).

When the threshold was set to 0.89, the real-time data, missed by one of the sequential neural networks, decreased from seven to four of the 10 inputs (Figure 10). For the four-neural-network option, nine out of the 10 inputs could be saved with a 90% success rate in learning new inputs for the data updates. At a threshold value of 0.46, the LSTM neural network could still not retain two inputs of the 10 incoming data, indicating a 20% input loss for the data updates and a 10% input loss for the system updates in total (Figure 10). However, all real-time data were retained by three sequential neural networks at a threshold value of 0.46 for updating the datasets and the system, without any need for further ceiling analyses of each model.

Another significant advantage of the four-neural-network option is that with the help of data and system updates, the system arrives at its mature state in a very short time; this multi-layered architecture enabled the system to train and operate a single neural network, CNN. To explore the recent capabilities,¹ a monitoring experiment was conducted on the system on 24 May 2021. It was observed that 1439 inputs out of 1440 unknown real-time data were retained as new training/validation data at the end of the day when using only a CNN with a threshold value of 0.80 and with an average prediction score accuracy of 99.93% for predicting daily activities after a few updates in the system with the assistance of the other neural networks (Table 3).

Most significantly, the increasing efficiency of the multi-layered system enabled the control and prediction of persistent outcomes of user occupancy that could be subsequently recognised in real time. Interactions (with more than one person) in the room for longer than 2 min, high gas sensor values in a 15-min period, a 1-h long duration for very high or very low temperatures, or day-long low health-check values are processed as persistent circumstances by the system. Accordingly, prediction outcomes for these longer periods are identified as critical for sending alerts through electronic mail to people using an IoT channel (Figure 11). This is seen as vital, particularly for people such as doctors, caregivers, experts, or family members who need to acquire alerts and big data regarding the critical circumstances of users and their wellbeing in indoor spaces (Hassanien et al., 2019; Nuhu et al., 2021).

Figure 11. Alert sent to the electronic mail of the IoT channel user.

5. Discussion

The increasing efficiency in real-time learning systems is regarded as critical to label-categorised activities and provide feedback about the wellbeing of the occupying users, who require intensive care in specially designed indoor spaces. With the rising influence of COVID-19, an inquiry into the air quality and air index values of indoor environments has been seen as equally critical as tracking user movements in developing learning models for healthcare and wellbeing purposes. The user occupancy patterns with air index and gas sensor values imply substantial results for energy-dependent modelling and even the development of IoT-based state-of-the-art air conditioning systems (Carpino et al., 2020; Guerra-Santin & Itard, 2010).

In the development of real-time learning system, the deployment of machine learning models and algorithms also played critical role. Besides the derived equations for three cases, regression models also help to distinguish three cases from anomalies with their RMSE results. Moreover, classification models also perform well with more than 98% accuracy in classifying all four cases that are derived from movement patterns of users. On the other hand, when comparing the performance of AI models with Machine Learning models on the initial 2-input data, most deep learning models, particularly sequential neural networks, perform better (Table 2).

The classification of motion tracking data also reveals the importance of physical configuration, which affects the learning models in the identification of user behaviour; and promises the application of learning outcomes in the active design of spaces for special care, such as hospital rooms. According to the associated physical constraints, the configuration angle, theta, and the distinguished cases of positioning (Figure 5) provide primary evidence for classifying user occupancy as behaviour labels and to develop machine learning models for specific usages. For example, depending on the physical configuration of the modules, anomalies provide trained and learned evidence for deep learning models to recognise and follow critical circumstances, e.g., more than one person in a room closely interacting with the patient. The classification of the four cases is also helpful for error regularisation of deep neural networks in distinguishing different data classification labels, such as 12, 15, and 17, 19, 20, identified through distinct cases (Table 1). Thus, machine learning models were also used to exclude the erroneous data from the dataset, detected as errors or false negatives by the regression and classification models, to optimise the training data and deep learning models in error regularisation steps.

It is also crucial to observe how multi-layered neural networks, optimised and ordered in a tandem configuration in the learning system, respond by different weighted prediction scores to the same real-time data through this research. Although each model is explored through ceiling analyses by achieving the same highest accuracy result (100%) against the training inputs, they show diverse performance in filtering and saving real-time data during the growth of the updating system (Table 2, Figure 10). This also enables the following weaknesses and strengths of each sequential neural network model for each categorical activity. By following the update numbers, weaknesses are found and developed for a versatile updating system against new real-time data without the need for further ceiling analyses. Recent observations revealed that 113,327 data entries to the IoT channel¹ were achieved as big data, and that the system also achieved 100% efficiency in the final session on 16.11.2021 (Table 3). The increasing efficiency in real-time learning systems was also significant as the prediction outcomes for apparent cases were used to reprogram new IoT devices in the same configuration of the experiment space.

Being different from widely used e-health applications, the developed real-time system has also enabled ubiquitous crowdsourcing of user-rated health-check values and the conditions of indoor spaces. For instance, the experiments conducted on 24 May 2021 revealed that 331 rated values were acquired perpetually with an average value of 4.3; and in the remaining 1109 data, the user did not prefer to activate the rating system. Thus, it can be concluded that the system generates online big data for remote smart health-care monitoring, which can be useful for the prediction of long-term health conditions and wellbeing (Jelodar et al., 2020).

6. Limitations and future work

In this research, the selected private indoor environment, with the physical configuration constraints, which was designed for well-being during infection control and healthcare, can also be considered as the primary limitation of this study. The lifecycle habits of the users were repetitive, and the developed learning system learned the activity patterns in a short period of time. This also limits the system to learn new patterns of behaviour even though the system is also developed to be operated in more complicated conditions.

As another limitation, imaging devices were avoided to ensure privacy in indoor spaces and to keep the costs and computational time low (Erişen, 2021; Maceli, 2021), because image processing does not provide evidence for air index and gas sensor values. Nonetheless, a real-time updating system has been developed as an associated learning model (Erişen, 2021) with multi-layered functionality. Thus, visual data can also be processed to train CNNs for versatile usages as another premise of this research. Security measurements for privacy-preserving data analytics (Alaqra et al., 2020) and context-aware data processing methods (Ogbuabor et al., 2020) are left as future studies.

On the other hand, the outcomes of this research imply that the learning system can also be applied through multiples nodes of larger scale smart health systems as well as more sophisticated smart health-care environments, such as hospital rooms. The networking capacity of the centralized learning system is designed to merge data from multiple special care contexts with decentralized IoT devices and modules (Alsamhi & Lee, 2021). The developed IoT-based sensor systems generate lightweight data, and are affordable enough to be distributed to multiple contexts. The real-time learning system returns fast and efficient prediction feedbacks and enables practical monitoring. Thus, in terms of future works, the developed multi-layered real-time learning and updating systems can be multiplied at different nodes of networks of smart health systems and in smart health-care facilities. They can also be applied to predict outcomes within larger-scale spatial and technological organisations by the affordable distribution of trained scalable hardware resources and IoT technologies (Xu et al., 2014) that can be developed through real-time learning systems for special tasks in industry, e-health, and public services.

7. Conclusion

The specific design of a special care context with IoT technologies and the real-time learning models that predict 10 critical behaviour labels to monitor activity and wellbeing is developed for the elderly and people with conditions such as asthma during infection control against COVID-19. The developed real-time learning system monitors, learns, and predicts activity patterns from environmental data, and sends electronic mail alerts to appropriate caregivers. The monitoring interface of the system can be accessed via IoT clouds, and the system returns critical data and feedback to caregivers, doctors, and family members that can instantly monitor the health status and interactions of users. Similar research can encourage the active participation of inhabitants in society to provide big data for smart health-care monitoring (Chui et al., 2019). In this regard, big data from the remote monitoring of wellbeing and infection control at the residential scale also generates the groundwork for larger scale crowdsourcing of e-health and smart health systems based on the active participation of users, as well as doctors, experts, and caregivers (Abbas et al., 2020; Chalmers et al., 2019; Habibzadeh et al., 2019; Ogbuabor et al., 2020).

The design and construction of smart spaces and new technologies are critical and should be multiplied to fight against the pandemic and other extreme circumstances that may require special care. Thus, embedded software of IoT technologies, with real-time updating learning systems in the selected experimental environment at private scale, offers greater potential for generating lightweight big data. Overall, the real-time learning model, developed using IoT data from smart systems with a specific physical setup, learns and derives specific user behaviour and anomalies to recognise critical circumstances, which are processed as alerts. The developed real-time learning system is evaluated as fast and efficient, with extremely lightweight yet comprehensive data with multiple inputs about user activity and facts from indoor space, which are then correlated to learn and predict different circumstances. This efficient real-time learning system enables a practical monitoring and alert system and should thus be distributed at multiple nodes of smart health-care systems and environments.

Abbreviations

Table

AI: Artificial Intelligence	GPR: Gaussian Process Regression	LCD: Liquid Crystal Display
AP: Access Point	GPS: Global Positioning System	LSTM: Long-Short Term Memory
Bi-LSTM: Binary Layered Long-Short Term Memory	HTML: Hyper-Text Mark-up Language	SVM: Support Vector Machine
CNN: Convolutional Neural Network	IoT: Internet of Things	WLAN: Wireless Local Area Network
EEG: Electroencephalography	RMSE: Root-Mean-Squared-Error

Acknowledgements

A brief report of the project conducted with this research is presented to the Ministry of Health of Turkish Republic and approved for scientific research. The author did not claim any funding for this research. The author also confirms that the participants who may be identifiable through the manuscript, who are over 65 years in age, and one is an asthma patient, have been given an opportunity to review the final manuscript and have provided written consent to publish.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The underlying research data for this article can be accessed at https://thingspeak.com/channels/1229234.

Additional information

Funding

There has been no direct funding received for this research.

Notes on contributors

Serdar Erişen

Serdar Erişen is an architect with Ph.D., M.Arch. and B.Arch. from Middle East Technical University. He studied new human-technology relations and intelligent spaces in his doctoral research. He is the author of “Incremental transformation of spatial intelligence from smart systems to sensorial infrastructures”. He also holds a B.Sc. in Business Administration from Anadolu University and a Professional Certificate on Quantum Computing & Quantum Internet from DelftX (joint programme offered by Delft & edx). He also holds a Micromaster Degree in Solar Energy Engineering offered by DelftX. He is a Professional Member of IEEE.

Notes

1. Recent IoT data can be accessed at https://thingspeak.com/channels/1229234