Detection of Affective States of the Students in a Blended Learning Environment Comprising of Smartphones

References

• AlZoubi, O., D’Mello, S. K., & Calvo, R. A. (2012). Detecting naturalistic expressions of nonbasic affect using physiological signals. IEEE Transactions on Affective Computing, 3(3), 298–310. https://doi.org/10.1109/T-AFFC.2012.4
   Web of Science ®Google Scholar
• Anolli, L., Mantovani, F., Confalonieri, L., Ascolese, A., & Peveri, L. (2010). Emotions in serious games: From experience to assessment. International Journal of Emerging Technologies in Learning (Ijet), 5(2010), 7–16. https://doi.org/10.3991/ijet.v5s3.1496
   Google Scholar
• Ashwin, T.S., & Guddeti, R. M. R. (2020). Automatic detection of students' affective states in classroom environment using hybrid convolutional neural networks. Education and Information Technologies, 25(2), 1387–1415.
   Google Scholar
• Bartneck, C., & Reichenbach, J. (2005). Subtle emotional expressions of synthetic characters. International Journal of Human-computer Studies, 62(2), 179–192. https://doi.org/10.1016/j.ijhcs.2004.11.006
   Web of Science ®Google Scholar
• Bartram, L., Patra, A., & Stone, M. (2017, May). Affective color in visualization. In Proceedings of the 2017 CHI conference on human factors in computing systems (CHI 2017) (pp. 1364–1374). ACM.
   Google Scholar
• Bauer, G., & Lukowicz, P. (2012, March). Can smartphones detect stress-related changes in the behaviour of individuals? In 2012 IEEE international conference on pervasive computing and communications workshops (pp. 423–426). IEEE.
   Google Scholar
• Beedie, C., Terry, P., & Lane, A. (2005). Distinctions between emotion and mood. Cognition & Emotion, 19(6), 847–878. https://doi.org/10.1080/02699930541000057
   Web of Science ®Google Scholar
• Boonroungrut, C., & Oo, T. T. (2019). Exploring classroom emotion with cloud-based facial recognizer in the Chinese beginning class: A preliminary study. International Journal of Instruction, 12(1), 947–958
   Web of Science ®Google Scholar
• Bosch, N., & D'Mello, S. (2019). Automatic detection of mind wandering from video in the lab and in the classroom. IEEE Transactions on Affective Computing.
   Google Scholar
• Botelho, A. F., Baker, R. S., & Heffernan, N. T. (2017, June). Improving sensor-free affect detection using deep learning. In International conference on artificial intelligence in education (pp. 40–51). Springer.
   Google Scholar
• Camurri, A., Lagerlöf, I., & Volpe, G. (2003). Recognizing emotion from dance movement: Comparison of spectator recognition and automated techniques. International Journal of Human-computer Studies, 59(1–2), 213–225. https://doi.org/10.1016/S1071-5819(03)00050-8
   Web of Science ®Google Scholar
• Chen, Y., Gao, Q., Yuan, Q., & Tang, Y. (2019). Facilitating students’ interaction in MOOCs through timeline-anchored discussion. International Journal of Human-Computer Interaction, 35(19), 1781–1799. https://doi.org/10.1080/10447318.2019.1574056
   Web of Science ®Google Scholar
• Ciman, M., Wac, K., & Gaggi, O. (2015, May). iSenseStress: Assessing stress through human-smartphone interaction analysis. In 2015 9th international conference on pervasive computing technologies for healthcare (PervasiveHealth 2015) (pp. 84–91). IEEE.
   Google Scholar
• Collier, G. L. (2007). Beyond valence and activity in the emotional connotations of music. Psychology of Music, 35(1), 110–131. https://doi.org/10.1177/0305735607068890
   Google Scholar
• D’Mello, S., Jackson, T., Craig, S., Morgan, B., Chipman, P., White, H., Person, N., Kort, B., El Kaliouby, R., Picard, R., & Graesser, A. (2008, June). AutoTutor detects and responds to learners affective and cognitive states. In Workshop on emotional and cognitive issues at the international conference on intelligent tutoring systems (pp. 306–308).
   Google Scholar
• Dao, M. S., Dang Nguyen, D. T., & Kasem, A. (2018). HealthyClassroom - a proof-of-concept study for discovering students’ daily moods and classroom emotions to enhance a learning-teaching process using heterogeneous sensors. In The 7th International Conference on Pattern Recognition Applications and Methods (ICPRAM 2018). (pp. 685–691). Funchal, Madeira, Portugal.
   Google Scholar
• dBaker, R. S., Gowda, S. M., Wixon, M., Kalka, J., Wagner, A. Z., Salvi, A., Aleven, V., Kusbit, G. W., Ocumpaugh, J., & Rossi, L. (2012, June). Towards sensor-free affect detection in cognitive tutor algebra. K. Yacef, O. Zaïane, A. Hershkovitz, M. Yudelson, & J. Stamper (Eds), In Proceedings of the 5th international conference on educational data mining, international educational data mining society. (pp. 126–133).
   Google Scholar
• Debener, S., Minow, F., Emkes, R., Gandras, K., & De Vos, M. (2012). How about taking a low‐cost, small, and wireless EEG for a walk? Psychophysiology, 49(11), 1617–1621. https://doi.org/10.1111/j.1469-8986.2012.01471.x
   PubMed Web of Science ®Google Scholar
• Douglas-Cowie, E., Cowie, R., Sneddon, I., Cox, C., Lowry, O., Mcrorie, M., Martin, J. C., Devillers, L., Abrilian, S., Batliner, A., & Amir, N. (2007, September). The HUMAINE database: Addressing the collection and annotation of naturalistic and induced emotional data. In International conference on affective computing and intelligent interaction (pp. 488–500). Springer.
   Google Scholar
• Epp, C., Lippold, M., & Mandryk, R. L. (2011, May). Identifying emotional states using keystroke dynamics. In Proceedings of the SIGCHI conference on human factors in computing systems (CHI 2011), (pp. 715–724). ACM.
   Google Scholar
• Fu, J., Ge, T., Li, M., & Hu, X. (2019). Affective computation of students’ behaviors under classroom scenes. In NeuroManagement and Intelligent Computing Method on Multimodal Interaction (pp. 1–6). Suzhou, Jiangsu, China.
   Google Scholar
• Gabrielsson, A. (2001). Emotion perceived and emotion felt: Same or different? Musicae Scientiae, 5(1_suppl), 123–147. https://doi.org/10.1177/10298649020050S105
   Web of Science ®Google Scholar
• Gaffary, Y., Jáuregui, D. A. G., Martin, J. C., & Ammi, M. (2015, September). Gestural and postural reactions to stressful event: Design of a haptic stressful stimulus. In 2015 international conference on Affective Computing and Intelligent Interaction (ACII 2015) (pp. 988–992). IEEE.
   Google Scholar
• Gan, C. L., & Balakrishnan, V. (2018). Mobile technology in the classroom: What drives student-lecturer interactions? International Journal of Human-Computer Interaction, 34(7), 666–679. https://doi.org/10.1080/10447318.2017.1380970
   Web of Science ®Google Scholar
• Gao, Y., Bianchi-Berthouze, N., & Meng, H. (2012). What does touch tell us about emotions in touchscreen-based gameplay? ACM Transactions on Computer-Human Interaction (TOCHI), 19(4), 1–30. https://doi.org/10.1145/2395131.2395138
   Web of Science ®Google Scholar
• Gilleade, K. M., & Dix, A. (2004, September). Using frustration in the design of adaptive videogames. In Proceedings of the 2004 ACM SIGCHI international conference on advances in computer entertainment technology (pp. 228–232). ACM, National University of Singapore.
   Google Scholar
• Gitinabard, N., Xu, Y., Heckman, S., Barnes, T., & Lynch, C. F. (2019). How widely can prediction models be generalized? Performance prediction in blended courses. IEEE Transactions on Learning Technologies, 12(2), 184–197. https://doi.org/10.1109/TLT.4620076
   Web of Science ®Google Scholar
• Glowinski, D., Dael, N., Camurri, A., Volpe, G., Mortillaro, M., & Scherer, K. (2011). Toward a minimal representation of affective gestures. IEEE Transactions on Affective Computing, 2(2), 106–118. https://doi.org/10.1109/T-AFFC.2011.7
   Web of Science ®Google Scholar
• González-Gómez, D., Jeong, J. S., & Rodríguez, D. A. (2016). Performance and perception in the flipped learning model: An initial approach to evaluate the effectiveness of a new teaching methodology in a general science classroom. Journal of Science Education and Technology, 25(3), 450–459. https://doi.org/10.1007/s10956-016-9605-9
   Web of Science ®Google Scholar
• Hammerla, N. Y., & Plötz, T. (2015, September). Let’s (not) stick together: Pairwise similarity biases cross-validation in activity recognition. In Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing (UbiComp 2015) (pp. 1041–1051). ACM.
   Google Scholar
• Hazlett, R. L. (2006, April). Measuring emotional valence during interactive experiences: Boys at video game play. In Proceedings of the SIGCHI conference on human factors in computing systems (CHI 2006) (pp. 1023–1026). ACM.
   Google Scholar
• Healey, J. A., & Picard, R. W. (2005). Detecting stress during real-world driving tasks using physiological sensors. IEEE Transactions on Intelligent Transportation Systems, 6(2), 156–166. https://doi.org/10.1109/TITS.2005.848368
   Web of Science ®Google Scholar
• Hertenstein, M. J., Holmes, R., McCullough, M., & Keltner, D. (2009). The communication of emotion via touch. Emotion, 9(4), 566–573. https://doi.org/10.1037/a0016108
   PubMed Web of Science ®Google Scholar
• Kambouropoulos, N., & Staiger, P. K. (2004). Personality and responses to appetitive and aversive stimuli: The joint influence of behavioural approach and behavioural inhibition systems. Personality and Individual Differences, 37(6), 1153–1165. https://doi.org/10.1016/j.paid.2003.11.019
   Web of Science ®Google Scholar
• Khanna, P., & Sasikumar, M. (2010). Recognising emotions from keyboard stroke pattern. International Journal of Computer Applications, 11(9), 1–5. https://d1wqtxts1xzle7.cloudfront.net/45795162/Recognising_Emotions_from_Keyboard_Strok20160519-7171-2jl0y0.pdf?1463727010=&response-content-disposition=inline%3B+filename%3DRecognising_Emotions_from_Keyboard_Strok.pdf&Expires=1608130508&Signature=K9D6npQI4lC7L4ZD7QWHFfpGfYAb83tBtBtFZ1Cv74Drqfjr~slpfd3LQVPhw1Eg34gpYD6nrOK2RbA7oQ~NO0fxPL41rKa7cgFZn2upccsxPuocqIkgWf29MdAOlGdXQj6P1y3AT-NLdXwoQAuBDwsSjiwjBUPBozV5-ADPz1Jd8EcF~pq4qZvd46lkJETJbtTHsri~5ozw0Z4zN~QURiN-DipLYGmCH1qSE6NT43RkbuHFL~S8G5-CW62iEQRJj8aeKPslYSecDMaPkRhCp5cim-gjMrSdzzu5F5JX70FOXIYdusCX9NG-ID6zbo725ioqb3I1OQs-tfK798p~sA__&Key-Pair-Id=APKAJLOHF5GGSLRBV4ZA
   Google Scholar
• Kim, Y., Soyata, T., & Behnagh, R. F. (2018). Towards emotionally aware AI smart classroom: Current issues and directions for engineering and education. IEEE Access, 6, 5308–5331. https://doi.org/10.1109/ACCESS.2018.2791861
   Web of Science ®Google Scholar
• Kolodyazhniy, V., Kreibig, S. D., Gross, J. J., Roth, W. T., & Wilhelm, F. H. (2011). An affective computing approach to physiological emotion specificity: Toward subject‐independent and stimulus‐independent classification of film‐induced emotions. Psychophysiology, 48(7), 908–922. https://doi.org/10.1111/j.1469-8986.2010.01170.x
   PubMed Web of Science ®Google Scholar
• Kwet, M., & Prinsloo, P. (2020). The ‘smart’classroom: a new frontier in the age of the smart university. Teaching in Higher Education, 25(4), 510–526. https://doi.org/10.1080/13562517.2020.1734922
   Web of Science ®Google Scholar
• Lang, P. J., Greenwald, M. K., Bradley, M. M., & Hamm, A. O. (1993). Looking at pictures: Affective, facial, visceral, and behavioral reactions. Psychophysiology, 30(3), 261–273. https://doi.org/10.1111/j.1469-8986.1993.tb03352.x
   PubMed Web of Science ®Google Scholar
• Lee, H., Choi, Y. S., Lee, S., & Park, I. P. (2012, January). Towards unobtrusive emotion recognition for affective social communication. In 2012 IEEE Consumer Communications and Networking Conference (CCNC 2012) (pp. 260–264). IEEE.
   Google Scholar
• Lee, P. M., Tsui, W. H., & Hsiao, T. C. (2014). The influence of emotion on keyboard typing: An experimental study using visual stimuli. Biomedical Engineering Online, 13(1), 81–92. https://doi.org/10.1186/1475-925X-13-81
   PubMedGoogle Scholar
• Li, J., Shi, D., Tumnark, P., & Xu, H. (2020). A system for real-time intervention in negative emotional contagion in a smart classroom deployed under edge computing service infrastructure. Peer-to-Peer Networking and Applications, pp. 1–14. https://doi.org/10.1007/s12083-019-00863-8
   PubMed Web of Science ®Google Scholar
• Li, L., Cheng, L., & Qian, K. X. (2008, September). An e-learning system model based on affective computing. In 2008 international conference on cyberworlds (pp. 45–50). IEEE.
   Google Scholar
• Lim, Y. M., Ayesh, A., & Stacey, M. (2014, August). The effects of typing demand on emotional stress, mouse and keystroke behaviours. In Science and information conference (pp. 209–225). Springer.
   Google Scholar
• Lim, Y. M., Ayesh, A., & Stacey, M. (2020). Continuous stress monitoring under varied demands using unobtrusive devices. International Journal of Human–Computer Interaction, 36(4), 326–340. https://doi.org/10.1080/10447318.2019.1642617
   Web of Science ®Google Scholar
• Loewenstein, G., & Lerner, J. S. (2003). The role of affect in decision making. In R. J. Davidson, K. R. Shekerer, & H. H. Goldsmith (Eds.), Handbook of affective science (pp. 619–642). Oxford University Press.
   Google Scholar
• Lv, H. R., Lin, Z. L., Yin, W. J., & Dong, J. (2008, June). Emotion recognition based on pressure sensor keyboards. In 2008 IEEE international conference on multimedia and expo (pp. 1089–1092). IEEE.
   Google Scholar
• Matsuda, Y., Sakuma, I., Jimbo, Y., Kobayashi, E., Arafune, T., & Isomura, T. (2010). Emotional communication in finger braille. Advances in Human-Computer Interaction, 2010(4), 1–23. https://doi.org/10.1155/2010/830759
   Google Scholar
• Miller, M. K., & Mandryk, R. L. (2016, November). Differentiating in-game frustration from at-game frustration using touch pressure. In Proceedings of the 2016 ACM international conference on interactive surfaces and spaces (pp. 225–234). ACM.
   Google Scholar
• Murray, I. R., & Arnott, J. L. (1993). Toward the simulation of emotion in synthetic speech: A review of the literature on human vocal emotion. The Journal of the Acoustical Society of America, 93(2), 1097–1108. https://doi.org/10.1121/1.405558
   PubMed Web of Science ®Google Scholar
• Nahin, A. N. H., Alam, J. M., Mahmud, H., & Hasan, K. (2014). Identifying emotion by keystroke dynamics and text pattern analysis. Behaviour & Information Technology, 33(9), 987–996. https://doi.org/10.1080/0144929X.2014.907343
   Web of Science ®Google Scholar
• Nortvig, A. M., Petersen, A. K., & Balle, S. H. (2018). A literature review of the factors influencing E-learning and blended learning in relation to learning outcome, student satisfaction and engagement. Electronic Journal of E-learning, 16(1), 46–55. https://files.eric.ed.gov/fulltext/EJ1175336.pdf
   Web of Science ®Google Scholar
• Paquette, L., Baker, R. S., Sao Pedro, M. A., Gobert, J. D., Rossi, L., Nakama, A., & Kauffman-Rogoff, Z. (2014, June). Sensor-free affect detection for a simulation-based science inquiry learning environment. In International conference on intelligent tutoring systems (pp. 1–10). Springer.
   Google Scholar
• Philippot, P. (1993). Inducing and assessing differentiated emotion-feeling states in the laboratory. Cognition and Emotion, 7(2), 171–193. https://doi.org/10.1080/02699939308409183
   PubMed Web of Science ®Google Scholar
• Polivy, J. (1981). On the induction of emotion in the laboratory: Discrete moods or multiple affect states? Journal of Personality and Social Psychology, 41(4), 803–817. https://doi.org/10.1037/0022-3514.41.4.803
   PubMed Web of Science ®Google Scholar
• Popescu, R., Ponescu, D., Roibu, H., & Popescu, L.C. (2018, September). Smart classroom-affective computing in present-day classroom. In 2018 28th EAEEIE Annual Conference (EAEEIE) (pp. 1–9). Hafnarfjordur, Iceland.
   Google Scholar
• Posner, J., Russell, J. A., & Peterson, B. S. (2005). The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Development and Psychopathology, 17(3), 715–734. https://doi.org/10.1017/S0954579405050340
   PubMed Web of Science ®Google Scholar
• Rao, K. S., Kumar, T. P., Anusha, K., Leela, B., Bhavana, I., & Gowtham, S. V. S. K. (2012). Emotion recognition from speech. International Journal of Computer Science and Information Technologies, 3(2), 3603–3607. http://cloud.politala.ac.id/politala/1.%20Jurusan/Teknik%20Informatika/19.%20e-journal/Jurnal%20Internasional%20TI/IJCSIT/Vol%203/ISSUE%202/ijcsit2012030264.pdf
   Google Scholar
• Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161–1178. https://doi.org/10.1037/h0077714
   Web of Science ®Google Scholar
• Sano, A., & Picard, R. W. (2013, September). Stress recognition using wearable sensors and mobile phones. In 2013 humaine association conference on affective computing and intelligent interaction (pp. 671–676). IEEE.
   Google Scholar
• Savvaki, C., Leonidis, A., Paparoulis, G., Antona, M., & Stephanidis, C. (2013, July). Designing a technology–augmented school desk for the future classroom. In International Conference on Human-Computer Interaction (pp. 681–685). Berlin, Heidelberg, Germany.
   Google Scholar
• Shah, S., Teja, J. N., & Bhattacharya, S. (2015). Towards affective touch interaction: Predicting mobile user emotion from finger strokes. Journal of Interaction Science, 3(1), 1–15. https://doi.org/10.1186/s40166-015-0013-z
   Google Scholar
• Söderlund, M., & Rosengren, S. (2008). Revisiting the smiling service worker and customer satisfaction. International Journal of Service Industry Management., 19(5), 552–574. https://doi.org/10.1108/09564230810903460
   Google Scholar
• Sokolova, M. V., & Fernández-Caballero, A. (2015). A review on the role of color and light in affective computing. Applied Sciences, 5(3), 275–293. https://doi.org/10.3390/app5030275
   Google Scholar
• Stein, A., Yotam, Y., Puzis, R., Shani, G., & Taieb-Maimon, M. (2018). EEG-triggered dynamic difficulty adjustment for multiplayer games. Entertainment Computing, 25(2018), 14–25. https://doi.org/10.1016/j.entcom.2017.11.003
   Web of Science ®Google Scholar
• Tikadar, S., Bhattacharya, S., & Tamarapalli, V. (2018, July). A blended learning platform to improve teaching-learning experience. In 2018 IEEE 18th International Conference on Advanced Learning Technologies (ICALT 2018) (pp. 87–89). IEEE.
   Google Scholar
• Västfjäll, D. (2001). Emotion induction through music: A review of the musical mood induction procedure. Musicae Scientiae, 5(1_suppl), 173–211. https://doi.org/10.1177/10298649020050S107
   Web of Science ®Google Scholar
• Vicencio-Moreira, R., Mandryk, R. L., & Gutwin, C. (2015, April). Now you can compete with anyone: Balancing players of different skill levels in a first-person shooter game. In Proceedings of the 33rd annual ACM conference on human factors in computing systems (pp. 2255–2264). ACM.
   Google Scholar
• Vizer, L. M., Zhou, L., & Sears, A. (2009). Automated stress detection using keystroke and linguistic features: An exploratory study. International Journal of Human-Computer Studies, 67(10), 870–886. https://doi.org/10.1016/j.ijhcs.2009.07.005
   Web of Science ®Google Scholar
• Wahyono, I. D., Saryono, D., Ashar, M., & Asfani, K. (2019, September). Face emotional detection using computational intelligence based ubiquitous computing. In 2019 International Seminar on Application for Technology of Information and Communication (iSemantic), (pp. 389–393). Semarang, Jawa Tengah, Indonesia.
   Google Scholar
• Whitehill, J., Serpell, Z., Lin, Y.C., Foster, A., & Movellan, J.R. (2014). The Faces of Engagement: Automatic Recognition of Student Engagementfrom Facial Expressions. IEEE Transactions on Affective Computing, 5(1), 86–98. https://doi.org/10.1109/TAFFC.2014.2316163
   Web of Science ®Google Scholar
• Woolf, B., Burelson, W., & Arroyo, I. (2007, July). Emotional intelligence for computer tutors. In Workshop on modeling and scaffolding affective experiences to impact learning at 13th international conference on artificial intelligence in education, (pp. 6–15).
   Google Scholar
• Woolf, B., Burleson, W., Arroyo, I., Dragon, T., Cooper, D., & Picard, R. (2009). Affect-aware tutors: Recognising and responding to student affect. International Journal of Learning Technology, 4(3–4), 129–164. https://doi.org/10.1504/IJLT.2009.028804
   Google Scholar
• Yannakakis, G. N., & Pavia, A. (2014). Emotion in games. In J. G. D’Mello & A. Kappas (Eds.), Handbook on affective computing (Vol. 2014, pp. 459–471). Oxford University Press.
   Google Scholar
• Zimmermann, P., Guttormsen, S., Danuser, B., & Gomez, P. (2003). Affective computing—a rationale for measuring mood with mouse and keyboard. International Journal of Occupational Safety and Ergonomics, 9(4), 539–551. https://doi.org/10.1080/10803548.2003.11076589
   PubMedGoogle Scholar