A framework for operator –  workstation interaction in Industry 4.0

References

• American Psychiatric Association. 2013. Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Arlington, VA: American Psychiatric Publishing.
   Google Scholar
• Antonakaki, P., D. Kosmopoulos, and S. J. Perantonis. 2009. “Detecting Abnormal Human Behaviour Using Multiple Cameras.” Signal Processing 89 (9): 1723–1738.
   Web of Science ®Google Scholar
• Ardanza, A., A. Moreno, A. Segura, M. de la Cruz, and D. Aguinaga. 2019. “Sustainable and Flexible Industrial Human Machine Interfaces to Support Adaptable Applications in the Industry 4.0 Paradigm.” International Journal of Production Research 57: 4045–4059.
   Web of Science ®Google Scholar
• Atallah, L., and G.-Z. Yang. 2009. “The use of Pervasive Sensing for Behaviour Profiling — a Survey.” Pervasive and Mobile Computing 5 (5): 447–464.
   Web of Science ®Google Scholar
• Baraglia, J., M. Cakmak, Y. Nagai, R. P. Rao, and M. Asada. 2017. “Efficient Human-Robot Collaboration: When Should a Robot Take Initiative?” The International Journal of Robotics Research. doi:10.1177/0278364916688253.
   PubMed Web of Science ®Google Scholar
• Behoora, I., and C. S. Tucker. 2015. “Machine Learning Classification of Design Team Members’ Body Language Patterns for Real Time Emotional State Detection.” Design Studies 39: 100–127.
   Web of Science ®Google Scholar
• Bergasa, L. M., D. Almería, J. Almazán, J. J. Yebes, and R. Arroyo. 2014. “Drivesafe: An app for alerting inattentive drivers and scoring driving behaviors.” Intelligent Vehicles Symposium Proceedings IEEE, June, 240–245.
   Google Scholar
• Brunelli, R., and D. Falavigna. 1995. “Person Identification Using Multiple Cues.” IEEE Transactions on Pattern Analysis and Machine Intelligence 17 (10): 955–966.
   Web of Science ®Google Scholar
• Buer, S. V., J. O. Strandhagen, and F. T. S. Chan. 2018. “The Link Between Industry 4.0 and Lean Manufacturing: Mapping Current Research and Establishing a Research Agenda.” International Journal of Production Research 56 (8): 2924–2940.
   Web of Science ®Google Scholar
• Burger, N., M. Demartini, F. Tonelli, F. Bodendorf, and C. Testa. 2017. “Investigating Flexibility as a Performance Dimension of a Manufacturing Value Modeling Methodology (MVMM): A Framework for Identifying Flexibility Types in Manufacturing Systems.” Procedia CIRP 63: 33–38.
   Google Scholar
• Calvo, M. G., A. Gutiérrez-García, A. Fernández-Martín, and L. Nummenmaa. 2014. “Recognition of Facial Expressions of Emotion is Related to Their Frequency in Everyday Life.” Journal of Nonverbal Behavior 38 (4): 549–567.
   Web of Science ®Google Scholar
• Cambria, E. 2016. “Affective Computing and Sentiment Analysis.” IEEE Intelligent Systems 31 (2): 102–107.
   Web of Science ®Google Scholar
• Clark, C. C., C. M. Barnes, G. Stratton, M. A. McNarry, K. A. Mackintosh, and H. D. Summers. 2017. “A Review of Emerging Analytical Techniques for Objective Physical Activity Measurement in Humans.” Sports Medicine 47 (3): 439–447.
   PubMed Web of Science ®Google Scholar
• Clarke, Sr., J. R., & Clarke, P. M. 1997. U.S. Patent No. 5,689,241. Washington, DC: U.S. Patent and Trademark Office.
   Google Scholar
• Cohen, P. R., and E. A. Feigenbaum, eds. 2014. The Handbook of Artificial Intelligence (Vol. 3). Oxford: Butterworth-Heinemann.
   Google Scholar
• Cohen, Y., M. Golan, G. Singer, and M. Faccio. 2018. “Workstation–Operator Interaction in 4.0 Era: WOI 4.0.” 16th IFAC Symposium on Information Control Problems in Manufacturing (INCOM-2018); Bergamo Italy; Session: “MoAT5 - Assembly System 4.0”; June 2018.
   Google Scholar
• Crowley-Koch, B. J., and R. V. Houten. 2013. “Automated Measurement in Applied Behavior Analysis: A Review.” Behavioral Interventions 28 (3): 225–240.
   Web of Science ®Google Scholar
• Crump, M. J., and G. D. Logan. 2013. “Prevention and Correction in Post-Error Performance: An Ounce of Prevention, a Pound of Cure.” Journal of Experimental Psychology: General 142 (3): 692–709.
   PubMed Web of Science ®Google Scholar
• Damiani, L., M. Demartini, G. Guizzi, R. Revetria, and F. Tonelli. 2018. “Augmented and Virtual Reality Applications in Industrial Systems: A Qualitative Review Towards the Industry 4.0 era.” IFAC-PapersOnLine 51 (11): 624–630.
   Google Scholar
• Dellaert, F., T. Polzin, and A. Waibel. 1996. “Recognizing Emotion in Speech.” Proceedings of ICSLP 1996, Philadelphia, PA, 1970–1973.
   Google Scholar
• Demartini, M., F. Tonelli, L. Damiani, R. Revetria, and L. Cassettari. 2017. “Digitalization of Manufacturing Execution Systems: The Core Technology for Realizing Future Smart Factories.” Proceedings of the Summer School Francesco Turco, 326–333.
   Google Scholar
• Draganski, B., C. Gaser, G. Kempermann, H. G. Kuhn, J. Winkler, C. Büchel, and A. May. 2006. “Temporal and Spatial Dynamics of Brain Structure Changes During Extensive Learning.” Journal of Neuroscience 26 (23): 6314–6317.
   PubMed Web of Science ®Google Scholar
• Du, Y., Q. Hu, D. Chen, and P. Ma. 2011. “Kernelized Fuzzy Rough Sets Based Yawn Detection for Driver Fatigue Monitoring.” Fundamenta Informaticae 111 (1): 65–79.
   Web of Science ®Google Scholar
• Eyben, F., M. Wollmer, T. Poitschke, B. Schuller, C. Blaschke, B. Farber, and N. Nguyem-Thien. 2010. “Emotion on the Road – Necessity, Acceptance and Feasibility of Affective Computing in the car.” Advances in Human–Computer Interaction 2010. doi:10.1155/2010/263593.
   Google Scholar
• Eysenck, M. 2012. Attention and Arousal: Cognition and Performance. Heidelberg, NY: Springer Science & Business Media.
   Google Scholar
• Fink, P., E. Ørnbøl, M. S. Hansen, L. Søndergaard, and P. De Jonge. 2004. “Detecting Mental Disorders in General Hospitals by the SCL-8 Scale.” Journal of Psychosomatic Research 56 (3): 371–375.
   PubMed Web of Science ®Google Scholar
• Gershwin, S. B. 2018. “The Future of Manufacturing Systems Engineering.” International Journal of Production Research 56 (1-2): 224–237.
   Web of Science ®Google Scholar
• Gorecky, D., M. Schmitt, M. Loskyll, and D. Zühlke. 2014. “Human-machine-interaction in the Industry 4.0 era.” 12th IEEE International Conference on industrial Informatics (INDIN) (2014), Porto Alegre, 289–294.
   Google Scholar
• Grant, A. M. 2008. “Does Intrinsic Motivation Fuel the Prosocial Fire? Motivational Synergy in Predicting Persistence, Performance, and Productivity.” Journal of Applied Psychology 93 (1): 48–58.
   PubMed Web of Science ®Google Scholar
• Happy, S. L., and A. Routray. 2015. “Automatic Facial Expression Recognition Using Features of Salient Facial Patches.” IEEE Transactions on Affective Computing 6 (1): 1–12.
   Web of Science ®Google Scholar
• He, M., B. Guo, H. Chen, A. Chin, J. Tian, and Z. Yu. 2016. “WhozDriving: Abnormal Driving Trajectory Detection by Studying Multi-Faceted Driving.” BigCom 2016, LNCS 9784: 135–144.
   Google Scholar
• Hu, Chao, M. Q. Meng, P. X. Liu, and X. Wang. 2003. “Visual Gesture Recognition for Human-machine Interface of Robot Teleoperation.” Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453), Vol. 2, 1560–1565.
   Google Scholar
• Hülse, M., and M. Hild. 2010. “Informatics for Cognitive Robots.” Advanced Engineering Informatics 24 (1): 2–3.
   Web of Science ®Google Scholar
• Jiao, Y., Y. Peng, B. L. Lu, X. Chen, S. Chen, and C. Wang. 2014. “Recognizing Slow eye Movement for Driver Fatigue Detection with Machine Learning Approach.” IJCNN (2014). International Joint Conference on neural networks, IEEE, July, 4035–4041.
   Google Scholar
• Kanfer, R., and P. Ackerman. 2000. “Individual Differences in Work Motivation: Further Explorations of a Trait Framework.” Applied Psychology 49 (3): 470–482.
   Web of Science ®Google Scholar
• Larovyi, S., J. L. M. Lastra, R. Haber, and R. del Toro. 2015. “From Artificial Cognitive Systems and Open Architectures to Cognitive Manufacturing Systems.” 2015 IEEE 13th International Conference on industrial Informatics (INDIN), IEEE, 1225–1232.
   Google Scholar
• Layons, M. L., S. Akamatsu, M. Kamachi, and J. Gyoba. 1998. “Coding Facial Expressions with Gabor Wavelets.” Proceedings, third IEEE International Conference on automatic face and gesture Recognition, Nara, Japan. IEEE Computer Society, April 14–16, 200–205.
   Google Scholar
• Lee, C. M., S. Narayanan, and R. Pieraccini. 2001. “Recognition of Negative Emotion in the Human Speech Signals.” Workshop on auto. speech recognition and understanding, Dec. 2001.
   Google Scholar
• Li, G., and W. Y. Chung. 2013. “Detection of Driver Drowsiness Using Wavelet Analysis of Heart Rate Variability and a Support Vector Machine Classifier.” Sensors 13 (12): 16494–16511.
   PubMed Web of Science ®Google Scholar
• Liu, J., Y. Xiao, Q. Hao, and K. Ghaboosi. 2009. “Bio-inspired Visual Attention in Agile Sensing for Target Detection.” International Journal of Sensor Networks 5 (2): 98–111.
   Web of Science ®Google Scholar
• Lora, M. 2017. “Validation of HMI Applications for Industrial Smart Display.” 2017 IEEE international high level design validation and test workshop (HLDVT), 23–30.
   Google Scholar
• Luis, J., V. Pelaez, G. Lopez, M. A. Fernandez, E. Elvarez, and G. Diaz. 2014. “An Automatic Data Mining Method to Detect Abnormal Human Behaviour Using Physical Activity Measurements.” Pervasive and Mobile Computing 15: 228–241.
   Web of Science ®Google Scholar
• Luneski, A., and E. K. P. D. Bamidis. 2010. “Affective Medicine: A Review of Affective Computing Efforts in Medical Informatics.” MIM 49 (3): 207–218.
   Google Scholar
• Montag, C., and J. Panksepp. 2016. “Primal Emotional-Affective Expressive Foundations of Human Facial Expression.” Motivation and Emotion 40 (5): 760–766.
   Web of Science ®Google Scholar
• Murukesan, L., M. Murugappan, M. Iqbal, and K. Saravanan. 2014. “Machine Learning Approach for Sudden Cardiac Arrest Prediction Based on Optimal Heart Rate Variability Features.” Journal of Medical Imaging and Health Informatics 4 (4): 521–532.
   Web of Science ®Google Scholar
• Omidyeganeh, M., A. Javadtalab, and S. Shirmohammadi. 2011. “Intelligent Driver Drowsiness Detection Through Fusion of Yawning and eye Closure.” 2011 IEEE International Conference on virtual environments human-computer interfaces and measurement systems (VECIMS), IEEE, 1–6.
   Google Scholar
• Onnasch, L., C. D. Wickens, H. Li, and D. Manzey. 2014. “Human Performance Consequences of Stages and Levels of Automation: An Integrated Meta-Analysis.” Human Factors: The Journal of the Human Factors and Ergonomics Society 56 (3): 476–488.
   PubMed Web of Science ®Google Scholar
• Park, K. S. 2014. Human Reliability: Analysis, Prediction, and Prevention of Human Errors. Vol. 7. New York, NY: Elsevier.
   Google Scholar
• Pekrun, R., and M. Frese. 1992. “Emotions in Work and Achievement.” International Review of Industrial and Organizational Psychology 7: 153–200.
   Google Scholar
• Perera, C., A. Zaslavsky, P. Christen, and D. Georgakopoulos. 2014. “Context Aware Computing for the Internet of Things: A Survey.” IEEE Communications Surveys & Tutorials 16 (1): 414–454.
   Web of Science ®Google Scholar
• Picard, R. W., E. Vyzas, and J. Healey. 2001. “Toward Machine Emotional Intelligence: Analysis of Affective Physiological State.” IEEE Transactions on Pattern Analysis and Machine Intelligence 23 (10): 1175–1191.
   Web of Science ®Google Scholar
• Pinto, J. M. O., P. F. Melo, and P. L. C. Saldanha. 2015. “Human-machine Interface (HMI) Scenario Quantification Performed by ATHEANA, a Technique for Human Error Analysis.” In Safety and Reliability of Complex Engineered Systems, edited by L. Podofillini, B. Sudret, B. Stojadinovic, E. Zio, and W. Kröger, 3111–3118. London: CRC Press.
   Google Scholar
• Poria, S., E. Cambria, R. Bajpai, and A. Hussain. 2017. “A Review of Affective Computing: From Unimodal Analysis to Multimodal Fusion.” Information Fusion 37: 98–125.
   Web of Science ®Google Scholar
• Rautaray, S. S., and A. Agrawal. 2015. “Vision Based Hand Gesture Recognition for Human Computer Interaction: A Survey.” Artificial Intelligence Review 43 (1): 1–54.
   Web of Science ®Google Scholar
• Romero, D., J. Stahre, T. Wuest, O. Noran, P. Bernus, Å Fast-Berglund, and D. Gorecky. 2016. Towards an Operator 4.0 Typology: A Human-Centric Perspective on the Fourth Industrial Revolution Technologies. CIE46, 1–11.
   Google Scholar
• Salmeron-Majadas, S., M. Arevalillo-Herráez, O. C. Santos, M. Saneiro, R. Cabestrero, P. Quirós, D. Arnau, and J. G. Boticario. 2015. “Filtering of Spontaneous and low Intensity Emotions in Educational Contexts.” In Artificial Intelligence in Education AIED 2015. Lecture Notes in Computer Science, vol 9112. edited by C. Conati, N. Heffernan, A. Mitrovic, and M. Verdejo, 429–438. Cham: Springer.
   Google Scholar
• Schutte, N. S., J. M. Malouff, and E. B. Thorsteinsson. 2013. “Increasing Emotional Intelligence Through Training: Current Status and Future Directions.” International Journal of Emotional Education 5 (1): 56.
   Google Scholar
• Shen, L., M. Wang, and R. Shen. 2009. “Affective e-Learning: Using “Emotional” Data to Improve Learning in Pervasive Learning Environment.” Educational Technology & Society 12 (2): 176–189.
   Web of Science ®Google Scholar
• Shruthi, S., K. Sona, and S. K. Kumar. 2016. “Classification on Hand Gesture Recognition and Translation From Real Time Video Using svm-knn.” International Journal of Applied Engineering Research 11 (8): 5414–5418.
   Google Scholar
• Strauch, B. 2017. Investigating Human Error: Incidents, Accidents, and Complex Systems. New York: CRC Press.
   Google Scholar
• Strozzi, F., C. Colicchia, A. Creazza, and C. Noè. 2017. “Literature Review on the ‘Smart Factory’ Concept Using Bibliometric Tools.” International Journal of Production Research 55 (22): 6572–6591.
   Web of Science ®Google Scholar
• Sudhkar, R. S., and M. C. Anil. 2016. “Emotion Detection of Speech Signals with Analysis of Salient Aspect Pitch Contour.” Emotion 3 (10): 138–142.
   Google Scholar
• Tao, J., and T. Tan. 2005. “Affective Computing: A Review.” In Affective Computing and Intelligent Interaction, ACII 2005. Lecture Notes in Computer Science, vol 3784. edited by R. W. Picard, 981–995. Berlin, Heidelberg: Springer.
   Google Scholar
• Wang, Z., X. Lou, Z. Yu, B. Guo, and X. Zhou. 2019. “Enabling non-Invasive and Real-Time Human-Machine Interactions Based on Wireless Sensing and fog Computing.” Personal and Ubiquitous Computing 23 (1): 29–41.
   Web of Science ®Google Scholar
• Wittenberg, C. 2016. “Human CPS Interaction Requirements and Human Machine Interaction Methods for the Industry 4.0.” IFAC PapersOnLine 49–19. 420–425.
   Google Scholar
• Yamamoto, E., S. Nakamura, and K. Shikano. 1998. “Lip Movement Synthesis from Speech Based on Hidden Markov Models.” Speech Communication 26: 105–115.
   Web of Science ®Google Scholar
• Yin, Y., K. E. Stecke, and D. Li. 2018. “The Evolution of Production Systems from Industry 2.0 Through Industry 4.0.” International Journal of Production Research 56 (1-2): 848–861.
   Web of Science ®Google Scholar
• Zhao, Y. Z., and X. Xu. 2010. “Enabling Cognitive Manufacturing Through Automated on-Machine Measurement Planning and Feedback.” Advanced Engineering Informatics 24 (3): 269–284.
   Web of Science ®Google Scholar
• Zhao, Q., X. Yuan, D. Tu, and J. Lu. 2015. “Eye Moving Behaviors Identification for Gaze Tracking Interaction.” Journal on Multimodal User Interfaces 9 (2): 89–104.
   Web of Science ®Google Scholar
• Zhou, K., T. Liu, and L. Zhou. 2015. “Industry 4.0: Towards Future Industrial Opportunities and Challenges.” The 12th international conference on Fuzzy systems and knowledge Discovery (FSKD).
   Google Scholar
• ZiLin, L., G. Tan, Y. Pang, Y. Tang, and K. Qian. 2017. Driving Fatigue Detection Based on Blink Frequency and Eye Movements (No. 2017-01-1443). SAE Technical Paper, 1–9.
   Google Scholar