Integrating Physical and Cognitive Ergonomics

The field of human factors and ergonomics (HF/E), since its inception, has been instrumental in developing methods, tools, and solutions when considering cognitive and physical systems independently. However, every human action is orchestrated by mind (and brain) and body interactions. To comprehensively understand how humans (from the neck up and down) interact with their environments, it is necessary to employ approaches that effectively identify, assess, and facilitate development of controls and remedial measures that address these mind-body interactions. The study of physical ergonomics is concerned with human anatomic, anthropometric, physiological, and biomechanical characteristics as they relate to physical work systems. The study of cognitive human factors is concerned with mental processes, such as perception, memory, reasoning, and motor response, as they affect interactions among humans and other elements of a system. Most work systems involve some level of mental or cognitive processing in addition to physical efforts, so that ideally physical and cognitive demands should be considered together when examining human behavior at work. High cognitive demands can influence physical capabilities, and physical demands can influence cognitive processing. So, while HF/E is a highly multidisciplinary field that considers humans relative to some aspect of their work environment, efforts are needed to integrate physical and cognitive subsystems during evaluation and (re)design when considering the human in the context of the work situation. The goal of this special issue is to contribute to the growing body of scientific literature on integrating physical and cognitive ergonomics that brings researchers closer as an interdisciplinary HF/E field. The breadth of topics covered includes studies that focus on quantifying human behavior when interacting with physical and cognitive subsystems, applied research that proposes predictive tools to assess multidimensional work demands, theoretical positions and new methodologies that challenge how we currently examine these interactions, and finally, evidence that highlights business outcomes of the interplay between physical and cognitive processes.

Mental workload, fatigue, and stress, stemming from an overloaded cognitive subsystem, have been shown consistently to affect several aspects of human physical capabilities. For example, cognitive distractors and social stress can alter biomechanical strategies during controlled processes such as upper extremity and low back exertions ( Marras, Davis, Heaney, Maronitis, & Allread, 2000; Mehta & Agnew, 2011; Mehta, Nussbaum, & Agnew, 2012) as well as automated processes such as walking (Beauchet et al., 2003; Osofundiya, Benden, Dowdy, & Mehta, 2016). Neuromuscular performance, such as muscular fatigue and recovery, also deteriorates with mental fatigue and workload (Yoon et al., 2009; Mehta & Agnew, 2012). Mechanistic investigations have revealed cardiovascular, endocrinal, neuromuscular, and perceptual pathways through which “non-biomechanical” work factors affect worker functional capacity. These include, but are not limited to, recruitment of same motor units during separate physical and mental work (Lundberg et al., 2002), increased neuromotor noise (Gemmert & Galen, 1997; Bloemsaat, Meulenbroek, & Galen, 2005) resulting in impaired motor coordination (Mehta & Agnew, 2011; Mehta et al., 2012; Mehta, 2015), altered detection, labeling, and attribution of sensations (Sauter & Swanson, 1996), and interference at the dorsolateral prefrontal cortex that is involved in cognitive processing and isometric motor contractions (Mehta & Parasuraman, 2014; Mehta, 2016).

The impact of exercise on cognitive functions, based on type and duration, is also well established and documented (reviewed in Tomporowski, 2003). Physical exercise influences the amount of attentional resources devoted to a given cognitive task, which follows an inverted U-shaped behavior of differences in physical intensity (Kamijo et al., 2004; Yanagisawa et al., 2010). Previous evidence has also demonstrated that the timing of cognitive tasks during exercise can regulate cognitive performance during dual-task scenarios (Audiffren, Tomporowski, & Zagrodnik, 2009). But what about modality of information processing? HF/E design strategies have regularly exploited sensitizing both auditory and visual systems for signal detection and information dissemination to enhance stimulus salience. In this issue, Pankok, Zahabi, Zhang, and Kaber (2016) examined the effect of physical exertions on concurrent cognitive task performance with different modalities of information presentation. Using traditional ergonomic techniques to assess physical exertions during running, the authors reported that in occupations where workers are heavily physically challenged, presenting cognitive information through visual modality may result in faster processing of inhibition responses than through auditory channels.

Because Pankok et al. (2016) simulated physical exertions in a controlled lab environment using a treadmill, the ecological validity of physical demands may be questioned. Indeed, it is possible that cognitive processes during running differ when one runs on a treadmill (and indoors) versus on a natural outdoor terrain. This, in fact, was examined in the next article in this issue by Blakely, Kemp, and Helton (2016). The authors sought to understand whether differences in natural terrain characteristics impacted cognitive and/or running performance. The study was conducted in naturalistic settings, in even and uneven natural terrains, and a tone-counting working task at two difficulty levels was employed to manipulate cognitive demands. Decrements in running performance were found as the cognitive load increased. Moreover, cognitive performance declined with increasing workload (i.e., greater cognitive difficulty and an uneven terrain).

Findings reported by Pankok et al. (2016) and Blakely et al. (2016) have important implications for warning signal designs in high stress scenarios. This is particularly true in scenarios that are associated with running/exercise and are extremely physically challenging, such as those experienced by law enforcement and military officers. One might inquire if the aforementioned relationships were true for physical tasks that require greater postural stability. Maintaining balance and postural stability, both seated and standing, are critical in minimizing the risk of falls in industrial settings, such as construction and oil and gas operations. The third article in this issue, by Cullen and Agnew (2016), investigated the effects of a multimodal dual-task paradigm (seated balance and auditory discrimination tasks) on performance, physiological, and subjective measures of workload. Contrary to the authors' hypotheses, the balance task did not affect the performance of the auditory task, and the presence of the auditory task led to better balance performance with unstable seating. While performance measures remained unchanged, and physiological workload indicator was only sensitive to the auditory task, subjective workload measures showed the greatest sensitivity to changes in workload due to both balance and the auditory task difficulty levels. As such, the authors underscore the need for employing multiple metrics of workload to better evaluate how operators interact with their multitasking environment.

The fourth article in this issue, by Murata (2016), employed multiple metrics, both behavioral and physiological, to predict subjective drowsiness during a simulated driving task. Driver fatigue and drowsiness is a critical HF/E challenge and current research (to practice) efforts have focused on real-time monitoring systems that can accurately predict driver drowsiness to avoid accidents. Murata (2016) integrated several behavioral measurements, such as neck bending angle, back pressure, and foot pressure, as well as physiological measurements, such as electroencephalography, heart rate variability, and blink frequency, to develop a model that offered ∼97% accuracy to predict subjective drowsiness. The author successfully employed an integrated approach using key information from different operator subsystems (cognitive and physical) during simulated driving tasks, through traditional ergonomics and human factors assessments as well as more recently developed neuroergonomic methods. The feasibility of obtaining different measurements, which vary in the level of technology requirements, intrusiveness, and precision offered, was also discussed in the context of developing real-time monitoring systems for tracking driver drowsiness. The objective of the next article in this issue, by Ye and Pan (2016), was to predict subjective recovery time, which can facilitate efficient physical and mental work shift schedules, using parameters that are feasible to obtain in naturalistic work environments. Their study built upon existing evidence on the interactions between physical and cognitive work demands to develop an estimation tool that utilized gender, relative body mass index, heart rate, perceived functional ability, and physical activity rating score, to predict when during the course of physical recovery workers can safely and effectively resume cognitive work.

One of the common discussion points raised in the aforementioned studies presented in this issue is the need for a better understanding of the impact of physical and cognitive stressors on various human subsystems. Neuroergonomics, the study of brain and behavior at work, is one of the numerous scientific impacts that Raja Parasuraman made on HF/E (Parasuraman, 2003; Parasuraman & Rizzo, 2007; Parasuraman & Wilson, 2008). The two Methods, Models, & Theories articles presented in this issue expand on this new subdomain of HF/E and as such aim to contribute to Raja's legacy. Hancock, Volante, and Szalma (2016) present an extension to Parasuraman's vigilance taxonomy (Parasuraman & Davies, 1977), in an attempt to defeat vigilance decrements, by emphasizing the need for simple design characteristics such as cuing and knowledge of results. Rather than redesigning existing poor designs, the authors recommend that the design of interfaces be informed such that real-world, operationally critical vigilance-inducing displays are never created in the first place. At the same time, displays should be designed in such a fashion that operators do not experience visual fatigue when working. Richter, Crenshaw, Domkin, and Elcadi (2016) present the utility of unique neuroergonomic research in quantifying visual effort, which has the potential to advance visual fatigue research in the emerging domain of visual ergonomics. The authors provide compelling arguments and supporting evidence that both the cognitive processes of the visual system and the biomechanical functions of the neck/shoulder area are impacted during visual fatigue development. The study further demonstrated the utility of functional near infrared spectroscopy (fNIRS) to uncover underlying neural mechanisms of visual fatigue. Another application of fNIRS is discussed subsequently by Cheng, Ayaz, Sun, Tong, and Onaral (2016). Their study employed fNIRS to examine functional connectivity (FC) between motor-related brain regions and high-level cognitive brain regions during distal upper extremity movements. Results of this study indicate that movement intention that requires collective neural efforts from cognitive-and motor-specific regions, during the transition period between rest and hand movement, can be accurately captured through FC changes. These findings have strong implications for improving the precision and latency of anticipation-based brain-computer interfaces, for neuroergonomic research related to human system integration and automation, and various neurorehabilitation approaches.

The final article in this issue, by Garrett et al. (2016), provided a unique perspective on the business impact of integration of physical and cognitive ergonomics. Standing desks have gained recent attention as a workplace intervention to minimize sedentary behavior and improve physical activity among employees. Garrett et al. (2016) examined productivity over time between a group of standing desk users and a seated control group in a call center, and reported significant improvements in productivity with standing desk usage. The authors emphasize that sustainability of office (physical) ergonomics solutions relies on whether these interventions consider cognitive impact on work, such as productivity and task interruption challenges.

The nine articles in this issue highlight studies that have combined facets of physical and cognitive ergonomics to better understand and assess human capabilities at work. However, integration of physical and cognitive ergonomics can simply be facilitated with how we design studies, analyze, and report study findings. For example, when investigating cognitive performance of different warning signal designs, studies should consider physical activity levels of the participant pool, physical environmental impact, and psychomotor requirements of the task. Similarly, in studies that focus on traditional physical ergonomic issues, such as lifting, the cognitive processes associated with the different task features (e.g., known versus unknown weights), use of visual/auditory aids for speed of work, etc., should be considered. Such integration efforts are even more critical when examining worker fatigue, operator situation awareness, and decision making, as well as understanding etiology of work-related musculoskeletal disorders.

ACKNOWLEDGMENT: IN MEMORY OF PROFESSOR RAJA PARASURAMAN

My most inspiring memory of Raja Parasuraman was when he shared his joy of publishing his first “physical ergonomics” article in 2014. After my initial surprise at this reaction of pure academic joy from someone who is truly a prolific scientist, I began to understand why. Raja was an “interdisciplinarian.” Raja integrated the field of human factors and ergonomics (HF/E) and neuroscience to develop a thriving new field of neuroergonomics, the study of brain and behavior at work, that has found home in several cognitive and physical ergonomic investigations alike. Thus, recognizing the mounting scientific evidence and the need for further examinations on the interactions of physical and cognitive demands on operator health and performance, Raja took charge to co-develop this special issue on “Integrating Physical and Cognitive Ergonomics.” He inspired me, and many others, through his science, his advice and mentorship, and (simply) his company. He challenged us to question the status quo and bridge disciplines such that the true nature of how and why humans and their various subsystems interact with complex work systems can be understood. After all, HF/E is a fusion of numerous (life) sciences. This special issue is dedicated to Raja and his efforts of bringing the HF/E community closer together through innovation and integration in theories, approaches, and tools. We miss you and hope to make you proud.