Inter-observer reliability of forceful exertion analysis based on video-recordings

Abstract

The objectives were to examine inter-observer reliability of job-level forceful exertion analyses and temporal agreement of detailed time study results. Three observers performed the analyses on 12 different jobs. Continuous duration, frequency and % time of lifting, pushing/pulling, power and pinch gripping exertions and estimated level of the exertions were obtained. Intraclass correlation coefficient and variance components were computed. Temporal agreement analyses of raw time study data were performed. The inter-observer reliability was good for most job-level exposure parameters (continuous duration, frequency and % time of forceful exertions), but only fair to moderate for the estimated level of forceful exertions. The finding that the between-observer variability was less than the between-exertion variability confirmed that the forceful exertion analysis method used in the present study can detect job exertion differences.Using three observers to perform detailed time studies on task activities and getting consensus of the majority can increase the between-observer agreement up to 97%.

Statement of Relevance: The results inform researchers that inter-observer reliability for job-level exposure measurement of forceful exertion analysis obtained from detailed time studies is generally good, but the observers' ability in the estimation of forceful exertion level can be poor. It also provides information on the temporal agreement of detailed forceful exertion analysis and guidelines on achieving better agreement for studies where accurate synchronisation of task activities and direct physiological/biomechanical measurements is crucial.

Keywords:

• forceful exertion
• inter-rater reproducibility
• observation method
• temporal agreement
• time study

Acknowledgement

We acknowledge the important contributions of Ruby Irving, Benjamin Hamilton, Cindy Orr, Jessica Keller and Larry Taingin assisting with the data collection in the field and of Caroline Smith in coordinating the field work. Blazej Neradilek provided statistical assistance. This research was funded in part by the US National Institute for Occupational Safety and Health (OH1007316) and the Washington State Department of Labor and Industries.