An epidemiological appraisal instrument – a tool for evaluation of epidemiological studies

Abstract

In recent years, ergonomics practices have increasingly relied upon the knowledge derived from epidemiological studies. In this regard, there is limited research devoted to the exclusive evaluation of the methodological qualities of ergonomics epidemiological studies. The aim of this study was to develop and test a general purpose ‘epidemiological appraisal instrument’ (EAI) for evaluating the methodological quality of existing or new ergonomic epidemiological studies using a critical appraisal system rooted in epidemiological principles. A pilot EAI version was developed and tested by a team of epidemiologists/physicians/biostatisticians, with the team leader being both epidemiologist and ergonomist. The pilot version was further tested with regard to other raters with/without a background in epidemiology, biostatistics and ergonomics. A revised version was evaluated for criterion validity and reliability. An assessor with a basic background in epidemiology and biostatistics would be able to correctly respond on four out of five questions, provided that subject matter expertise is obtained on specific items. This may improve with the article's quality. Training may have an effect upon assessors with virtually no background in epidemiology/biostatistics, but with a background in ergonomics. In this latter case, the inter-rater degree of agreement is largely above 90% and assessors can resolve their differences in a subsequent round. The EAI proved to be a valid and reliable appraisal instrument that may be used in various applications, such as systematic reviews and meta-analyses.

Keywords:

• Design
• Epidemiological appraisal
• Evaluation
• Observational studies