Observed vs. self-reported agricultural activities: Evaluating 24-hr recall in a pilot study

Abstract

Few studies have evaluated the validity of self-report of work activities because of challenges in obtaining objective measures. In this study, farmers’ recall of the previous day’s agricultural activities was compared to activities observed by field staff during air monitoring. Recall was assessed in 32 farmers from the Biomarkers of Exposure and Effect in Agriculture Study, a subset of a prospective cohort study. The farmers participated in 56 visits that comprised air monitoring the day before an interview. The answers for 14 agricultural activities were compared to activities observed by field staff during air monitoring (median duration 380 min, range 129–486). For each task, evaluated as yes/no, overall agreement, sensitivity, specificity, and kappa were calculated. Median prevalence of the 14 activities was 8% from observation and 13% from participants (range: 2–54%). Agreement was generally good to perfect, with a median overall agreement of 95% (range: 89–100%), median sensitivity of 84% (50–100%), median specificity of 95% (88–100%), and median kappa of 0.65 (0.31–1.0). Reasons for disagreement included activities occurring when the field staff was not present (i.e., milking cows), unclear timing notes that made it difficult to determine whether the activity occurred the day of and/or day before the interview, definition issues (i.e., participant included hauling in the definition of harvesting), and difficulty in observing details of an activity (i.e., whether hay was moldy). This study provides support for accurate participant recall the day after activities.

Keywords:

• Farming tasks
• occupational questionnaires
• recall validity

Acknowledgments

Amy Miller, Kate Torres, Sarah Woodruff, Marsha Dunn, and other staff at Westat, Inc. (Rockville, MD) contributed to the study coordination and data management; Susan Viet contributed to the expert review of the activity logs. We also thank Anne Taylor from Information Management Services, Inc. (Rockville, MD) for data management support, and the field research team in Iowa, including Charles Lynch, Debra Lande, Debra Podaril, and Jennifer Hamilton. Finally, we gratefully acknowledge the participation of the Biomarkers of Exposure and Effect in Agriculture Study participants that made this work possible.

Additional information

Funding

Funding for this work was provided by the Intramural Research Program of the National Institutes of Health (Z01CP010119), with support from National Cancer Institute Director's Intramural Innovation Award Program. Felicia Hung was funded by the Yale School of Public Health summer scholarship funding.