The perils of ignoring design effects in experimental studies: Lessons from a mammography screening trial

Abstract

Objective : Threats to external validity, including pretest sensitisation and the interaction of selection and an intervention, are frequently overlooked by researchers despite their potential to significantly influence study outcomes. The purpose of this investigation was to conduct secondary data analyses to assess the presence of external validity threats in the setting of a randomised trial designed to promote mammography use in a high-risk sample of women.

Design : During the trial, recruitment and intervention, implementation took place in three cohorts (with different ethnic composition), utilising two different designs (pretest–posttest control group design and posttest only control group design).

Results : Results reveal that the intervention produced different outcomes across cohorts, dependent upon the research design used and the characteristics of the sample.

Conclusion : These results illustrate the importance of weighing the pros and cons of potential research designs before making a selection and attending more closely to issues of external validity.

Keywords:

• external validity
• research design
• methodology
• cancer screening
• randomised controlled trial