Retrospective Pretests: Recent Use in Visitor Studies Research and Ways to Make Them More Informative

Abstract

Many visitor studies researchers use Retrospective Pretest (RPT) ­methods to document outcomes. Research literature compares the validity of RPT with traditional pretest-posttests. This article reviews visitor study and informal learning literature about how RPT has been used to evaluate programs in museums, aquariums, parks/recreation, zoos, and tourism. It furthers discussions about response shift bias, arguing that response shift can be either intentional or inadvertent. The type of response shift is important in determining when RPT should be used. We argue that RPT is the best choice when the program intends to shift participants’ understanding of constructs being measured, and when comparing intervention with non-intervention groups. The literature review found that RPT results usually focus on statistical significance testing. RPT data offer much more learning when the results are also examined from other perspectives. This article describes additional analyses of RPT data that can assess the applicability of programs to intended audiences.

Keywords:

• Evaluation
• informal learning
• literature review
• retrospective pretest

Additional information

Notes on contributors

Melanie Hwalek

Melanie Hwalek is CEO of SPEC Associates and Assistant Professor of Psychology at Michigan State University. She specializes in research design, psychometrics, and evaluation management. She has evaluated programs across the human services spectrum for several decades. Address correspondence to: Melanie Hwalek, Social Program Evaluators and Consultants Inc, Southfield, Michigan, USA. E-mail: [email protected].

Cassandra Solomon-Filer

Cassandra Solomon-Filer is an evaluator advocate with BECOME: Center for Community Engagement and Social Change and also works as an independent evaluation consultant. Her evaluation experience is mostly focused on youth programs, particularly evaluations of informal learning, civic engagement, and vocational training programs.

Deborah Wasserman

Deborah Wasserman is Senior Researcher at the COSI center for research and development. Her work primarily involves applying self-determination theory to research and evaluation of youth development programming, human service delivery systems, visitor studies, and informal science education.