Evidence of an Output Bias in the Judgment of Public Performance: A Replication and Extension

Abstract

Despite calls for an evidence-based focus on outcomes as a way to enhance accountability for public performance, findings from a prior study suggest that the public may be more impressed by high frequency (low cost) but ambiguous outputs (such as people served) rather than more meaningful but costly outcomes (causal effects). We attempt to replicate and extend the investigation of this output bias through a pair of survey experiments involving judgments about two evidence-based, highly effective social programs: one, an HIV/AIDS prevention program (adapted from the prior study), the other, a program for special needs high school students (Check and Connect). Our findings confirm that respondents viewed both programs more favorably when given information about mere outputs (people served) in comparison with more rigorous outcomes (causal effects). We then tested an extension of the Check and Connect experiment in which we modified the framing of cost and performance information in ways that reduced the tendency toward an output bias. We speculate on the possible mechanism that may lead to an output bias, and we discuss the implications of our findings for evidence-based public policy and management.

Keywords:

• accountability
• evidence-based policy
• experimental methods
• government performance
• program evaluation

Additional information

Notes on contributors

Gregg G. Van Ryzin

Gregg G. Van Ryzin is professor in the School of Public Affairs and Administration, Rutgers University-Newark (USA). His work employs experimental and behavioral approaches to various issues in public management, including citizen satisfaction, coproduction, performance measurement, representative bureaucracy, and organizational behavior. He is author (with Dahlia Remler) of Research Methods in Practice (SAGE) and editor (with Oliver James and Sebastian Jilke) of Experiments in Public Management Research (Cambridge).

Ashley Grosso

Ashley Grosso is assistant professor in the Department of Urban-Global Public Health, School of Public Health, Rutgers University (USA) and a member in residence at the Rutgers Institute for Health, Health Care Policy and Aging Research. Her research is focused on social and structural determinants of health among people living with or at risk for HIV infection. She holds a Ph.D. from the Rutgers School of Public Affairs and Administration.

Étienne Charbonneau

Etienne Charbonneau is Canada Research Chair in Comparative Public Management at École Nationale d'Administration Publique, Montreal (Canada). His current research focuses on public management, accountability and surveillance. He holds a Ph.D. from the Rutgers School of Public Affairs and Administration.