How and When Democratic Values Matter: Challenging the Effectiveness-Centric Framework in Program Evaluation

Abstract

Performance information is overwhelmingly used in program evaluation by both public managers and external stakeholders. In the market-based New Public Management movement, effectiveness is public programs’ major selling point. However, this approach may marginalize the role of democratic values in governance. In the current complex society with anti-government sentiments, we embrace the idea of New Public Service to reiterate the importance of democratic values. Using a conjoint experiment, we compare the effects of effectiveness and democratic values in predicting public program evaluation, conditioned on citizens’ trust in government. Our results show that effectiveness and democratic values contribute similar effects in explaining policy preferences. Distrust in government strengthens the effect of democratic values but reduces the effect of effectiveness. Our findings challenge the prevalent effectiveness-centric framework in public management. We suggest that citizen-state interaction should not rely only on performance merits, but also on inclusiveness and openness values.

Keywords:

• new public service
• democratic values
• conjoint experiment
• performance management

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Acknowledgments

We thank Gregory Porumbescu and Chengxin Xu for their feedback during different stages of this project. We thank technical advice from Ivan Lee and Chien-shih Huang. We are also grateful for insightful comments from participants at the 2019 Annual Conference of the Association of Public Policy Analysis & Management (APPAM).

Data availability statement

The data that support the findings of this study are openly available in Harvard Dataverse at doi:10.7910/DVN/FJ82VU.

Notes

1 We had H3b in the pre-registration report that tests for moderating effects of low trust on democratic values. Here, we posit moderating effects of low trust on inclusiveness and openness separately and posit two hypotheses. Both H3b and H3c in this section stay aligned with the logic of H3b in the pre-registration report.

2 We conducted a pilot test to validate the statistical power of our LT treatment. We recruited 197 subjects from MTurk, We used the same design as our formal experiment, which randomly assigned subjects into the low trust treatment group (showing three negative government information) or the control group (no information). We then measure their levels of trust in government by asking: Do you trust the U.S. local governments? (0–100, from definitely not to definitely yes). The negative information cues result in a reduction of 14.21 points (S.E. = 3.23, p = 0.00, Cohen’ d = 0.63) in the participants’ trust in U.S. local governments, which is similar as results in our formal experiment.

3 The overall LT effect on choice is untestable, because every pair of program comparison generates one approved program profile and one disapproved profile in the forced choice mechanism. So, eventually 50% of program profiles would be approved in both the LT and control groups. Therefore, the p-value in a two-sample t-test on choice equals to 1.

4 As Leeper et al. (2020, p. 6) argued, “…the differences between conditional AMCEs are used as a way of descriptively characterizing differences in preferences (i.e., levels of support) between the groups rather than differences in causal effects on preferences in the groups.” In this study, causally detecting the LT effect is crucial for us to disentangle the complex theoretical mechanisms between trust, value, effectiveness, and program support. In this sense, difference-in-MMs between the control and LT group is preferred.

Additional information

Notes on contributors

Yixin Liu

Yixin Liu is a post-doctoral fellow at the Maxwell School of Citizenship & Public Affairs at Syracuse University.

Heewon Lee

Heewon Lee is a post-doctoral researcher in the Askew School of Public Administration and Policy at Florida State University.

Frances Berry

Frances Berry is the Frank Sherwood Professor of Public Administration in the Askew School of Public Administration and Policy at Florida State University.