Broad Impacts and Narrow Perspectives: Passing the Buck on Science and Social Impacts

Abstract

We provide a critical assessment of the National Science Foundation’s (NSF) “broader impacts criterion” for peer review, which has met with resistance from the scientific community and been characterized as unlikely to have much positive effect due to poor implementation and adherence to the linear model heuristic for innovation. In our view, the weakness of NSF’s approach owes less to these issues than to the misguided assumption that the peer review process can be used to leverage more societal value from research. This idea, although undoubtedly well‐meaning, is fundamentally flawed. Retooling or refining the Broader Impacts Criterion does not alter the fact that conventional peer review, based on specialized scientific and technical expertise, is not up to the task of ensuring adequate judgements about social impact. We consider some possible alternative approaches to providing greater social impact in science and include in our assessment past and current efforts at NSF and throughout the federal research establishment that address, in some cases having addressed for decades, the intentions and goals of the Broader Impacts Criterion, albeit using alternate mechanisms. We conclude that institution‐building and explicit and targeted policy‐making are more useful and democratically legitimate approaches to ensuring broad social impacts.

Keywords:

• Science Policy
• National Science Foundation
• Broader Impacts
• Peer Review

Acknowledgement

The authors are grateful to Dan Sarewitz for his helpful comments on an earlier draft of this paper.

Notes

[1] Broader impact examples are available from http://www.nsf.gov/pubs/gpg/broaderimpacts.pdf; INTERNET.

[2] The NSF Grant Proposal Guide is available from http://www.nsf.gov/pubs/policydocs/pappguide/nsf08_1/gpg_index.jsp; INTERNET.

[3] Links to these NSF documents and to other related web sites are available from http://www.ndsciencehumanitiespolicy.org/workshop/links.php; INTERNET.

[4] The “Broader Impacts Toolbox” is available from http://broaderimpacts.info/; INTERNET.

[5] The data were drawn from a national survey of university scientists that was conducted from August 2003 to July 2004 by a team in the Research Value Mapping Program at Georgia Tech. The sample design for the survey included tenured and tenure track scientists and engineers employed in “Research Extensive” (formerly “Research I”) universities, as defined by the Carnegie Classification (Carnegie Foundation for the Advancement of Teaching 2002). The sample was stratified by academic discipline, academic rank, and gender. For alternate research purposes, Experimental Program to Stimulate Competitive Research universities and Historically Black Colleges and Universities faculties were included in the original sample, but these two groups were not used for the analysis in this paper. For the subgroup of respondents used, n = 1647 with a response rate of 38%.

[6] Responses were based on a four‐point Likert scale, where 1 = “strongly disagree”, 2 = “disagree somewhat”, 3 = “agree somewhat”, and 4 = “strongly agree”. For this paper, the scale was reduced to two points, “agree” and “disagree”. Accordingly, the 64% is comprised of 18% respondents indicating that they “strongly disagree” and 46% indicating that they “disagree somewhat”.

[7] The 63% is comprised of 16% respondents indicating that they “strongly agree” and 47% indicating that they “agree somewhat”.

[8] Bozeman and Boardman (2004) observe that this “culture clash” soon re‐emerged with the establishment of the Engineering Research Centers programme, which thrives to date.