Implementation of the National Science Foundation’s “Broader Impacts”: Efficiency Considerations and Alternative Approaches

Abstract

The National Science Foundation (NSF) has, since 1997, attempted to diversify and enrich science research and education in the USA through the Broader Impacts Criterion (BIC), also known as “Criterion Two” or the “Second Criterion”. In doing so, NSF has so successfully integrated BIC into its discovery grant funding programmes that it has become difficult to assess the efficiency (in an economic sense) of BIC activities, as opposed to cataloguing its products (number of trainees, publications, etc.). Moreover, current practice at NSF requires that each and every Principal Investigator receiving a discovery grant address both Science, Technology, Engineering and Math activities and broader impacts, despite the fact that their formal training is most likely to be in only one of these areas. Against this backdrop, I consider NSF spending on broader impacts, conduct a microeconomic analysis of effectiveness of BIC expenditures, and discuss alternative funding models and Principal Investigator profiles and expertise sets that might not only accelerate the goals of expanding NSF’s broader impact, but additionally enhance the quality of science funded by this agency.

Keywords:

• National Science Foundation
• Broader Impacts
• Academic Specialization
• Criterion Two
• Discovery Grants

Acknowledgements

The author is grateful to Robert Frodeman and J. Britt Holbrook for extensive discussions on NSF’s BIC, as well as the broader topic of the role of science and technology in society. Much appreciated are frank and candid discussions with an anonymous NSF staff member. The author greatly appreciates comments on drafts of this manuscript by Robert Frodeman, J. Britt Holbrook, Kathryn Gould Cullivan, Michael Monticino and John Quintanilla. The author is supported by NSF operating grants IOB‐0614815 and IOS‐0942287, although they may be his last if they actually read this article.

Notes

[1] Staff members at NSF dislike the term “Second Criterion”, which suggests that it is a lesser criterion.

[2] Panel deliberations and scores are not made public. The author reaches this conclusion based on anecdotal information from interviewing panellists and NSF staff members.

[3] A well‐known phenomenon in epidemiology is the appearance of an increase in disease incidence simply due to an enhanced ability to diagnose that disease. This raises the question of whether NSF is now funding more relevant research due to the implementation of BIC, or whether proposers simply have had to go to greater pains to point out the broader significance and connection of their research.

[4] The author did not conduct a formal poll and presents these preliminary data as anecdotal—but nonetheless quantitative.

[5] This discussion is not intended to suggest that equal emphasis is (or should be) placed on the scientific merits and broader impacts of a project. NSF’s instructions indicate that the reviewers should make the determination of the relative weights for each criterion. While both need to be prominently featured in any discovery grant proposal, this narrative does not address the probable large variation in relative emphasis by individual reviewers or panels.

[6] Of course, it could be argued that NSF funding rates are currently so low that panels only have to identify those unusual individuals who can indeed specialize in the current NSF duality (e.g. Quadrant 1 in Figure 1). However, this cynical view fails to recognize that left on the table are large numbers of exciting, high‐impact proposals with either high intellectual or broader impact merit, but not both as currently required.

[7] The NCBI PubMed database contains a relatively high proportion of journals likely to publish National Institutes of Health‐funded rather than NSF‐funded articles, and so a decreased emphasis on BIC‐related search terms is not surprising. However, the fact that only approximately 0.1% of the papers in that database contain common descriptors of BIC activities speaks to the issue of little coverage of BIC activities in mainstream STEM journals.