The Math and Science Partnership Program Evaluation: Overview of the First Two Years

Abstract

This study describes the Math and Science Partnership Program Evaluation (MSP-PE) during the project's first two years and provides the evaluation framework being used to assess the National Science Foundation's MSP Program. The study conveys the MSP-PE's ongoing design and implementation. To show how they reflect the nature of the MSP Program, the study addresses the following questions: (a) What are the MSP Program's main themes? (b) What kinds of activities have the program's awardees been putting into place? (c) What are the awardees doing to assess K-12 student achievement outcomes? and given the preceding conditions, (d) What is the framework and design for the MSP-PE? The study shows how the framework and the emerging evaluation derive from the program's main themes and its early activities, also giving readers a glimpse of the program's activities. The study traces the rationale behind a multi-institutional framework that covers a series of pathways in the K-20 span of mathematics and science education. This systems approach calls for a series of substudies that collectively address the multifaceted interorganizational and intraorganizational relationships in the MSP Program. The evaluation's framework provides a unifying scope for the series of substudies—all of which have been undertaken as part of the MSP-PE. Some of MSP-PE's early substudies are contained in this special issue.

ACKNOWLEDGMENT

This study is one in a series of substudies for the Math and Science Partnership Program Evaluation (MSP-PE) conducted for the National Science Foundation's MSP Program. The MSP-PE is conducted under Contract No. EHR-0456995. Since 2007, Bernice Anderson, Ed.D., Senior Advisor for Evaluation, Directorate for Education and Human Resources, has served as the National Science Foundation Program Officer. The author, from COSMOS Corporation, is Robert Yin.

The MSP-PE is led by COSMOS Corporation in current partnership with George Mason University (GMU) and Brown University. Robert K. Yin (COSMOS) serves as Principal Investigator and Jennifer Scherer (COSMOS) serves as one of three Co-Principal Investigators. Additional Co-Principal Investigators and their collaborating institutions (including discipline departments and math centers) are Patricia Moyer-Packenham (USU, formerly GMU) and Kenneth Wong (Brown).

Notes

¹The MSP Program is concerned with all primary and secondary school grades, pre-Kindergarten through Grade 12. For convenience's sake, “K-12” is used throughout to reference this range of grades rather than the more cumbersome “pre-K-12.”

²These were the first two strategic priorities, respectively, in the NSB's vision statement for NSF. The third strategic priority was to build the nation's basic research capacity by making critical investments in infrastructure, including advanced instrumentation, facilities, cyberinfrastucture, and cutting-edge experimental capabilities (NSB, 2005).

³For instance, strengthening the focus and funding to improve K-12 mathematics education in particular also appears as a priority in the legislation related to the American Competitiveness Act (e.g., Domestic Policy Council, 2006; Sroufe, 2006).

⁴The priority given to the MSP was sufficiently high that the NSF-MSP has a counterpart initiative—the “Mathematics and Science Partnerships”—at the U.S. Department of Education (ED-MSP). A U.S. House Committee report described the complementarity of the two initiatives as follows: Whereas NSF's program is to fund “innovative programs to develop and establish new models of education reform, thereby remedying the lack of knowledge about math and science research,” ED's program is aimed at “broadly implementing and disseminating new teaching materials, curricula, and training programs” (U.S. House of Representatives, 2003, p. 4). The ED-MSP is being separately evaluated, falling outside of the purview of the MSP-PE and hence of this study.

⁵The MSP Program distinguishes among four types of awards. Of the 53 awards still active from the first two cohorts, 35 were either “comprehensive” (an awardee covers the entire K-12 grade span) or “targeted” (an awardee covers a selected number of grades). Only 1 award fell into the third category of teacher institutes (an awardee focuses on teacher training institutes). The remaining 17 awards fell into the fourth category of “research, evaluation, or technical assistance” awards (an awardee chooses to study, support, provide tools for, or otherwise collaborate with one or more of the “comprehensive,” “targeted,” or “institute” awardees, and the collaboration also may include similar projects not funded directly by the NSF-MSP). Although the MSP-PE evaluation framework and design embrace all four types of awards, the description of the MSP activities in this study is limited to the first two types of awards only. The evaluation initially intends to focus separately on the other two types of awards, later bringing and synthesizing the lessons learned from all four types into a fuller comprehensive assessment of the entire MSP Program.

⁶The six questions and topics appear in the original solicitation for the MSP-PE award and define its goals.

⁷Each MSP has an ongoing “project” evaluation to serve this purpose. To date, most of the project evaluators are collecting data and providing “formative” feedback to their host MSP. Some but not all of the evaluators eventually plan to conduct “summative” evaluations. Regardless, the project evaluations will not address cross-project findings or lessons, which are the main thrust of the MSP-PE.