A Pilot Study to Identify Comparison Schools for Math and Science Partnership Participating Schools: Preliminary Findings on One Math/Science Partnership

Abstract

This pilot study proposes a set of analytical steps for comparing schools that participate in the National Science Foundation's Math and Science Partnership (MSP) Program and their nonparticipating peers in the same state. This pilot is part of a larger effort to evaluate the MSP Program's role in student achievement, with two companion analyses. Although our pilot study uses a comparative approach, the study by Dimitrov in this issue follows a within-group design. The third analysis by Yin and his associates in this issue covers the varied designs used by the MSPs themselves in their own evaluations.

In this pilot, we focus on a sample of participating schools in one MSP in one state. The nonparticipating schools were carefully matched with the program participating schools on eight demographic variables to form a comparison group. This article offers detailed documentation on how we operationalize two matching methods for comparative purpose. We conclude that carefully executed matching methods are promising for large scale comparative analysis on the effects of the MSP Program across different states.

ACKNOWLEDGMENT

This pilot study is one in a series of substudies for the Math and Science Partnership Program Evaluation (MSP-PE) conducted for the National Science Foundation's MSP Program. The MSP-PE is conducted under Contract No. EHR-0456995. Since 2007, Bernice Anderson, Ed.D., Senior Advisor for Evaluation, Directorate for Education and Human Resources, has served as the National Science Foundation Program Officer. The authors are Kenneth K. Wong, Ph.D., of Brown University, and Ted Socha, M.A., of National Center for Education Statistics.

This illustrative study draws on publicly accessible school-level data files from one state and from data available at the National Center for Education Statistics' Common Core of Data. In addition, the study consults secondary materials only: available literature together with all of the annual reports, evaluation reports, MSPnet documents, and Web site information reported by the individual Math and Science Partnerships (MSPs) in the MSP Program accessible through the school year 2004–05.

We express gratitude to the Brown research team consisting of Joshua Marland and Erikson Arcaira. Francis X. Shen also provided valuable support on data analysis.

Notes

¹This is a log-rank weighting function. For further explanation as to rreg's methodology, see Statistics With STATA by Hamilton (2006)

²The particular MSP analyzed was targeted to focus only on middle and high schools, not elementary schools

^a Percentage calculated from a raw number

^b Variable dropped due to extensive missing data

^a Dummy variable where 1 = Title I Eligible School

^b 1 = Large City, 2 = Mid-Size City, 3 = Urban Fringe of Large City, 4 = Urban Fringe of Mid-Size City, 5 = Large Town, 6 = Small Town, 7 = Rural, outside [Core Based Statistical Area], 8 = Rural, inside [Core Based Statistical Area]

^a Dummy variable where 1 = Title I Eligible School

^b 1 = Large City, 2 = Mid-Size City, 3 = Urban Fringe of Large City, 4 = Urban Fringe of Mid-Size City, 5 = Large Town, 6 = Small Town, 7 = Rural, outside [Core Based Statistical Area], 8 = Rural, inside [Core Based Statistical Area]

^a Dummy variable where 1 = Title I Eligible School

^b 1 = Large City, 2 = Mid-Size City, 3 = Urban Fringe of Large City, 4 = Urban Fringe of Mid-Size City, 5 = Large Town, 6 = Small Town, 7 = Rural, outside [Core Based Statistical Area], 8 = Rural, inside [Core Based Statistical Area]

^a Dummy variable where 1 = Title I Eligible School

^b 1 = Large City, 2 = Mid-Size City, 3 = Urban Fringe of Large City, 4 = Urban Fringe of Mid-Size City, 5 = Large Town, 6 = Small Town, 7 = Rural, outside [Core Based Statistical Area], 8 = Rural, inside [Core Based Statistical Area]

^a Dummy variable where 1 = Title I Eligible School

^b 1 = Large City, 2 = Mid-Size City, 3 = Urban Fringe of Large City, 4 = Urban Fringe of Mid-Size City, 5 = Large Town, 6 = Small Town, 7 = Rural, outside [Core Based Statistical Area], 8 = Rural, inside [Core Based Statistical Area]

^c Out of a possible 361

^a Dummy variable where 1 = Title I Eligible School

^b 1 = Large City, 2 = Mid-Size City, 3 = Urban Fringe of Large City, 4 = Urban Fringe of Mid-Size City, 5 = Large Town, 6 = Small Town, 7 = Rural, outside [Core Based Statistical Area], 8 = Rural, inside [Core Based Statistical Area]

^c Out of a possible 368

^a Dummy variable where 1 = Title I Eligible School

^b 1 = Large City, 2 = Mid-Size City, 3 = Urban Fringe of Large City, 4 = Urban Fringe of Mid-Size City, 5 = Large Town, 6 = Small Town, 7 = Rural, outside [Core Based Statistical Area], 8 = Rural, inside [Core Based Statistical Area]

^c Out of a possible 361

^b 1 = Large City, 2 = Mid-Size City, 3 = Urban Fringe of Large City, 4 = Urban Fringe of Mid-Size City, 5 = Large Town, 6 = Small Town, 7 = Rural, outside [Core Based Statistical Area], 8 = Rural, inside [Core Based Statistical Area]