From improvement to relationship management: a case study of data artifact creation and use in a school improvement network

ABSTRACT

Improvement networks are a relatively recent phenomenon in US education that create interorganizational networks of educators working together to improve specific educational problems. A shared emphasis of these networks is the use of data to support the improvement process, but little is known about their data use in practice. This study takes an in-depth look at the 800+ data artifacts created and used in a single focal network’s 3rd year, applying a mixed methods case study design. The findings show that while data artifacts are predominantly created to support the improvement process, they are frequently used for relationship management as network hub members navigate the sociopolitical dynamics typical of a complex change effort – if they are used at all. The study has implications for practitioners as they decide what data artifacts to create and use, as well as for researchers’ theory building about productive data use in improvement efforts.

KEYWORDS:

• Improvement networks
• improvement science
• continuous improvement
• data use

Disclosure statement

No potential conflict of interest was reported by the authors.

Ethical approval

Ethical approval was obtained for this project from the University of Pittsburgh’s IRB: STUDY18120016.

Data availability statement

Data not available – participant consent: The participants of this study did not give written consent for their data to be shared publicly, so due to the sensitive nature of the research, supporting data are not available.

Notes

1 The Networks for School Improvement Initiative was launched by the Gates foundation in 2018 to support organizations bringing together and serving as intermediaries for networks of middle and high schools. These networks work together to improve high school graduation and college success rates for their Black and Latino students and students experiencing poverty through the use of continuous improvement methods, proven indicators of future student outcomes, and unwavering commitment to equity (Bill & Melinda Gates Foundation, n.d.). To date, the initiative has supported the formation and operation of over 40 networks for school improvement.

2 The first and second authors of this study were both researchers in the focal network’s hub team, which afforded access to network information and special insight into network functioning. The first author was a graduate research assistant who joined the focal network in its 3rd year and continued as a network member through the end of the network’s 5th and final year. The second author was involved with the network from its initiation, serving as a faculty member affiliated with one of the research centers.

3 Network hubs engage in a number of key processes associated with organizing and operating a high-functioning networked improvement community. The Carnegie Foundation for the Advancement of Teaching has a framework for describing these hub functions, such as site-level improvement routines and capacity building.

4 For 63 artifacts (8% of the sample), we felt that there was a strong secondary intended use and included that secondary use into a Combined Intended Use variable for our analyses. While the inclusion of the secondary intended use did more accurately reflect the complex nature and tensions of the focal network’s multi-stakeholder environment, it did not significantly impact the results of our analyses.

Additional information

Funding

The study was supported by a grant from the Bill & Melinda Gates Foundation. The findings and opinions expressed here are those of the authors, not the foundation.

Notes on contributors

Anna E. Premo

Anna E. Premo is a doctoral candidate in the Learning Sciences and Policy program and graduate research assistant in the Learning Research and Development Center at the University of Pittsburgh. Her research interests include organizational learning, data and measurement use, and research–practice partnerships, and she strives to reframe complex organizational change and transformation as learning processes. Premo began her career as a researcher in international development, receiving a Fulbright to conduct research in South Africa shortly upon completion of her master’s degree. She later transitioned to strategy and transformation consulting in sales, delivery, and operational roles. Most recently, she served as the Global Head of Business Transformation for the banking industry in a multinational technology services and consulting company. She holds an MCP in international development and an SB in urban studies and planning from the Massachusetts Institute of Technology.

Jennifer Lin Russell

Jennifer Lin Russell is a professor of leadership, policy, and organizations, whose research focuses on organizing educational systems for improvement. She examines policy and other educational improvement initiatives through an organizational perspective and seeks to reshape the relationship between educational research and practice in order to accelerate improvement in the field. Russell recently served as professor of educational foundations, organizations, and policy at the University of Pittsburgh, where she was chair of her department and associate director for educational research and practice at the Learning Research and Development Center. She received her PhD in education: policy, organization, measurement, and evaluation from the University of California, Berkeley in 2007. She also holds an MA in curriculum and instruction from the University of San Francisco and a BA in political science and urban studies from Northwestern University.

Megan Duff

Megan Duff is a postdoctoral researcher on the Measuring Network Health Project at Peabody College, Vanderbilt University. Her research uses mixed methods, qualitative, and quantitative approaches to examine the implementation of policies aimed at school and system improvement with a particular emphasis on creating more equitable educational opportunities in urban school systems. Her recent work focuses on two primary concerns: (1) how policy implementation is mediated by local contexts and (2) whether and how policies can accelerate educational improvement through interorganizational collaboration. Duff holds a PhD and MA in education policy from Teachers College, Columbia University, a MS in middle school education from Bank Street College of Education, and a BA in English from Columbia College, Columbia University. In addition, she has worked as a Middle School English teacher and teacher leader in New York City traditional and charter schools.