Locating data use in the microprocesses of district-level deliberations

ABSTRACT

While there is an abundance of data-use literature available, there is still a need to develop methodological approaches for studying naturally occurring data use in decision-making processes over time. The central contribution of this paper is a strategy to understand the use of data in long-term observations of educational leaders’ policy-making deliberations. Using longitudinal and observational data, we created ‘decision trajectories’ that traced microprocesses of deliberation around specific decisions over time. We employed frame theory to locate when and how data entered decision-making processes within these trajectories. Our approach addresses the use of data as they arise in the context of longitudinal observations – a method that provides insight into how data may be used to inform, frame, or justify educational decisions.

KEYWORDS:

• Data use
• decision making
• frame theory
• qualitative methods
• longitudinal

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. We use the term deliberation to refer to discussions – centred on problems or choices to be made – that occurred within and often extended across district meetings.

2. There are exceptions, including Cosner (2011), and our own prior work (e.g. Coburn et al., 2008; Coburn et al., 2009).

3. Our dataset was gathered from August 2012 through September 2014 when we observed district meetings related to mathematics over two years across a range of district subunits (n = 300 h). We interviewed all mathematics leaders and all other district administrators who had instructional responsibilities in mathematics (n = 87 interviews with 44 administrators). We gathered 1826 artefacts, including documents from the meetings we observed and those related to district mathematics policy. The comprehensiveness of data collected contributes significantly to the methods described herein, as their longitudinal and observational nature supported the novel analytic techniques we explain.

4. We recognise that not all researchers are able to invest the resources necessary to conduct the time-intensive data collection we recommend here.

5. All names – of districts, people, and programmes – are pseudonyms.

6. During each step of the coding process, we developed criteria for our analytical decisions, including what constitute an episode, a frame, and a reason. These were explicit criteria that our team used as guides with which to frequently calibrate our coding – a process of coding in groups, individually, and then regrouping to check one another’s work. This is an approach we suggest for achieving reliability using this methodology.

Additional information

Funding

This work was supported by the Spencer Foundation [201500139].