Missing the Mark: Assessing Dispositions and the Reification of Whiteness in Teacher Preparation

Abstract

In order to explore whether the Watermark^TM Educator Disposition Assessment (EDA) is an equitable assessment for all teacher candidates regardless of race, this study used a Quantitative Critical Race Theory (QuantCrit) methodological design to analyze a pilot implementation of the assessment representing 650 discreet disposition assessments undertaken in 24 designated courses in five distinct programs of educator preparation at a midsized Midwestern university. Chi-square analysis and descriptive statistics from fall 2019–spring 2020 archival data indicate a statistically significant association between the variable race and the average scores assigned to students in multiple EDA assessments, with marked disadvantages for Black teacher candidates. Through this analysis and a careful and critical review of language embedded in each rubric row, the authors argue that the Watermark^TM EDA advances and reifies a set of normative practices that align with dominant White, middle-class cultural values under the guise of objectivity.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 A vocal minority of faculty identified items on the Watermark™ EDA which were subjective in nature and which were likely to be interpreted from a White, middle class cultural lens that could potentially negatively impact BIPOC students. This vocal minority spoke truth to power, but institutional momentum to satisfy conditions of CAEP accreditation overrode concern. The vocal minority was, however, successful in asserting that a complementary document—an Equity Implementation Guide—be created as a companion to the assessment, alerting assessors to the jeopardy of biased interpretation of rubric rows. The Equity Implementation Guide was adopted along with the newly required rubric for the assessment of dispositions, but no training on the Equity Implementation Guide was provided to assessors, as was urged by its authors.

2 The Technical Guide (Almerico et al., 2018) includes a note about addressing bias. They state “The EDA team conducted a check for racial bias, gender and ambiguity within the measure. A group of experts (N = 125) was gathered and broken into nine teams of 2 to 3 participants. They were tasked with rating the perceived bias, gender and ambiguity present in the indicator descriptions. There were two areas of concern identified. Seventy-seven percent of the experts reported racial bias evident in “Oral Communication.” Specifically, they noted that the use of the phrase “Standard English” in the oral communication rubric did not allow for a widen use of Ebonics or dialects. The resulting descriptor now reads “Demonstrates strong professional oral communication skills as evidenced by using appropriate language, grammar, and word choice for the learning environment.” The second concern was the use of the word “cordial” to describe written communication. Cordial was rated to be ambiguous by 68% of the experts so it was changed to “conventional” (p. 25). We, the authors, argue that the newly selected words retain the underlying bias they sought to supplant.

3 Per the Technical Guide (Almerico et. al., 2018), “cordial” was changed to “conventional,” a term that remains problematic as it prioritizes the dominant, normative practice.

4 It is worth noting a few points here. First, the effectiveness of drawing upon community, family, and student knowledge to plan for instruction is well documented (see literature pertaining to culturally relevant teaching, culturally responsive teaching, culturally sustaining teaching, community responsive teaching, and community engaged teaching). Further, for the purposes of candidate lesson plans, instructors often want candidates to provide citations for the sources they used to plan specific activities and not necessarily draw upon scholarly literature to support their instructional choices. For example, if a lesson plan drew upon an idea on Pinterest, the expectation is that the candidate would provide a link to the webpage.

5 An overview of the implementation system can be found at https://www.bsu.edu/-/media/www/departmentalcontent/teachers/otes/new%20website/dispositions/dispositions.pdf?la=en&hash=7B4E7D24C3193B43D845BF79767A4AF3D91CDC7D. It should be noted that there is no formal orientation to this system. How candidates learn of the disposition assessment and related alert system varies from program to program.