Exploring Charter School Innovation: A Comparison of Popular Charter School Models

ABSTRACT

This paper expands on previous work on charter school typology and presents disparities in standardized test outcomes across models by using standardized Z-Scores weighted by NAEP performance. Analyses indicate that in ELA, Classical schools have the highest relative performance, followed by Montessori and Art schools. In math, Classical schools once again have the highest relative performance, followed by Montessori and STEM schools. For reasons discussed in the paper it is premature to posit causality, so the results should instead be viewed as descriptive. We suggest a more pluralistic testing framework may be appropriate when evaluating the performance of specialized schools.

KEYWORDS:

• Authorizers
• charter schools
• edfacts
• evaluation
• foci
• NAEP
• performance

Background

The question of what defines a high-quality education has been a subject of ongoing debate among educators, parents, and policy makers. Despite numerous endeavors to assess student progress and outcomes, this underlying divergence of opinion persists. Nonetheless, policy makers require effective methods for evaluating public programs in order to optimize the allocation of public resources. Standardized testing, grounded in a foundational set of criteria, has proven instrumental in illuminating educational disparities and serving as a means to appraise schools and educators. The inherent drawback of this approach lies in society’s failure to consider the diverse array of objectives and aspirations held by parents. The utilization of uniform standards proves impractical since the identification of an ideal educational institution remains unattainable without well-defined and mutually agreed-upon objectives. The determination of an ideal school depends on the individual goals of the parents. As parents do not universally agree on the goals of public education, those of us in the education space are left with an unworkable system incapable of properly evaluating a school’s performance (Ackerman & Egalite, 2017). Some thinkers in the educational space have suggested that school choice programs that embrace the pluralistic nature of parental desires could help solve this issue (Egalite & Wolf, 2016; Hanushek et al., 2007; Kisida & Wolf, 2010). Others have argued that school choice would also yield the benefits of competition in education (Hanushek, 2006). For the past 30 years, the public charter school movement has presented parents with a plethora of schooling options and therefore serves as the perfect testing ground for these lines of reasoning.

While school choice sometimes elicits mixed emotions, charter schools and other forms of school choice continue to receive broad support (Shuls, 2018). Roughly two- thirds of charter schools are specialized in a wide array of foci including STEM, Arts, CTE, Language Immersion, etc (White & Huang, 2022). When a school chooses to focus on a particular mission, it gives researchers a valuable opportunity to assess whether that model is, in any way, superior to the baseline average school in some way. If that is the case, then there may be an opportunity for the education sector to derive successful policies, curricular choices, pedagogical approaches, and other key decisions around how to best support families in the community, staff, and most of all, students. They can accomplish this because charter schools are public schools.

When a private citizen wants to open their own public school, a state-appointed actor may approve them or a third party to independently “charter” or run a public school. Those charged with overseeing charter schools are called authorizers. Authorizers come from a variety of sources including state governments, local governments, municipal governments, state- approved independent chartering boards, and in rare cases, nonprofit organizations, or universities (NACSA, 2021). Each state creates its specific laws detailing who these authorizers can be. As of 2022, 45 states, D.C., Puerto Rico, and Guam all have charter laws (NAPCS, 2022). In exchange for the freedom and autonomy to innovate, charter schools are required to meet agreed-upon accountability standards. If a school fails to meet all of the standards, an authorizer may choose to revoke the charter, which effectively closes the public charter school.

Thus, the way authorizers evaluate the performance of a school is a critical policy decision. They typically look to state-determined standardized exams in ELA and Math for insight into how a school is performing. 22 states have some form of History or Civics Exam (NCES 1, 2018). All states test for some science at some point and in some form, but only 6 do so more than twice in a child’s 12 years of schooling (NCES 2, 2021). Both the irregularity of administration and the wide variance of tested material make comparing these data difficult. If charter schools offer countless specialized schools, then it may place those charter schools at a disadvantage as an aggregate when researchers evaluate them based on the same criteria as for other generic public schools. However, by recognizing the many forms of specialization, authorizers might, for example, more accurately assess charter schools based on their missions, models, and attributes (Maranto & Shakeel, 2021). To further investigate, we look to see if the top school models demonstrated performance disparities in the 2018–19 school year. While this does not necessarily indicate that the models themselves cause the disparities, determining which models performed the best descriptively should provide some direction for researchers interested in exploring innovation and what lessons can be gleaned from the charter school sector.

Comparing models presents a challenge due to the potential influence of families and communities that opt for specialized schools on student outcomes. In other words, there may exist notable distinctions between families attending classical schools and those selecting Art schools, for instance. Nevertheless, conducting comparisons within a given model can help mitigate selection bias and elucidate the effectiveness of specific innovations within that model. As researchers categorized charter schools based on self-identification rather than strict adherence to a predetermined set of model characteristics, variations in the adoption of innovations emerge even among schools classified under the same model. This diversity provides an opportunity to examine which innovative elements of a model correlate with improved student performance.

While there are many metrics that could be used for evaluating a given school’s performance, the most used metrics are ELA and Math proficiency. While we hope to one day expand the scope of inquiry to include measures of growth, civic engagement, long-term outcomes, and executive functioning, data limitations prevent us from doing so in this paper.

Data

In a previous paper, the researchers coded all public charter schools in the 2018–19 school year into 68 models. Each model needed to have five schools that identified with the same set of foci within their mission statements, charter applications, and promotional materials. Roughly a third of charter schools did not self-identify with a specialized focus, therefore researchers coded them as “general” (White & Huang, 2022). Adding to this dataset of charter schools and their models, researchers leveraged data collected and published by The National Center for Educational Statistics (NCES): EDfacts. This dataset provides the valuable Reading Language Arts (RLA aka. ELA) and math proficiency scores used in this paper. Edfacts does have some limitations which need to be mentioned. Proficiency data from Edfacts was merged with school model data using the NCESSCH ID.

Data limitations

The underlying model data were the best available, but not necessarily perfect. Although the effort to collect the data was certainly laborious in nature, it was not exhaustive, and information was surely missed. Additionally, because it required a school to explicitly state their model, it is possible that some schools were missed if the school chose not to publicly divulge their educational strategies. Additionally, there are some limitations when it comes to the performance data.

1. Testing data comes from schools’ grades 3-8 and one year of high school. This meant that of the 7,572 charter school records, 739, or roughly 10% did not have performance data for either RLA or Math. We would prefer to have performance data for every school at every grade level. However, the Edfacts data should nevertheless prove reasonably representative as evidenced by the countless other papers and government reports that leverage these same data (Ahearn, 2008) (Reardon et. al., 2016) (Ginsburg & Smith, 2019).

2. For student confidentiality reasons, Edfacts reports performance for some schools with low enrollment as bands such as 13%-17% proficient instead of the actual average. When the number of participants is low, Edfacts, often reports the proficiency data in a range such as “60-64%.” For these ranges, we averaged the numbers and coded the school as 62%. Because we account for the number of students in our model, the effects of these averages should be minimal. Some public schools had so few participants that Edfacts marks the range as Less than 50 or 20 or Greater than 50 or 20. This is particularly prevalent in sub-group data where small participant numbers are more common. These schools were removed from the analysis. They represented 3,043 2018-19 ELA records and 3,550 Math records or 3.4% and 3.9% of the total public school records respectively. Additionally, a further 1,336 records in the ELA file and 1,350 in the Math file were suppressed all together. Thus, the total useable ELA data was 95.1% of records and 94.6% of records for Math.

3. Across the datasets produced by NCES, multiple charter schools are sometimes reported as one charter school record when they represent several schools. Conversely, some charter schools are reported as multiple records when really those records all belong to the same school (White, 2019). We utilized the National Alliance for Public Charter School’s method of accounting for charter schools (White, 2019). This means the adoption of what they call the modified count system. This system is designed to adjust the ways in which NCES reports charter data in a way that more accurately reflects the on-the-ground situation.

4. Comparing schools nationally is not as straightforward as one might think. Each state government chooses which standardized test to adopt. Education reformers previously attempted to unify the nation around a consortium test such as PARCC or Smarter Balance (Jochim & McGuinn, 2016) (DeNisco, 2016). To make matters more complicated, some states administer customized versions of the PARCC exam (Gewertz, 2017). In 2018-19 only 16 states and D.C. use this test, 32 states use non-consortium tests, and three use a mixture of PARCC and Smarter Balance and their own questions. With so many different tests administered across the U.S. and its territories, it is very difficult for researchers to evaluate school performance nationally. We address this by evaluating a school’s performance in two ways. First, we compared models by the frequency in which they out-performed the school district and state averages. When combined with the model data previously collected and presented in this article (White & Huang, 2022), we were able to crudely compare the top ten most popular models. Second, we attempt to normalize the Edfacts data using Z scores weighted by NAEP performance. Previous researchers have used a similar strategy for district-level analyses (DeArmond et. al., 2015; Monarrez & Chingos, 2020; S. Reardon et al., 2016; S. F. Reardon et al., 2016). However, as we needed school-level data for our analysis of charter school models, our method differs somewhat.

5. Proficiency distributions vary across states. For example, the average proficiency score in math for Virginia was more than 81% whereas the average proficiency score for New Mexico was less than 22%. To standardize the data across states, we convert proficiency scores to Z-scores by using within-state averages and standard deviations.

6. The NAEP assessment data also suggests that some states outperform others on standardized assessments. We adjust the Z-scores to account for state differences identified by NAEP by applying a state multiplier, which is equal to the average percentage difference the state scored in 4th and 8th Grade. One limitation here is that we did not have NAEP high-school scores for 2018-19 and had to assume the average between 4th and 8th would nevertheless serve as a reasonable adjustment. To see the NAEP Weights for each state please see appendices 1 and 2.

NAEP-weighted Z-scores

To understand the process of generating NAEP weighted Z-scores researchers used the following formula:

$\left(\frac{SchoolLevelProficiency-StudentWeightedStateAverage}{StateStandardDeviation}\right)NAEPWeight=NAEPWeightedZScore$

With this completed, researchers aggregated NAEP Weighted Z-Scores at the model level while adjusting for participation to reduce the noise from the aforementioned Edfacts standardization limitation.

Research methods

Researchers began by calculating district and state averages in ELA and math. District and state averages were calculated using all available public schools across the nation and weighting performance by the number of students. Subsequently, the researchers compiled a comprehensive roster encompassing all charter schools that employed one of the top ten models in relation to student enrollment. Finally, they compared each school’s proficiency scores to their state and district averages to get binary variables that tell us whether the school is outperforming its state or district. This method allowed for the aggregation along model lines and across states.

The results show disparities across models. The highest performing model in 2018–19 was the Classical model. Nearly 70% of classical schools outperformed their districts and states in ELA. In math, more than 60% outperformed their districts and states. Classical schools were not the only successful model. STEM, Montessori, and Art schools did well, too. A slight majority of High-Expectations schools also outperformed their districts in math and reading. Please note the researchers are not claiming that the model itself causes the disparity. That question goes beyond what this paper explores.

Tables 1 & 2 depict national disparities descriptively and Tables 3–6 depict disparities across the four largest charter states.

Table 1. Percentage of Charter Schools Outperforming Local School District by Model.

Model	Percent Better than District - ELA	Percent Better than District - Math
Art	62%	45%
Classical	70%	62%
CTE	34%	21%
High Expectations	53%	53%
Home School	NA	NA
Language Immersion	48%	44%
Montessori	64%	47%
Progression	46%	41%
Project-Based Learning	51%	39%
STEM	65%	61%

Note: The homeschool model includes cyber charter schools which draw students from across the state. For that reason, we excluded them from Table 2 and Figure 1.

Table 2. Percentage of Charter Schools Outperforming The Stateby Model.

Model	Percent Better than State - ELA	Percent Better than State - Math
Art	54%	34%
Classical	70%	63%
CTE	26%	15%
High Expectations	33%	36%
Home School	24%	4%
Language Immersion	40%	38%
Montessori	65%	47%
Progression	37%	30%
Project-Based Learning	46%	34%
STEM	50%	50%

Table 3. Arizona’s Participation, Proficiency, and Average Scores by Model.

State	Model	Total ELA Participation	Total Math Participation	Average student score - Math	Average student score - ELA	State Weighted Math Performance - Public	State Weighted ELA Performance - Public
AZ	Art	2913	2920	35.1	45.9	43.7	45.0
AZ	Classical	12,153	12,177	63.2	64.6	43.7	45.0
AZ	CTE	168	167	17.3	16.8	43.7	45.0
AZ	High Expectations	10,237	10,242	58.9	61.1	43.7	45.0
AZ	Home School	1567	1483	30.0	36.8	43.7	45.0
AZ	Language Immersion	2,318	2,319	64.4	65.2	43.7	45.0
AZ	Montessori	2,556	2,564	52.9	65.1	43.7	45.0
AZ	Progression	3,932	3,941	50.0	52.4	43.7	45.0
AZ	Project-Based Learning	1,729	1,731	50.4	55.6	43.7	45.0
AZ	STEM	10,436	10,323	66.6	66.2	43.7	45.0

Table 4. California’s Participation, Proficiency, and Average Scores by Model.

State	Model	Total ELA Participation	Total Math Participation	Average student score - Math	Average student score - ELA	State Weighted Math Performance - Public	State Weighted ELA Performance - Public
CA	Art	8,989	8,957	42.2	56.4	39.5	50.7
CA	Classical	3,354	3,347	52.1	64.3	39.5	50.7
CA	CTE	1,897	1,875	42.4	57.1	39.5	50.7
CA	High Expectations	18035	18036	36.2	47.4	39.5	50.7
CA	Home School	18211	18150	21.7	43.9	39.5	50.7
CA	Language Immersion	8850	8862	38.8	49.4	39.5	50.7
CA	Montessori	4353	4337	34.6	52.7	39.5	50.7
CA	Progression	11541	11555	34.3	52.6	39.5	50.7
CA	Project-Based Learning	5,716	5,728	44.2 40.9	58.8 51.5	39.5	50.7
CA	STEM	16,028	16,084			39.5	50.7

Table 5. Florida’s Participation, Proficiency, and Average Scores by Model.

State	Model	Total ELA Participation	Total Math Participation	Average student score - Math	Average student score - ELA	State Weighted Math Performance - Public	State Weighted ELA Performance - Public
FL	Art	8,870	8,458	68.7	68.1	58.8	55.8
FL	Classical	4,160	4,085	78.0	78.3	58.8	55.8
FL	CTE	2,717	2,437	73.9	72.6	58.8	55.8
FL	High Expectations	1081	1048	54.8	47.0	58.8	55.8
FL	Home School	733	723	43.0	61.0	58.8	55.8
FL	Language Immersion	1,842	1,839	68.6	66.9	58.8	55.8
FL	Montessori	1,534	1,519	64.1	74.8	58.8	55.8
FL	Progression	33,275	32,471	60.8	60.3	58.8	55.8
FL	Project-Based Learning	149	148	59.2	63.9	58.8	55.8
FL	STEM	17,881	17,525	62.6	62.5	58.8	55.8

Table 6. Texas’s Participation, Proficiency, and Average Scores by Model.

State	Model	Total ELA Participation	Total Math Participation	Average Student Score - Math	Average Student Score ELA	State Weighted Math Performance	State Weighted ELA Performance
TX	Art	1,627	1,582	46.0	51.9	50.5	46.6
TX	Classical	10,517	10,197	52.4	57.2	50.5	46.6
TX	CTE	2,878	2,778	40.7	36.3	50.5	46.6
TX	High Expectations	24,478	22,850	49.9	43.6	50.5	46.6
TX	Home School	4,196	3,901	29.1	51.4	50.5	46.6
TX	Language Immersion	14,665	14,428	43.9	45.8	50.5	46.6
TX	Montessori	299	299	40.8	50.5	50.5	46.6
TX	Progression	4,031	3,729	37.4	39.0	50.5	46.6
TX	Project-Based Learning	728	582	18.5	23.1	50.5	46.6
TX	STEM	24,693	23,197	54.3	50.3	50.5	46.6

Figure 1. Top Ten Charter School Models by 2018-19 NAEP-Weighted Z Scores - RLA.

Based solely on the standardized ELA and Math exams, it appears there are large numbers of Classical and STEM charter schools outperforming their district and state peers in both ELA and math, while Montessori and Art School out-perform their local districts in ELA in most cases. However, a major limitation of this method of evaluation is that it does not show us the degree to which these schools are outperforming their peers. To get a better understanding of model performance, we next look at the distributions of schools within the model using NAEP-weighted Z-scores that allow the researchers to aggregate across states by model. Figures 1 and 2 show how well the top ten models perform in RLA and Math.

Figure 2. Top Ten Charter School Models by 2018-19 NAEP-Weighted Z Scores - Math.

Looking at the percentage of schools performing at least a standard deviation higher than the mean, we see Classical schools once again emerge at the top in both ELA (36%) and math (30%) too. One interesting finding here is that all models show potential for further investigation. With so many schools performing more than a standard deviation higher than the average, the education sector might learn countless lessons with further exploration into what underlying mechanisms are at play.

In addition to overall performance, breaking the data out by subgroups reveals some additional interesting findings, too. Tables 7 & 8 show how well the top charter school models did serving Children with Disabilities (CWD), Economically Disadvantaged (ECD) students, students classified as having Low English Proficiency (LEP), and 4^th and 8^th grade. Please note that we used only the 4^th and 8^th grade NAEP weights instead of the average when adjusting the 4^th and 8^th grade Z Scores.

Table

Top Models by 2018–19 NAEP-Weighted Z-Scores - RLA
Top Models	LEP Model Z-Score	CWD Model Z-Score	ECD Model Z-Score	4th Model Z-Score	8th Model Z-Score
Classical	0.06	0.36	0.45	0.56	0.81
Montessori	0.08	0.56	0.14	0.31	0.64
Art	0.12	0.05	−0.07	0.05	0.45
STEM	−0.08	−0.07	0.12	0.11	0.40
Progression	0.06	−0.03	0.10	0.14	0.11
Project-Based Learning	−0.27	0.10	−0.35	−0.12	0.53
Language Immersion	0.00	−0.22	−0.25	−0.05	0.05
CTE	-0.14	-0.09	−0.34	0.24	−0.26
High Expectations	0.05	−0.48	0.01	−0.14	−0.10
Home School	−0.45	−0.21	−0.30	−0.48	−0.13
General	−0.08	−0.20	−0.16	−0.06	0.03

Table

Top Models by 2018–19 NAEP-Weighted Z-Scores - Math
Top Models	LEP Z-Score	CWD Z-Score	ECD Z Score	4th Grade Z-Score	8th Grade Z-Score
Classical	−0.12	0.21	0.21	0.47	0.61
STEM	−0.03	−0.11	0.12	0.12	0.43
High Expectations	0.10	−0.39	0.13	−0.03	0.07
Montessori	−0.17	0.01	−0.15	−0.19	0.23
Progression	−0.10	−0.20	−0.06	−0.02	−0.06
Art	−0.11	−0.14	−0.34	−0.13	0.16
CTE	−0.11	−0.23	−0.39	0.01	−0.03
Project-Based Learning	−0.45	−0.18	−0.54	−0.18	0.36
Language Immersion	−0.16	−0.33	−0.37	−0.12	−0.21
Home School	−0.78	−0.64	−1.08	−0.88	−0.81
General	−0.10	−0.24	−0.20	−0.11	0.02

In RLA, CWD perform more than half a standard deviation (0.56 stdv) better in Montessori schools, the highest of any model. Students with LEP see the largest gains (0.12 stdv) in Art-focused charter schools, but also see gains in other models as well.

Classical charter schools demonstrate impressive gains with CWD and ECD students in RLA (0.36 and 0.45 stdv). In math, these populations performed 0.21 stdv higher in classical schools. High Expectations schools show gains for LEP and ECD populations in math (0.10 and 0.13 stdv) but notably do not show similar gains for students with disabilities.

Discussion

Given the myriad of data limitations, this work will require additional replication and verification, but the preliminary findings are still interesting. The authors were particularly surprised by the emergence of the Classical, STEM, Montessori, and Art models as some of the highest achieving on ELA and math exams. This paper does not reach a conclusion on why these models outperform others. They could be exceptionally effective, or there could be something unique about the families that choose these education environments. Alternatively, these models may uniquely align with math and reading exams in a way that other models like CTE, Language Immersion, and others may not. There are a variety of hypotheses that deserve testing, but this paper simply seeks to establish that these disparities exist. The fact that these disparities exist raises an important concern.

Until the charter sector can establish why these disparities exist the continued use of ELA and Math exams, alone, to evaluate the performance of specialized schools is problematic. It may be the case that authorizers are revoking charters using a framework that inequitably evaluates schools. One temporary solution to this would be a more wholistic approach to evaluation. Authorizers should work with schools to establish alternative metrics that are in line with their mission. These could include specialized exams, classroom inspections, or the inclusion of alternative outcome measures.

The authors hope that this first step will lead to new research that explores which educational components lead to success within models and what lessons might be shared and replicated across a wider array of schooling options. In this way, charter schools can leverage their autonomy to innovate for the betterment of all students regardless of their family’s choice in school.

Acknowledgments

We would like to thank James Shuls for the feedback he provided on a previous draft of this paper, as well as the attendees at the International School Choice and Reform Conference for their ideas and suggestions.

Disclosure statement

No potential conflict of interest was reported by the authors.

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Appendices Appendix 1

Table

State	Average ELA Score	Weight
DoDEA	257.5	1.071
Massachusetts	252	1.048
New Jersey	248.5	1.033
Connecticut	247	1.027
New Hampshire	246	1.023
Utah	246	1.023
Colorado	246	1.023
Wyoming	246	1.023
Vermont	245	1.019
Ohio	244.5	1.017
Idaho	244.5	1.017
Indiana	244	1.015
Florida	244	1.015
Wisconsin	243.5	1.012
Montana	243.5	1.012
Pennsylvania	243.5	1.012
Washington	243	1.010
Maine	243	1.010
Nebraska	243	1.010
Minnesota	243	1.010
Virginia	243	1.010
South Dakota	242.5	1.008
Maryland	242	1.006
North Dakota	242	1.006
Kentucky	242	1.006
North Carolina	242	1.006
Illinois	241.5	1.004
Iowa	241.5	1.004
Oregon	241	1.002
Kansas	241	1.002
Rhode Island	241	1.002
New York	241	1.002
National public	240.5	1
Missouri	240.5	1.000
Michigan	240.5	1.000
Tennessee	240.5	1.000
Georgia	240	0.998
Delaware	239	0.994
Hawaii	238	0.990
Nevada	238	0.990
Arizona	237.5	0.988
South Carolina	237.5	0.988
California	237.5	0.988
Mississippi	237.5	0.988
Arkansas	237	0.985
Oklahoma	237	0.985
Texas	236	0.981
West Virginia	234.5	0.975
Louisiana	233.5	0.971
Alabama	232.5	0.967
District of Columbia	232	0.965
New Mexico	230	0.956
Alaska	228	0.948

Appendix 2

Table

State	Average Math Score	Weight
DoDEA	271	1.040
Massachusetts	270.5	1.038
Minnesota	269.5	1.035
New Jersey	269	1.033
Virginia	267	1.025
New Hampshire	266	1.021
Wyoming	266	1.021
Wisconsin	265.5	1.019
Indiana	265.5	1.019
Connecticut	264.5	1.015
North Dakota	264.5	1.015
Pennsylvania	264.5	1.015
Nebraska	264.5	1.015
Utah	264.5	1.015
South Dakota	264	1.013
Idaho	264	1.013
Ohio	263.5	1.012
Colorado	263.5	1.012
Vermont	263	1.010
Washington	263	1.010
Montana	262.5	1.008
North Carolina	262.5	1.008
Florida	262.5	1.008
Texas	262	1.006
Maine	261.5	1.004
Iowa	261.5	1.004
National public	260.5	1.000
Kansas	260.5	1.000
Illinois	260	0.998
Tennessee	260	0.998
Missouri	259.5	0.996
Maryland	259.5	0.996
Arizona	259	0.994
New York	258.5	0.992
Georgia	258.5	0.992
Kentucky	258.5	0.992
Michigan	258	0.990
Oregon	258	0.990
Delaware	258	0.990
Rhode Island	257.5	0.988
Mississippi	257.5	0.988
Hawaii	257	0.987
Oklahoma	256.5	0.985
South Carolina	256.5	0.985
California	255.5	0.981
Nevada	255	0.979
Arkansas	253.5	0.973
Alaska	253	0.971
District of Columbia	252	0.967
West Virginia	251.5	0.965
Louisiana	251.5	0.965
New Mexico	250	0.960
Alabama	249.5	0.958
Puerto Rico	203.5	0.781