The aesthetic effects of a new lesson design approach: Mathematical stories

Abstract

Research suggests that high school students often have negative experiences with mathematics. To address this challenge, this paper shares findings of a design-based research project in which researchers and teachers developed and used a narrative approach to lesson planning in order to design lesson experiences that provide opportunities for high school students to become captivated with mathematical content (“CMLs”). The goal of this approach is to provide students positive aesthetic opportunities, such as inspiring student curiosity, while maintaining cognitive demand and coherence. Overall, students reported more positive, varied aesthetic experiences (e.g., suspense, surprise) in CMLs than in other lessons with the same teacher and students. These findings provide evidence that designing lessons as mathematical stories shows promise and can offer students more positive aesthetic experiences in mathematics.

Keywords:

• Curriculum
• aesthetics
• mathematical story
• affect
• emotion
• beliefs and attitudes
• narrative

There is ample evidence of negative student experiences within the secondary mathematics classrooms (e.g., Mullis et al., 2016). Consistently, attitudes toward mathematics decrease from 4th grade to 8th grade, according to both national (for the United States) and international measures (e.g., Mullis et al., 2016, 2020). For example, whereas 20% of international students indicated a strong dislike of mathematics at 4th grade, the proportion in 8th grade students was more than double that (41%), according to the 2019 TIMSS Assessment (Mullis et al., 2020). Similarly, when asked about their classroom experiences with mathematics, 34% of 15-year old students in nine countries participating in the 2018 PISA survey (Bulgaria, Georgia, Hong Kong, Ireland, Mexico, Panama, Serbia, Spain, and United Arab Emirates) indicated that they experienced “quite a bit” or “extreme” boredom in mathematics class (2018 Database - PISA, n.d.). These results are consistent with our own studies in high schools in the northeast United States; when we asked over 450 high school students across grades 9 to 12 in 2018 and 2019 to describe their typical experience during mathematics class, the most common response was “boring.” This discouraging state of affairs makes us determined to learn how mathematical learning experiences could be designed and enacted so that high school students instead describe them with terms such as “thought-provoking” or “intriguing.”

To reverse this trend, we advocate for the design of mathematical learning experiences to take into account the aesthetic opportunities for students. By aesthetic, we refer to the way in which a student is moved by an experience to act, such as feeling compelled to ask a question or being motivated to find an explanation for an unexpected result (Dewey, 1934; Sinclair, 2001). Note that the aesthetic dimensions of a learning experience can be either positive or negative; just as a mathematical lesson can be compelling to an individual (i.e., to gasp or ask a question), it can also repel (i.e., lead an individual to disengage). Lessons that succeed in offering students compelling mathematical experiences are aesthetically-rich, what Sinclair (2001) describes as those learning environments that “enable children to wonder, to notice, to imagine alternatives, to appreciate contingencies and to experience pleasure and pride” (p. 26). Thus, aesthetically-rich mathematical learning experiences position students as active drivers of the content while also potentially improving student views toward mathematics. There is a need to understand how to design and enact aesthetically-rich mathematical lessons for high school students.

One way to design aesthetically-rich mathematical learning experiences is by drawing on the affordances of what makes literary stories compelling and pleasurable. Analyses of high school mathematics lessons with heightened positive aesthetic responses (e.g., exclamations of “Wow!) have linked narrative moves of mathematics lessons (e.g., foreshadowing, misdirection) with positive student aesthetic reactions, such as anticipation, curiosity, and surprise (Dietiker, 2016; Dietiker et al., 2016). Building on this work, the current study explores whether using a narrative approach to designing mathematical lessons can improve student experiences with mathematics while addressing mandated curriculum topics. Working with six high school teachers, we have engaged in design-based research (Cobb et al., 2003; Edelson, 2002) through three year-long cycles. The teachers and researchers have designed multiple lessons that provide opportunities for students to become captivated with mathematical content (“Captivating Mathematics Lessons” or “CMLs”) using an emerging design framework. Our results suggest that even in their first iteration, this is a promising approach.

Theoretical framework

Sequences of mathematical content designed to unfold across a lesson or course can be interpreted as a form of narrative (Borasi & Brown, 1985; Dietiker, 2013; Sinclair, 2005). Certainly, elements of mathematics curriculum have always been associated with narrative (i.e., in the case of word or “story” problems). However, we conceptualize all unfolding content in a lesson as a mathematical story: a sequence of mathematical events (such as tasks or discussions) that connects a beginning with its end (Dietiker, 2013, 2015). Although it may be uncommon, the metaphor of mathematics-as-story has been increasingly used to conceptualize and study mathematics curriculum (see, for example, Sinclair (2005), which compares mathematical textbooks to literary stories that are “monotonous” and “uninspired”)¹. This metaphor is particularly useful because theorists of literature have long focused on ways to characterize both how events are sequenced and how their unfolding potentially creates aesthetic opportunities, such as building suspense or generating surprise. Framing mathematics lessons as stories integrates both coherence and aesthetic; when stories are incoherent, readers may not be able to follow or gain understanding, potentially leading them to quit reading. Yet when a story is coherent, the aesthetic dimensions of a story can compel a reader to keep reading and work at making sense of the story (Nodelman & Reimer, 2003).

To illustrate how a mathematics lesson can be interpreted as a story, we describe how one lesson temporally unfolded for enhanced aesthetic opportunities. The lesson began when the teacher distributed 14 cards, each with a representation of a linear function (e.g., tables, graphs), and challenged students to sort the cards into sets. Without any guidance on how to form the sets, the students, in groups of 4 or 5, began finding multiple pairs of cards that appeared to contain representations of the same function. Over time, some groups began to identify sets of three cards. After all student groups had identified multiple sets of cards, the teacher called the class together to share their findings. As students presented their sets of cards, spirited disagreements arose as proposals appeared contradictory (i.e., A with B versus A with C). Over time, through argumentation, the class resolved these disagreements, often expanding the set once students were convinced the cards represented the same function (i.e., A with B and C). At some proposals, the teacher expressed surprise or doubt (e.g., “What! You think A goes with K? That’s crazy! What do you mean?”). After this continued for 12 minutes, one student called out: “All of them go together.” Other students expressed surprise, and the teacher questioned the proposal (“You actually think that?”). After more students began to agree that the functions were indeed all the same, the teacher tasked them with proving it. In small groups, students excitedly explored whether cards that looked markedly different (e.g., because the teacher had designed the scales of different graphs so that the lines appeared to have different slopes) were actually the same. During a second full-class discussion marked by high student participation and enthusiasm, students presented convincing arguments that all the representations were of the same function. In the final moments of class, students proposed new questions about whether the y-intercepts and slopes were all the same, given that all the cards contained representations of the same function and yet appeared distinctly different.

Of course, not all mathematical stories will offer as compelling experiences as this example. Since mathematical stories can vary by their aesthetic qualities, this raises the question of how to decide whether certain aesthetic qualities are experienced differently than others. For example, is a lesson students describe as “enjoyable” aesthetically pleasing in the same way as one which the same students describe as “intriguing?” To characterize this distinction within literary stories, Barthes (1974) proposes the idea of writerly texts, those that invite creative imagination, encourage multiple interpretations, and stimulate inquiry—positioning the reader as a producer of meaning. In contrast, a literary text that is comparatively closed to multiple interpretations, leads the reader to obvious conclusions, and does not stimulate a reader’s imagination is deemed a readerly text. Drawing on this framework, we propose that some lessons position students as coauthors of the mathematical story, whereby they are afforded generative opportunities to create mathematics through inquiry, as described by Borasi et al. (1990) and Sinclair (2005)). Therefore, in this paper, we take writerly mathematical stories to be those aesthetically-rich lessons, like the linear function lesson described above, that engage students in thinking creatively about mathematical phenomena, asking mathematical questions, and persevering through complexity. Likewise, readerly mathematical stories are those that are not compelling and do not invite creative participation by students.

In this study, we also assume that aesthetic experiences vary by individual (Sinclair, 2001). Furthermore, we assume that the meaning generated throughout a mathematical story, similar to literary story, is a mixture of a reader’s goals, experience, and purpose, and thus is individual in nature (Rosenblatt, 1994). Yet, despite individual differences, we also recognize that stories can have broad appeal in general (as evidenced by best-selling novels and box-office success). Since the audience of an enacted mathematical story is arguably the students, their evaluation and description of an experience with the unfolding mathematical content is an appropriate measure of its aesthetic value. Thus, a successful mathematical story is one that holds broad aesthetic appeal to students. Broad aesthetic appeal, in turn, reflects a consensus amongst students that a mathematical story offers both coherence and captivation, even when the content is challenging.

Designing CML lessons

Working in pairs with a group of researchers, the participating teachers attended to three dimensions of mathematical stories (i.e., captivation, coherence, and complexity) to design multiple CMLs for one or more of their classes. To design lessons that are both captivating and coherent, the designers sequenced content as they attended to the emerging mathematical story (i.e., “How could we sequence the activities to enable various aesthetic experiences, such as surprise?” or, “Does this mathematical story make sense?”). In addition, to encourage the design of CMLs that maintain mathematical complexity as they build curiosity or create suspense for students, the designers also attended to the cognitive demand of CMLs (Stein et al., 2000), which describes the complexity of students’ engagement with mathematics (from low, which involves mostly rote memorization and simple procedures, to high, which involves more complex forms of reasoning such as justification and making connections).

At the beginning of the design process, the teachers selected a mathematical topic from their mandated curriculum (i.e., state standards). They often identified content areas that had been routinely challenging to make interesting to students, such as an introduction to the Rational Root Theorem or imaginary numbers. Once the topic was selected, the design group then spent time exploring the topic, attending to any notable aesthetic elements that they experienced. Next, the design group decided on an aesthetic goal they hoped students would have during the lesson, such as suspense about a mystery, or surprise at a plot twist. As the designers determined how to enable students to experience these stories, they considered both traditional aspects of lesson planning (e.g., Will students address this question in pairs or individually?) and guiding heuristic questions focused on shaping content, such as those in Figure 1. Throughout this process, the teachers and researchers endeavored to make the mathematical ideas captivating, as opposed to bringing in interesting contexts or games. That is, CMLs were designed to enable students to become curious about a mathematical object or relationship (e.g., How can these seemingly different representations of functions represent the same function?), so that students would be able to experience fascination with mathematical phenomena (e.g., concepts).

Figure 1. Guiding heuristic design questions.

We also utilized several specific tools intended to prompt the teachers to articulate and develop their current thinking about their emerging lessons. First, after each meeting, we asked teachers to reflect in writing to two questions: What is your vision of the CML at this stage? What are you still figuring out? Teachers were also encouraged to generate a representation for how the mathematical content would unfold within their mathematical stories, such as a storyboard (for example, the storyboard created for the Equivalent Linear Functions CML, described earlier, is shown in Figure 2).

Figure 2. Storyboard of a mathematical story investigating equivalent linear functions.

Methods

This study uses a design-based research approach (Cobb et al., 2003; Edelson, 2002) to begin to answer the questions: Does designing lessons as mathematical stories shift the aesthetic experiences of students, and if so, how? To what extent do CMLs account for shifts in student interest after accounting for other potential factors, such as gender and disposition toward mathematics? We aim to characterize how lessons that were designed as mathematical stories (“CMLs”) impacted student experiences in terms of interest, aesthetics, understanding, and challenge. We are also interested in learning whether potential improvements only exist for certain sub-populations of students. This paper reports on the first of three iterative cycles of instructional design and classroom-based analysis represented in Figure 3.

Figure 3. First cycle of instructional design and classroom-based analysis.

To inform the initial design process, our teachers participated in an extensive professional development prior to the school year during which the lesson observations reported in this paper were collected (2018–2019). Then, during the 2018–2019 school year, the CMLs were designed by three pairs of mathematics teachers from three high schools, along with university-based researchers, over the course of between 3 and 5 meetings of approximately an hour. The lesson topics, selected by the teachers, were sequenced within the course so that the design of each CML could take into account assumptions about previously learned content and could prepare students for future content. Because real-world contexts (e.g., rocket travel), games, and specialized technology can boost student engagement (Durik & Harackiewicz, 2007; Renninger et al., 2002) and could therefore prevent us from tying student descriptions of interest and experience with the aesthetic dimensions of the mathematical content, the CMLs were designed devoid of these elements.

When each CML was enacted in the classroom with students, researchers attended the lesson to observe. In addition to observing CMLs, researchers also observed multiple lessons for each teacher that were not designed through this process (“non-CMLs”), selected periodically throughout the school year, to offer a control group of lesson observations with the same teacher and students. The non-CMLs were “everyday” lessons that teachers taught. Just like CMLs, these lessons addressed standard curricular concepts. No restrictions were placed on their design, so they vary in terms of whether they are based on textbooks or whether teachers had enacted them in previous years.

To learn whether and how designing lessons as mathematical stories can improve the experiences of high school students, we compared the reported student experiences for CMLs with non-CMLs. Because only CMLs were restricted from having real-world scenarios, posing a higher bar to clear to increase student positive experiences, we can connect any improvement for student experiences to the aesthetic affordances of the design of the mathematical story (as opposed to the game or worldly context).

The remainder of this section describes in detail the participating teachers and schools, the data collection tools, and how the data were analyzed.

Participants

Six experienced mathematics teachers, who each had between four to seven years of teaching experience, were selected from three participating high schools (serving students aged 14–18 years old) in the Northeast of the United States so that each teacher would have a colleague² from the same school with whom to collaborate on the designs of their lessons. The schools were selected to offer contrasts: (1) a small independent charter school with mostly Latinx students and a subject-specific curriculum, (2) a large public school with a very diverse student body (representing multiple ethnic groups) and an integrated curriculum, and (3) a large public school with a majority white student body and a subject-specific curriculum. In addition to selecting schools to reflect a diversity in terms of curriculum and student populations, efforts were made to select classes³ so that the observed lessons comprised a wide mixture of subjects (e.g., geometry), selectivity (i.e., honors or non-honors), and grade levels. Since each CML was designed to take into account the teacher’s knowledge of their students (i.e., what might interest them) and their prior understanding, the diversity of classroom contexts (i.e., content, grade level, and school demographics) enables us to identify design principles that might function beyond the specific classrooms of our participants.

Each teacher designed three CMLs for at least one of their courses. This resulted in only seventeen CMLs designs because two teachers co-designed a CML that they each enacted in their own classes. The distribution of lesson observations for each teacher, along with the course subject, predominant grade level of students, and number of observations for CMLs and non-CMLs is provided in Table 1. Across the 8 classes, approximately 80% of students participated in the study, for a total of 141 students. The number of participants in each class was between 12 to 25 students, with an average of 17.6 students (sd = 4.4).

Table 1. Participating teachers and classes with subject, grade levels, and numbers of CML and non-CML lesson observations.

Teacher	Class	Subject	Grade**	Number of students	Number of CMLs	Number of non-CMLs
A	A1	Integrated Math 1	9	17	3	2
B	B1	Geometry	10	13	1	1
	B2	Algebra 2	10	12	2	2
C	C1	Algebra 2	10	21	2	2
	C2	Algebra 2	10	21	1	1
D	D1	Algebra 2	11, 12	17	2	2
E	E1	Integrated Math 3 H*	9, 10, 11	25	2	2
F	F1	AP Calculus*	12	15	3	1

* Indicates course has selective enrollment (i.e., enrollment is restricted only to some students based on criteria).

** In the United States, grade 9 typically represents the first year of high school, and for the schools in this study, includes students aged 14–15 years old. Grade 12 is the last year of high school.

Data collection and measures

This study draws from a mixture of data: disposition surveys, CML and non-CML lesson observations, post-lesson surveys of student experiences, and post-lesson student interviews. Each of these data collection methods is described here.

Student disposition survey

To explore student-related factors that might influence student experiences with CMLs, at the start of each school year, all students were given a Disposition Survey. In addition to questions to collect demographic information (i.e., gender identification, ethnic and racial identities, and home language(s)), this survey included a mixture of Likert survey items from Trends in Mathematics and Science Study (Mullis et al., 2016) and the TRIPOD tool (Ferguson & Danielson, 2015) to measure perceptions of mathematics (e.g., does the student like math?) and perceptions of their learning experiences (e.g., does the student think the teacher cares about them?) (see Riling et al. (2018) for more information on the creation and testing of this tool). Because we suspected that a student’s enjoyment of any particular mathematics lesson was likely influenced by that student’s view of mathematics, an aggregate captivation measure was generated for each student by taking the average of students’ responses to three items about whether or not they agree with the statements (using a scale of 1 (strongly disagree) to 4 (strongly agree)): math is boring⁴, I learn interesting things in mathematics, and I like to think about mathematics or solve puzzles outside of school.

Lesson observations

Each CML and non-CML was videotaped and observed by researchers, who took fieldnotes. A single protocol was used to collect data for all enactments so that students would not be aware of which lessons were designed with the mathematical story design framework. These observations enabled us to select students for the post-lesson interview (see later section). The observation recordings and fieldnotes were consulted when analyzing the survey and interview results to connect specific mathematical activities (e.g., when a teacher and students simplify a complicated composition of a function and its inverse) with the student aesthetic reports (e.g., a student describing surprise when the messy expression simplifies to just x).

Student Lesson Experience Survey

Immediately after each lesson, a Lesson Experience Survey (see Figure 4) was administered to each student to be taken independently using Qualtrics. Across the 29 observations, 443 surveys were completed, 238 of which were for CML lessons. To measure students’ perceptions of the lesson, this survey asked students multiple questions to measure their level of interest in the lesson, such as the number of positive descriptors selected in #1, the category chosen to describe their feeling during the lesson in #2, and agreement with the statements “time flew by” and “I wish more lessons were like this one” in #4. In addition, it prompted students to rate their level of understanding, the degree of challenge, and the degree of relevance. It also prompted students to select three descriptions that describe their experience during the lesson (e.g., intriguing, just OK, dull), which were displayed in a random order. These descriptors include terms that reflect positive, neutral, and negative experiences, which were generated from pilot surveys conducted with students to identify terms they use to describe an array of aesthetic experiences (Riling et al., 2019).

Figure 4. The Lesson Experience Survey (LES).

Although the pilot study results confirmed our assumptions about which descriptors represent positive and negative experiences (e.g., dull was negative and intriguing was positive) there were two descriptors, frustration and not special, that had unexpected classifications. Frustration, which originally was assumed to describe negative experiences, was instead classified as a neutral descriptor because it was used by students to describe both positive and negative experiences. That is, some students selected it along with other positive descriptors and indicated a high interest level, while others selected this descriptor along with negative descriptors and indicated a low interest level. In addition, not special, which was originally classified as a neutral descriptor was identified as a negative descriptor because it was selected overwhelmingly by students who also selected dull or boring and indicated a low interest level.

Cronbach’s alpha results indicate an acceptable level of reliability for the survey items related to lesson interest (α = 0.773). In addition, an analysis of the relationship between student interest and the qualitative descriptors showed strong validity; students with high interest measures selected descriptors associated with high interest (e.g., amazing) while students with low interest measures selected descriptors associated with low interest (e.g., dull). See Riling et al. (2019) for more information about the design and test of the Lesson Experience Survey.

To enable the comparison of experiences that, although different (e.g., amazing, suspenseful), could reflect shifts from negative to positive experiences overall, we created an aggregate positive descriptor measure for each student survey based on the number of positive descriptors selected (between 0 and 3). We similarly generated aggregate neutral and negative descriptor measures. Finally, we generated a net descriptor measure for each survey response that was the sum across the descriptors, where each positive descriptor was weighted +1, each neutral descriptor was 0, and each negative descriptor was −1, resulting in a measure between −3 and 3. For example, for a student who used one positive descriptor and two negative descriptors, the net descriptor measure would be −1.

Student post-lesson interviews

After students responded to the LES, between 2 and 5 students were selected for individual interviews. The students were selected to inform our understanding of a mixture of aesthetic experiences. That is, to learn more about the types of experiences students described with a variety of descriptors, we selected a mixture of students that appeared to have different experiences during the lesson. For example, if researchers heard an audible gasp or an exclamation (e.g., “Wow!”) during the lesson observation, we selected this student to learn more about student perspectives on surprise or other positive aesthetic opportunities. We also selected students who appeared uninterested to learn how lessons can be dull or boring. Care was taken so that the set of students interviewed would reflect the class demographics in terms of gender, race, and ethnicity. Interviewers asked students to describe any feelings they remembered having and to connect these feelings with what was happening in the lesson. Often, further questions were posed to clarify the connection between the events of the lesson, the mathematical ideas involved, and the aesthetic experiences they described. Overall, 71 interviews were conducted across the lessons. These interviews were analyzed for the writerly or readerly qualities of the student mathematical experiences, and how these qualities corresponded to the student’s selected descriptors on their Lesson Experience Survey. Specifically, we analyzed how the interview described aesthetic experiences (i.e., what were they compelled or repelled to do?) and how these experiences provided opportunities for creativity, curiosity, and perseverance.

Data analysis

To learn about the impact of lessons designed as mathematical stories on student aesthetic experiences, we first compared students’ descriptions of their experiences in CMLs in comparison to non-CMLs. To learn if CMLs received a different proportion of each type of descriptor, we conducted a paired sample t-test of the net descriptor measure for students who had at least one CML and one non-CML (n = 115) to compare students’ use of different descriptors. Additionally, we identified trends in the students’ selection of descriptors for individual lessons (e.g., are certain descriptors selected more often for CMLs or non-CMLs?).

Next, we investigated the impact of CMLs, as compared to non-CMLs, on students’ lesson interest. This enabled us to see how much of any increased student interest in CMLs could be attributed to CML design, rather than other factors such as captivation, gender identity, and course selectivity. To learn whether the benefits of CMLs could be explained by a lack of challenge or because students found the lessons more relevant, we also used paired t-tests at the student level. These enabled us to explore potential differences between the CMLs and non-CMLs in terms of other student-reported perceptions of the lesson: relevance, challenge, and student understanding.

To learn whether and how CMLs impact students’ interest in lessons, we created a model to predict student interest in a lesson using Linear Mixed Models (LMM), an extension of linear regression models that can include random effects (i.e., class effects) and correlated errors (West, 2009). We chose to use LMMs because this approach allowed us to take advantage of the fact that the data are nested and correlated, as students from different classes repeatedly participated in Lesson Experience Surveys at the end of each observed lesson, which violates the assumption of independence. Also, not all classes participated in the survey for the same number of times or at the same interval, making for data sets of unequal sizes. LMMs can accommodate such unbalanced design, missing data, as well as the study’s nested data structure (i.e., the Lesson Experience Surveys were collected at class level, but analysis was conducted at student level).

This model of lesson interest was developed by successively testing whether different factors (e.g., class selectivity) improved the model. We were particularly interested in how CMLs might impact the aesthetic experiences of students who are historically disenfranchized from mathematical learning, with regard to gender identification, race, and first language. Unfortunately, the structure of our data does not enable us to assess the impact of student racial identification or first languages on students’ lesson experiences; the student demographics of each school in the study is so different (with little overlap) that it is not possible to distinguish between the effects of a student’s school and their racial identification or first language. We also are not able to assess any impact of grade level, as the participating teachers taught students in different grades and thus it is not possible to distinguish between teacher and grade level effects. On the other hand, because there were students who self-identified as males or as females⁵ enrolled in each of the three schools and all six teachers’ classes, we investigated the relationship between gender identification and student lesson experiences⁶.

Therefore, we started building our model by testing the random factor of student class (i.e., the class in which the student was enrolled). Next, a sequence of fixed factors was added to the model, testing whether each factor improved the model: student captivation, course selectivity, lesson type (CML or non-CML), and student gender (male or female). In addition, the interactions between student captivation and lesson type, student gender and lesson type, and student captivation and gender were tested as potential factors. The goodness of fit of two models after adding each fixed and random factor was determined by using a likelihood-ratio test and its associated p-value. When the newest additional factor was fixed, we calculated the χ² distribution by comparing the maximum likelihood (ML) of the new model to that of the existing best-fit model. For random factors, we considered the change in the restricted maximum likelihood (REML). A new model was considered a better fit if χ² > 3.84, indicating $p\le 0.05$ (at 1 degree of freedom, due to adding one factor upon each iteration of the model) (West, 2009).

Findings

To describe the ways CMLs shifted student aesthetic experiences with mathematics, we begin this section by presenting overall similarities and differences in how students described their experiences in CMLs and non-CMLs on the Lesson Experience Survey. To better characterize these shifts and represent the complex experiences student surveys can sometimes oversimplify, we also provide student descriptions of their experiences from the post-lesson interviews. We then describe the similarities and differences in the aesthetic experiences offered by individual CMLs and non-CMLs at the lesson level. We end this section by describing whether and how different characteristics of students (e.g., gender, captivation with mathematics), class (e.g., selectivity), and lesson design (e.g., CML or not, complexity) act as factors that influence student interest and how students perceive their experiences in CMLs.

Overall shifts in student-reported aesthetic experiences with mathematics

Collectively, our analysis indicates that CMLs improved the quality of the aesthetic experiences of students with mathematics. A higher percentage of students used positive aesthetic descriptors to describe their experience in CMLs (57.8%) than non-CMLs (46.8%). Furthermore, the average number of positive descriptors used by students after CMLs (1.71) was greater than those selected after non-CMLs (1.48), and this difference was statistically significant (t(114) = 2.75, p < 0.01). This improvement also held for net descriptor measures; the descriptors selected after CMLs were more positive overall even after neutral and negative descriptors were factored in (1.37 for CMLs compared to 1.02 for non-CMLs), and this difference was statistically significant (t(114)= 2.59, p < 0.05). Moreover, the average student interest measure was greater for CMLs (2.92) than non-CMLs (2.77), and this difference was statistically significant (t (114) = 2.90, p < 0.01).

Although, overall, students picked more positive descriptors for CMLs, there was a wide variation in which descriptors were selected. Figure 5 displays the percentage of students who selected each descriptor for CMLs and non-CMLs in decreasing order from left to right according to the average interest (in parentheses) of the students who selected the descriptor. From the graph, it can be seen that students selected the positive aesthetic descriptors thought-provoking, amazing, fun, suspenseful, surprising, intriguing, funny, and fascinating more frequently when describing their experience with CMLs. Non-CMLs received a higher percentage of two positive descriptors (enjoyable and satisfying), and all neutral and negative descriptors, with the exception of frustrating. Despite these overall differences, the differences for most individual descriptors are small and not statistically significant. This is likely because aesthetic reactions differed across CMLs and non-CMLs and because students were only allowed to pick three descriptors. Yet some descriptors are significantly impacted by CMLs; when comparing CML and non-CML descriptors at a student level, two descriptors show statistically significant differences: intriguing (t(114)= 2.66, p < 0.01), which was selected more frequently for CMLs, and dull (t(114) = −2.06, p < 0.05), which was selected more frequently for non-CMLs.

Figure 5. Proportion of responses for descriptors on the Lesson Experience Surveys, along with corresponding average student interest levels.

We also found evidence in the student interviews that these shifts in aesthetic descriptors indicated stark differences in the mathematical experience of the students. That is, students who selected one or more of the descriptors more common to CMLs (thought-provoking, amazing, fun, suspenseful, surprising, intriguing, funny, and fascinating on the LES) often described a mixture of strong emotions they felt in relation to the emerging mathematical ideas in the interview. For example, one student, who selected suspenseful, surprising, and enjoyable on the LES, described her spontaneous curiosity, even posing a question, after listening to another group’s proposal that many seemingly different linear functions were representations of the same function. She explained, “I was like, ‘HOW?!’⁷ because like, they cannot be, like, connected to each other.” Other students who selected these positive descriptors shared how they felt fascinated by the unfolding mathematical ideas such as a new emerging pattern. For example, after a lesson introducing the Rational Root Theorem, a student, who selected the descriptors thought-provoking, fascinating, and fine in the LES, described how the mathematical ideas inspired him to propose a conjecture about the roots of polynomials. He explained,

Well, see, the way I do stuff, figure out stuff, is like seeing if there’s like a pattern, cuz patterns usually means that something’s right, cuz, it’s a pattern, you feel me? So when I saw the coefficients, with the pattern of the roots, I thought that maybe like, it’s kinda crazy, but maybe the coefficients are related to the polynomial, in the roots.

He was so captivated by the connection that he continued to ponder his idea throughout the interview, suggesting additional nuance that he thought might explain why the pattern that he noticed did not repeat in other examples. As these two examples demonstrate, students who selected these descriptors experienced moments in which they were compelled to author mathematical ideas.

Although there were two positive descriptors, enjoyable and satisfying, that students used more frequently to describe non-CMLs as compared to CMLs, students seemed to use them to describe more readerly experiences in non-CMLs. For example, one student who selected enjoyable, fine, and just ok on the Lesson Experience Survey explained, “it’s like one of the typical classes we tend to have so it was, you know, I’m used to this type of class.” Another student, who selected satisfying, fun, and fine and on the survey explained that they found the lesson “enjoyable” because they appreciated working in a group with other students, saying, “We did an activity with the group, and like when we were in a group I learned, um, a bit more about how to solve some of the factoring problems that we were doing.” This remark suggests that the student participated more as a receiver, rather than an author (i.e., generator), of mathematical ideas. This quality was echoed by a different student who described their satisfaction of getting the correct answer to a problem assigned to them, stating, “It was satisfying… because like, I was able to solve out the whole problem and then get an answer.” Unlike the students who used words such as fascinating and surprising, these students did not describe moments of mathematical creativity, suggesting that although these experiences may have been pleasant, they were not centered on student creation of mathematics.

As stated previously, neutral terms (i.e., fine, just ok, and frustrating) were selected frequently for both types of lessons, but more frequently after non-CMLs than CMLs. Our analysis indicates that these descriptors are not indicative of a common experience. For example, 29% of students selected fine with one (24%) or two (5%) negative descriptors, while 77% of students selected fine with one (40%) or two (37%) positive descriptors. We interpret this to mean that fine has both positive and neutral connotations and likely was selected on a sizable number of surveys because students were required to make three selections. Furthermore, the term frustrating appears to have different connotations for students across the lessons. It was frequently considered to be negative in non-CMLs. In one non-CML, a student who selected dull, boring, and not special on the Lesson Experience Survey used frustrating during a post-lesson interview to describe her dissatisfaction, stating, “I was kind of bored. It said it was a challenge but it really wasn’t. … When I’m in math class, I like to be challenged and like, if I get it right, I’m happy about it, because I like the satisfaction of getting it right.” In contrast, frustration and challenge in CMLs was often appreciated. For example, in the Equivalent Linear Functions lesson described previously, a student who selected fun, frustrating, and fascinating in survey explained during a post-lesson interview that the CML “was frustrating because it had numbers that were tricky… like I didn’t know what to do with those [but] towards the end, I knew they all went together [and] had to figure out how.” This student went on to explain that their frustration was “a good one” because it enabled a satisfying reveal.

The shifts in variation of collective student aesthetic experiences

Since CMLs could improve student experiences overall without shifting the types of experiences offered to students (i.e., students indicate that the lessons are more interesting but do not agree on the type of aesthetic experiences as indicated by the selection of descriptors), we explored whether the CMLs or non-CMLs had broad appeal, and, if so, what the nature of that broad appeal was (i.e., whether these lessons enabled certain types of aesthetic experiences for many students). When comparing the descriptors students used to describe individual lessons, there are notable differences when CML lessons are compared with non-CMLs. Except for fine, the descriptors that were selected by a large proportion of students varied greatly by lesson, indicating that different lessons offered different types of experiences. These differences are visible in Table 2, which lists the proportion of the participating students who selected each descriptor for each lesson⁸. The CMLs and non-CMLs are grouped and listed by decreasing average lesson interest. The dominant aesthetic qualities, as determined by the proportions of student survey responses for the lesson (i.e., the highest percent of students selecting the descriptor for the row, excluding fine⁹) or a dominant presence of the descriptor across lessons (i.e., the percent of students is greater than or equal to the 90th percentile for the column), are shaded. For example, although 36% of students in the “Geometric transformation” lesson selected thought-provoking, which was greater than the proportions of the students who selected the dominant aesthetic qualities suspenseful (14%) and amazing (21%), this proportion is relatively small when compared with the proportions of thought-provoking for other lessons (for which 54.7% is the 90th percentile). Yet the terms suspenseful and amazing were each above the 90th percentile (12.9% for suspenseful and 15.7% for amazing), so they were highlighted for this lesson, unlike thought-provoking. On the other hand, although the 40% of students describing the “Area of trapezoids” lesson as thought-provoking is below the 90th percentile threshold, it is a dominant aesthetic quality of the lesson because it was the most frequent descriptor, other than fine, selected by the students to describe their perceptions of this lesson.

Table 2. Proportion of descriptors selected by students after each lesson, with dominant aesthetic qualities* for each lesson highlighted.

		Positive										Neutral			Negative
Topic	Interest	Suspenseful	Amazing	Fascinating	Fun	Funny	Enjoyable	Satisfying	Intriguing	Surprising	Thought-provoking	Fine	Just OK	Frustrating	Not special	Dull	Boring
CMLs
Equivalent linear functions	3.69	23%	8%	8%	62%	38%	62%			31%	31%	15%		23%
Geometric transformations	3.43	14%	21%	14%	29%		64%	21%	36%	7%	36%	43%	14%
Rational Root Theorem	3.19	5%	5%	24%	24%	5%	38%	19%	24%	14%	48%	57%	29%		10%
Volume of solids	3.14	7%			64%	21%	57%	21%	21%	21%	21%	29%	21%	14%
Polynomial division	2.94		22%	33%	50%	6%	33%	11%	17%	6%	39%	44%	22%	6%	6%		6%
Exploring tile patterns	2.93		7%	13%	13%		27%		27%	13%	33%	47%	60%	40%	13%	7%
Logarithmic identities	2.90	20%		10%	20%	10%		10%	40%		40%	50%	40%	10%	20%	10%	20%
Extraneous solutions	2.89		6%	6%	11%		28%	6%	39%	6%	61%	61%	33%	17%	11%	6%	11%
Function horizontal translation	2.88			6%	24%		24%	29%	35%		12%	76%	35%	18%	18%	12%	12%
Derivatives of exp. functions	2.82			9%	9%		27%	27%	64%		64%	45%	27%			9%	18%
Imaginary number intro	2.80			20%	7%	7%	20%	7%	20%	33%	60%	47%	27%	20%	13%	13%	7%
Area of trapezoids	2.80		10%		10%		30%	20%	20%	10%	40%	60%	10%	20%	30%	10%	30%
Riemann sums intro	2.79				7%		21%	14%	29%	14%	50%	50%	36%	29%	21%	7%	21%
Introduction to the unit circle	2.77		5%		18%		27%	18%	27%	9%	41%	45%	50%	9%	14%	18%	18%
Logarithmic identities	2.73						27%	7%	33%	13%	47%	87%	40%		27%	7%	13%
Introduction to inverses	2.36		9%	18%	9%	18%	9%	9%	9%	9%	18%	55%	55%		45%	18%	18%
Non-CMLs
Scatter plots and trend lines	3.18		12%	6%	53%	6%	59%		12%	29%	18%	59%	35%			6%	6%
Logarithm basics	3.04	4%	4%	8%	20%		44%	28%	16%	12%	32%	60%	44%		20%		8%
Graphing linear inequalities	2.86		14%	21%	29%	7%	64%	7%	7%		29%	64%	29%	7%	7%	7%	7%
Rational expressions	2.81		25%	13%	19%	13%	50%	6%	25%	6%	44%	13%	31%	31%	6%	6%	13%
Exponential decay (with dice)	2.80	7%			27%		60%	20%	27%	7%	27%	73%	20%		20%	7%	7%
Systems of inequalities	2.71			6%			24%	24%	18%		41%	82%	59%	18%	12%	12%	6%
Props. of logs, exp. eqns.	2.69	13%		6%		6%	25%	6%	38%	6%	38%	63%	31%	13%	25%	19%	13%
Surface area of cones	2.67				25%		42%	25%	25%	8%	17%	42%	33%		25%	25%	33%
Percent change (review)	2.65				17%	4%	22%	13%	4%	9%	30%	65%	30%	13%	48%	43%
Differentiation with Prod. Rule	2.60			7%	13%	7%	20%	7%	27%		53%	67%	53%	13%	20%	7%	7%
The unit circle	2.54			8%	8%	8%	15%	15%	15%	15%	31%	69%	62%		23%	15%	15%
Inverse functions	2.36			9%	18%			18%		9%	9%	45%	45%	27%	36%	27%	55%
Logarithm change of base	2.18				8%			8%		17%	25%	33%	50%	33%	42%	25%	33%

* Note that shading indicates that the aesthetic quality of lesson was dominant based on student survey responses for the lesson (i.e., the highest percent of students selecting the descriptor for the row, excluding fine) or a dominant presence of the descriptor across lessons (i.e., the percent of students is greater than or equal to the 90th percentile for the column).

Not surprisingly, different dominant aesthetic descriptors were selected by students for different lessons. Since CMLs were designed with different aesthetic goals (e.g., surprising students, inspiring curiosity), we expected some aesthetic descriptors, such as surprising, to be selected by students to describe some CMLs but not others. Some of these descriptors that describe particular types of aesthetic experiences, such as suspenseful and amazing, were not selected routinely by students after all lessons, but were chosen by a relatively high proportion of students after some CMLs. For example, although the descriptor suspenseful was only selected on 3% of surveys overall, it was selected by 23% of students to describe the CML on equivalent linear functions (described previously). Suspenseful was also selected by 20% of students to describe a lesson focused on logarithmic identities, during which students worked in groups to identify patterns and develop rules about logarithms through arithmetic exploration. In comparison, most CMLs and non-CMLs were not described as suspenseful by any students.

The patterns in the dominant aesthetic descriptors selected for CMLs and non-CMLs were similar to those found when comparing the descriptors across the two sets of lessons. While individuals experience mathematical stories differently, for CMLs, the dominant qualities were more positive and less negative overall when compared with non-CMLs. As there was a greater number of CMLs overall, we expected dominant aesthetic descriptors to emerge for more CMLs, and so we identified descriptors that were dominant in at least two CMLs more than non-CMLs. Using this criteria, we found that overall, more CMLs had a dominant aesthetic quality of being suspenseful, fun, funny, intriguing, and thought-provoking. In contrast, non-CMLs were more frequently described as dull and boring. This is not to say that non-CMLs did not offer positive experiences. For example, more non-CMLs (6) were described as enjoyable when compared to CMLs (2). Overall, however, the CMLs successfully offered students an array of aesthetic experiences that were not often offered by non-CMLs.

Impact of CMLs on student interest when considering other factors

The improvement of aesthetic experiences with mathematics for students in CMLs were observed in all eight classes, for all grade levels, and for all mathematical courses. Table 3 summarizes the average student measures per class for CMLs and non-CMLs. On average, students in all 8 classes indicated more interest in CMLs when compared to non-CMLs. In addition, six classes showed an increase in the number of positive descriptors selected by students to describe CMLs compared to non-CMLs, whereas two remained relatively constant.

Table 3. Impacts of CMLs across distribution of classes (where class “A1” = teacher “A,” class “1”).

Class	Weighted average positive descriptors			Weighted average student lesson interest measure
	Non-CML	CML	Difference	Non-CML	CML	Difference
A1	1.87	2.10	+0.23	3.04	3.33	+0.29
B1	1.42	1.40	−0.02	2.67	2.80	+0.13
B2	0.73	1.29	+0.56	2.33	2.62	+0.29
C1	1.13	1.45	+0.32	2.64	2.89	+0.25
C2	2.00	2.17	+0.17	2.81	2.94	+0.13
D1	1.55	1.50	−0.05	2.74	2.77	+0.03
E1	1.34	1.74	+0.40	2.85	2.98	+0.21
F1	1.33	1.90	+0.57	2.60	2.92	+0.32

Moreover, CMLs influenced the experiences of students in different types of courses and with different mathematical dispositions (“captivation”) as indicated by the Disposition Survey. For example, the greatest shifts in both positive descriptors and lesson interest occurred in very different curricular contexts: an AP Calculus class (F1) and a nonselective Algebra 2 class with the lowest perception of lessons in general (B2). Furthermore, the CML with the highest level of student interest overall (the Equivalent Linear Functions lesson, listed in Table 2) was in Class A1, a freshman-level nonselective mathematics course addressing topics from Algebra 1 and Geometry. Therefore, this analysis provides some evidence that CMLs improve student interest regardless of whether the course has selective enrollment or is of an advanced level (i.e., AP Calculus versus Math 1).

It is reasonable to question whether the effect of CMLs on student interest was in part due to a reduction in the degree of mathematical challenge or by increasing relevance (i.e., connecting content to student lives). However, a paired t-test at the student level showed that there was virtually no difference between non-CMLs and CMLs in terms of student perceptions of lesson relevance (mean = 2.29 and 2.17, respectively, t(114) = 1.87, p = 0.06), mathematical challenge (mean = 2.46 and 2.42, respectively, t(114) = 0.55, p = 0.58), or their understanding of the content (mean = 2.53 and 2.54 respectively, t(114) = −0.23, p = 0.82). The fact that student reports of relevance of CMLs and non-CMLs are not different is surprising given that non-CMLs were able to include engaging contextual scenarios, while CMLs were not. The lack of statistically significant difference in understanding or challenge in the two groups of lessons suggests that the increased interest in CMLs as compared to non-CMLs was likely not due to easier topics being selected, lessons being less frustrating, or lessons being taught in a manner that reduced the perception of challenge. This is consistent with our results described earlier that the descriptor frustrating was selected in higher proportion to describe CMLs, and that it was selected by 23% of students to describe the lesson that had the highest student interest overall.

In light of our assumption that students with higher captivation would be more likely to express greater interest in mathematics lessons overall, we explored how much of the increased interest for CMLs can be attributed to the CMLs, rather than other factors such as captivation. Our LMM analysis shows that student captivation, gender, and the interaction of gender and captivation explain a portion of student interest (see Table 4). In fact, an increase of 1 captivation level predicts lesson interest will increase by 0.46, confirming our assumption. Students’ gender also impacted their interest level, where females saw a reduction in interest of 0.55 on average (p < 0.05). Although captivation improved the lesson interest for both males and females, these effects were qualified by an interaction between gender (female) and participant captivation (0.27, p < 0.01). That is, captivation improved females’ interest to a greater extent. Furthermore, incorporating class as a random variable did improve our model overall; it explained a small amount of variance between classes after all other variables were taken into account (0.02 points, p = 0.119). Factors that did not improve the model include course selectivity, interaction between captivation and lesson type, and interaction between gender and lesson type (see Appendix). After taking all these additional factors (i.e., captivation, gender, interaction between gender and captivation, and class) into account, a CML, on average, improves student experience by 0.24, and this improvement is statistically significant (p < 0.01).

Table 4. Model of student interest.

Factors	Impact on Student Interest (Standard Error)
Residual Variance	0.32 (.02)**
Intercept	1.82 (.18)**
Random effects
Class	0.02 (.01)
Fixed effects
Captivation	0.46 (.07)**
Non-CML	0 (.00)
CML	0.24 (.06)**
Gender (Female)	−0.55 (.24)*
Gender (Male)	0 (.00)
Female* Captivation	0.27 (.09)**
Male* Captivation	0 (.00)

* p < 0.05, ^**p < 0.01.

Discussion

With persistently negative views of school mathematics, particularly at the high school level, there is a pressing need to develop strategies that give students aesthetically-rich experiences in mathematics. From our first year of design-based research, we are encouraged by evidence that it is possible to design and teach mathematics in a way that students find more engaging and interesting, and therefore more “writerly.” Mathematical stories in settings such as coordinate planes and algebraic spaces can captivate high school students. Although the structure of our data did not allow us to measure differential effects CMLs had on grade level or certain student demographics (i.e., race or home language), the mathematical stories designed and enacted by our participating teachers showed heightened student interest across all eight classes, for each grade level, each course, and regardless of whether students were selectively enrolled. We suspect that our results might even downplay the difference that CMLs could make for students, as the participating teachers were already predisposed to incorporate challenges within their lessons and had experience using the teaching practices needed to enact these challenges in their classes. In other classroom settings, where CMLs may stand out as more novel, mathematical stories may have an even larger positive impact on students. Our study also shows that, in addition to improving student interest in CMLs overall, lessons designed as mathematical stories can indeed broaden aesthetic opportunities for students such as offering suspense and surprise. Overall, high school mathematics lessons can create what Sinclair (2001) refers to as “aesthetically-rich learning environments” (p. 26) by enabling students to wonder (i.e., finding a lesson fascinating, intriguing, or thought-provoking), to imagine alternatives and appreciate contingencies (i.e., surprising, suspenseful), and to experience pleasure with mathematics (i.e., amazing, fun). These results offer us hope that low student dispositions in high school mathematics can be improved.

Another goal was to position students as creators of mathematics—that is, students who can pursue solutions to problems they themselves have posed. Our findings suggest that the CMLs were successful in enabling this to happen. First, the increased level of student interest during CMLs indicates that students were more likely to be engaged in the mathematical content. In addition, consider that the descriptors thought-provoking, intriguing, and suspenseful were used more for CMLs than for non-CMLs. The increased frequency of the descriptor thought-provoking, which was statistically significant, suggests that students encountered something in the lesson that pushed them to think beyond the ideas presented by their teachers. The selection of intriguing suggests that students were curious about the answer to one or more questions. Lastly, the descriptor suspenseful could be interpreted as a heightened state of intrigue. Even though in these lessons, the students ultimately learned about topics introduced by their teachers and mandated by curricular policy, their selection of these descriptors indicates that they took ownership of the questions that arose. We suspect that in many cases, students even posed the questions before their teachers did so; exploring this is part of the future work we intend to do with these lessons.

One concern might be that enacting aesthetically rich mathematical learning experiences in the classroom could come at the expense of student learning and mathematical rigor. Although the variations in the content of the lessons in our data set did not allow us to measure student learning, we are happy that CMLs overall were able to increase student interest without decreasing their perception of rigor (i.e., degree of challenge) or understanding. Since the mathematical story design framework supports the design of lessons that alternately heighten and reduce tensions between what is known and not known by the students, CMLs are likely to be perceived as challenging while still being interesting to students. This interest may enable students to persevere in making sense of the challenging mathematical ideas. Further, because designing a CML includes designing how the story will move past the conflict, the participating teachers may have paid more attention to how students would be able to successfully move beyond any perceived challenge.

Study limitations and future research

We acknowledge that the design of this study limits our ability to fully understand the effects of CMLs on student interest. For example, due to the fact that the teachers in this study designed these CML lessons with other teachers and researchers with extensive time and feedback, these aspects of the study likely contributed to their success. Thus, this study shows that these high school mathematics teachers, who each had extra planning time and design support (e.g., colleagues, researchers, and the mathematical story framework), designed and taught CMLs that raised student interest overall. Additional research is now needed to learn whether and how having extra planning time or design support contributed to the success of CML lessons. In addition, these lessons were designed specifically for the students and their curricular contexts. As this was not an experiment, we do not claim that these CML lessons would necessarily appeal to students in other classrooms.

The structure of our data also limited what we were able to analyze. For example, although the design of our study did not enable us to explore the nested nature of our data (i.e., the students within classes, within schools), an expanded study is needed to learn how the benefits of the CML design approach is moderated by the school or classroom context within which the CMLs are enacted. The fact that our groups of participants at each of the three schools in our study had very distinct compositions with respect to student race, home language, and grade level prevented us from analyzing the impact of CMLs on these groups. We hope future research can augment the data to enable the study of the impact of CMLs on student interest across different student populations, focusing on those groups who have traditionally experienced marginalization within mathematics.

We recognize that this study is just the beginning; future work also needs to explore other potentially positive aesthetic experiences for high school students in mathematics. While we are heartened to learn that students described CMLs as suspenseful, fun, funny, intriguing, and thought-provoking in greater proportions, we wonder what other types of positive aesthetic goals teachers could design for in mathematics.

Concluding thoughts

We imagine a future in which high school students regularly describe mathematics as inherently “intriguing” or “captivating.” If topics like the rational root theorem or polynomial division can be taught so that students describe them as “fascinating,” then we call on the field to explore further aesthetically-rich opportunities for students with advanced and abstract mathematical content. By learning more about what a narrative approach can reveal about the design of mathematical content, we hope to move mathematics education toward a more expansive mindset when it comes to what gets taught (and deemed important) in high school mathematics classrooms. We look forward to a day when all students eagerly look forward to the next mathematical topic—not because it is required, but because they want to know where the story will take them next.

Disclosure statement

The authors report there are no competing interests to declare.

Additional information

Funding

This material is based upon work supported by the National Science Foundation under Grant #1652513. We would also like to thank Miriam Gates and Anna Moss for their help in collecting this data and share our appreciation for the many contributions of the teacher participants in this study.

Notes

1 For the interested reader, this metaphor is used to make sense of mathematical sequences in a variety of areas of mathematics education, such as Borasi and Brown (1985), Darby-Hobbs (2013), Gadanidis and Hoogland (2003), Meyer (2011), and Zazkis and Liljedahl (2009).

2 Although the participating teachers at the same school taught within similar contextual and curricular constraints, they did not necessarily teach the same course.

3 Note that in this study, course will describe the name of the subject (e.g., Algebra 2) being learned, while class will refer to a particular group of students and teachers who experience the course.

4 This item was reverse-coded in the analysis.

5 In this study, all references to a student’s gender (e.g., female) reflects the student’s identification of gender, unless otherwise stated.

6 It should be noted that when student gender was part of an analysis, only students who identified as male or female were included. That is, we chose not to include the one student who reported a non-binary gender identity in order to not suggest that this student represents all non-binary students.

7 Emphasis added in transcription to reflect the way the student emphasized the exclamation.

8 The percentages in each row sum to 300%, with error due to rounding, because each student was required to select three descriptors for each lesson.

9 Since our analysis (described in the previous section) shows that fine was likely a descriptor selected for all types of lessons because students were required to select three descriptors, it was not indicated as the dominant aesthetic quality unless its proportion met or exceeded the 90th percentile for this term (74%).

Appendix:

Model evaluation (where gray indicates the model is improved), where each row represents a test of the previous model (numbered 0 through 5) with a new factor

Table

Model	New factor	Fixed or random	Factors in this iteration of the model	ML^†	REML	Degrees of freedom	χ^2 distribution^‡
0				767.913	772.947	1	base
1	Class	Random	Class	747.155	750.539	1	22.408
2	Captivation	Fixed	Class, Captivation	703.508	711.366	1	43.647
	Selective	Fixed	Class, Captivation, Selective	703.158	712.854	1	0.350
3	CML	Fixed	Class, Captivation, CML	687.507	699.305	1	16.001
	Captivation*CML	Fixed	Class, Captivation, CML, Captivation*CML	685.612	700.430	1	1.895
4	Gender^β	Fixed	Class, Captivation, CML, Gender	683.237	698.826	1	4.270
	Captivation*CML	Fixed	Class, Captivation, CML, Gender, Captivation*CML	681.608	700.222	1	1.629
	Gender*CML	Fixed	Class, Captivation, CML, Gender, Gender*CML	682.892	700.941	1	0.345
5	Gender*Captivation	Fixed	Class, Captivation, CML, Gender, Captivation*Gender	674.576	693.152	1	8.316

^† When the newest additional factor was fixed, we calculated the χ² distribution by comparing the maximum likelihood (ML) of the new model to that of the existing best-fit model. For random factors, we considered the change in the restricted maximum likelihood (REML).

^‡ χ² distribution of 3.84 or above indicates the new iteration of the model is improved, $p\le 0.05$. χ² distribution of 6.63 or above indicates the new iteration of the model is improved, $p\le 0.01$.

^β All tests with gender assigned females = 0 and males = 1.