Development and implementation of Concept-Test questions in abstract algebra

Abstract

In tertiary mathematics courses, students often have difficulties acquiring an understanding of the mathematical concepts covered. One approach to address this problem is to implement so-called Concept-Tests. These are multiple-choice questions whose distractors represent common problems and misconceptions related to the concepts. While there exist lots of such questions for calculus, Concept-Test questions focusing on basic concepts of abstract algebra are still rare, although previous research has shown that students have many problems with these. We therefore developed such questions for important concepts of basic group and ring theory in the years 2020–2022. In this paper, we first want to present the questions and the developmental process. Furthermore, we want to present an empirical study investigating to what extent the questions helped students in a proof-oriented abstract algebra course to acquire an understanding of the concepts covered. This study especially indicates that the developed Concept-Test questions provided good starting points for conceptual changes.

KEYWORDS:

• Abstract algebra
• Concept-Tests
• groups and rings
• conceptual change

1. Introduction

Students in tertiary mathematics courses are often able to carry out routine procedures correctly but have an insufficient understanding of the underlying mathematical concepts. This is well known for important concepts of calculus such as limit or derivative (e.g. Davis & Vinner, 1986; Orton, 1983), but has also been found for basic concepts of abstract algebra such as group and subgroup, isomorphism, coset, or normality (e.g. Asiala et al., 1997; Dubinsky et al., 1994; Leron et al., 1995).

One approach to address this problem is the use of Concept-Tests (Mazur, 2017). These are multiple-choice questions whose wrong answer options represent misconceptions students might have. Several implementation scenarios of such questions are conceivable and were tried out in the past, e.g. in lecture-classes with peer discussions for fostering student activities (Alcock, 2018; Miller et al., 2006; Pilzer, 2001), in tutorials for helping students deepen their understanding of the concepts introduced in the lecture (Bauer et al., 2023), or in formative quizzes (Cronhjort et al., 2018; Feudel & Unger, 2022). Despite these manifold possibilities to implement Concept-Tests into mathematics courses, there are indications in the literature that these can actually help students gain a better understanding of the concepts covered (Bauer et al., 2023; Cronhjort et al., 2018; Miller et al., 2006; Pilzer, 2001).

While there exist many Concept-Test questions for calculus that are publicly available (Bauer, 2019; Hughes-Hallet et al., 2005; Terrell et al., 2003), such questions for basic concepts of abstract algebra are still rare. We therefore developed based on previous research Concept-Test questions that focus on basic concepts of group and ring theory. These questions especially provide concrete instantiations of the concepts, which can help students gain an understanding of them (Alcock, 2010).

In this paper, we first want to present the questions and the developmental process. Afterwards, we want to present results from a study, in which we implemented the questions into an abstract algebra course via formative quizzes, and investigated whether these have helped students to acquire an understanding of the concepts covered. The developed Concept-Test questions presented here are especially meant to be helpful tools for instructors of abstract algebra courses who wish to promote an understanding of the concepts taught, as this is often one major challenge in algebra courses at university (Dubinsky et al., 1994), but also for researchers investigating students’ difficulties with basic concepts of abstract algebra.

2. Literature review

2.1. Students’ difficulties in understanding basic concepts of abstract algebra

Dealing with abstract algebra is difficult for many students due to its abstract nature. In particular, the corresponding mathematical concepts are defined on the basis of their properties alone (Dubinsky & Leron, 1994). It is therefore not surprising that many students have difficulties to gain an understanding of these. Several problems and misconceptions related to basic concepts of abstract algebra have been shown in previous studies, which we will now summarise.

2.1.1. Problems concerning the concepts of group, subgroup and binary operation

Dubinsky et al. (1994) were one of the first who investigated learners’ understanding of fundamental concepts of group theory. They analysed written assessments of 24 high school teachers taking part in a summer workshop on abstract algebra, and conducted additional interviews with 10 of them. In this study, they identified several difficulties. The first one was that some participants did not coordinate the two components ‘set’ and ‘operation’ of groups and subgroups properly. One of them, for example, considered ${\mathbb{Z}}_3$ to be the set $\{0,1,2\}$, $\{1,2,3\}$, $\{0,2,3\}$, or $\{0,2,4\}$, from which the authors concluded that the participant just looked at the set while ignoring the operation. Another problem was that some participants tended to equip a set given with a certain operation by themselves with which they were familiar or which had been an operation on the set in former tasks. One of them, for example, equipped the subset $\{0,1,2\}$ with the operation ‘addition mod $3$’ in a task to decide whether it is a subgroup of ${\mathbb{Z}}_6$. Furthermore, some participants were unsure about what needs to be checked for deciding whether a subset is a subgroup of a given group. Similar problems were also found in later studies (Brown et al., 1997; Hazzan, 1999; Iannone & Nardi, 2002; Melhuish, 2018).

Regarding the operations of groups, a further problem became visible in a recent study on students’ understanding of binary operations (Melhuish et al., 2020). With a survey administered in two abstract algebra classes ($n=12$) from two universities in the US and additional interviews with half of the participants, Melhuish et al. investigated which attributes students judged as critical for identifying a given relationship between inputs and outputs as a binary operation. They found that several students based their judgments on rather superficial features, for instance, the occurrence of certain signs like $+$ or an element-operator-element formatting, but not on properties that are essential for being a binary operation such as closure or the necessity to have two inputs. Hence, students might be misguided by superficial features like symbols when asked about group operations and their properties.

2.1.2. Problems concerning the concepts of isomorphy and isomorphism

Also regarding these concepts, several difficulties are documented in the literature. Leron et al. (1995), for instance, investigated students’ understanding of isomorphy and isomorphism with task-based interviews at a university in Israel. In the tasks, the students had to decide whether groups given were isomorphic, explain how they can tell whether groups are isomorphic or not, and what an isomorphism is. Leron et al. first found that the students tended to proceed rather procedurally when checking for isomorphy: check the order of the groups, find the identity elements, determine the orders of the elements, investigate whether the groups are cyclic or commutative (p. 165). Although isomorphic groups necessarily share these properties, several students thought that having some of these in common is sufficient for isomorphy. Furthermore, the participants had much trouble with the concept of isomorphism. Several thought that finding a bijective mapping between two groups is sufficient for isomorphy. In addition, participants got stuck when trying to construct isomorphisms when there was more than one way to proceed. The authors argued that this might have occurred because students might think that an isomorphism must be unique, or that there must be a canonical algorithm for its construction.

Furthermore, Weber and Alcock (2004) suggested on the basis of a study on the construction of proofs involving isomorphy that students might not have an intuitive understanding of the concept, for example as ‘the same group except the elements and operations may have different names’. They argued that this might be one reason why the undergraduates they observed were often not able to prove or disprove that groups are isomorphic with alternative approaches than trying to construct an isomorphism.

2.1.3. Problems concerning the concepts of coset, normality and quotient group

In the study with 24 high school teachers already mentioned, Dubinsky et al. (1994) also investigated possible problems in understanding the concepts of coset, normality, and quotient group. They first found that although their participants could often determine cosets correctly, they had problems to consider a coset as an entity (e.g. the coset $3$ in ${\mathbb{Z}}_{18}$), which suggests that these participants did not develop an understanding of coset as an object. Furthermore, one participant thought it suffices to determine the sets $h\circ H$ with $h\in H$ for finding all cosets of a group $G$ with respect to a subgroup $H$. Concerning normality, Dubinsky et al. especially found that participants mixed it with commutativity. In addition, one participant thought it suffices to check $h\circ H=H\circ h$ for all $h\in H$ only. Concerning quotient groups, Dubinsky et al. found that the participants did not have a deep understanding of coset multiplication. For instance, although many course participants could carry out this operation in ${\mathbb{Z}}_{18}$ correctly – often rather mechanically by adding the first two elements of two cosets in ${\mathbb{Z}}_{18}$ and then taking the coset containing this sum, they failed to carry out the coset multiplication in $D_3$, because they could not find a similar mechanical procedure.

These results have been deepened and extended in later studies, for example, in Asiala et al. (1997) who investigated students’ understanding of coset, normality, and quotient group at a large midwestern university in the US. Besides the problems already mentioned, they especially found that students were uncertain about the role normality plays in connection with quotient groups. Further problems related to quotient groups were found in a later study by Siebert and Williams (2003) on students’ understanding of $\mathbb{Z}/n\mathbb{Z}$. Besides the issue that their participants could not explain why the method of representatives for coset multiplication works, they sometimes inserted numbers into the set part of the coset notation $k+n\mathbb{Z}$, from which the authors concluded that they considered the elements of $\mathbb{Z}/n\mathbb{Z}$ as numbers and were not aware that these are actually sets. This suggests that students might not be aware of the underlying structure of quotient groups, which is called quasi-structural understanding in the literature (Sfard, 1991).

Finally, we want to mention students’ problems in connection with Lagrange’s theorem. Hazzan and Leron (1996) found that students tended to use it inappropriately by applying it for showing that a given subset is a subgroup. The authors suggested that these students might have considered a ‘naive converse’ of Lagrange’s theorem as true, or – as Hazzan later pointed out – they might have tried to take refuge in a context with which they were much more familiar than with abstract algebra: divisibility of natural numbers (Hazzan, 1999).

2.1.4. Problems concerning concepts of ring theory

Besides group theory, basic abstract algebra also covers some concepts of ring theory (Dubinsky & Leron, 1994). Many of these have counterparts in group theory, and therefore, the same problems might be expected. However, ring theory involves further concepts, in particular, zero divisor and unit. We only found two studies on students’ understanding of these two concepts. Cook (2014) investigated how two students discovered these during a guided reinvention scenario. The students’ quotes in this scenario especially suggest that they regarded the two concepts as opposing. This can be a fruitful view, but can also lead to the misconception that an element of a ring must be either a unit or a zero divisor. Concerning zero divisors, Cook later also found a further problem: Students might have problems to unpack its definition correctly and mix it with its converse (Cook, 2018) – similar to Lagrange’s theorem.

Overall, the literature indicates that students have many problems and misconceptions in connection with important basic concepts of abstract algebra.

2.2. The use of Concept-Test questions for promoting understanding

One approach to help students detect and remedy problems and misconceptions such as the ones mentioned above is to implement multiple-choice questions that explicitly take these up in their wrong answer options. Mazur (2017) introduced the term ConcepTest for such questions. Several suggestions regarding the implementation of such Concept-Test questions are documented in the literature. Most of these followed the idea of Mazur to implement them into lecture-classes together with peer-discussion phases about the answer options (Alcock, 2018; Lucas, 2009; Miller et al., 2006; Pilzer, 2001; Schlatter, 2002). Sometimes, Concept-Tests were also implemented as quizzes, for instance, pre-lecture quizzes in a flipped classroom (Cronhjort et al., 2018; Jungić et al., 2015). Finally, Bauer et al. (2023) have also tried out integrating Concept-Tests into rather traditional tutorials with the instructor explaining the solution of the questions and possible misconceptions directly after a first vote.

There are some indications in the literature that the use of Concept-Tests can help students in tertiary mathematics courses gain an understanding of the concepts covered. First, several authors who implemented Concept-Tests with peer-discussions reported that the students improved a great deal in a second vote after the discussion phase, e.g. Miller et al. (2006) or Cronhjort et al. (2013). Other authors compared classes using Concept-Tests with other classes, e.g. Pilzer (2001). However, in these comparisons, it is often not clear to what extent the classes with the Concept-Tests differed from the other classes, for instance, with respect to the content covered or the examination.

A study that tried to address this issue was conducted by Cronhjort et al. (2018). They compared different classes of the same calculus course (but with students from different study programmes) at a University in Sweden in 2015. All of these used the same textbook and had the same final examination. Some classes followed an interactive teaching approach, while others were lecture-based. The interactive classes were organised as a flipped classroom and especially involved Concept-Tests with peer-discussions. Cronhjort et al. then especially found that the students from the classes with the Concept-Tests and peer discussions had on average higher normalised gains in a test consisting of 15 conceptual multiple-choice questions, which they administered as pre- and post-test. However, it remained open to what extent also other features of the interactive classes influenced the results, for example, their organisation as a flipped classroom.

A study that tried to investigate the effect of peer discussion in connection with Concept-Tests was conducted by Bauer et al. (2023). They implemented these into tutorials of an Analysis course at a German university in 2018. In four groups, the Concept-Tests were implemented with peer discussions. In the other four, the peer-discussion phase was omitted. Instead, one student with the correct answer was asked to explain her/his reasoning, and afterwards, the tutor gave extensive explanations about which answer was correct or wrong and why. The students were assigned randomly to one of the two group types. In a survey at the end of the semester, the students in both groups perceived the Concept-Tests as helpful for gaining an understanding of the concepts covered – without significant differences between them. Furthermore, the results in the final exam, which had a strong emphasis on conceptual questions, also did not differ significantly between the two group types. This indicates that the use of Concept-Test questions can support students in gaining an understanding of the concepts covered also in rather traditional settings based on instructional explanations.

Overall, the literature indicates that an integration of Concept-Tests into tertiary mathematics courses can help students acquire an understanding of the concepts covered. However, essential for this are suitable questions that actually address students’ difficulties. While there exist lots of Concept-Test questions for calculus that are publicly available (Bauer, 2019; Hughes-Hallet et al., 2005; Terrell et al., 2003), such questions for abstract algebra are still rare, although students have many problems with the corresponding concepts. We therefore developed Concept-Test questions relating to basic concepts of abstract algebra based on previous research. The questions especially aim at targeting problems and misconceptions documented in the literature and are meant to be helpful tools for promoting an understanding of the concepts covered in them.

3. Theoretical framing

As we tried to construct Concept-Tests that aimed at promoting an understanding of mathematical concepts, we first want to clarify what we mean with ‘understanding a mathematical concept’ from a theoretical point of view.

At university, mathematical concepts are usually introduced via a formal definition. However, Vinner (1983) pointed out that although students might be able to recite such formal concept definitions, their thinking of the corresponding concepts is rather based on associations. Tall and Vinner (1981) introduced the term concept image ‘to describe the total cognitive structure that is associated with the concept, which includes all the mental pictures and associated properties and processes’ (p. 152). It is built up over years through experiences, changing as the individual matures. Therefore, similar to Melhuish (2019), we claim that understanding a mathematical concept does not only mean to know its definition, but to have a rich concept image. Such a concept image can especially help to assign a meaning to a mathematical concept. But it is also important for creating proofs that cannot be constructed by unpacking definitions and manipulating symbols alone (Weber & Alcock, 2004).

But what constitutes such a rich concept image, and how can the building of such be fostered? Selden and Selden (2008) pointed out that it comprises examples, non-examples, facts, properties, relationships, diagrams, and visualisations. Mason and Watson (2008) especially highlighted as a significant component of a person’s concept image a collection of examples and non-examples to which she or he has access, which they called example space. Therefore, students should be exposed to many different examples of a concept, which also vary in some features that do not contribute to being an example (Mason & Watson, 2008), for enriching and extending their concept images.

However, also a rich concept image may contain parts that stand in conflict to the formal definition of the concept. It is therefore also important that students build up a connection between their concept image and the concept definition in their mind (Vinner, 1991). One possibility to promote the building of such might be tasks demanding students to judge whether a given object is an example or not on the basis of the definition. A second important approach to tighten the connection between a student’s concept image and the concept definition is to provoke cognitive conflicts resulting from recognised discrepancies between these two (Tall & Vinner, 1981). This can lead to a modification of the concept image, and hence to a conceptual change (Vosniadou & Verschaffel, 2004) – meant here as a developmental process of one’s concept image so that it becomes consistent with the formal concept definition.

We especially took up the following aspects of the framework in our Concept-test approach to foster students’ understanding of basic concepts of abstract algebra:

• For fostering an acquisition of rich concept images of the concepts, the students should especially be exposed to many different examples that might help them create an example space. In addition, relationships between the different concepts should be made explicit.

• The students should be offered opportunities to build connections between their concept image and the formal concept definition, for instance, by requiring them to check whether an object fulfils really all properties given in the concept definition.

• The students’ understanding of the concepts can be improved if they recognise discrepancies between their concept image and the formal concept definition, which can lead to a cognitive conflict and an adaptation of their concept image.

These aspects led to three overall design principles of our Concept-Test questions that we will present in the next chapter. Furthermore, in a subsequent evaluation of the effect of the Concept-Test questions, we especially tried to evaluate to what extent aspects of students’ concept images that were meant to be addressed in the questions have changed from the middle of the semester to the end.

4. Description of the designed Concept-Test questions

4.1. Design principles of the Concept-Test questions

The Concept-Test questions we designed had the aim to help students develop rich concept images of basic concepts of abstract algebra that are closely connected with the formal concept definitions in the students’ minds. As just said, this led to three design principles for the development of our questions.

First, to foster the development of rich concept images, our Concept-Test questions involved multiple examples of groups and rings. In the case of groups, these covered especially: the residual group ${\mathbb{Z}}_n$ as a simple example of a finite group, the symmetric group $S_n$ and the dihedral group $D_n$ as examples of non-commutative groups, and $\mathbb{Z}/n\mathbb{Z}$ as prototypical example of a quotient group. In the case of rings, these additionally covered matrices as examples of non-commutative rings. In addition, several questions focused on relationships between different concepts of abstract algebra such as ideal and subring or zero divisor and unit.

Second, to help students build a connection between their concept images and the formal concept definitions, the questions often focused on a specific property in the definitions of the concepts addressed. Thereby, we especially used in several questions non-examples for which just one of the properties is violated. In one question, for instance, the students had to decide whether the set consisting of the identity and the two reflections that map a rectangle on itself with the composition as operation is a group. Here, only the closure is violated.

Third, we tried to foster conceptual changes by inducing cognitive conflicts between the students’ concept images and the formal concept definitions. For this, we especially tried to create the distractors of the Concept-Test questions on the basis of problems and misconceptions documented in the literature. An example is the misconception that two groups are isomorphic if there exists a bijective mapping between them. For making the misconceptions better visible, we often equipped the false answer options with reasoning that is clearly related to the corresponding misconception, for instance, a distractor containing the argument that two groups given are isomorphic because a bijective mapping between them can be found.

Of course, not all the questions could fulfil all design principles. Nevertheless, we tried to use these as an overall guideline for the design of our questions.

4.2. Developmental process

1st step – collecting problems and misconceptions we wanted to address

When starting to develop the Concept-Test questions in 2020, we first collected important problems and misconceptions related to the basic concepts of abstract algebra from Section 2 that are documented in the literature. These were in the case of groups:

• Considering a group as a set without operation or assigning some kind of canonical operation

• Focus on superficial features such as signs in connection with group operations

• Uncertainties about which properties are necessary for justifying that a subset is a subgroup of a given group

• Assumption that two groups are isomorphic if they have certain properties in common or if there exists a bijective mapping between them

• Assumption that homo- or isomorphisms must be unique

• Confusion of normality with commutativity

• Belief that a naive converse of Lagrange’s theorem is true

• Difficulties in understanding the structure of a quotient group, especially the nature of the elements as sets or the role normality plays in connection with such groups.

Concerning rings, we developed similar questions that addressed like problems for the analogous basic concepts of ring theory (ring, subring, ideal, quotient ring). In addition, we added extra questions that tried to address two important problems with units and zero divisors mentioned in Section 2.1: the misconception that a ring element needs to be either a zero divisor or a unit and difficulties to unpack the formal definition of zero divisor correctly.

2nd step – development of a first version of the questions

In 2020, we developed for each of the problems mentioned above a first version of Concept-Test questions addressing these. If possible, we based the Concept-Test questions on open questions that had been used in the studies discovering them originally. Thereby, we decided to develop the questions immediately as multiple-choice questions according to the problems/misconceptions from the literature. Nevertheless, for exploring whether our distractors really addressed these, we additionally provided two empty lines for students’ justifications. A sample question of the first round can be seen in Figure 1 (all material was translated by the authors for this paper). This question addressed normality and its possible confusion with commutativity.

Figure 1. Sample item addressing the problem of mixing normality with commutativity.

3rd step – implementation of the questions

For checking the validity of our questions and answer options, we implemented the questions into an abstract algebra course for future teachers at a large German university in the years 2020, 2021, and 2022 via formative quizzes, and examined whether students’ reasoning for choosing certain answer options suited to our intentions. An example is shown in Figure 2 illustrating that the student really mixed normality with commutativity – despite being able to cite the formal concept definition of normality correctly.

Figure 2. Sample of a student’s reasoning on a Concept-Test question on normality.

4th step – refinement of the questions

Depending on this validity check, we then refined the questions in two ways. First, we adapted the formulation of a question if we recognised that it had been misunderstood or if the formulation made the question too nit-picking. Second, we modified the answer options by including students’ reasoning explicitly into them. An example illustrating this development is shown in Figure 3 – based on the original question of whether ${\mathbb{Z}}_3$ is a subgroup of ${\mathbb{Z}}_6$ from Dubinsky et al. (1994).

Figure 3. Illustration of the development of the questions for an item on students’ problems with coordinating a group and its operation.

The correct answer is d), because whether $\{0,2,4$} is a subgroup depends on the operation the set $\{0,1,\dots ,5\}$ is equipped with. The students of the course had experienced the latter with at least two operations: addition mod 6 and multiplication mod 6. In the first version of the question in 2020, we stayed close to the formulation of the original question in Dubinsky et al. (1994), and the distractors were rather general to find out how the students might argue. When refining the questions, we first omitted the notation ${\mathbb{Z}}_6$ because this notation is sometimes also in the literature tacitly equipped with the addition mod 6 as operation (even if not in the course). But more importantly, we included the students’ reasoning into the distractors, which then point out directly possible misconceptions. Distractor a) focuses on a consideration of a group as a set, and distractors b) and c) focus on the problem that students might equip a set with some operation themselves although not given.

Similarly, we refined the other questions: from a first version in 2020 over an intermediate version in 2021 to the final version in 2022. In the latter version, the students then justified their responses basically in the way we intended when developing our distractors.

4.3. The final Concept-Test questions

The development resulted in 24 Concept-Test questions on basic concepts of group theory and 17 further questions on basic ring theory. Tables 1 and 2 give an overview on the questions with the concepts and problems/misconceptions we intended to address with them, and the literature source of the corresponding problems. The questions themselves are shown in Appendix (Figures A1–A7 at the end of the article).

Table 1. Overview on the Concept-Test questions on basic group theory.

Item no	Concept/ theorem focused	Problems/ misconceptions addressed	Literature foundation of the problem
1	Group	Consideration of a group as a set without being aware that an operation is necessary	Dubinsky et al. (1994)
2, 3	Group/subgroup	Consideration of a group as a set and/or equipment of a set with a familiar operation although not given	Dubinsky et al. (1994)
4–6	Group	Focus on superficial properties like signs in connection with group operations (here: considering $e$ as neutral element or $+$ as usual addition on the integers)	Melhuish et al. (2020)
7–9	Group/Subgroup	Not being aware that closure is a necessary property for being a group or subgroup	Brown et al. (1997); Melhuish et al. (2020)
10, 11	Subgroup	Not being aware that inverses must also lie in a subset for being a subgroup	Brown et al. (1997); Dubinsky et al. (1994)
12	Subgroup	Not being aware that inverses must lie in a subset for being a subgroup, forgetting to check that the candidate must be actually a subset	Brown et al. (1997); Dubinsky et al. (1994)
13–15	Isomorphy and isomorphism	Consideration that groups are isomorphic if they have certain properties in common or if there exists a bijective mapping between them, assumption that homo- or isomorphisms must be unique	Leron et al. (1995); Weber and Alcock (2004)
16–18	Normality	Mixture of normality with commutativity, uncertainties about the range of elements that need to commute with the subgroup	Asiala et al. (1997); Dubinsky et al. (1994)
19–21	Lagrange’s theorem	Mixture of Lagrange’s theorem with a naive converse, misuse of the theorem in inappropriate situations	Hazzan and Leron (1996)
22	Quotient groups	Not being aware that elements of quotient groups are sets	Siebert and Williams (2003)
23	Quotient groups	Problem to consider cosets as an entity	Item exactly taken from Melhuish (2019)
24	Quotient groups	Uncertainties about the role of normality in connection with quotient groups	Asiala et al. (1997)

Table 2. Overview on the Concept-Test questions on basic ring theory.

Item no	Concept focused	Problems/ misconceptions addressed	Literature foundation of the problem
1, 2	Ring	Consideration of a ring as a set and/or equipment of a set with familiar operations although not given	Dubinsky et al. (1994) for groups
3	Ring	Focus on superficial properties like signs in connection with ring operations (here: considering $0$ as neutral element for addition or $+$ and $\cdot$ as the usual operations on the rational numbers)	Hazzan (1999); Melhuish et al. (2020)
4	Ring	Focus on superficial properties in connection with ring operations (here: the sign $\cdot$), not being aware that two distributive laws need to hold	Melhuish et al. (2020) for the first issue
5–6	Subring	Not being aware that closure is a necessary property for being a subring (for $+$ or $\cdot$)	Brown et al. (1997) and Dubinsky et al. (1994) for groups
7–8	Subring	Not being aware that the additive inverses must also lie in the subring	Brown et al. (1997) and Dubinsky et al. (1994) for groups
9	Quotient rings	Not being aware that the elements of quotient rings are sets	Siebert and Williams (2003)
10	Ring homomorphism	Assumption that ring homomorphisms map the one of a ring on the one of the other ring	-
11, 12	Unit and zero divisor	View of zero divisors and units as opposing, considering the one of a ring as the only unit	Cook (2014) for the first issue
13	Zero divisor	Problems to unpack the definition of zero divisor logically correctly	Cook (2018)
14, 15	Ideal	Not being aware that ideals need to fulfil two properties (subgroup property with respect to $+$ and closure with respect to $\cdot$ for all ring elements)	Analogous to problems with subgroups as in Brown et al. (1997) and Dubinsky et al. (1994)
16, 17	Ideal	Uncertainties about the relationship between ideal and subring	-

5. An accompanying study on the effect of the Concept-Test questions

We tried to investigate the effect of the Concept-Test questions in an abstract algebra course in the years 2020–2022, into which we integrated them via optional formative quizzes. For this, we gathered in these three years data regarding the quizzes each student has taken, their answers to the Concept-Test questions, and their scores in the final examination. We then found a positive correlation between the score in the final examination and the number of quizzes the students had taken in 2020 as well as in 2021: 0.369 and 0.341 $(p=0.009\mathrm{and}p=0.010)$. However, since we did not know whether this correlation may have just occurred because the ones who had taken the Concept-Tests were mainly the stronger or the more engaged students, we decided to conduct a systematic study with additional student data on the effect of the Concept-Test questions in 2022, which we will now describe.

5.1. Description of the course, its participants, and the implementation of the Concept-Test questions

We implemented the Concept-Test questions into a proof-oriented course ‘Algebra and number theory’ at a large German university in the years 2020–2022. Its participants were prospective teachers who would later teach mathematics up to the end of the secondary level. The students attending the course had all taken the following mathematics courses before: ‘Linear algebra and analytical geometry I and II’, ‘Analysis I and II’ and a course in axiomatic geometry.

The course was based on the textbook by Kramer and Von Pippich (2013), of which there also exists an English version (Kramer & Von Pippich, 2017). The topics covered were:

1. Natural numbers (axiomatic definition, divisibility, and prime factorisation)

2. Basics of group theory (groups, subgroups, homo- and isomorphisms, cosets, Lagrange’s theorem, normality, quotient groups, the fundamental homomorphism theorem, the construction of groups from semigroups with cancellation laws, and the construction of the integers as a special case)

3. Basics of ring theory (rings, zero divisors, integral domains, units, subrings, ring homomorphisms, ideals, factor rings, the construction of quotient fields, and the construction of the rational numbers as a special case)

4. Real numbers (the construction of the real numbers using Cauchy sequences of rational numbers).

The course consisted of two 90-minute lectures and one 90-minute tutorial per week and lasted 14 weeks. In addition, the students were obliged to submit written assignments each week (homework problems) and needed at least 50% of the attainable points over the whole semester for admission to the final examination. In 2020 and 2021, the sessions of the course were given online via ZOOM due to the COVID-19 Pandemic. In 2022, they were given face-to-face.

The Concept-Test questions were implemented into this course via formative quizzes: one quiz on the basics of group theory covering the concepts of group, subgroup, and isomorphy/isomorphism, a second quiz on group theory covering cosets, normality, and quotient groups, and one quiz covering ring theory. Hence, the 24 group theory items from Table 1 were split into 2 quizzes, while there was just 1 quiz on ring theory. All the quizzes were made available after the respective topics had been fully covered in the course, and the students could complete the quizzes within one week. Just as in the sample question in Figure 1, they were asked to select the right option and to justify their answer. They then received feedback by the first author on their answers and their justifications. In 2020 and 2021, the quizzes were completely optional. In 2022 – the year with our systematic study – the quizzes were offered only to a group who volunteered to take all of them. This group covered 20 out of the 86 students actively taking part in the course including the examination.

5.2. Methodology of the study in 2022

We tried to investigate the effect of the Concept-Test questions on students’ understanding of the concepts covered in two ways.

Firstly, we compared the examination results between the 20 students who took the Concept-Test questions with the others. The final exam focused on mathematical concepts as well and on proof. For controlling students’ prior qualification and their effort, we conducted an additional anonymous mid-semester survey. In the corresponding questionnaire, we asked the students for the grades from the previous courses ‘Analysis II’, ‘Linear algebra and analytical geometry II’, or alternatively from another course if they could not remember these grades. Furthermore, we asked for the time spent for post-class processing the lectures and working on the homework problems. Finally, we asked them whether they took the quizzes comprising of the Concept-Test questions, and if so, how much time they needed for the first quiz. For identifying the subgroup of the survey participants in the exam later, we asked all exam participants whether they had taken part in the survey (without tracking them individually).

Secondly, the first author conducted interviews after the semester with 5 of the 20 students of the Concept-Test group for investigating the development of their understanding of the concepts addressed in the Concept-Test questions. As an introduction, he asked the students if they perceived the Concept-Tests as helpful, and to what extent these have contributed to gaining an understanding of the concepts covered. In the main part, he gave them a copy of their own uncorrected responses (including justifications) to the 24 questions of group theory and asked them to point out the questions to which they would now respond differently and why. This way, we wanted to find out whether a conceptual change – meant as development of the concept image – might have taken place.

5.3. Data collection

Twenty students volunteered to submit the three formative quizzes comprising of the Concept-Test questions from Tables 1 and 2. They all consented that their data will be used for this study and published in an anonymized form.

The anonymous survey on students’ prior qualification and effort for the course took place in the middle of the semester directly after the deadline of the first formative quiz. The questionnaire was administered in the lecture and the tutorials. Forty-two students took part in the survey, who also consented that their data would be used for investigating the effect of the Concept-Tests. This is about half of the participants taking the final examination. Hence, the survey sample might be a bit biased to students who were at least willing to take part in the face-to-face sessions during the semester. These 42 survey participants especially included 16 of the 20 students taking the Concept-Test questions. And finally, 5 of the 20 students who had taken the Concept-Tests participated in the post-semester interview. Thereby, the interviewer chose such students who had shown clear misconceptions in their responses during the semester.

5.4. Data analysis

5.4.1. Analysis of the effect of the Concept-Test questions on the examination

Step 1: We first compared the average scores and the grades between the ones who had taken the Concept-Test questions and the others. For testing whether differences in the average scores were significant, we used a $t$-test after having tested both groups for normality with the Kolmogorov–Smirnov-Test. In addition, we compared the distributions of the examination scores with a Whitney-U test.

Step 2: In the second step, we investigated on the basis of the survey data whether the ones who had taken the Concept-Tests were just the stronger students or the ones who invested more time. For this, we compared the means of the self-reported grades from previous courses and the average time spent per week for post-class processing the lectures and working on the homework problems between the ones who had taken the Concept-Tests and the others. Again, we used $t$-tests for investigating whether these averages differed significantly between the two groups after having tested for normality. Finally, we repeated the comparison from Step 1 by restricting the cohort on students of whom we also had survey data on their prior qualification and time investment.

5.4.2. Analysis of the interview data

The interviews consisted of 2 parts: an introductory part focusing on the general question of the perceived benefit of the Concept-Tests and the main part focusing on the development of the interviewees’ concept images regarding the concepts addressed in the 24 questions on group theory. Concerning the introductory part, we categorised students’ answers according to the benefits of the Concept-test questions they mentioned. The corresponding categories evolved from the data. Concerning the main part of the interview focusing on possible changes in the students’ concept images, we analysed for each group theory question whether the students changed responses compared to what they had answered during the semester, and if they could provide adequate reasoning for their response – and maybe even reflect on misconceptions they might have had earlier.

5.5. Results of the study

5.5.1. Comparison of the examination results

Table 3 shows the means of the scores and the median of the grades achieved in the final exam by the students who took the Concept-Test questions and the others. In total, 60 points were attainable. The t-test showed that the means in the scores differed significantly between the two groups ($p<.001)$. A Whitney-U Test furthermore showed that the corresponding distributions differed significantly as well $(Z=3.528,p<.001)$.

Table 3. Students’ mean scores and grades in the final examination, grades: $1=$‘very good’, $2=$‘good’, $3=$‘satisfactory’, $4=$‘sufficient’, $5=$‘fail’.

	Students taking the Concept-Tests $(N=20)$	Other students $(N=66)$
Mean score in final examination	36.5	22.7
Median of the grade	3.65	5

While the results of the exam were overall rather weak, which is not surprising as this course is very demanding for future teachers, the students who had taken the Concept-Test questions scored on average a lot better.

The survey data furthermore indicate that these differences did not only occur because the ones who had taken the Concept-Tests were only stronger students or the ones who invested much more time. While among the survey participants, the Concept-Test group had on average a slightly better grade in previous courses and also invested on average a little more time during the semester (see Table 4), the $t$-tests showed that these differences were not significant $(p=.137$ for grades and $p=.627$ for time investment). Even if one adds the time spent additionally for the Concept-Test questions – estimated via $3\times$ (time spent on the first of the 3 quizzes with the questions) $÷$ (the 14 weeks of the semester) – the difference in time investment remains insignificant.

Table 4. Mean of students’ grades in previous courses and average of the total time spent per week for post-class processing and homework, grades: 1 = ‘very good’, 2 = ‘good’, 3 = ‘satisfactory’, 4 = ‘sufficient’, 5 = ‘fail’.

	Survey participants taking the Concept-Tests $(N=16)$	Other survey participants $(N=26)$
Average grade in previous courses	$2.27$	2.63
Time spent on average per week on post-class processing and homework problems	482 min	449 min

Finally, Table 5 shows a comparison of the examination results between the group who had taken the Concept-Tests and the others – restricted to the 36 students who stated on the cover sheet of the exam that they had also taken part in the mid-semester survey. The differences in the mean of the scores and the median of the grades are similar to Table 3 – only the $p$-values for the $t$-tests for different means in the score and the Whitney-U test comparing the distributions of the scores were a little higher ($p=.007$ and $p=.011$). As the previous grades and the time investment did on average not differ significantly between the two groups, this is a further indication that the differences in Table 3 did not only occur because mainly stronger students or students who invested much more time took the Concept-Tests. Nevertheless, since our sample in Table 5 is rather small, and since it was perhaps the more engaged students who took the Concept-Tests, one still has to be a little cautious about generalising these results.

Table 5. Mean scores and grades in the final examination of the survey participants, grades: $1=$‘very good’, $2=$‘good’, $3=$‘satisfactory’, $4=$‘sufficient’, $5=$‘fail’.

	Survey participants taking the Concept-Tests $(N=16)$	Other survey participants $(N=20)$
Mean score in final examination	37.3	23.5
Median of the grade	3.65	5

5.6. Results of the interviews

5.6.1. Perceived usefulness of the Concept-Test questions

All five interviewees – pseudonymised as Mark, Dora, Peter, Pia, and Tom for this article – considered the Concept-Tests as helpful. Mark, for instance, highlighted that these helped him check his understanding and motivated him to spend additional time with the lecture’s content:

I would say that they helped me in two ways: On the one hand, one could check again what one has learned in the lecture. And one could see, have I mastered this or not? Because this was always a little hard to evaluate. And they also motivated to spend an additional evening reviewing things.

He also emphasised that the Concept-Test questions helped him detect gaps in understanding, for instance, related to the concept of isomorphy.

Dora also mentioned the usefulness of the Concept-Test questions for checking her own mastery of the content. But she furthermore emphasised that these helped her to engage with the content continuously during the semester, especially because working on the questions was less time intensive than the homework problems:

Ehm, I found them very helpful. Simply that you do not lose track of the content. And since you have smaller tasks and – different from the exercise series – did not need hours for solving a task, the motivation did not vanish.

Finally, we want to highlight two benefits mentioned by Peter. He argued that the Concept-Test questions were good occasions for communication about the content with peers, and that they were a good help for preparing the examination, because you get an idea of what might be expected there.

Overall, the students mentioned the following perceived benefits of the Concept-Test questions:

• Check their own mastery of the content (Mark, Dora)

• Detect gaps in understanding (Mark, Pia, Tom)

• Source for additional examples that help to gain an understanding of the concepts covered in the lecture (Peter)

• Foster engagement with the content already during the semester (Mark, Dora, Peter, Tom)

• Provide occasions for communicating about the content with peers (Pia, Peter)

• Help to prepare for the examination (Pia, Peter).

This especially suggests that the Concept-Test questions – administered via formative quizzes – did not only help the students check their mastery of the content or detect and remedy gaps in understanding, but also fostered an active and continuous engagement with the content during the semester.

5.6.2. Changes in students’ conceptions

As mentioned in Section 5.2, the 5 interviewees were provided with a copy of their uncorrected responses to the 24 questions on group theory during the interview again, and asked which questions they would now answer differently. Table 6 shows for all items for which the students had not chosen the correct answer option during the semester whether they changed their response to the correct one in the post-semester interview. Nobody changed an answer option from the correct one to an incorrect one in the post-semester interview.

Table 6. Changes in the students’ responses to the questions on group theory that they had not answered correctly during the semester (see Appendix for the items).

Student	Items answered incorrectly during the semester	Change in the responses
Mark	3, 4, 7, 11, 13, 15, 23	He adjusted all his false responses to correct ones in the post-semester interview, and provided adequate justifications
Dora	4, 14, 15, 16, 17, 22, 23, 24	She adjusted almost all her false responses to correct ones in the post-semester interview and provided adequate justifications – except item 24 on the role of normality in connection with quotient groups
Pia	1, 3, 6, 9, 11, 14, 15, 22, 23, 24	She adjusted her false response to a correct one with an adequate justification for item 3 on the problem that the neutral element is not denoted with $e$. Furthermore, she was able to identify all the items she had answered wrongly during the semester in the post-semester interview by herself, but was still unsure about which answer option was correct
Peter	3, 4, 7, 11, 13, 22, 24	He adjusted his false responses to correct ones with adequate justifications for most of the items (items 4, 7, 11, 13, 24). He furthermore remembered during the interview that his former response on item 22 – that the elements of $\mathbb{Z}/n\mathbb{Z}$ are numbers of the form $r+n\cdot q$ – was incorrect, but could not provide a reason. He did not recognise his former mistake in item 3
Tom	1, 4, 9, 13, 14, 17, 20, 22, 23	He adjusted some responses to correct ones with adequate justifications (items 1, 4, 14). For the items 13, 17, and 22, he recognised that his former responses had been wrong, but was still not sure about the right option in the interview. He did not refer to the incorrectly answered items 9, 17, 20, and 23 when asked for which items he would change his response

Table 6 indicates that some conceptual changes – meant as a development of the students’ concept images – have taken place between the first time the students had taken the Concept-Test questions during the semester and the interview after the semester. We want to illustrate this explicitly with two sample students: Mark and Dora. Mark had refined his concept image regarding subgroups. For instance, during the semester, he had still equipped the set ${\mathbb{Z}}_3$ with the operation ‘addition mod 3’ when asked whether it was a subgroup of ${\mathbb{Z}}_6$. He then became aware that a subgroup always needs to be considered with the same operation as the overall group. Furthermore, he had recognised with this item the necessity to check for closure:

It was not clear to me that closure is a criterion. In the definition you provided, it was just not contained.

Also Dora’s quotes indicate a development of her concept images based on the Concept-Test questions. For instance, when taking the Concept-Tests, she had the idea that homomorphisms/isomorphisms must be somehow unique:

Ehm, I even believe that for 14 and 15, I had the problem that I thought that only one mapping can exist when we find a homomorphism or an isomorphism. Ehm, and that I always said ‘Okay, if this map does not work, it cannot be the case’. But there may be still other mappings.

She emphasised that she recognised this mistake with the Concept-Test questions. She furthermore adapted her concept image of normality, which she had at first mixed with commutativity (items 16 and 17, which she had answered wrongly during the semester, see Figure A3):

D:

Ehm, I thought that it required commutativity.

I:

The normal subgroup?

D:

Because I was confused with the definitions that we had. But it only requires that the left cosets are equal to the right cosets, and this does not mean commutativity.

Besides the development of the students’ concept images suggested by changes in the answer options, two students also referred to items for which they would not change the answer option, but now justify the response differently. Tom, for instance, stated in the interview that he could now justify why $(\mathbb{Z},+)$ and $(\mathbb{Q},+$) are not isomorphic (item 15 in Appendix in Figure A3), while during the semester, he was only able to reason why all other options must be incorrect.

However, Table 6 also shows that not all misconceptions have been remedied. In some cases, the interviewees did not recognise during the interviews that their former responses had been incorrect, which suggests that they might still have the problems/misconceptions addressed in these items, for instance, the problem to mix the order of an element of a quotient group with the order of the corresponding set. In other cases, students became aware that their conceptions had been not fully correct but did not sustainably modify them adequately. An example is Pia who reported to have recognised with the Concept-Test questions that the existence of a bijective mapping is not sufficient for isomorphy, but still did not know what the additional property must be in the post-semester interview.

Furthermore, we do not always know for sure whether it was actually the Concept-Test questions that caused the changes in the interviewees’ concept images so that they were able to adjust formerly false responses to correct ones in the post-semester interview. Nevertheless, their explanations suggest that the questions and the feedback provided at least helpful starting points for conceptual changes – either by making them realise misconceptions explicitly or by prompting them to review certain topics again.

6. Summary, discussion, and outlook

6.1. Summary and discussion

As mentioned in the introduction, implementing Concept-Test questions is one approach to address the problem that students in tertiary mathematics courses often do not acquire a solid understanding of the mathematical concepts covered (Alcock, 2018; Bauer et al., 2023; Lucas, 2009; Miller et al., 2006; Pilzer, 2001; Schlatter, 2002). In the case of abstracta algebra, however, such questions are still rare although previous research has shown that students have lots of problems with these concepts (see section 2.1). We therefore developed from 2020 to 2022 such questions on the basis of former research on students’ understanding of these concepts. These especially aimed at helping students build up rich example spaces to the abstract concepts, which are an essential component of a student’s concept image, and at helping them detect and remedy problems/misconceptions related to the concepts found in previous research. Therefore, we constructed questions and answer options on the basis of the studies that originally identified these problems. By administering the questions in an abstract algebra course via formative quizzes and requiring students to justify their responses, we checked the validity of the questions and the distractors and refined them based on students’ reasoning. This finally resulted in 24 questions on concepts of basic group theory and 17 questions on basic ring theory (see Appendix).

For trying to investigate whether the developed questions were useful for promoting an understanding of the concepts covered, we first looked at the data of the final exams of the algebra course into which we had implemented the questions via optional formative quizzes in 2020 and 2021. In this data, we found a positive correlation between the usage of the quizzes and the score in the final examination. In 2022, we furthermore conducted an empirical study, in which we tried to also take students’ prior qualification measured by their grades in previous examinations and their effort measured by the time spent weekly for the course into account. These data indicate that the better performance of the students who took the Concept-Test questions did not only occur because these were much stronger than the others or invested much more time with the content during the semester. Furthermore, we conducted interviews to explore whether the questions helped to detect and overcome problems with basic concepts of group theory. The interview data suggest that these could initiate a development of the students’ concept images in some aspects, especially regarding the following problems that had been shown in former studies (Asiala et al., 1997; Brown et al., 1997; Dubinsky et al., 1994; Leron et al., 1995; Melhuish et al., 2020; Siebert & Williams, 2003):

• Consideration of a group as a set or assignment of a ‘canonical’ operation by the students

• Focus on superficial features such as signs in connection with group operations

• Forgetting closure as property to check when investigating whether a subset is a subgroup

• Uncertainties about the operation to consider when investigating whether a subset is a subgroup

• Assumption that the existence of a bijective mapping is sufficient for isomorphy or that groups are isomorphic if the orders of the elements coincide

• Assumption that homo- or isomorphisms need to be unique

• Confusion of normality with commutativity

• Difficulties in understanding the structure of a quotient group, especially the nature of the elements as sets that form an entity.

Furthermore, the interview data are an indication that Concept-Test questions may not only be useful when implemented in lectures in connection with peer discussions as often done (Alcock, 2018; Lucas, 2009; Miller et al., 2006; Pilzer, 2001; Schlatter, 2002), but also in other course components such as formative quizzes. Our data suggest that the use of Concept-Test questions in such formative quizzes can not only help students detect gaps in understanding, but also motivate them to engage with the content of the course continuously, which is also denoted as an important potential of formative quizzes in general (Gaspar Martins, 2017; Lim et al., 2012).

Last but not least, we want to emphasise that our questions are not meant to be a diagnostic test like the questions in the Group Theory Concept Assessment by Melhuish (2019). Therefore, we did not try to make sure that the examples occurring in the questions are covered in every course. Instead, the questions are meant as a tool that might help instructors foster an understanding of the concepts covered and help students overcome certain difficulties. In particular, it is very reasonable that instructors of abstract algebra courses who want to promote an understanding of the concepts covered choose just some of our questions that fit best to what they had covered in their courses.

6.2. Limitations and outlook

Finally, we want to mention some limitations of the research presented here, which yield starting points for future research.

Firstly, although the questions have been developed on the basis of previous studies, research on students’ understanding of abstract algebra is still going on, and new issues might be found that are not yet addressed in the questions. Therefore, the questions should in our opinion not be seen as a finalised product, and should be developed further or extended in the future. We especially welcome researchers and instructors to do so.

Secondly, we did not carry out a factor-analysis to find out if the questions actually address different facets of students’ understanding of basic concepts of abstract algebra. However, since the questions are not meant to be used as a diagnostic test, but as a tool for promoting students’ understanding of the concepts, we do not consider overlaps and certain redundancies in the questions as a problem.

The most important limitation, however, concerns our evaluation of the effect of the Concept-Test questions. It has only taken place in one course, and the sample of 20 students taking part in the empirical study in 2022 as experimental group was rather small. Similarly, the number of students not taking the Concept-Tests of whom we could collect data on their prior qualification and their effort during the semester covered only half of the course participants in 2022. Hence, the samples might have been biased, and it is therefore important to investigate the effect of the Concept-Test questions in other courses and settings as well.

And finally, although our interview data show that our participants’ concept images developed between their first completion of the Concept-Test questions and the post-semester interview, the data only provide some indications about how the questions might have fostered this development – mainly in the form of the perceived benefits of the Concept-Test questions, for instance, that they helped to detect gaps, provided additional examples or that they fostered communication about the concepts with peers. Therefore, an important issue for future research is to explore in more detail how these and other Concept-Test questions help students improve their understanding of the abstract concepts addressed in them.

Nevertheless, despite these limitations, our data suggest that the developed Concept-Test questions might be one useful and adaptable tool for instructors who want to promote an understanding of basic concepts of abstract algebra in their courses, as well as for researchers who might want to detect certain problems or misconceptions related to these concepts.

Disclosure statement

No potential conflict of interest was reported by the authors.

Appendix

(a) Concept-Test questions focusing on basic concepts of group theory

These are shown in Figures A1–A4 (questions translated into English by the authors).

Figure A1. Concept-Test questions 1–6 on group theory.

Figure A2. Concept-Test questions 7–12 on group theory.

Figure A3. Concept-Test questions 13–18 on group theory.

Figure A4. Concept-Test questions 19–24 on group theory.

(b) Concept-Test questions focusing on basic concepts of ring theory

These are shown in Figures A5–A7 (questions translated into English by the authors).

Figure A5. Concept-Test questions 1–6 on ring theory.

Figure A6. Concept-Test questions 7–12 on ring theory.

Figure A7. Concept-Test questions 13–17 on ring theory.