Assessing ethics education through national tests—an advantage or not?

ABSTRACT

The purpose of this study is to describe, critically analyse and discuss the Swedish system of assessing ethics education in compulsory school through national tests. The publicly available tests from 2013 for grades six and nine have been studied as have the assessment instructions for teachers. Staff responsible for the test construction have been interviewed. The aims, core content and knowledge requirements of the curriculum were also studied. The concept ‘ethical competence’ was used as an analytical tool in the qualitative content analyses. Through the design of this study, the actual test, its process of construction and the curriculum were examined. The results suggest that ethics education, given (a) the curricular construction of what ability to assess, (b) complexities of test construction in ethics and (c) possible teach-to-the-test effects, runs the risk of being limited to an argumentative, conceptual competence, with ethics education being emptied of crucial content. However, being included in national testing can strengthen the position of a school subject. Is it then an advantage for ethics education to be tested in this way? The critical problems the study raises make the author conclude it to be a disadvantage for ethics education to be tested through national tests.

KEYWORDS:

• Ethics education
• compulsory school
• national testing
• character education
• ethical competence

Introduction

‘What may be learnt in ethics education?’ is a crucial question to pose, not least in relation to contemporary ethical societal challenges such as migration, climate change, responsibilities towards future generations and the equality and rights of human beings. The urgency of present ethical challenges such as these raises questions of how ethics is taught and learnt in school. That this varies in compulsory schools in Europe is shown by Korim and Hanesová (2010), whose study demonstrates how ethics education is differently shaped within curricula in the European countries. In this study, Swedish ethics education for the compulsory school is at the centre.

In 2013, additional national tests were introduced for grades six and nine in several school subjects, including the subject knowledge of religion [religionskunskap], in which ethics is tested. In this article, a critical analysis is undertaken of the testing of ethics education through the 2013 national test within its curricular context, i.e. the goals expressed in the new Swedish curriculum from 2011. The research interest revolves around the effects that the curriculum and a system of national tests may have for a particular subject, in this case, for a humanistic, philosophically oriented subject like ethics.

Purpose, research questions and contribution

The purpose of this paper is to describe, critically analyse and discuss the Swedish system of assessing ethics education in compulsory school through national tests. This is to identify potential effects on ethics education in compulsory school of the system of national tests introduced in 2013, in turn based on the new Swedish curriculum from 2011. Methodologically speaking, the study consists of qualitative content analyses of documents, mainly curricula, syllabuses, national tests and assessment instructions. Additionally, interviews were carried out with the two main members of staff responsible for the process of construction of the 2013 national tests in ethics. The concept of ‘ethical competence’ serves as the analytical focus for this article.

The initial research questions of this study were (a) What ethical competence is asked for in the 2013 national test? and (b) What conceptions of ethical competence are left out? In addition to these two research questions, the study also revolves around a second set of critical questions: (c) Can assessing ethics through national tests be understood as an advantage or not? This question in turn implies another one: (d) Under what conditions can national testing be said to be valuable for a school subject? Consequently, the more general contribution of this study consists of a critical discussion of the value of national tests for a school subject like ethics.

Background and research overview

Ethics education—in varying curricular contexts

When overviewing European ethics education in compulsory school, Korim and Hanesová (2010) note that ethics education, due to historical, cultural, ideological, political and economic factors, is organized differently in different countries. In some curricula, in 14 European countries in fact, ethics is a separate subject and is mainly devoted to character education. However, in 17 European countries, ethics is either a general concern of the entire curriculum or of a group of subjects, e.g. social science subjects, or is part of one single subject. In six of these countries, Sweden being one, the subject of which ethics is a part is a non-confessional religious education or the subject of philosophy. These approaches differ from those in southern European countries, where ethics education is integrated into confessional Christian education, or former Eastern European states where ethics is not combined with studies of religions. Curricula also vary regarding whether the emphasis within ethics education is more generally on values education or on a philosophical introduction to ethics (Korim & Hanesová, 2010).

Since 2011, Sweden has a new curriculum, Curriculum for the compulsory school, preschool class and the recreation centre 2011 [hereafter Curriculum, 2011]. Ethics is mentioned in the initial parts of the curriculum¹ as an overall important goal for the education of Swedish students (Curriculum, 2011, p. 9–21) but it also has its specific place in the syllabus of the non-confessional subject religionskunskap [literally: knowledge of religion], hereafter religion education (RE).

Assessment of ethics education and ethical competences

With Curriculum 2011, the assessment of Swedish students through national tests was expanded, starting in 2013. To the already ongoing tests in mathematics, Swedish and English, tests in social and natural science subjects were added. In RE, which includes ethics education, national tests were introduced in grades six and nine in 2013. However, in 2015, the tests for grade six became optional and were abandoned in 2016, which was also the case with tests in grade six for the other social and natural science subjects.

In relation to the expanded Swedish national tests, a certain amount of research has been conducted. A think-aloud study related to ethics education and its assessment was performed by Osbeck, Franck, Lilja and Lindskog (2015). In the study, six teachers were audio-recorded while analysing students’ responses to ethics tasks from the 2013 national tests. The teachers each reassessed 12–15 students’ tests, from both grades six and nine. The results show that the teachers had difficulties with the assessment instructions, due both to inconsistencies within the instructions and the teachers themselves not agreeing with the instructions. In sum, the results of Osbeck et al. (2015) show ambiguities regarding the assessment of students’ responses and indicate that the standards of assessment were not clear.

In a follow-up study, Osbeck (2017) chose one task in the grade nine 2013 national test² and analysed 50 students’ responses to it. The task was an open task regarding the word forgiveness. The students were asked to ‘reason about’ why forgiveness could be of importance for two partners in a situation involving forgiveness. Osbeck’s main interest was to study the presence of possible ‘ethical insight’ in the students’ responses. The results show that such an insight, as defined by Osbeck, was present in the responses, as well as an existential understanding of the phenomenon of forgiveness. However, such competences are not valued in the assessment instructions, which is also the case for normative dimensions in the students’ answers. Osbeck discusses ethical insight and existential understanding as sub-competences of a more general ethical competence, as well as analytical, argumentative and normative sub-competences, and argues that the RE syllabus, national tests and assessment instructions are silent in relation to competences like ethical insight and existential understanding. In a second follow-up study to Osbeck et al. (2015), Franck (2017) develops another scheme of sub-competences based on different material and analyses. Franck describes five sub-competences: reflective, conceptual, normative, analytical and action sub-competences. A different approach to ethical competence is demonstrated in Sporre (2017), with an interest in how global responsibility is expressed in the curriculum and tested in the national tests. Based on the initial parts of Curriculum 2011, Sporre defines the ethical competence as expressed there and then compares that competence to the one mentioned in the syllabus for RE and the 2013 test. The complex ethical competence expressed in the introductory parts of the curriculum is simplified in the syllabus of RE and the test. Regarding a global responsibility, it is not tested as an ethical concern through the national tests in RE, nor in any other social science or natural science subjects, nor is global responsibility paid attention to on a syllabus level to the same extent as in the introductory part of Curriculum 2011, where it is given considerable attention.

The three studies above exemplify different approaches to a discussion of ethical competence by approaching it as a totality (Sporre, 2017) or by discussing it in terms of sub-competences (Franck, 2017; Osbeck, 2017). Larsson (2017) demonstrates another approach by arguing for critical thinking as the ethical competence to be tested in his discussion of what ought to be the focal point of national tests. Stern (2017), in turn, discusses the ethics of testing ethics and the potential of alternative forms of assessment to national tests. In a wider discussion, Fancourt (2017) links the development of the Swedish system of national tests to the present neoliberal steering of educational systems, with its orientation towards seeing education as preparation for working life and its focus on competence being part of such a development, initiated and influenced by the OECD.

Mathematics education, assessment and competence

Mathematics is a school subject that, in contrast to ethics, has been tested for a long time in Sweden. In a study similar to the present one, with an interest in test questions and competences, Boesen, Lithner and Palm (2018) analyse a number of test questions in Swedish national tests from 1995 and onwards. Available test questions from various years and grades are analysed with regard to how well they represent the competences and competence-related activities within mathematics, against the background of ongoing international didactical discussions about mathematics. The analyses of Boesen et al. (2018) point to limitations in the test items as compared to the richer descriptions of competences in the didactical research discussions. Boesen et al. (2018) have, compared to this study, many more test items to base their analyses on, as mathematics has been tested for more years and in more grades (three, six and nine). However, for both studies, access to the tests is restricted as only test items for certain years are openly available for research, and other test tasks are kept secret in order to allow them to be reused. It is also worth noting that the research-led discussion on competence and sub-competences in didactics of mathematics has been developing over decades, whereas in ethics education, such a discussion has begun more recently.

Swedish students taking (more) tests

Silfver, Sjöberg and Bagger (2016) (see also Silfver, Sjöberg & Bagger, 2013) have studied how the macro-trends of performativity filter down to the mathematics classroom and affect grade three children’s understanding of themselves as test-takers, their proficiency in mathematics, and their relationship to the teacher in relation to tests. Some children understand themselves as ‘appropriate’ test-takers, others less so. Furthermore, the test interrupts the usual learning climate, as cooperation between children is no longer promoted, and the role of the teacher changes from someone who assists to someone who may seem not to care about his/her students (Silfver et al., 2016).

Using group interviews with students in grade six, Löfgren and Löfgren (2015, 2016, 2017) have followed up on students’ experiences of the expansion in numbers of national tests in Sweden. Their studies document students’ narratives in relation to the tests in social and natural science subjects. Löfgren and Löfgren (2015) point to the importance that students, and their parents, give to the tests, but it is also obvious that the students in the new test situation to a large extent are left alone in preparation for the tests—not assisted by parents nor teachers. In another study, Löfgren and Löfgren (2017) analyse the narratives of children to see how the performative policy is understood and enacted by the students. Students’ expressions of their identity, of the classroom atmosphere and teachers’ agency are obviously influenced by the performativity policy. In a third study, the focus is on variations in students’ narratives regarding whether to understand the tests as high or low stake (Löfgren & Löfgren, 2016).

Assessment of ethics education and curricular variations

When searching for research regarding assessment or evaluation of ethics education, it becomes obvious that much of the research does not deal with compulsory school but rather with higher education programmes, e.g. business education (cf. Nicholson & DeMoss, 2009). A study by Avci (2017), aiming to determine aspects of quality in ethics education, identified 34 relevant scholarly articles from 2010 to 2015, mainly from other disciplines than education. Avci proposes that the important quality factors for ethics education are that curricula are well structured and that theory and practise are integrated.

A research field relating to ethics education that has been widely researched throughout the years, see e.g. the overview by Lee and Taylor (2013), is the field of moral education and moral development. However, studies in this field differ from the present one, since in the research tradition of moral development, the interest in testing is to test the moral development of individuals, whereas in the Swedish case, the testing through national tests has the purpose of testing students’ knowledge in relation to stated curricular goals that do not focus on individual moral development as such. However, in a more general sense, research from this research field can provide a useful background.

When approaching the issues at stake here, international curricular comparison could be of interest. One recent example of curricular development that has received interest within research is The Ethics and Religious Culture Program (ERC) of Québec State, Canada (http://www.education.gouv.qc.ca/en/contenus-communs/parents-and-guardians/ethics-and-religious-culture-program/). This comprehensive programme was introduced in 2008 and includes ethics education, with developed goals, for 12 years of primary and secondary schooling. In a study focusing on the ethics part of the programme, Bouchard and Morris (2012) evaluate the programme drawing on a Habermasian understanding of ethics. They find shortcomings in the ERC programme regarding its integration of the self-understanding of students and existential dimensions of their understanding of themselves in the social world, including a communal telos. These shortcomings are seen as deficits resulting from a narrow ‘academic approach’ that has replaced a broader one that was present in the preparatory work for the ERC programme.

Another example regarding ethics education in the curriculum for compulsory school comes from Lehrplan PLUS, Grundschule, Bavaria, Germany (https://www.lehrplanplus.bayern.de/fach/ethik#). The structure of the ethics syllabus is described in three dimensions which make up a competence structure. The three dimensions are (1) central object areas linked to the ethical concepts: norms, values, mind and moral; (2) four process-related competences to be developed: to recognize and understand, to reflect and judge, to empathize and take interest and to act and communicate; (3) four thematic areas: being human, living together, religion and culture, and the modern world. Through this competence structure, ethics education is described in terms of both content and ethical competences. There is a strong emphasis on pro-social values in the syllabus, emphasizing, for example, encounters that allow the student to meet diversity. The ethics subject in Bavaria is an alternative for students not participating in Protestant or Catholic RE.

In a German research project, ETiK, both examples of a conceptual structure of ethical competence and examples of test tasks related to it are presented (Benner & Nikolova, 2016). Four building blocks of an ethical competence are described. The first of them links basic knowledge in ethics to a discerning competence and an action-preparing competence, a second one deals with the identification of individual morals in relation to those of others, the third one handles volitional dimensions and the fourth treats argumentation through identification of various models of ethical decision-making in relation to moral philosophers (Benner, 2016).

National tests in Sweden—part of contemporary trends

This research overview has contributed examples of alternative ways of viewing competence in ethics education and pointed to various ways of constructing curricula and syllabuses. It has also shed some comparative light from the national testing in mathematics education and the competence discussion therein. In addition, students’ experiences of the new situation with more testing have been exemplified. Studies have also indicated that schools, teaching and learning can be evaluated in other ways than through national tests.

Regarding the Swedish system of (expanded) national tests, it is an example of international trends of high-stakes testing as discussed by, for example, Au (2011) and in the studies referred to above by Löfgren and Löfgren (2015, 2016, 2017) and Fancourt (2017). The developments in the Swedish school system of the last 25 years, with its decentralization and the introduction of free choice in the school sector, are brought up in an international study by Burns, Blachenay and Köster (2016) to provide the basis for a critical discussion on neoliberal trends and how accountability and school development can be achieved locally. Also, discussing accountability in contemporary educational systems, Sahlberg (2010) questions the role of tests as a basic factor in the development of public schools. In an extensive research overview of evaluation and assessment in Swedish education, Forsberg and Lindberg (2010) note that the concept assessment originally used to refer to individual performance, while evaluation mostly referred to the evaluation of schools and systems, but nowadays the terms have come to be used more interchangeably. This may be seen as part of a more general trend where the assessment of individuals, through (inter)national testing, nowadays plays a more crucial role in the evaluation of schools and national school systems.

Methodological and theoretical considerations

Methodologically speaking, this study is a qualitative study analysing the content of documents, with ‘ethical competence’ as the analytical focus. The material for the study is the ethics questions in the Swedish national RE test 2013, and the 2013 assessment instructions to teachers for grade six and nine (Bedömningsanvisningar årskurs 6, 2013; Bedömningsanvisningar årskurs 9, 2013; https://idpp.gu.se/forskning/utvecklingsprojekt/nationella-prov/religionskunskap). However, Curriculum 2011, including the RE syllabus (p. 176–188), is essential for an understanding of the ethical competences asked for in the national test, and it is presupposed in this study that the test is expected to test the goals of the curriculum. Also of interest are the agreements between the University of Gothenburg (GU), in charge of the construction of the test, and the Swedish National Agency of Education (SNAE), giving the guidelines for the test construction (‘Agreement’, Bilaga 1, 2011; ‘Agreement’, Bilaga 1, 2012). The annual reports provided to SNAE by the university (Lindskog, 2013, 2014) are also of value.

The analytical tool ‘ethical competence’ is used heuristically, meaning that no particular theoretical understanding has guided my use of the concept. In the texts, terms like skills, abilities, knowledge requirements but also general curricular or syllabus goals express what here is interpreted as indicating ‘ethical competence’.

The content analysis has been carried out in line with a thematic analysis (Vaismoradi, Turunen, & Bondas, 2013) rather than content analysis, quantitatively understood. The texts have been read, and reread, several times and on a detailed level, drawing attention to formulations in order to interpret their meaning. Furthermore, in the interpretation, the texts from SNAE have been understood as directing the processes of test construction.

It is obvious that ethical theories can influence the way ethics education is understood, constructed and described in school curricula and tested. Within contemporary ethical theory, discourse ethics (Benhabib, 1992), neo-Aristotelian perspectives (Nussbaum, 1990), relational ethics (Løgstrup, 1956) and utilitarianism (Singer, 1993/2003) represent varying theoretical approaches. However, as formulations in test questions, curricula and syllabuses are brief, it can prove quite difficult to convincingly show theoretical influences in material such as the texts under study here, as works of moral philosophers and national tests/curricula are texts with clearly different purposes. Still, it can be of interest to observe whether any tendencies emerge. In general, it can be held that for a national curriculum for a compulsory school, a broad orientation in the varying traditions of ethical theory can be expected. Such a normative presupposition undergirds the present study.

In a general understanding of contemporary curricular trends and development, the author shares critical perspectives as expressed in the works of Stephen Ball (e.g. Ball, 2012; Ball & Olmedo, 2013) and as expressed by Löfgren and Löfgren or Fancourt in the research overview above. However, the main research interest here is not sociological but rather tending in the direction of didactics of ethics.

When approaching the task of critically evaluating the national test and its questions, a model intended for evaluation of early childhood education inspired the design of the study. The model was developed by Pascal and Bertram (2016) and emphasizes that evaluation should not focus only on results or outcomes but consider context, process and outcome as a whole. Had this study only analysed the questions of the test, not taking into the consideration the process that steered and directed the test construction including the curricular context, it would have given a limited result. The model by Pascal and Bertram directed the study towards thorough analyses of curricula, agreements and interviews with the university staff who produced the tests. In reporting the results, the three keywords from the model are used. Under context, I will refer to the curricular conditions for the test, under process I report the conditions for how the tests were constructed and under outcome the test questions are presented. Thereafter, a discussion on ethical competence follows.

The two interviews with the staff responsible for the construction of the 2013 national test lasted 1.5 h each. The interviews have been transcribed, anonymized and thereafter analysed. The interviewees were asked about the process and work with the construction of the tests, including interpretations of Curriculum 2011, and to comment on the individual test questions. In addition to being asked for their consent to be interviewed, they were offered the opportunity to read through and comment on the transcripts. The staff also provided the agreements between GU and SNAE and the reports by Lindskog mentioned above.³

Results—context, process and outcome

The context

Curriculum 2011 has a threefold structure for all syllabuses, namely aim, core content and knowledge requirements. The curriculum does not provide examples of lessons or more detailed instructions for teachers, so-called educative curriculum materials (cf. Davis, Palincsar, Smith, Arias, & Kademian, 2017) but is quite general in nature. Under aim, a general text applicable to all 9 years of compulsory school is given. The texts under core content and knowledge requirements are split up and are different for grades one to three, four to six and seven to nine. Knowledge requirements for grades six and nine specify the knowledge required for each of the marks from A to E (A being the highest and E indicating pass, with F used for non-passing). Under core content in the subject of RE, the four different themes are religions and other outlooks on life, religion and society, identity and life issues and ethics. Under each of these themes, two to six bullet points indicate what the core contents are. The three bullet points under core content for ethics, grade six, read:

● Some ethical concepts, such as right and wrong, equality and solidarity.

● Daily moral questions concerning the identities, roles of girls and boys, and gender equality, sexuality, sexual orientation and exclusion and violation of rights.

● Questions about what a good life can be and what it may mean to do good (Curriculum, 2011, p. 179).

And for grade nine, there are four points mentioned under core content:

● Daily moral dilemmas. Analysis and argumentation based on ethical models, such as consequential and deontological ethics.

● Views of the good life and the good person are linked to different kinds of ethical reasoning, such as virtue ethics.

● Ethical questions and the view of people in some religions and other outlooks on life.

● Ethical concepts which can be linked to questions concerning sustainable development, human rights and democratic values, such as freedom and responsibility (Curriculum, 2011, p. 180).

Knowledge requirements, in turn, describe what the required knowledge is. The progression for marks E–A in the text is described as being about going from a ‘simple’ (E), to a ‘developed’ (C), and a ‘well developed’ (A) capacity. In the case of the ethics theme, the knowledge requirement for mark E, grade six, is formulated thus:

Pupils can apply simple reasoning about everyday moral issues, and what it might mean to do good. Pupils make reflections which basically relate to the subject and use some ethical concepts in a basically functional way. (Curriculum, 2011, p. 181. Bold in original)

For grade nine, mark E, it reads:

Pupils can reason and argue about moral issues and values by applying simple and to some extent informed reasoning, and use ethical concepts and models in a basically functional way. (Curriculum, 2011, p. 183. Bold in original)

It is obvious that the knowledge requirements put a strong focus on reasoning and the use of concepts and models. However, under aim regarding teaching in RE, it initially says:

Teaching should help pupils to develop their knowledge of how different religions and other outlooks on life view questions concerning gender, gender equality, sexuality and relationships. Pupils should, in addition, be equipped to analyze and determine their standpoint in ethical and moral questions. Teaching should also contribute to pupils developing an understanding of how people’s values are linked to religions and other outlooks on life. It should also contribute to pupils developing their capacity to act responsibly in relation to themselves and their surroundings. (Curriculum, 2011, p. 176)

Here, one can note that through the teaching, students are to be ‘equipped’ to take a stance on ethical and moral issues and develop ‘an understanding of how people’s values are linked to religions and other outlooks on life’. The capacity to act responsibly with regard to oneself and one’s surroundings should also be developed through the teaching. It is obvious that this overall aim is broader than the more specific wording of the knowledge requirements for ethics, with their focus on reasoning, concepts and models. The most probable reason for this discrepancy is that in the next paragraph of the aim, after the quoted one, five abilities are named that students are to develop through RE education. The fourth one, the one relevant for ethics, states that students are to develop the ability to

reason and discuss moral issues and values based on ethical concepts and models (Curriculum, 2011, p. 176)

Here, a simplification takes place within the aim for the syllabus of RE. There is a move from a more general goal emphasizing a personal stance in moral issues, the development of an understanding of how values, religions and worldviews interact and preparation for personal responsible agency, to a narrower formulation of one single ability. This ability is then followed up on in the knowledge requirements, while the broader description is present to some extent under the paragraphs of core content but is left out in the knowledge requirements. Additionally, the initial, wider description in the aim is also more in line with the introductory parts of the curriculum (Curriculum, 2011, p. 9–21) with its broader understanding of an ethical competence. It includes a competence ‘to be able to consciously formulate and express an ethical stance based on knowledge where the values of democracy and human rights as well as personal experience are of vital importance’ (Sporre, 2017, p. 122). The competence also includes empathy towards other people, their situations and well-being. In addition, critical thinking and independence in relation to others is emphasized in the formulation of the stance. With regard to ethical competences, this competence is an obvious contrast to the narrow ability as it is expressed in the aim for RE and its knowledge requirements, and which later appears in the assessment instructions for the test.

The process

When staff at the GU started to develop the tests in RE in 2011, the new curriculum was an important factor. This was controlled through the agreements between the university and the SNAE. The first agreement guaranteed that the university would work on the project until 2014 and the goal was to produce a test to take place in spring 2013, thereafter to be followed by annual ones. The agreement also stated that GU was to report annually on their working process, in a ‘verksamhetsberättelse’ [report on activities] here referred to as Lindskog (2013, 2014). The overall purpose of national tests, according to the agreement, was to assist in equality of assessment and setting of marks and to provide a basis for analysis of how the ‘knowledge aims’ are achieved on the school, provider/district and national levels. (‘Agreement’, Bilaga 1, 2011, p. 1).

More detailed instructions in the agreement clarify that in addition to the construction of the tests, GU was also to develop teacher instructions and a model for how to combine marks from different parts of the test to produce a single mark. The university was also to assist the SNAE in the distribution of the 2013 tests with testing beforehand to make sure that the actual test of 2013 had a ‘very good coverage’ of the content of the syllabus of the subject (‘Agreement’, Bilaga 1, 2011, p. 2). GU was also to participate in meetings organized by SNAE and cooperate with other universities assigned the task of developing tests in other school subjects. This was to achieve a high level of similarity between the different tests in construction of tasks and teacher instructions (‘Agreement’, Bilaga 1, 2011, p. 2).

Consequently, the staff at GU had certain factors to take into consideration. They were to develop the test in cooperation with SNAE and with groups of staff at other universities, in their case colleagues in the other social science subjects: geography, history and civics/social studies. This should be done with respect for the specific profile of the RE subject but still striving towards uniformity. One paragraph of the SNAE instructions also stated that the tests were to be constructed so that they took into consideration the varying ages and levels of maturity of students. Additionally, core content left out the first year was to be tested the year after (‘Agreement’, Bilaga 1, 2011, p. 1–2).

Studying the reports from GU (Lindskog, 2013, 2014), it can be seen that a number of pre-tests were carried out from the autumn of 2011 to 2012 until the spring of 2013, when the first test took place. The pre-testing involved full-scale tests as well as mini tests, the latter with just 25–90 students and fewer items, and the other tests with larger numbers of students and tasks. From the reports, it is also obvious that experienced RE teachers were involved in the process of assessing the test questions. They also assisted in construction and testing of items, as well as the development of teacher instructions (Lindskog, 2013, p. 10).

As mentioned above, regarding Curriculum 2011, five abilities, with the fourth one valid for ethics education, are included in the aim for RE (Curriculum, 2011, p. 176). In the assessment instructions for teachers in the 2013 national test, it is obvious that these five abilities are introduced as the abilities that are to be tested through the national test (Bedömningsanvisningar årskurs 6, p. 4; Bedömningsanvisningar årskurs 9, p. 4). In these texts, the abilities from the aim are combined with the knowledge requirements of the syllabus. The ability to ‘reason and discuss moral issues and values based on ethical concepts and models’ (Curriculum, 2011, p. 176) is combined with parts of the knowledge requirements as indicated below in Table 1. The selected and condensed parts of the RE syllabus in Curriculum 2011 are therefore what came to constitute the core of what was regarded to be important to test and assess when ethics education was to be evaluated through the 2013 national tests.

Table 1. Ability and knowledge requirement for ethics education according to teacher instructions, grades six and nine respectively.

Grade	Ability	Selected parts of knowledge requirements
6	Reason and discuss moral issues and values based on ethical concepts and models	The pupil can apply … reasoning about everyday moral issues, and what it might mean to do good. The pupil makes reflections … and uses some ethical concepts in a … functional way
9	Reason and discuss moral issues and values based on ethical concepts and models	The pupil can reason and argue about moral issues and values by applying … informed reasoning, and using ethical concepts and models…

The outcome

The complete RE test in 2013 for grade six consisted of 21 tasks, out of which 4 tested ethics. The test for grade nine consisted of 25 tasks and 3 tested ethics. The tasks are presented below, first for grade six (Table 2) and thereafter for grade nine (Table 3).⁴ Based on the interviews, comments from staff involved in developing the tests are presented below revealing deliberations and general reflections in relation to the actual process of constructing the tasks and the difficulties involved.

Table 2. Tasks testing ethics in the 2013 national test, grade six.

Grade 6, task no.	Content (abbreviated)	Concepts/Models suggested for use	Type of task
3	Maria and Keyla, have been friends for a long time. Maria has a problem with sweating. Should Keyla talk to Maria about this or not? Their friends regard this to be a problem. Task: write down what Keyla should do and argue also why and what consequences there might be of what she does. Use two of the suggested concepts	Responsibility Wrong Right Duty Fair	Open, free writing
9	Ethical concepts. Combine a statement with one of the three suggested concepts and argue which one fits best. (1) Boys and girls should be treated equally (2) If I see that someone else is feeling left out/an outsider, I show that I want to be a friend to that person (3) Where I live, there is a sports hall. Teams of boys and girls practise there. All the teams have 5 h a week for training (4) In Sweden same-sex marriages are allowed. This means that you are allowed to marry someone of the same sex as yourself	Suggested concepts per task (1) Empathy, solidarity, equality (Sw. jämlikhet) (2) Justice, solidarity, equality (3) Justice, solidarity, empathy (4) Empathy, justice, equality	One of the three concepts to be ticked, and then one should argue for one’s choice. Assessment instructions indicate that the construction of reasoning is what matters not which concept was chosen
10	The task is introduced with the heading ‘Violation’ (Sw. kränkning). Students are instructed to invent a situation that exemplifies a violation, to briefly describe the situation and argue for why it is a violation and then to suggest what could be done to improve the situation		Open, free writing
13	The task is introduced by a chat conversation between two students. One has been absent from school the same day and asks the other one for notes he/she has taken during the day. The student does not seem to want to hand over the notes, and ends the conversation saying the computer must be used by the student’s mother. The task is to discuss the situation and possible consequences it might have. At least two of five suggested concepts are to be used	Responsibility Wrong Right Fair Duty	Open, free writing

Table 3. Tasks testing ethics in the 2013 national test, grade nine.

Grade 9, task no.	Content (abbreviated)	Concepts/Models suggested for use	Type of task
12	Forgiveness is introduced and called an ethical concept, and reference is made to students perhaps having experienced being forgiven or having forgiven. Task: discuss why forgiveness can be of importance to both the one being forgiven and the one that forgives	Forgiveness	Open, free writing
24	It is stated that models can be used as tools for ethical discussion. Three kinds of ethical theories (A–C) are described as models. Students are then to match each description with the name of three ethical theories (1–3). The theories are also given synonyms (here in parenthesis)	(1) Intentionalist ethics/(disposition ethics) (2) Consequentialist ethics/(effect or result ethics) (3) Deontological ethics/(rule ethics)	Combine A–C with 1–3
25	In this task, the death penalty is introduced through a short text ending by stating that one can argue for or against it. In the task, students are to discuss whether the death penalty is right or wrong using the models/ethical theories from the earlier task	Intentionalist ethics Consequentialist ethics Deontological ethics	Open, free writing

Two matters become obvious from this overview: (1) five out of the seven tasks are constructed to be open tasks, for free writing; (2) knowledge of concepts and models plays a crucial role in terms of the content that is asked for and/or the assistance given. Another observation is that the number of questions testing ethics is 3 out of 25 in the grade nine test, where 2 of the 3 questions are also linked to one another (no. 24 and 25), and 4 out of 21 in the grade six test. This means that there were not many tasks testing ethics.

From the interviews, certain matters stand out.⁵ One of these is the reflection from Interviewee 1 (I1) concerning the difference between constructing a test for one’s class and constructing national tests. In constructing a test for your own class, as a teacher, you know your students, you know what you have taught recently or earlier, so when constructing a test and identifying the appropriate level of testing, you have an advantage as compared to when you construct a test for 25,000 unknown students. The interviewee also reflected on how one can construct tests that are fair in the assessment of knowledge of students in different situations, with different backgrounds and with, for example, varying reading and writing abilities. I1 exemplified various types of questions when reflecting on this. At the same time as having all these variations in mind, one must also, according to the instructions, construct tests where the individual tasks can discriminate between different levels of marks, A–F, in the students’ responses. Another aspect, raised by both interviewees and specific to ethics, is that certain questions cannot be asked. For instance, certain ‘classical’ ethical themes like abortion, or euthanasia, may be impossible to put into a test, as they can be emotionally sensitive to individual students and carry a risk of unfairness in the test situation. However, given matters such as these above, the many pre-tests in schools of different kinds, the involvement of experienced teachers and joint discussions facilitated the process. The work together with the SNAE, and staff at other universities who were constructing tests in the other social science subjects, was also important but still, for those working with the RE test, their ‘own’ RE process mattered more, as differences between the subjects were relevant in test construction.

When discussing the construction of tasks for grade six, I1 pointed to difficulties regarding the way the ability and knowledge requirements in ethics are formulated, as the students are to discuss, reflect on what they discuss as well as use specific concepts. The pre-tests contained tasks where the students’ written discussions were very advanced, but when asked to use certain specific concepts, this complicated the students’ answering of the test questions. According to I1, the problem is not that the students may not have an adequate understanding of concepts. If that in itself had been tested, by for example having descriptions of concepts where the task was to combine the term/concept with a description, that may very well have worked better. However, the staff were instructed to test for the ability in one and the same task and not split the aspects of the ability up across more tasks, which would have been easier for the younger students to deal with—more in line with their maturity.

Commenting thereafter on the four individual tasks of 2013, I1 considers that two out of four functioned well, no. 3 and 13 (see Table 2). They generated many responses from students who could show their knowledge. Task no. 10 demanded that the students first come up with a situation, then describe it, then justify their choice and finally suggest possible solutions. Given the limited time in a test situation, such a task demands too much, according to I1. In task no. 9, the suggested concepts were somewhat close to one another in their significance and were not put before the students in a systematic way, which caused them difficulties. The interviewee also reflected on the character of the RE subject and the ethics-related parts of it in comparison with geography or history, subjects where there is seemingly more matter-of-fact knowledge to be tested. It was also the case that the group working with RE very strictly looked at abilities and knowledge requirements in preparation for the 2013 test, rather than at what is stated in the core content of Curriculum 2011.

The second interviewee (I2) commenting on the grade nine test discussed early on in the interview the frequent use of the Swedish verb ‘resonera’ [in English: discuss, reason] in the knowledge requirements for RE. In the ethics-related questions, it is also combined with the verb ‘argumentera’ [in English: argue], which in Swedish has connotations of giving arguments for or against a certain matter. In the group developing the test, a lot of discussion was spent on how this ability, to ‘resonera’, can be tested. In the knowledge requirements, furthermore, ‘concepts’ and ‘models’ are mentioned (see above), so this is also what was to be tested according to I2. With Curriculum 2011, there was also a change, as ‘ethical models’ were now to be taught in grades seven to nine, while before they were to be taught in upper secondary school (grades ten to twelve). When asked about how other competences than to ‘resonera’ and ‘argumentera’, for example a competence for action, could be expressed in the tests, I2 stated that to learn how to argue for or against a certain moral action is preparation for action, pointing also to shifting of perspectives as being an important aspect of an argumentative capacity.

Looking at the three ethics-related questions of the grade nine test, I2 expressed appreciation of task no. 12, as the task brings together the chosen ethical concept (forgiveness) with a reflective process where the reasoning about the concept reflects shifting of perspectives and allows students to express deeper insights. The other two tasks, no. 24 and 25 (the linked tasks), were, according to the interviewee, constructed so that no. 24 would help students in solving no. 25. This meant that task no. 24 could be seen as a facilitating task that provided information about the various ethical theories to be used in task no. 25. The choice of deontological and consequentialist ethical theories was justified, as they are explicitly mentioned in the core content of the RE subject. Another choice in the construction of the task—to use the death penalty as an example—came after discussions where other possible examples were rejected. In response to a question from the interviewer about whether the death penalty could have been discussed in relation to the value of the intrinsic and inviolable value of all human beings expressed in the introductory part of Curriculum 2011, I2 responded that the task was to test the knowledge requirements of the syllabus for RE, not the general parts of the curriculum. The respondent also commented that certain ethical issues like sustainable development often met with reactions that this was not part of RE. Additionally, looking back to the test from 2013, the interviewee found that more tasks could have been dedicated to testing ethics.

Summary and discussion

Research questions (a) and (b)

In response to the first research question, (a) What ethical competence is asked for in the 2013 national test?, it is obvious that the questions in the test asked students to reason, argue and discuss using suggested concepts and thereby demonstrate an argumentative and conceptual ethical competence. In grade nine, the students’ use of ‘ethical models’, or theories, was also tested. This is consistent with the one ability indicated under aim, articulated in the knowledge requirements of Curriculum 2011 and expressed in the assessment instructions. Coming to the second research question, (b) What conceptions of ethical competence are left out?, a few matters can be noted. The competences mentioned in the aim of the RE syllabus, like an understanding of how people’s values are linked to their religions or worldviews, or the formulating of a personal standpoint in moral and ethical questions, and the development of a capacity to act responsibly in relation to oneself and one’s surroundings, were not tested or given any credit. In the assessment instructions, the focus is rather on the way the arguments are formed, and the presence of perspectival shifts or described consequences is also emphasized. What was accredited and given points, to later be summarized and converted into marks for the whole test, is the number of concepts, or arguments for or against, or models/theories used.

In her study of responses to the task on forgiveness (task no. 12) in the grade nine test, Osbeck (2017) searched for ethical insight as a competence and found this competence, together with an existential understanding of the phenomenon of forgiveness, to be present in the students’ responses, as well as a personal stance on matters of forgiveness. Even though this could be important in a discussion of forgiveness, it was not accredited in the assessment instructions, which focus on the argumentative competence. Franck (2017) argues that preparation for responsible agency, a capacity to act, is crucial in ethics education. In the test, such dimensions can be said to have been present in tasks where students have been asked to comment on or suggest alternative actions; however, it must be acknowledged that a capacity to act, or even willingness to do so, is not easy to evaluate through a test. This points to practise-related dimensions of ethics education. There are also other problems when it comes to testing in ethics education, as you may come close to registering students’ personal opinions on matters where their integrity could be at risk. Additionally, the analysis of the introductory part of the Swedish curriculum by Sporre (2017) draws attention to other dimensions of ethical competence, namely how a personal stance needs to include knowledge or facts, a capacity to empathize with people and recognize their well-being, a critical capacity, as well as independence vis-à-vis others.

To conclude, in relation to the first two research questions, the ethical competence tested was the ability to reason and argue with a focus on the use of concepts and ethical theories. Ethical competences that were not tested were the taking of and arguing for a personal standpoint based on values, knowledge and facts, a preparedness for responsible agency and identification of people’s values in relation to their religion or worldview. Neither was an ethical insight or existential knowledge of the tested phenomena accredited. The ‘construction’ of the ability and knowledge requirements in Curriculum 2011 as compared to the aims and initial parts of the curriculum is crucial in explaining this result.

Research questions (c) and (d)

The two remaining research questions are (c) Can assessing ethics through national tests be understood as an advantage or not? and (d) Under what conditions can national testing be said to be valuable for a school subject?

Given the plurality of possible dimensions of ethical competence as exemplified above, it becomes obvious that it can be difficult to test ethics education. In the Swedish case, obvious limitations follow when the wider goals under aim are reduced to one ability, namely the ability to ‘reason and discuss moral issues and values based on ethical concepts and models’ (Curriculum, 2011, p. 176). For the RE subject, five abilities are stated, with the ability in ethics being one of the five. This leads to the question: What if ethics on the curricular level was a subject of its own, would it then not be possible to state, say five abilities that would better respond to and cover more dimensions of a school subject like ethics? From this it follows that it could be an advantage for ethics education to be tested as a subject of its own—to have a different curricular standing.

Or is it rather the case that testing ethics through national tests, as alluded to in the reasoning of the interviewees I1 and I2, implies limitations e.g. in terms of what issues can or cannot be brought up in tests? Is the subject, and the way you can teach and learn it, bound to concrete contexts, local conditions, familiar to the teacher and would the evaluation of students consequently be favoured if it took place locally? Are there additional problems in using a national test format where the ambition is to make tests function equally through the direction of a national body like the SNAE? Alexander (2016) raises the matter of how character education can be measured. Although character education per se is not what is at stake in Swedish ethics education, part of Alexander’s (2016) discussion of measuring can still be relevant to ethics education. Alexander distinguishes between two kinds of measuring, where a more quantitative one may actually become superficial in relation to ethics education. In line with Alexander’s argument (2016), one can ask the following questions: How to handle intentions, purposes and volitional aspects of moral and ethical issues? Does a quantitative measuring contribute towards making superficial what is to be measured? Consequently, is the Swedish model for national tests adequate vis-à-vis ethics as a subject? The proportionally high frequency of open questions in the ethics part of the RE test indicates the complexities of assessing ethics and that it is not a subject that lends itself to quantitative measurement. This argument is also supported by the results from Osbeck et al. (2015), where the teachers were in disagreement with the assessment instructions about what to measure and how to interpret the instructions. This may be interpreted as pointing to the inadequacy of the system: in spite of a number of experienced teachers being involved in the construction of the tests, trying out tasks etc., the instructions were still unclear.

Avci (2017) has emphasized as possible criteria for the quality of ethics education that the curriculum should be well structured and that theory and practise should be integrated. The problem for ethics education with Curriculum 2011 is the contradictory way that ethics education is handled, given the simplification that takes place within the curriculum itself, whereby a more complex ethical competence is reduced and limited. The placement of ethics education within RE furthermore contributes to the problems, as one single ability is formulated as the valid one for ethics. Additionally, the role of the numeric test system in producing points for each task, to be summarized into one mark, is also complex when handling open questions.

To summarize, from the point of view of ethics education, the expansion of the national tests in 2013 into more subjects, ethics included, raises questions. For example, it can be questioned whether the system of national tests really serves the development of Swedish ethics education and whether it is limited by the narrow argumentative and conceptual ability that this study shows is asked for in the national tests. Is there a risk of a possible teach-to-the test effect? Can alternative more complex curricula like the ones already mentioned, from Bavaria, Germany, or the ERC curriculum from the Province of Quebec, Canada, serve as examples where complex goals are set for the ethics education of future citizens? And where assessment is not carried out through national tests but through other forms of evaluation? Additionally, it seems as if the results from the German ETiK project, with its systematic relation between the suggested ethical competence and discussion around test construction, could contribute to the development of Swedish curriculum and test construction.

The above results in relation to questions (c) and (d) point to several disadvantages in the testing of ethics through the Swedish national tests. Furthermore, the condition for national tests in this form to be valid and fair is that the subject to be tested has a structure that matches the test construction. For ethics education in Sweden, this is hardly the case.

Ethical theories, the ability, the national test and ethics education—concluding remarks

How then do the 2013 RE test and Curriculum 2011 correspond to varying perspectives from ethical theories? What can be said about the focus of the 2013 test with seven ethics-related questions and the singular argumentative ability with a focus on concepts and ethical models that were to be tested? From an ethical theoretical point of view, one could tentatively test the argument that the focus on a verbal, argumentative capacity may seem to be in line with a utilitarian or deontological way of understanding ethics, where the purpose of normative reasoning is to prepare for knowing which action is the right one. However, there are not consistent tendencies in the material to support either of these two conclusions. Additionally, the knowledge requirements of the curriculum and national testing do not take ethics education towards a relational ethics where a concern for other (human) beings’ needs is central, nor in the direction of a neo-Aristotelian focus on the good life and the character of human beings, nor towards a discourse ethics involving negotiating deliberations in the public sphere. Even though some examples of test questions activate students’ emotions, or the questions are framed in relational settings, it is an argumentative, conceptual capacity that is recognized. Consequently, a comparison with ethical theories indicates a need to deepen and expand the Swedish national test and the curricular knowledge requirements into more aspects of ethics, in order to provide a plurality of ethical theoretical perspectives crucial for a compulsory school in a multicultural society with a plurality of world views.

This points to the complex character of ethics, and the question remains of how productive national testing such as the one researched here is for a school subject like ethics. Additionally, how is personal integrity and independence vis-à-vis others in the formulation of a personal stance preserved in a test situation where you as a student are to provide correct answers? Given the many limitations of the Swedish national tests 2013 in relation to ethics education, it is reasonable to conclude that it is a disadvantage for ethics education to be tested in such a way.

Disclosure statement

No potential conflict of interest was reported by the author.

Additional information

Funding

The work was supported by The Swedish Research Council [Grant Number Dnr 2014-2030].

Notes on contributors

Karin Sporre

Karin Sporre is professor of Educational Work with a focus on values, gender and diversity. With a disciplinary background in ethics, she carries out policy studies, but her research interests also include studies of students’ and teachers’ work and views on matters related to ethics. Children’s existential issues, human dignity/rights and sustainability within education are additional themes of interest to her.

Notes

1. Note that what is referred to here as the initial part of the curriculum is the more general part of the curriculum [läroplan], not specific to any subject, while the later parts of the curriculum contain the syllabuses [kursplaner] for the individual subjects.

2. Osbeck and researchers at the University of Gothenburg have been granted access to students’ responses as their department has been responsible for the production of the national test in RE. The test questions and assessment instructions from the 2013 national tests are openly available for research, but those from subsequent years are not. Questions are kept secret and can be used a second time. The tests from 2013 and the assessment instructions are available in Swedish at https://www.npsoportal.se/amnesproven-ak-9-2013/.

3. These documents are available through the author.

4. The tasks are, as indicated, translated by the author and slightly abbreviated when the introductory texts were long. For further elaboration on the tasks, see Sporre (2017).

5. The interviews took place in December 2015 meaning that time had elapsed since the construction of the tests from 2013. Both interviewees had participated in the development of further tests. Their reflections on the 2013 tests were to some extent influenced also by these experiences.