Facing the validity gap. Perspectives on the final degree project

ABSTRACT

This study detected patterns in teacher educators’ and pre-service teachers’ beliefs and practices regarding the degree project. Requirements in policy documents were also considered. The analysis centred on validity, teacher cognition and grounded theory. Conversations were conducted with instructors and students at three departments for teacher education in Sweden. Findings showed inadequacies with regard to construct- and consequential validity for instruction and learning, timing, pre-service teachers’ prior knowledge and attitudes to educational research. Recommendations for improvement involved having teaching practice and the project run parallel and part-time during a term. Further proposals tallied with national regulations about autonomy and academic literacy for teachers. For example, the pre-service teachers responded positively to introducing crash courses and providing opportunities for supervision prior to the course. Improved validity was deemed imperative for developing long-term literacy and motivation towards incorporating science and empirical data in the teaching profession.

KEYWORDS:

• Validity
• teacher cognition
• grounded theory
• document analysis
• degree project

Introduction

Validity plays a crucial role in criterion-referenced assessment. It measures to what extent various assessments show evidence of being logical, reliable and useful outcomes of aligned instruction and learning from the perspectives of instructors, students and further stakeholders. As Kane defined it, validity is “the extent to which the proposed interpretations and uses of test scores are justified. The justification requires conceptual analysis of the coherence and completeness of the claims and empirical analyses of the inferences and assumptions inherent in the claims” (2016, p. 198). In other words, outcomes need to be analysed and justified in light of formative and summative teaching-and-learning processes.

The degree project at the final stage of teacher education in Sweden can be defined as a test. The course encompasses 15 ECTS and the intended learning outcomes cover expectations to do with selecting a research issue, suitable theories, methods/methodologies as well as analysing and critiquing the data. Self- and peer assessment are integral parts of the process in the various stages. There are opportunities and challenges for both academic staff and students regarding validity of the final degree project in the Bridging Teacher Education Programme (BTEP). Supervisors and examiners will guide, assess and grade the outcomes in accordance with the intended learning outcomes. Then, the students are expected to take individual initiatives and be self-reliant in academic projects that last a mere ten weeks. In light of the fact that the legal framework for school in the Swedish context stipulates that education “must be based on scientific knowledge and proven experience”, the course includes implications for being proactive (SFS 2010:800, Chapter 1, §5; Skolverket, 2014). The degree project can be a relevant preparation for implementing research and theory in professional teaching.

The aim and rationale of this exploratory study are to detect perspectives on final degree projects articulated by teacher educators and pre-service teachers in light of legal regulations. The research questions that follow emerged in the study:

1. What do the interview conversations reveal about the stakeholders’ perspectives on the final degree project?

2. What does the legal framework stipulate that concerns the final degree project?

At the centre of the aim are questions about validity and alignment in target-oriented instruction and testing, including opportunities and challenges for self-determined learning to develop in light of the requirements. Borg’s approach to pre- and in-service teachers’ cognition and Charmaz’s constructivist framework for grounded theory will be employed in the analysis. No such studies have been conducted previously.

Theoretical framework

Validity

Details in the definitions of the term vary, but it is reasonable to argue that validity relies on inferences from assessment in light of strict requirements for aligned instruction (Kane, 2016; Martone & Sireci, 2009; Messick, 1989). Messick’s ground-breaking argumentation advocated that tests should measure what they purport to measure, that the way in which they are conducted should be aligned with teaching and learning and that test designers must consider their impact. In other words, “validity is an inductive summary of both the existing evidence for and the consequences of score interpretation and use” (1979, 1989, p. 13). This means that constructing and conducting tests are crucial phases. Concerns about teaching, learning as well as consequences of testing in the extended sense need to be considered carefully. What will tests lead to in terms of knowledge and skills for the learners when they finish their studies, apply for, and occupy, positions in society? Hence, construct validity, in Messick’s words “the meaning of the measure” or the raison d’être of the summative assignment, is of central importance both for interpreting and justifying the use of tests (1989, p. 17). Then there is consequential validity or the consequences and usefulness of tests in the extended sense of time and space to consider (Messick, 1989, pp. 17–18 and passim). Similarly, the degree project is supposed to facilitate a deep understanding of research approaches and methods in teacher education and beyond.

Self-determined learning

Teacher education is accountable for promoting and facilitating self-determined learning. For example, academic staff are expected to instruct future teachers about instructional- and constructive alignment (Biggs & Tang, 2011; Cohen, 1987). Alignment between standards or requirements in policy documents, instruction, assessment and developing strategies for deep, active learning is imperative both for instructors and students (Entwistle, 2018; Marton, 2017). Similarly, Richter et al. advocated “active and self-regulated learning” in high-quality teacher education and argued that “teacher educators who perceive their tasks more from a learner-centred perspective show higher levels of constructivist-oriented beliefs, self-efficacy, and job satisfaction” (2021, p. 7). Yet another way of articulating such requirements in target-oriented instruction is to define relevant learning perspectives. Heutagogy, or self-determined learning, outlined by Blaschke emerges from a “central focus on learner agency [and] encourages the development of skills of autonomy and exploration, reflection and critical thinking, and innovation and entrepreneurship” (2021, p. 1630).

On the one hand, a condition for fostering such reflective, autonomous learning when producing and defending a degree project requires careful planning and instruction to secure the students’ knowledge base and academic literacy with regard to formalities. On the other hand, in heutagogy learners are equally responsible for reaching the targets and taking responsibility for their own learning. That is to say, requirements and expectations with regard to alignment and heutagogy should serve the same purpose for instructors as for students. Bandura advocated “self-efficacy” and “agency” in learning processes and argued that “perceived self-efficacy refers to beliefs in one’s capabilities to organize and execute the courses of action required to produce given levels of attainments […] Efficacy belief is, therefore, the foundation of agency” (2009, pp. 16–17). To be able to take responsibility for their own learning, students need advanced instruction on how this can be achieved. Academic staff are also expected to practise what they preach when conducting courses. At times, however, these goals may be difficult to reach when students face different approaches to teaching, learning and research in various disciplines (de Joyce & Feez, 2016; Fuchs, 2018; Sjølie & Østern, 2021). Such differences may pose validity problems unless pre-service teachers are provided with aligned guidance in academia.

Methodology

In this study, teacher cognition, constructivist grounded theory and document analysis form a methodological approach to analysing the findings.

Pre- and in-service teacher cognition

Borg’s ground-breaking teacher cognition is a “construct” in which pre- and in-service teachers’ knowledge, beliefs, emotions, etc., can be analysed for raising awareness of, and “considering alternative positions” for, teaching and learning (S. Borg, personal communication, 17 January 2022). Placing such findings in a “broader context” in terms of time and space for the individual and for the community may be necessary for understanding the complexity of teacher cognition (Borg, 2015, pp. 320–321; Borg, 2019, pp. 1154–1155). In the present study, such concerns will revolve around the participants’ attitudes to, and beliefs about, opportunities and challenges for instruction and learning in light of regulations and research.

Grounded theory

Grounded theory determines the methodology and the method. Co-constructed by the participants, the data generate the findings and the analysis. When dealing with participants’ knowledge and beliefs as well as formal regulations in a country, a dynamic perspective on what emerges from various data may be productive (Glaser & Strauss, 1967). In Charmaz’s words about her constructivist approach, “the research process is fluid, interactive, and open-ended” and “a confluence of conditions in the research situation frames the data we collect” (2014, p. 23 and p. 320). This approach aims to provide an organic, progressive perspective on what the data convey and acknowledges or even welcomes the idea that researchers participate in, and construct, their research (Charmaz, 2014, p. 13; Cohen et al., 2018, p. 247). Thus, it challenges notions about achieving complete neutrality and objectivity in research conducted by human beings in specific conditions.

Moreover, this study goes beyond strict demands for induction in grounded theory. Thornberg (2012) even argued that the “grounded theorist has to accept the impossibility of pure induction and at the same time recognize the analytical power of the constant interplay between induction (in which he or she is never tabula rasa) and abduction” (pp. 247–248). Similarly, the authors of the present study employed a flexible, interactive approach to the selected texts. Hence, a theoretical framework was included for the sake of scope, delimitations and clarity.

Document analysis

Furthermore, the authors employed qualitative document analysis. When analysing findings from the interview conversations, regulations function as “supplementary sources of data” or alternative “texts” that may shed further light on the participants’ perspectives (Charmaz, 2014, p. 45). The selection of documents was based on the regulatory framework for instructors and pre-service teachers in teacher education but also on what is required for qualified secondary teachers in school.

Method

Sources of data

This qualitative study was conducted in the BTEB at a Swedish university between the years 2020 and 2021. Eleven semi-structured interview conversations and policy documents on the national level were included. The study employed “a systematic strategy of simultaneous data collection and analysis” or what is called an “iterative process” in which “sharp distinctions between data collection and analysis phases of traditional research are intentionally blurred” (Charmaz, 2014, p. 343). In other words, instead of applying a theory to the data, potential messages and patterns in them were first detected and coded so that a local perspective on the findings evolved. Charmaz’s definition of “interview conversations” that follows is relevant to this study:

Discourses accomplish things. People not only invoke them to claim, explain, and maintain, or constrain viewpoints and actions, but also to define and understand what is happening in their worlds. Thus, discourses serve purposes but not all these purposes are strategic. Interviews offer one way of eliciting discourses, which may be multiple, fragmented and contradictory as well as coherent and consistent. And research participants can use interviews to find, piece together, or reconstruct a discourse to make sense of their situation. (Charmaz, 2014, p. 85)

Employing interview conversations in the present study similarly aimed to let participants articulate and discover their honest thoughts, beliefs or attitudes and values. Charmaz defined intensive interviewing as a way of “gathering accurate ‘facts’” as far as it is possible, which is also relevant here (2014, pp. 56–59). Hence, lived, subjective experiences in the phenomenological sense were regarded as data in the present study. Both authors transcribed and interpreted the conversations verbatim. Using NVivo, the texts were then coded individually and collaboratively in recurrent sessions.

The document analysis covers education acts for higher education and for school as well as ordinances dealing with the national system of qualifications for lower- and upper-secondary school teachers and the curricula for school (see details about the sources in Findings from the regulatory framework for teacher education and for school).

Instructors

Interview conversations were conducted with four instructors. IA, IB, IC and ID represent two faculties and three teacher-education departments. All of them have extensive experiences from teaching and educational research connected with their various subjects.

Pre-service teachers

The seven pre-service teachers included in this study are PE, PF and PG, PH, PI, PJ and PK. Interview conversations with them include two focus-group conversations and two individual ones. The majority of the students studied their various subjects between ten and twenty years ago in Sweden or abroad.

Findings

Findings from the interview conversations will be outlined below. First, we examined aspects introduced by the instructors and the students. Secondly, we conducted a document analysis in light of the requirements and expectations in the national policy documents for higher education and for school.

The qualitative interviews were delimited to the categories mentioned in the subheadings below. First, we created several “initial codes” and then we decided on a couple of central aspects in a process that Charmaz called “focused coding” (2014, p. 113). In the latter phase, the most relevant areas formed a frame for the analysis.

Instructors’ comments on challenges in the short programme

All of the instructors mentioned aspects to do with challenges regarding course design in the BTEP. For example, ID underlined the fact that there are extensive areas in teaching and learning that need to be covered in a programme that merely spans three terms. He agreed with the authors about some of the challenges regarding the academic literacy of the students. On the one hand, they come from various disciplines and may not be well-versed in educational sciences. On the other hand, he stated, there is one exception – students who specialise in Swedish as a second language have come across this research design in previous courses. Then, ID outlined some general thoughts on the degree project:

One of the most important aspects of this course, I find, is to be able to critique educational sciences and be able to take an interest in new research [but] then you will also need to […] have seen how such a process can develop. This is why many of them do not really succeed. It takes time to collect data. You are supposed to study the literature etc. and then you are expected to produce an analysis but which ends up being a mere descriptive presentation.

The limited amount of time and practice for writing a proper study impinges on the quality, according to him. However, incorporating some elements for preparatory stages during practicum may lead to improvements, ID added.

Moreover, in the focus group conversation with IA and IB, they expressed awareness of a number of problems to do with understanding and motivation. IA said that:

In course evaluations a number of students and the number may vary [from year to year] who are very critical and […] often they are actually not critical towards the design or because they dislike the course. Instead, they think that: We do not have time to be dealing with academic mumbo jumbo like this; we need to practice teaching! […] This is one of the concerns and the other one, […] as course coordinator I inform the students during the term prior to the course about the responsibility that teachers may have for development and that to produce a degree project is to learn how to function in that way when they start working. I have said that this course is the jewel in the crown of sorts where they are supposed to merge everything.

In this statement, IA commented on problems and offered a solution – empowering and proactive pep talk to the students. The reason why students can react in a negative way regarding the course, she added, has to do with the various disciplines and what is required. Coming from a scientific background, “the students are uninformed about research methods in social sciences and they think that most of what they study, come across and what we introduce is not real research […] and they are simply lost in an unfamiliar research tradition”, IA argued.

Finding solutions to challenges

Regarding the second coded category “finding solutions to challenges” in spite of a number of internal and external structural challenges, IC mentioned what follows.

We have tried to find solutions for handing in proposals earlier, but then there has been criticism levelled at us from other departments. They do not condone the fact the students are expected to hand in something for future courses, since it affects the particular course they are attending. Then, we have had to accept such a decision. So, instead we have tried to invite the students to participate in projects that we initiated ourselves. We have detected a limited interest for this, since they prefer to focus on their own projects.

In a similar fashion, ID mentioned that preparations can be arranged in a couple of stages. First, the students are expected to study research articles and empirical studies in relevant courses both for seminars and for examinations. Aspects such as content, research questions, implications of the findings and how to apply such knowledge to the teaching profession are relevant. Secondly, time is allocated for reflection and analysis of the students’ own teaching unit planning.

IA stated that two of her colleagues had begun to introduce articles and research but that the results of this strategy were not yet known. Then, there were some issues that she had been meaning to raise with a colleague in the department about how to find solutions to the criticism levelled at the degree project, but they had not yet been discussed.

Proposing improvements, guidelines and structures

The third category about improvements, local guidelines and structures resulted in twenty references in NVivo. IC mentioned how the coordinators had taken the initiative to list ongoing projects led by colleagues at her department, but that merely “two to three students in two years’ time” had shown interest in participating in such a project.

IB underlined a need for preparations in teacher education. He advocated the introduction of meta-understanding for the students and while reading about research on teaching and learning in other courses, they should be encouraged to analyse the methods employed. When one of the interviewers posed a question about whether a crash course in education research should be considered, he replied that it would be a challenge in the BTEP that “spans one and a half years, in the worst-case scenario soon one year” and that caters for career change work in terms of the required content. Instead, IB argued that aspects of how research in educational sciences can be conducted should be embedded explicitly throughout the programme. IA agreed and added that it is of utmost importance that students can appreciate findings from educational studies in their future profession.

In addition, all of the instructors in this study preferred having clear guidelines for the degree project and adjusting them, when necessary, but they were also keen listeners who showed empathy with students’ perspectives, including their frustrations, in course evaluations. Moreover, they were aware of the limitations to do with time and resources primarily and tried to solve local issues in these courses for each department over time. For example, IC mentioned challenges for the department with regard to having access to suitable supervisors. In an atmosphere of both frustration and irony, she said that: “I usually tell the students that the best thing for them is to have a knowledgeable supervisor and an ignorant examiner and that the worst thing would be the other way around.” Still, she added that students in the BTEP are more advanced in terms of academic writing than the pre-service teachers for primary school and that they therefore often can manage producing and defending their projects in an adequate manner.

Everyone agreed that there were advantages to be gained from various kinds of deliberation and decision-making across department boundaries, even if there were slight differences between the ways in which the courses were conducted at the three departments. One concrete, progressive proposal for course design can be to let courses interact with regard to instruction and learning. ID presented plans underway – in cooperation with other departments in the programme – for establishing new guidelines. The idea is to let two courses, a teaching practice and degree project, run parallel and part-time during one entire term. This is an experiment, he added, to see how such an extended arrangement would function when providing “more time for the students to think”. That is to say, the students can get the opportunity to meet their supervisors already in September during the autumn term and can collect data in a structured way during teaching practice. Three departments and a couple of training schools would be involved in this experiment.

Pre-service teachers’ comments on literacy and solutions to challenges

Turning to responses from the pre-service teachers, what follows emerged in the interview conversations. In the two sections below, perceived literacy challenges and comments on academic writing as well as proposals to solve problems will be addressed.

Being aware of challenges regarding academic literacy

Responses from the pre-service teachers revealed either confidence in, or frustration at, their academic literacy and knowledge base for producing/defending a degree project in education or at the format or the discipline-specific conventions for educational sciences. There were eleven references to the students’ experienced challenges and seven comments on academic writing.

In a focus-group conversation, PE mentioned that she had learned some aspects of how researchers can conduct their studies in educational sciences from previous courses. PG added that, even if it had been several years since she studied comparative literature, she could outline differences between literary studies that focus on written texts and projects in educational sciences that often rely on empirical findings. Contrasts between essays in the humanities and the social sciences, for instance, include the notion that details about “method” can be embedded in the reasoning and that the perspectives on academic writing and research in the two disciplines diverge, PG mentioned. Then, PF argued that he had acquired a good understanding of research design, but he raised a few challenges connected with various academic cultures in Europe. He compared conditions in France and in Sweden. In the former country the expectations involved a more extensive study and more exacting standards than in the latter one, he stated. For example, in a French master thesis, the research problem must be formulated in an optimal way with regard to how a study can contribute to previous research. That is to say, the academic discourse must include an evaluation of previous research and show how the study in question can function as a relevant contribution to a particular research field. PF concluded that authors in France are expected to proceed from a problem that has eluded other researchers and that all of the above-mentioned aspects would be necessary for a French graduate-level thesis.

PG contended that she knew “almost nothing” about how to conduct research in educational sciences. She had reread her assignments in comparative literature and found:

So, in principle [the degree project] is completely different – this is entirely new to me and at the beginning of this course I am left to my own devices and am trying to learn about methods in the social sciences and am feeling rather unprepared, actually. I do not know all that much about qualitative analysis and very little about real methods and absolutely nothing about quantitative analysis. So, this is the beginning of the learning curve, but you will need to face this task in the same way as you do for other tasks. That is to say [PE and PF laugh] you try to comprehend what to do and what you need to know to manage and, well, it will just have to work.

Similarly, in an individual conversation, PH complained about feeling completely unprepared for the assignment and stressed the fact that he had not written an academic essay since the 1990s in comparative literature. In contrast to the three fellow students mentioned above, however, PH finished the degree project recently and was in the position to comment on the process with regard to planning, conducting and defending a project as well as on the supervision and examination. Some of his concerns are paraphrased by the authors below:

1. degree projects in education/social sciences appear to rely heavily on formatting, rather than on content

2. in undergraduate courses, it is hardly reasonable to expect advanced literacy in research methods from students

3. none of the courses in teacher education provided basic instructions for producing a degree project and PH had to learn about the format [APA] on his own.

In a further interview conversation, PI was surprised to find that his instructors encouraged the students indirectly and from the start to opt for interviews as a suitable method and that he was unfamiliar with such approaches. He contended that there was no introduction to the expected research design, which affected his motivation. Then, PI received rather severe critique during his defence, he said, and added that aspects regarding design surely should have been raised during the supervision.

In the case of a focus-group conversation with PJ and PK, the reactions against the course were even more scathing. PJ experienced a gap between the format that he had learned as a civil engineer and the one that he was expected to be able to manage in teacher education. He had consulted with fellow students in economics, technology and engineering who were also in their 40s or 50s and contended that there were immense differences between what is expected in those areas compared with expectations in social sciences – the focus was set on the end result rather on formalities in engineering, PJ stressed. PK agreed and added that the formatting is considered less important in the former areas than in educational sciences.

Proposals for solving problems

There are similarities between the students’ comments with regard to the third category “proposals for solutions to problems”. Having the opportunity to prepare for the degree project during practicum would probably be useful, they responded. To be able to “reflect on experiences during teaching practice could help” and why not make good use of the observations in this regard, PE said. PF also expressed a positive attitude to such initial preparations. PG agreed and added that this is exactly what she had already done of her own accord. When observing her mentor in school, she discovered a useful theme for her project. Such preparations should nevertheless not be all that extensive, PG warned, since teaching practice may turn out to be rather stressful for the students. Still, all three students detected opportunities for embedding some preparation in compulsory observations during practicum.

In the individual conversation with PH, he asked for some oral and written introduction to the task at hand during the second term, which precedes the degree project. If such a solution could be implemented, then students would probably not have to face the examiners’ lengthy lists of areas that need improvement during the defence, he underlined. Turning to PI, when asked a question about whether it would be an improvement to have the opportunity and time to reflect on themes during teaching practice, he thought this would be a good idea. An alternative solution would be to introduce a crash course, he proposed.

Similarly, PJ and PK responded that they would welcome a crash course in research methods for educational sciences. They also asked for more streamlined supervision and PJ told a story about his plans during teaching practice. He received an initial blessing from the course coordinator and some assistance regarding references, but then he realized that he was not allowed to collect data until his plans were condoned by a supervisor. There was a lack of guidance that made preparations impossible, he thought, and he had to wait until he was invited to supervision. Hence, PJ experienced a Catch-22 situation that generated disappointment. Both PJ and PK stressed the need for more flexible structures with regard to preparations and the collection of data.

All of the seven students in this cohort saw potential areas for development in the degree project at three departments. These areas primarily cover the preparatory phases. Having the opportunity to receive guidance and to be expected to propose a theme at an early stage in the programme, is one concrete proposal from the students. Hence, they found that there would be room for some initiatives to establish a proactive interaction between supervisors on campus and student teachers during practicum.

Findings from the regulatory framework for teacher education and for school

The Swedish Higher Education Act states that courses on the advanced level are expected to “further develop the ability of students to integrate and make autonomous use of their knowledge” (SFS 1992:1434, Ch. 1, Section 9). Similarly, policy documents for school rest on the above-mentioned democratic approaches (see SFS 2010:800, Ch. 3, §2). Lifelong learning, transparency, equivalence and mobility are also recommended for the implementation of the Bologna process (European Commission/EACEA/Eurydice, 2020, p. 39, 42, 80, 124 and 166). However, in a global report, the World Bank argued that “education systems are often poorly aligned with learning goals”, which may depend on the complexities of education systems both in terms of structure and interaction (2018, p. 170). In addition, Lewin and Lundahl underlined some aggravating circumstances that revolve around complex steering mechanisms and the implementation of school reforms in Sweden (SOU 2014:5; Lundahl, 2019).

Nevertheless, the regulations and recommendations in the Higher Education Act, the Higher Education Ordinance, the Ordinance on Supplementary Educational Programmes as well as in the Education Act and the curricula for school are clear. The Act underlines that higher education institutions are accountable for establishing “courses and study programmes based on scholarship [..,] on proven experience, and research […] as well as development work” (SFS 1992:1434, Ch. 1, Section 2). The Ordinance states that secondary teachers in Swedish schools must “demonstrate specialised knowledge of the theory of knowledge and qualitative and quantitative research methods as well as the relationship between the disciplinary foundation and proven experience and its significance for professional practice” (SFS 1993:100, Annex 2, Outcomes, para. 4, n.p.). A similar requirement regarding fundamental values is mentioned in the Education Act for school which states that the education must be based on scientific knowledge and proven experience, but in the curriculum for upper secondary school this requirement has turned into a recommendation “the education should be based on scientific grounds and proven experience” (SFS 2010:800, Ch. 1, §5; Skolverket, 2011; p. 4. See also Skolverket, 2014, p. 4 and 7). The consensus in the Swedish target-oriented approach to teaching and learning is that autonomy, reflection and deliberation will lead to optimal local decisions (Skolverket, 2014, p. 7 and passim). However, it is common knowledge that research may not be part of day-to-day practice in schools for various reasons (Skolverket, 2014, p. 10).

For the Higher Education Diploma, there are two alternatives, either Master of Arts or Master of Science for lower secondary and upper secondary school in the first and second cycles. Central knowledge areas listed for future teachers in the Ordinance revolve around the school system; curriculum theory, teaching and learning; special needs education; theory of knowledge and research methodology; leadership; assessment and grading; and evaluation and development (SFS 1993:100, Annex 2, n.p.). Then, a number of expected outcomes are listed for the core areas: knowledge and understanding; competence and skills; judgement and approach, such as this example for an MA in upper secondary education:

• the student shall demonstrate the subject knowledge required for professional practice, including both broad knowledge of the field and a considerable degree of specialised knowledge in certain areas of the field as well as specialised insight into current research and development work. (SFS 1993:100, Annex 2, Outcomes, para. 2, n.p.)

In other words, there are extensive literacy requirements regarding the specific school subjects in question as well as research and school development. Furthermore, the Ordinance on Supplementary Educational Programmes lists requirements regarding a degree project of 15 credits for each of the subjects studied during the programme (SFS 2011:686, §9b, 10, 11, 12). There are no further national guidelines for how such projects should be designed. Hence, deliberation and liberal democratic approaches are expected to prevail for how they should be planned and conducted by the students under the guidance of appointed supervisors and assessed by the examiners. That is to say, on the national level, there are no academic restrictions for the formalities with regard to format, style and structure. Instead, local curricula, syllabi and grading criteria are developed by the university or college in question.

Conclusions

In this exploration, teacher cognition as well as constructivist grounded theory facilitated the process of letting the participants reflect on their experiences, beliefs and perceptions as well as to “define and understand what is happening in their worlds” (Borg, 2015; Charmaz, 2014, p. 85). In light of national requirements, the conversations revealed threats against established demands for test validity, task alignment and pre-service teachers’ self-determined learning.

First, instructors argued that they faced challenges in view of pre-service teachers’ inadequate academic literacy with regard to educational research and, as a result, their negative attitudes or low motivation to conduct such studies. The test construction for degree projects per se may not be an acute issue for the students in the BTEP, but the above-mentioned inadequacy will affect construct and consequential validity in a negative way. In other words, if these students lack knowledge and understanding of the final degree project, then the inferences from this test, that is to say, the construct validity, will lead to limited values in the evaluation. In addition, inferences from the consequential validity in teacher education may also prove to be problematic, if the students develop negative attitudes to conducting, or even taking a future interest in, research in educational sciences.

Secondly, the study raised relevant issues about discrepancies between national/local expectations and students’ previous studies as well as current knowledge base. The latter aspect appears to be the main impediment to the students’ “self-efficacy” and learner “agency”, which Bandura underlined in another context as necessary for taking responsibility for one’s own learning (2009, pp. 16–17). Hence, the findings of the present study raise questions about to what extent students in the BTEP can take initiatives for furthering their own literacy with regard to the educational sciences. If they lack the knowledge base, then they can hardly be expected to develop self-determination and a passion for pursuing research on teaching, assessing and learning.

Challenges, solutions and improvements

There are a number of validity concerns involved for the stakeholders regarding pre-service teachers’ literacy in educational sciences. In terms of pre- and in-service cognition, the level of frustration for the instructors vis-à-vis the students in this study may differ in that they emerge from various perspectives, but in general, they amount to a rather substantial dilemma. The degree project in the final stage of teacher education for these students is supposed to be a gateway and a bridge to the teaching profession. Future teachers are expected to be ambassadors for a scientific approach to teaching and learning, but how can this expectation come to fruition?

The instructor ID mentioned that the majority of the students come from various disciplines and may not be well versed in educational sciences. He added that the degree project is supposed to provide opportunities for learning, to reflect on and “critique educational sciences”. However, to succeed in this endeavour students need to develop the required skills, to believe in and understand the relevance of including planning for, and conducting teaching based on knowledge of science and experience, as the policy documents for higher education and for school stipulate. As mentioned previously, qualified secondary teachers must “demonstrate specialised knowledge of the theory of knowledge and qualitative and quantitative research methods as well as the relationship between the disciplinary foundation and proven experience and its significance for professional practice” (SFS 1993:100, Annex 2, Outcomes, para. 4, n.p. See also SFS 2010:800, Chapter 1, §5; Skolverket, 2011; Skolverket, 2014, p. 4 and p. 7). Hence, there are strict expectations with regard to basing teaching on research and validated experience.

The question is how such demands may be met if students in the BTEP neither have the advanced literacy with regard to educational sciences, nor the necessary motivation or understanding of its significance. IA and IB revealed how results from course evaluations over the years invariably include sceptical comments on educational sciences, or – what some students call – “academic mumbo jumbo”. This expression raises another red validity flag and leaves the instructors perplexed. IA added that she has tried her best as a course coordinator to inspire students during the term prior to degree project. Developing insights into how expected scientific approaches can be implemented by school teachers and the importance of summarising what they have learned in the programme are examples of what she would say to invigorate the students.

However, with a number of departments involved in teacher education, there are challenges for teacher educators when trying to improve the conditions for degree projects. A cognitive dilemma emerged from the instructors’ perspective in that they had to contend both with the tight circumstances for the courses in terms of time and space as well as with pre-service teachers’ complaints and/or needs. For example, the students would have wanted opportunities for preparing such courses in deliberation with supervisors at an early stage as well as with a better understanding of what the assignment involves. Hence, one of the solutions to the problem can be to arrange possibilities, for example, during teaching practice to provide space for developing themes or ideas for the degree project.

On the one hand, current plans, such as ID’s proposal to let on-campus courses and teaching practice run parallel during a term, will most likely lead to substantial advantages both for supervisors on campus, students and mentors in school. There will be opportunities for student teachers to reflect on themes and methodologies under the guidance of tentative supervision on campus as well as for mentors to offer advice on potential aspects that need scientific scrutiny. On the other hand, challenges such as the ones presented by the pre-service teacher PG, who stated that she knew “almost nothing” about how to conduct research in educational sciences, and PI, who complained about feeling completely unprepared for the degree project in the BTEP, require attention.

Document analysis

Teacher education is accountable for offering “courses and study programmes based on scholarship [..,] on proven experience, and research […] as well as development work” (SFS 1992:1434, Chapter 1, Section 2). Acquiring knowledge and skills with regard to “qualitative and quantitative research methods as well as the relationship between the disciplinary foundation and proven experience and its significance for professional practice” (SFS 1993:100, Annex 2, Outcomes, para. 4, n.p.) are further requirements stipulated on the national level. For school, there are also responsibilities and expectations with regard to scientific approaches to teaching and learning in the Education Act (SFS 2010:800, Chapter 1, §5; Skolverket, 2011; p. 4. See also Skolverket, 2014, p. 4 and p. 7). For teacher education, this aspect includes extensive expectations both with regard to various aspects of valid instruction, such as what is taught and how the courses can be conducted for optimal long-term learning.

A report by the Research Council defined educational sciences as an interdisciplinary field that can include research from all of the faculties (Vetenskapsrådet, 2006, p. 3). Regarding scientific approaches and research in education, the Government underlined the importance of conducting research on teaching and learning at the various departments in teacher education and of considering research in disciplines such as pedagogy, psychology, political science, economy and philosophy in the instruction (Vetenskapsrådet, 2006, pp. 7–8; see also SOU 2018:19). Such concerns will probably provide both opportunities and challenges for instructors. For example, one key issue is how to deal with and build bridges between a plethora of approaches to doing research in various disciplines in light of the fact that the students are expected to produce a thesis or a report in educational sciences during the latter stages of their professional teacher training.

With this background in mind, addressing issues about validity in teacher education is necessary, that is to say, how courses can cover the variety of requirements stipulated for knowledge and skills about subjects as well as about research and scientific approaches. What does the evaluation say about degree projects in terms of validity? To what extent can such courses and the way in which they are planned, conducted, assessed and examined provide the knowledge, skills and motivation that are required for developing a scientific approach to teaching in school? Is there a progression between the various courses in the programme to enable such a development for the students?

In a decentralised, egalitarian education system, such as the Swedish one, courses in pedagogy that span three terms at various departments and at practicum in schools will inevitably lead to a number of challenges both for the academic staff and the students. First, the requirements and guidelines on the national level will have to be adjusted for needs on the local levels. Secondly, BTEP adult students may not even be familiar with educational sciences but may be trained in other scientific traditions, such as engineering science, economics and finance, comparative literature or linguistics. Thirdly, the brief and disparate courses in the programme may render it difficult to collaborate. This concerns conditions for course coordinators and instructors for on-campus courses as well as for mentors and students in teaching practice.

Implications for further research

One of the most surprising findings of this study is that the instructors and the pre-service teachers’ perspectives both displayed similar concerns and were rather disparate in an unintentional way, which prompts a need for convergence. It is clear, for instance, that construct- and consequential validity issues need to be addressed in view of PJ and PK’s harsh criticism of circumstances for producing a degree project. Their attitudes toward educational sciences and, hence the scientific approaches to teaching and learning, raise a number of questions. PG’s calm, or even neutral, statement about not having a clue about how to conduct a study in educational sciences is also a cause for concern. If these notions and experiences are shared by fellow students in the BTEP, then this circumstance needs to be examined further.

After all, the process of writing and defending a degree project should provide opportunities for students to develop and deepen their knowledge about aspects which concern a profession that will form a central part of their future lives. Perhaps crash courses in educational sciences and/or alternative designs of courses can lead to increased levels of validity in the BTEP. There may be several solutions to the problems detected in the present study, but it is obvious that something needs to be done to improve the situation.

Consent from participants and ethical considerations

Informed consent with regard to participation and publication of the interview data have been obtained from the participants.

In line with the ethical codex for qualitative research and the research ethics for qualitative research in the Swedish context, participants were informed about the aim of the study and that they could choose to opt out of the study at any time (Vetenskapsrådet, 2017). For integrity, participants’ names were replaced with pseudonyms.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Notes on contributors

Birgitta Fröjdendahl

Dr Birgitta Fröjdendahl, Senior Lecturer at the Department of Teaching and Learning at Stockholm University, is doing qualitative research mainly on formative and summative assessment in teacher education.

Ylva Sandberg

Ylva Sandberg, Phil. Lic., is a lecturer in language education at the Department of Teaching and Learning, Stockholm University. Her research interests include language learning and teaching, multilingualism, multimodality, teacher cognition and professional development.