Secondary teachers’ views about teaching and assessing the diversity of scientific methods in practical science

ABSTRACT

A significant issue in the teaching, learning and summative assessment of practical science is that the underpinning model of scientific methods is based on a fairly simplistic and linear account that does not represent how scientists actually do practical work. In this paper, we examine how science teachers in England and Wales view scientific methods included in teaching and summative assessment of practical science. Following on a survey of 51 secondary teachers’ views, we focus on the outcomes of a professional development programme where science teachers were introduced to a framework called Brandon’s Matrix that presents a diversity of scientific methods in the context of Project Calibrate. Through in-depth qualitative analysis, we investigated how three teachers engaged with this framework having taught it to their students. The findings illustrate that teachers conceptualise practical science as a mainly hands-on activity, while they express the need for the inclusion of more minds-on skills in the high-stakes assessment of practical science. In addition, the findings show that teachers are quite receptive to adapting new frameworks, such as Brandon’s Matrix, for their teaching. Although the teachers did not see a significant departure of the assessment questions from the regular high-stakes tests taken at age 16, they did recognise that there was a valuable element of thinking skills embedded in them.

KEYWORDS:

• Practical science
• summative assessment
• teacher views
• scientific methods

Introduction

Practical science has been an essential part of science education for a long time (Layton 1973). For example, England has a long tradition of integrating practical science in science classrooms to facilitate students’ understanding of scientific practices and methods (e.g. Abrahams and Millar 2008). Despite the fact that it is an integral part of the national science curriculum, the term ‘practical science’ has been defined in various ways, rendering the discussions about the topic problematic (Dillon 2008). For example, ‘practical activities’ (Gatsby 2017), ‘independent enquiry’ (Pearson, 2017a) and ‘experimental work’ (Pearson, 2017b) are terms that are often used interchangeably to refer to practical work that takes place in science lessons.

A significant issue in the teaching, learning and assessment of practical science is that the underpinning model of scientific methods is based on a fairly simplistic and linear account that does not represent how scientists actually do practical work (Lunetta, Hofstein, and Clough 2007). The issue of the distorted representation of scientific methods has been a topic of discussion among scientists for decades. For instance, Peter Medawar (1978) argued that the scientific paper (by analogy to the scientific method) ‘is a fraud as it gives a totally misleading narrative of the processes of thought that go into the making of scientific discoveries’ (p.43). Woodcock (2013) discussed such ‘myths’ of the scientific method and highlighted a wide variation in the content of scientific method representations that range from 2–3 steps to 11 steps. The scientific method is depicted as testing of a hypothesis through a careful consideration of independent and dependent variables through a step-wise and linear process. Blachowicz (2009) illustrated that textbooks tend to present the scientific method as a stepwise process in a simple empiricist view of science.

In this paper, we examine how secondary science teachers in England view scientific methods included in teaching and summative assessment of practical science in the context of a 3-year funded project called Project Calibrate (Erduran 2021; Erduran, Ioannidou & Baird, 2021; Ioannidou & Erduran, 2021) In order to answer our research questions, an online survey was designed and distributed to secondary science teachers in England. Following on a survey of teachers’ views (n = 51), we focus on the outcomes of a professional development programme where three science teachers were introduced to a framework called Brandon’s Matrix (Brandon 1994) that aims to go beyond a simplistic account to be inclusive of the diversity of methods in science. In other words, the study aims to address the limitations of the traditional stepwise and linear process of hypothesis testing based on manipulation of variables by engaging teachers in explorations of a framework that is inclusive of a diversity of scientific methods. The diversity of scientific methods is represented in a framework proposed by Brandon (1994), which is inclusive of non-manipulative approaches to methods in science, such as parameter measurements in observations. We investigated through in-depth qualitative analyses how three teachers engaged with an adaptation of Brandon’s framework in the format of a matrix in order to introduce it to their students during practical investigations. In doing so, we investigated their views of summative assessments designed to capture the diversity of scientific methods. The ultimate aim of the paper is to contribute to understanding how science teachers’ views and experiences of new perspectives on practical science can be used to improve teaching and summative assessment of practical science.

Literature review

Defining practical science

Practical science has been investigated from a range of perspectives (Hofstein 1988; Lock 1988). Wellington (1998) categorised the aims of practical science into three main arguments: cognitive arguments, suggesting that practical science can facilitate students’ understanding of laws and theories, affective arguments, describing how practical work can inspire and enthuse students and lastly, skills arguments, referring to the dexterity skills, as well as the transferable skills, such as observation and prediction. Subsequently, Hofstein and Lunetta (2004) proposed five themes to summarise the aims of practical science: understanding of scientific concepts; interest and motivation; scientific practical skills and problem-solving abilities; scientific habits of mind; and understanding of the nature of science.

The goals are mirrored to a large extent in England’s national curriculum (Department for Education 2014), in which practical science falls within ‘Working Scientifically’ (WS). Learning and assessment objectives under this heading include: (a) development of scientific thinking; (b) experimental skills and strategies; (c) analysis and evaluation; and (d) scientific vocabulary, quantities, units, symbols and nomenclature. In other parts of the world, the teaching and learning of practical science has also been promoted, for instance in the case of the Next Generation Science Standards in the USA (NGSS Lead States 2013), the Korean Science Education Standards in South Korea [Ministry of Education (MOE), Ministry of Science and ICT (MSICT), & Korea Foundation for the Advancement of Science and Creativity (KOFAC) 2019] and the Curriculum Guidelines of 12-Year Basic Education: Science [Ministry of Education (MOE) 2018] in Taiwan although the particular specifications about practical work can vary (Erduran et al. 2020^b).

While practical science as a learning objective has received attention by researchers and policymakers, a number of studies attempted to shed some light on teachers’ understanding of the goals of practical work (e.g. Abrahams and Saglam 2010; Kerr 1964; Pekmez, Johnson, and Gott 2005). Kerr (1964) identified 10 aims of practical work as perceived by science teachers, among which accurate observation and careful recording were perceived as the most important aims for practical science. Studies conducted in a last decade indicate that teachers’ views regarding practical science have generally remained in agreement with the aims described by Kerr, Boulind, and Rolls (1963) almost four decades ago (Johnstone and Al-Shuaili 2001).

For instance, drawing on Kerr’s themes, Dillon (2008) summarised the main purposes for practical work in four aims: to promote accurate observation and precise description, to provoke and maintain interest, to stimulate a logical method of reasoning and to present phenomena in more authentic contexts for the students. In an empirical study investigating the views of 393 secondary science teachers on practical science, Abrahams and Saglam (2010) concluded that teachers teaching at the lower secondary level believed that the main goal of practical science is to ‘arouse and maintain interest in the subject’. However, the same study illustrated that teachers’ views at upper secondary level were differentiated since teachers indicated that practical science is used in classrooms ‘to prepare pupils for assessed practical work’.

Teaching practical science

Despite teachers’ positive attitudes towards the aims and impact of practical science on student learning, relevant studies have shown that secondary science teachers’ beliefs about practical science are often not reflected in their teaching (e.g. Abrahams and Millar 2008). Although teachers recognise that practical work can facilitate students’ understanding of the nature of science and scientific thinking, teaching practical science has often been described as inflexible, monotonous and predictable (Erduran and Dagher 2014). For instance, Abrahams and Millar (2008) concluded that when conducting practical investigations secondary science teachers tend to focus on the manipulation of physical objects and equipment, rather than cognitive skills, such as the link between observations to conceptual ideas or understanding of scientific methods and practices.

Other relevant studies have raised questions about the extent to which this recipe-like teaching style, which typically focuses on hands-on skills, can facilitate students’ understanding of ‘how science works’ since students often follow instructions mindlessly without reflecting on the steps and the purpose of the practical (Abrahams 2009; Abrahams and Millar 2008). According to previous studies, the ‘cookbook approach’ to practical science is likely to promote the idea of a single, valid ‘scientific method’ as a linear straightforward process (e.g. Hodson 1996), which often begins with the formulation of hypothesis and concludes with the confirmation or rejection of the stated hypothesis. This depiction of scientific inquiry has been criticised by several researchers for inaccurately portraying scientific practices as a linear, step-wise process, while undermining the breath of scientific methods used by scientists (Erduran and Dagher 2014; Wivagg and Allchin 2002).

The impact of high-stakes summative assessments on teaching practical science

It is commonly accepted that teaching is often driven by high-stakes assessment (Childs and Baird 2020). Thus, one can argue that the inconsistency between teachers’ views of practical science and the teaching strategies that teachers follow can be explained by the accountability pressures that they often face with regard to student outcomes in high-stakes examinations. These pressures have been particularly evident in England, following the introduction of a new science national curriculum in 2016 and the subsequent changes in the high-stakes examination of practical science at age 16 that took effect in 2019 (Ofqual 2015).

This reform proposed the summative assessment of practical science through written examinations that target specific ‘required practicals’ that teachers are asked to perform in their classrooms (Childs and Baird 2020). In some cases, teachers were provided with a ‘Required practical handbook’, which included instructions and guidelines for teachers, as well as student methods sheets with step-by-step instructions on how to perform an experiment. It has been argued that the introduction of such methods sheets would endorse a teacher-centred cookbook approach to teaching practical science by promoting mindless recall and passive acceptance of information and ideas (Childs and Baird 2020).

Apart from its format, the content of the summative assessments for practical science can have an impact on the way in which practical science is currently taught. In a recent study, Cullinane, Erduran, and Wooding (2019) showed that there is an imbalance in the way in which scientific methods are represented in chemistry GCSE papers in England. More explicitly, the study reported that although hypothesis testing and manupulation of variables tend to be emphasised in teaching, the assessments mainly focused on non-manipulative parameter measurement.

A new framework for teaching and assessing practical science

In order to promote more creative and inquisitive approaches to teaching practical science, Abrahams (2011) suggested a more minds-on approach that would encourage students’ meta-cognitive skills that take place while doing science. By following a minds-on approach teachers can provide opportunities for students to reflect on the decision-making processes that occur during scientific investigations (Ioannidou and Erduran, 2021).

Departing from the abovementioned suggestions, the authors led a research project that aimed to design new summative assessments for practical science in order to provide a more balanced representation of scientific methods in high-stakes assessments. The new assessments were designed using a conceptual framework, named Brandon’s Matrix (Brandon 1994). The matrix illustrates the variety of scientific methods by grouping them into four categories based on whether they involve variable manipulation and/or hypothesis testing Table 1.

Table 1. Diversity of scientific methods (reproduced from Erduran and Dagher, 2014).

	Experiment/Observation
	Manipulate	Not Manipulate
Test Hypothesis	Manipulative Hypothesis Test	Non-Manipulative Hypothesis Test
Measure Parameter	Manipulative Description or Measure	Non-Manipulative Description or Measure

The framework addresses two key questions about whether or not an observation or experiment involves (a) hypothesis testing or parameter measurement, and (b) manipulation of variables. The outcome is a matrix that considers these various combinations in order to clarify what methods are followed in science. This categorisation highlights that the scientific method is not a single linear process, but rather scientists use a breath of scientific methods depending on the research question, resources and expertise (Brandon 1994). Erduran et al. (2020^a) have used the following contemporary example from the Covid-19 pandemic to illustrate the different categories of Brandon’s Matrix:

Consider the current investigations around Covid-19 infections. Some data are collected around how the virus might be influencing a patience’s breathing over a length of time. Such observation is simply based on the recording of parameters where there is no manipulation of variables in the sense of an experimental design. Likewise, sometimes these data might be subjected to hypothesis testing about correlation between incubation period and extent of lung disease but without having been part of an experiment, resulting in some non-manipulative hypothesis testing. Eventually scientists will have carried out some randomised control trials where a drug could be treated as a variable in interventions where there are also control groups to test the placebo effect. The important point is that all these different approaches are essential to the conduct of science and there is no one single method but rather a diversity of scientific methods.

Brandon’s Matrix illustrates how different methods are needed in science to reach conclusions that help explain phenomena. It also illustrates that there is no single method but rather a range of methods that contribute to the explanatory conclusions in science. We should note that although practical science is inclusive of Brandon’s Matrix categories, it is also a broader concept than simply scientific methods. In other words, while Brandon’s Matrix emphasises scientific methods and, as such, is directly linked to practical science, practical science includes much more than just scientific methods. For instance, practical science is inclusive of other aspects such as equipment, apparatus and other resources.

Brandon’s Matrix was used as a conceptual framework for the development of new summative assessment questions that would include a wider spectrum of scientific methods (Erduran, Ioannidou & Baird, 2021). For this purpose, a group of trained examiners developed six sets of assessments for GSCE level (osmosis, ecology, electromagnetic spectrum, circuits, chromatography, mixtures and distillation) (Erduran 2021; Erduran & Wooding 2021).

Aim of the study

Although teachers play a key role in preparing their students for summative assessment, there is currently limited literature in England on teachers’ views on how practical science should be summatively assessed (Abrahams, Reiss, and Sharpe 2013). This empirical study, thus, investigates teachers’ views of practical science and the way in which it is currently assessed in England and Wales. This would provide us with some insight on how teachers perceive the implementation of a recent reform in the summative assessment of practical science. Moreover, it explores their views on a new theoretical framework, Brandon’s Matrix, and its pedagogical affordances towards a more mindful approach to teaching practical science.

Research questions

1. What are teachers’ views of practical science and the way in which it is currently assessed at high-stakes assessments in England and Wales?

2. How do science teachers perceive the use of Brandon’s Matrix as a pedagogical tool for teaching and assessing practical science?

Methodology

In order to answer the research questions, a multilevel use of approaches was followed and ‘different types of methods at different levels of data aggregation’ were applied (Tashakkori, Teddlie, and Teddlie 1998 18). The research design consisted of a baseline study (Study 1) and an in-depth interview study (Study 2). The purpose of Study 1 was to explore science teachers’ general views of practical science and its assessment at GCSE level. The participants of this study were secondary science teachers from across England and Wales. Subsequently, to address the second research question, three teachers were interviewed individually. They had participated in a workshop organised in the context of Project Calibrate (https://projectcalibrate.web.ox.ac.uk) and were able to use assessments and Brandon’s Matrix in their teaching. These teachers used the resources that were presented in the workshop to plan and teach their practical science lessons. Taken together, the two studies provide an insight on how teachers approach practical science and its assessment. The first study provides an overview from a sample of teachers while the second study illustrates in more depth how teachers who have taught practical science having participated in professional development used a robust framework underpinning scientific methods in practical science Figure 1.

Figure 1. Overview of studies and data collection.

In order to ensure that appropriate ethical standards were upheld, the methods applied during the data collection were approved by an ethics committee. Furthermore, all the participants completed an informed consent stating the ways in which confidentiality and anonymity would be maintained throughout the research project.

Study 1: Teachers’ views of practical science and its assessment

The first study aimed to investigate science teachers’ views on practical science. For this purpose, a number of science teachers teaching in England and Wales were invited via emails and social media to take part in an online survey. Thus, this study used nonprobability voluntary response as a sampling method (Stern et al. 2017). The sample consisted of overall 51 science teachers, whose average teaching experience was 8.53 years (SD = 6.71) and ranged between 0 and 25 years. Given that teaching experience could be one of the factors influencing teacher responses, it can be argued that the sample, although relatively small, could demonstrate a certain degree of representativeness. More than half of the sample (29/51) taught combined subjects (all three science subjects or a combination of two) while 9 taught only chemistry, 6 only physics and 7 only biology. Teaching more than one subjects was a desirable characteristic in this study, as participants would be more likely to demonstrate a more well-rounded understanding of how practical science is currently being taught and assessed at upper secondary science classrooms across different subjects.

The survey questions were developed by the research team and the content validity of the survey was checked by a panel of experts (Grant and Davis 1997) specialised in science education and assessment. More explicitly, the panel of experts, which consisted of science teacher educators, provided feedback on the item style and the comprehensiveness of the survey. After the validation phase, five questions were included based on two dimensions: (a) teachers’ views on teaching practical science and (b) teachers’ views on the assessment of practical science. Since the aim of Study 1 was to explore secondary science teachers’ views in light of themes discussed in current and previous relevant literature, the majority of the questions that were included were open-ended. The survey questions and the corresponding dimensions are presented in Table 2.

Table 2. Survey questions for Study 1.

Dimension	Survey question
A. Teaching of practical science	1. What is your definition of practical science? Can you give some examples? 2. Can you tell us how you normally plan and teach a practical science lesson at GCSE level?
B. Assessment of practical science	3. Has the move to written summative assessments influenced your teaching? If so, how? 4. What resources do you commonly use for the summative assessment of your students' learning in practical science lessons? 5. What practical science skills do you think should be assessed summatively at GCSE? Why?

As illustrated in Table 2, the survey included questions targeting teachers’ definitions of practical science and the way in which they usually teach practical science. These questions would enable us to investigate whether science teachers perceive practical science as a ‘hands-on’ activity and whether they follow teacher-centred ‘cookbook’ approaches to teach practical science, as described by previous research (Abrahams 2009; Abrahams and Millar 2008). In order to investigate the ‘washback effects’ of high-stakes assessments on practical science teaching (e.g. Childs and Baird 2020), teachers were asked to indicate the resources that they use to teach practical science. The aim of this question was to examine the extent to which past exam papers are used to teach practical science compared to other resources. In addition, participants were asked to reflect on whether their teaching approaches have changed due to the new format of summative assessment for practical science. Finally, the last question aimed to explore secondary science teachers’ views on the need for adopting a more ‘minds-on’ approach to teaching and assessing practical science as expressed in previous research (e.g. Abrahams, 2011), in contrast to the hands-on skills (e.g. manipulation of equipment) currently promoted by the high-stakes assessments.

Teachers’ answers were examined through thematic analysis (Braun and Clarke 2006) following inductive as well as deductive coding. The answers were firstly deductively coded based on three coding categories: (a) definitions of practical science, (b) teaching approaches for practical science, and (c) teachers’ views on the summative assessment of practical science. In the second coding phase, the transcripts were coded using axial coding (Strauss and Corbin 1990) in order to search for further sub-themes emerging from the data. The emerging sub-themes were discussed, refined and clustered until consensus was reached. All teacher answers were coded again by two independent raters, until their inter-rater agreement was higher than 95%.

Study 2: Brandon’s Matrix as a professional development and pedagogical tool

After conducting the online national survey, the research team organised a workshop for science teachers. The first goal of the workshop was to present Brandon’s Matrix as a new framework for teaching scientific methods within practical science lessons. The second aim was the introduction to a set of new assessment questions designed by using Brandon’s Matrix as a guiding framework. Examples for each quadrant were introduced and were drawn by all three science subjects currently assessed at high-stakes assessments in England and Wales (biology, chemistry and physics). The professional development resources (Wooding, Cullinane and Erduran 2020) can be downloaded at the Project Calibrate website.

Three science teachers who participated in the workshop expressed interest in presenting Brandon’s matrix and the designed assessments to their students. The engaged teachers selected one of the topics included in the designed assessments in order to teach it in their classroom. After approximately three months, the teachers were contacted again for a semi-structured interview that lasted 10 to 20 minutes. The teachers who implemented the framework as a teaching tool, were three science teachers teaching in state-funded schools located in different regions in England and Wales. Teacher 1 was the head of science in his school, while he had experience in educational research, as he completed a graduate course after his teacher training. Teacher 1 and Teacher 2 taught in all-girls schools, while Teacher 3 taught in a mixed-gender school. More details on participants’ demographic information are presented in Table 3.

Table 3. Participants’ demographic information for Study 2.

Teacher	Subject specialism	Teaching experience	Gender
Teacher 1	Biology	3 years	Male
Teacher 2	Physics	1 year	Female
Teacher 3	Biology	4 years	Female

The interview aimed to investigate the ways in which teachers adapted Brandon’s Matrix and used it as a teaching tool in their classrooms in order to teach the topics that were presented in the designed assessments. Moreover, the goal of the interview was to examine teachers’ views on the designed assessments and the perceived student performance with regard to the assessments. For this reason, researchers encouraged teachers to describe the procedures that were applied in their classroom in order to capture the reasoning processes that took place during the adaptation of the matrix for their own teaching. The interview questions were developed based on five dimensions: (a) teachers’ overall views on Brandon’s Matrix, (b) overall views on assessments, (c) lesson preparation, (d) teaching strategies, and (e) perceptions of student performance. Similar to Study 1, the interview data were analysed using thematic analysis (Braun and Clarke 2006).

Findings and Discussion

Study 1: Teachers’ views on practical science and its assessment

The results of the study showed that almost half of the participants defined practical science as a combination of hands-on and minds-on skills (n=26), while some participants perceived it as an activity that involves only hands-on skills (n=20) such as the use of equipment. The rest did not provide relevant answers. This finding illustrates that although curriculum recommendations (e.g. ‘Working Scientifically’) ask for a combination of minds-on (e.g. scientific thinking) and hands-on (experimental skills) skills, many teachers still tend to define practical science as a solely hands-on activity.

Teachers who referred to hands-on only skills tended to refer to knowledge gaining by ‘carrying out experiments’, or ‘activities involving equipment’, while some teachers used the specific term ‘hands-on’, for instance: ‘Hands-on learning that requires students to handle and use equipment. On the contrary, teachers who referred to both hands-on and minds-on skills, described practical science as an activity that combines the use of equipment as well as problem-solving, hypothesis testing and engaging in scientific methods in order to understand how science works:

‘Solving a problem using practical work to make observations or collect results. These are then used to answer the aim of the practical work. Or, the purpose could be to learn how to use equipment safely and properly. It is really important that students know and understand why they are doing the practical before they start.’ (science teacher)

Teachers’ responses to the question ‘How do you normally plan and teach practical science?’ showed that while the majority of the teachers (n=48) described the teaching approaches applied during the lesson, fewer teachers (n=8) referred to the planning that takes place before the lesson. This may be indicative of the reduced time that teachers spend in reflecting on their lesson planning, as a result of the recent introduction of the ‘required practicals’ (Childs and Baird 2020).

In terms of the teaching approaches, the majority of the participants, (33/48) described teacher-led approaches to teaching practical science, 16 of which mentioned that they usually demonstrate the practical in the classroom before allowing students to perform the practical themselves. Fewer teachers (n=9) referred to a combination of teacher and student-led approaches reporting that they adapt the lesson structure and the teaching based on various factors, such as the characteristics of the specific practical or the ability level of the group. The teachers who referred to student-led only approaches (n=3), described how they elicit student ideas by giving the ‘big picture of the practical’. Teachers’ preference towards more teacher-centred approaches can be explained by the recipe-like approaches in which practical science is currently taught (Childs and Baird 2020; Hofstein and Lunetta 2004), in which teachers demonstrate the practicals in a prescriptive and repetitive manner.

With regard to the resources that teachers use to summatively assess student performances, the majority of the participants selected past examination questions (34/51), followed by school-based written tests (5/51), teacher feedback (5/51) and checklists to recorded presentations (4/51). This finding illustrates that the preparation and delivery of practical lessons are oriented towards exam preparation, rather than the promotion of creativity and deep learning. This supports the claim that for upper secondary school teachers, the primary goal of practical lessons is ‘to prepare pupils for assessed practical work’ (Abrahams and Saglam 2010). This narrowing of the taught curriculum to what is assessed has previously described as a frequently unintended and negative washback or backwash effect (e.g. El Masri, Erduran, and Ioannidou 2021; Alderson and Wall 1993).

When asked about the impact of the new summative assessment format for practical science to their teaching, many participants (17/51) stated that their teaching has not been affected, although seven of them mentioned that they did not have enough teaching experience in order to be able to answer the question. In contrast, a number of teachers mentioned a ‘washback effect’ of the new assessments (13/51) describing a greater emphasis on exam questions and recall of the procedures that take place in class.

‘Teaching style has reverted back to recap, review and recall to ensure the knowledge is embedded. There is unfortunately less room to be creative at GCSE level.’ (science teacher)

This result supports criticisms raised in recent publications that the introduction of ‘required practicals’ may strengthen the -already commonly adopted- ‘cookbook’ teaching approaches for practical science that primarily promote recall and repetition (e.g. Erduran et al. 2020^b; Childs and Baird 2020).

Regarding teachers’ views on the skills that should be summatively assessed at GCSE, the majority of the participants (30/51) believed that more minds-on skills should be summatively assessed, such as the ‘method evaluation’, ‘linking to theory’, ‘planning the investigation’ and ‘evaluating the results’. In contrast, teachers who referred to hands-on skills (21/51) emphasised the ‘use of equipment’ and the ability to take accurate measurements. It can be argued that this finding illustrates teachers’ need for the inclusion of elements of self-reflection and scientific thinking in the design of assessment questions. Thus, teachers’ views are consistent with concerns previously raised by researchers regarding the need for more minds-on approaches to teaching and assessing practical science (e.g. Abrahams 2011; Erduran, Childs, and Baird 2020). This finding also provides another perspective to the interpretation of teachers’ views of practical science. Although teachers seem to be influenced by traditional teaching practices and conceptualisations of practical science as a ‘hands-on’ activity, they recognise the need for inclusion of more mindful approaches to teaching and assessing practical science.

Study 2: Brandon’s Matrix as a pedagogical tool

The results of the thematic analysis produced three main themes: (a) teachers’ choice of pedagogical strategies; (b) teacher responses to Brandon’s Matrix and assessments; and (c) teacher perceptions of student response. Within these themes, a series of subthemes were identified as outlined in Figure 2.

Figure 2. Themes and sub-themes from the thematic analysis of the interview data.

Theme 1: Teachers’ choice of pedagogies

The first theme from the interview data outlines the different choices teachers made when introducing the concept of Brandon’s Matrix to their pupils and the way they prepared them for the practical assessments. The data fall under two subthemes, the resources and planning teachers chose as well as their reflections on their teaching practice. The way the three teachers introduced Brandon’s Matrix was similar across the sample, with a reliance on the materials from the workshop or online seminar as a basis for their teaching. For the delivery of the practical lesson itself, the resources available in school and those that had been used in the past were chosen; described by Teacher 2 as ‘just the usual ones’. When reflecting on their teaching and the use of Brandon’s Matrix to frame the practical lesson, all the teachers provided positive statements about the experience and were aware of the additional ‘minds on’ thinking it encouraged (Table 4).

Table 4. Sub-themes and examples for teachers’ choice of pedagogies.

Sub-theme	Axial quotation
Planning & Resources	I used basically a lot of the resources that we'd had so from the presentation, I basically used similar examples and talked about the idea of the Matrix. (T2)
Reflection on teaching	[the students] got more from that practical than they did from a similar one last week where I didn't use the hypothesis and that style of questioning. (T1)
	…it made the practical go better because it made the children think about what they were doing. (T3)

There were also a number of suggestions made by the teachers regarding how they might alter the way they deliver the same content in the future. This included allowing extra time to prepare the students or a simplified introduction to the concept of the matrix would also be useful for lower ability pupils.

Theme 2: Teacher responses to Brandon’s Matrix and summative assessments

The teachers’ own personal responses to the concepts introduced through Brandon’s Matrix and the utility of the assessments was a main theme within the data. Three subthemes emerged from this main theme including teacher understanding of the matrix, teacher views on the matrix and teacher views on the assessments. The teachers demonstrated a sound understanding of the matrix when asked about the concepts of hypothesis testing and manipulating variables. Teacher 2 also commented on the use of the scientific method and the need for broader thinking to allow for scientific progress. The teachers expressed some positive views on the use of Brandon’s Matrix in the classroom that encouraged a ‘minds on’ approach to practical science. This was a particularly important finding, as it is in line with our expectations regarding Brandon’s Matrix affordances to address the need for deeper understanding of scientific methods during practical science.

However, limitations around the time needed to embed the use of the matrix were also highlighted. Overall, the teachers perceived the assessments as suitable resources. Yet there were concerns that the inclusion of the matrix may be an additional piece of content for students to study in preparation for their GCSEs. These concerns can be explained by the teachers’ tendency to ‘teach to the assessment’, as described in Study 1, which oftentimes does not accommodate innovative teaching methods Table 5

Table 5. Sub-themes and examples of teacher responses to Brandon’s Matrix and summative assessments.

Sub-theme	Axial quotation
Teachers’ understanding of Brandon's Matrix	I think certainly you don't have to have a hypothesis for science… and in terms of manipulation of a variable, with physics, that'd being impossible because there is stuff we can't manipulate (T2)
	If you look back and see that an awful lot of scientific discovery was done by mistake. …if you had to say that people's discovery only came from what we thought was possible, then you wouldn't make any progress. (T2)
Views on Brandon's Matrix	I love the idea of looking at science that's not hypothesis driven. I think in schools that's literally all we do [… .]. So I liked looking at all the other angles. (T3)
	I think that as a concept of getting students to think deeper about practical work, I think is very useful. (T2)
	I don't think it's necessarily particularly useful for a single practical. If it had been something that was part of the curriculum, I'd have spent more time on it. (T2)
Views on assessments	…the only way you could do it would be actually to have it in the curriculum and then see if it works or not. (T1)
	I didn't see that much difference between questions we have already had. (T2)
	It did feel like it was just a bit shoehorned in there as well. It's kind of an extra thing to think about. (T3)

.

Theme 3: Teachers’ perceptions of student response

The teachers all provided comments on their perceptions of the student response to the matrix and the assessments. Within this main theme, two subthemes emerged including perceptions of student performance and perceptions of student enjoyment. The teachers recognised some additional challenge faced by their students in the practical lesson and assessments with an awareness that introducing this additional content may be better suited to higher ability students. Finally, the teachers perceived mixed levels of enjoyment from their students. Teacher 3 believed their students viewed the additional content negatively while Teacher 1 perceived a much more positive response from their students. These findings are in line with findings from a recent large-scale study on student perceptions and enjoyment of Brandon’s Matrix according to which students may hold positive attitudes towards the matrix and the assessments Table 6

Table 6. Sub-themes and examples of teachers’ perceptions of student response.

Sub-theme	Axial quotation
Perceived student performance	The students who would have and given any feedback generally like implied that it was hard. (T2)
	They managed the first two parts. So, it was the first making of a hypothesis that they had to make. But that very last one, where they had to design almost the whole experiment, I think it is where we lost where I lost them. (T1)
	They performed the practical very well…But they are a top set so they are able to perform practicals well. But this time it was tangible that they had to think about every step. (T3)
Perceived student enjoyment	They just felt it was another thing to do. It's just not another thing that we’ve now got to learn. (T3)
	It was almost like they felt like they had more ownership of it because they had more say in what they were actually doing. So, they understood the logic behind it. (T1)

.

Conclusion

Overall, the findings of the two studies emphasise how science teachers view practical science and its summative assessment. Findings from Study 1 illustrate that, despite recent attempts in the national curriculum to include hands-on, as well as minds-on skills in teaching practical science (Department for Education 2014), teachers mainly refer to traditional hands-on conceptualisations, such as the use of equipment.

Since the majority of the participants use past exam papers for the summative assessment of their students’ practical science skills, teachers’ conceptualisations of practical science as a solely hands-on activity can be explained by the fact that high-stakes assessments often emphasise such skills, neglecting more higher-order skills, such as students’ understanding of scientific methods (El Masri, Erduran, and Ioannidou 2021). In addition, these findings strengthen the claim that teachers ‘teach to the assessment’ (e.g. Childs and Baird 2020). Furthermore, the majority of the participants reported that they usually follow teacher-centred approaches, which, as described by previous literature (e.g. Abrahams 2009; Abrahams and Millar 2008) may promote mindless step-by-step procedures for teaching and learning practical science. Another finding of Study 1 showed that when asked about the skills that should be assessed in the summative assessment of practical science, participants focused on the minds-on skills, such as problem-solving. Since participants defined practical science as a hands-on activity, it can be argued that this finding illustrates teachers’ need for the inclusion of elements of self-reflection and scientific thinking in the design of assessment questions.

In Study 2 Brandon’s Matrix was introduced to a number of teachers as a new framework aiming to facilitate a more minds-on approach to teaching and assessing practical science and scientific methods, in particular. The in-depth investigation of teachers’ views about scientific methods showed that they are quite receptive to Brandon’s Matrix for their teaching. Although the teachers did not see a significant departure of the assessment questions from the regular high-stakes questions, they did recognise that there was a valuable element of thinking skills embedded in them. Overall, the paper contributes to understanding of how the teaching and assessment of practical science and in particular scientific methods can be improved. The studies presented highlight insights into science teachers’ views about addressing the limitations of the conventional simplistic model of the scientific method.

Funding

Research reported in this paper was jointly funded jointly by Wellcome Trust, Gatsby Chariable Foundation and Royal Society (Grant Number 209659/Z/17/Z awarded to Sibel Erduran as the Principal Investigator).

Disclosure statement

No potential conflict of interest was reported by the author(s).