Using model-based scaffolds to support students solving context-based chemistry problems

ABSTRACT

Context-based learning aims to make learning more meaningful by raising meaningful problems. However, these types of problems often require reflection and thinking processes that are more complex and thus more difficult for students, putting high demands on students’ problem-solving capabilities. In this paper, students’ approaches when solving context-based chemistry problems and effects of systematic scaffolds are analysed based on the Model of Hierarchical Complexity. Most answers were initially assigned to the lowest level of the model; higher levels were reached without scaffolds only by few students and by most students with scaffolds. The results are discussed with regard to practical implications in terms of how teachers could make use of context-based tasks and aligned scaffolds to help students in this activity.

KEYWORDS:

• Problem solving
• scaffolding
• context-based learning
• open-ended problems
• upper secondary chemistry

Introduction

Students come in touch with chemistry and use products based on chemistry in many different contexts, often without explicit awareness. In learning processes, these contexts can be taken as starting points to raise chemistry-related questions, to apply and deepen knowledge of and about chemistry, and to develop competencies such as argumentation, again with regard to chemistry. Context-based problems are usually not simple questions that can be answered by rote learning or algorithmic scripts but they require different problem-solving abilities. Solving problems that are embedded in authentic contexts often implies specific challenges and demands to students’ abilities (Leou, Abder, Riordan, & Zoller, 2006).

Students’ approaches to solving problems in chemistry education have been analysed from several perspectives (e.g. Bodner & McMillen, 1986; Hayes, 1989; Nakhleh & Mitchell, 1993). The importance of problem-solving competencies has also been highlighted as a generic skill in the society being one of the foundations of everyday life (OECD, 2003). Therefore, the problem-solving levels and strategies students use and develop in school are important for their future. For this study, we aimed to support students in the difficult endeavour of solving context-based problems by providing systematic scaffolds. The structure and format of these scaffolds were designed according the idea of stepped supporting tools (Fach, de Boer, & Parchmann, 2007) and the levels of the Model of Hierarchical Complexity in Chemistry, MHC-C (Bernholt & Parchmann, 2011). The quality of students’ responses to open-ended context-based tasks was analysed with regard to their accuracy and complexity, but also how students elaborate on their answers when being triggered by the predesigned scaffolds.

Background

Context-based learning

Context-based learning (CBL) approaches have been introduced in many countries as a means of increasing students’ interest and their content knowledge in specific subjects. Although there is no full consensus regarding the meaning of ‘context’, a useful definition applied in several approaches is that a context ‘is the red thread along which the investigation of the issue in question develops’ (Nentwig et al., 2007, p. 1441). The rationale is that the learner will start off from the context, work on problems and tasks directly related to the context, and thereby become aware of the content knowledge in demand to understanding the issue in question on a ‘need-to-know’-principle (Bulte et al., 2006). In consequence, context-based learning as well as context-based assessments need to provide opportunities for students to link contexts and concepts in what King and Ritchie (2013) referred to as ‘fluid transitions’, i.e. a back-and-forth movement across activities in order to make connections between contextual and conceptual information while working in context-based courses or on context-based problems (Bellocchi, King, & Ritchie, 2016). The context thus is not only a mere decoration in the beginning or in the end to illustrate something or motivate students, but context and concept are intertwined, and making use of contextual information is a requirement when working on contextual questions.

Hence, CBL targets both the affective and the cognitive domains of learning as a context can both raise or hinder motivation and interests as well as conceptual explanations and problem-solving approaches (cf. Bennett, Lubben, & Hogarth, 2007; de Jong & Taber, 2014). Research indicates that context-based approaches are more motivating and interesting than conventional alternatives for both teachers and students, and teachers report that students who completed such courses exhibit levels of chemical content knowledge that are equal to those achieved with conventional courses (King, 2012; Sadler, 2009; Schwartz, 2006). According to a meta-analysis by Bennett et al. (2007), however, effects on learning outcomes were less coherent even though CBL approaches were perceived more enjoyable and relevant to the students in comparison to more conventional chemistry courses.

Although several studies have highlighted the advantages of CBL approaches, demands and criticisms have also been raised (e.g. Taasoobshirazi & Carr, 2008; Ültay & Calik, 2012). As Whitelegg and Parry (1999) emphasise, the selection of appropriate contexts is vital for CBL approaches to actually increase students’ motivation and their perception of relevance. Sevian and Talanquer (2014) stress the importance of not neglecting content knowledge or losing sight of the need to convey chemical ideas and practices when concentrating on the context (also cf. Gilbert, Bulte, & Pilot, 2011; Prins, Bulte, & Pilot, 2016). The student-centred emphasis commonly applied within CBL approaches also often results in learning environments providing less guidance to students, which in turn might also slow down students’ learning by making it difficult to see the chemical concepts in the context (Parchmann et al., 2006; Wickman, 2014). Especially with regard to connecting contexts and concepts, many students need scaffolds to learn to identify links in a context and to apply concepts to solve context-based problems.

In sum, the persistently contradictory results of CBL on learning outcomes might be due to the resulting different affordances of CBL tasks in different approaches. More research is needed to better understand the characteristics of context-based problems, of students’ approaches when dealing with context-based problems, and on effects of suitable scaffolds (Demetriadis, Papadopoulos, Stamelos, & Fischer, 2008; Hmelo-Silver, Duncan, & Chinn, 2007). The next paragraphs will therefore explore the functionality of a context in a problem with regard to the application of conceptual explanations, as well as systematic models to describe and to scaffold steps of conceptual learning and problem-solving.

Conceptual learning and problem-solving

In science education research, several approaches have been advanced to address the issue of students’ (un)successful development and application of domain-specific conceptual understanding under the label of conceptual change research. A major controversy in this research field applies to conflicting findings concerning the issue of context sensitivity of students’ conceptions, i.e. that students do not perform coherently across contexts, and that they often provide different explanations when working on differently contextualised problems (Brown, 2014; Clark, 2006; Ozdemir & Clark, 2009).

Unlike other conceptual change approaches (Chi, Slotta, & de Leeuw, 1994; Vosniadou & Brewer, 1992), the ‘intuitive fragments’ perspective (diSessa, 1993) acknowledges that students often do not apply their (alternative) conceptions coherently across contexts but that they answer differently in different contextual settings. According to this approach, conceptions are regarded as consisting of various elements which generally act together more or less coherently, but also might respond quite differently to contextual features. Based on their individual prior knowledge, students ‘extract’ (diSessa, Sherin, & Levin, 2016) specific information from the contextual setting at hand. Some of these aspects might be generally prominent for a student constructing scientific explanations; others might be cued by specific contextual features (Kapon & diSessa, 2012). In consequence, a person selects from all available information in a situation the particular information needed to infer the applicability of a concept in this situation. Such processes necessarily vary across contexts, especially in early stages of learning a specific concept (diSessa, 1993). Repeated practice and experience in applying a concept then leads to a more reliable ‘extraction’ of information and thereby to a higher degree of coherence of students’ answers across different contexts (without assuming context-free abstractions or schemas as in other approaches of knowledge transfer; Wagner, 2010).

Solving a context-based problem might therefore not only be a question of knowing the underlying concept, but also of ‘seeing’ the relevance and suitability of a concept in a given context. Dawson (2014) argues that learners’ conceptions might be best described as neuronal coexisting elements, representing both common-sense and normative scientific views, both being optionally activated in dependence of contextual factors. In accordance with diSessa and Wagner (2005), he concludes that ‘science educators need to aim more actively at strengthening the learner’s executive processes which select contextually appropriate responses and inhibit inappropriate ones’ (Dawson, 2014, p. 389). Thus, the following questions arise: how and to which extent do students make use of contextual aspects when solving context-based problems, and how can students’ problem-solving processes be supported by specific scaffolds?

Problem-solving and scaffolding approaches

In the problem-solving process, the starting point is the problem. Here we focus on open-ended context-based chemistry problems, sometimes also defined as ill-structured problems (Byun, Lee, & Cerreto, 2014; Kelly, McLoughlin, & Finlayson, 2016), such as CBL problems. Based on the framework explored above, studentś problem-solving processes can be analysed systematically with regard to the information students extract from the context and the explanations they develop starting from this. An evaluation of the quality of such an explanation raises a need for models providing characteristics that can both be evaluated and addressed by instruction and scaffolds.

Various models exist for evaluating the quality of an explanation. For example, Taber and Watts (2000) propose three key aspects to consider when evaluating a response to be an explanation or not: if the response has the structure of an explanation (by stating words like ‘because’ and ‘therefore’); if the response is logically consistent; and if the response is correct or not. Bernholt and Parchmann (2011) have proposed a complexity model with five hierarchical levels (Unreflected Everyday Experiences, Facts, Descriptions of Processes, Uni-variate Empirical and Logical Causality, and Multivariate Empirical and Logical Causality). The model brings to mind the logical conjunction of facts and processes in order to establish causal and interdependent relations (cf. Jonassen & Ionas, 2008). With regard to CBL, these conjunctions both incorporate contextual and conceptual aspects. The complexity of a task depends on the explanatory power of the expected argumentation for a successful solution. Additionally, the structure of the model ties in with students’ preconceptions and experiences as the starting point. Similar approaches were put forward by Shavelson, Ruiz-Primo, and Wiley (2005) as well as Biggs and colleagues (Biggs & Collis, 1982; Biggs & Tang, 2007).

In addition to research focusing on the results of students’ approaches to scientific problems, i.e. their solution or explanation, numerous studies also focused on the process of students’ problem-solving. For instance, Overton, Potter, and Leng (2013) found that students are often unable to identify details of the problem, they get stuck if they lack data, and they are distracted by the context and their own lack of knowledge. The authors highlight many difficulties in the solving process when students do not get support, and from this, scaffolding and tutoring from a teacher or a peer are important to consider. Both lines of research, i.e. on the product and the process of students’ problem-solving, are needed to not only better characterise problem-solving processes in CBL, but also to develop suitable scaffolds. Already forty years ago, Wood, Bruner and Ross (1976 , p. 89) discussed the tutorial process in problem solving, where ‘an adult or “expert” helps somebody who is less adult or less expert’. From this tutoring or scaffolding, students can be guided to higher-order thinking. A Korean quasi-experimental study analysed how effective three different question-prompts strategies were to promoting cognitive scaffolding when university chemistry students solved ill-structured problems (Byun et al., 2014). The results show that instructor-generated prompts from an expert or teacher made the students solve the chemistry problems significantly better than students who developed their own peer-prompts or students who at first developed their own prompts and thereafter revised them with help from the instructor. In a US study on general chemistry at university level, a hierarchical taxonomy was used to design a course aiming to scaffold students’ learning through encouraging higher-order thinking (Toledo & Dubas, 2016). Early observations from that study indicate that students evaluated the different instruments implemented in their courses (scaffolding instruction, scaffolding assessment, and feedback) quite positively, claiming that all course materials had a positive impact on their learning.

In sum, empirical results with regard to the effectiveness of CBL approaches in fostering students’ conceptual understanding are inconclusive. An explanation could be that students have difficulties in seeing the chemical concepts in the context (Parchmann et al., 2006; Wickman, 2014), an assumption that is also supported by cognitive theories of conceptual change (cf. Dawson, 2014; diSessa & Wagner, 2005). This also raises the importance of providing suitable scaffolds and analysing their effects on the support of problem-solving processes having been identified as (too) demanding for many students without extra scaffolds.

Consequently, the aim of this study was to provide deeper insights into the problem-solving process of students when working on context-based problems, and particularly how systematic scaffolding can help students in this activity. Using open-ended context-based tasks, we aimed to require students to solve problems in everyday contexts that are seen as relevant and meaningful to students. We incorporated a set of predefined, model-based scaffolds in order to support the students’ problem-solving processes. Two research questions were investigated based on problem-based interviews:

1. How do students approach open-ended context-based chemistry problems and do these approaches differ between contexts?

2. To what extent is scaffolding based on a hierarchical model of complexity effective with regard to supporting students’ problem-solving processes?

Methodology

For this study, 15 context-based chemistry problems were developed using clear design criteria (Broman & Parchmann, 2014). The 15 problems dealt with five different topics (medical drugs, fats, fuels, energy drinks, and soaps and detergents) and three different contextual framings for these topics (personal, societal, and professional applications). The problems focused on the same chemical content area (structure–property-relations) and could be solved by similar approaches of chemical argumentation. The problems, besides being written in an everyday-life language without explicit chemistry hints, always contained molecular structures to emphasise the chemistry focus of the problems (Broman & Parchmann, 2014). Examples of problems from the topic of energy drinks in three different contextual settings (personal, societal, and professional) are presented in Figure 1.

Figure 1. Examples of open-ended context-based chemistry tasks on the topic of energy drinks in three different contextualizations.

Sample

Twenty students (age 19) at the end of upper secondary school solved two problems each; in total, 40 approaches were analysed. The students were sampled from three different schools in a mid-size city in Sweden to avoid a teacher effect. The students voluntarily agreed to take part in the study and their teachers assessed them as medium to high achievers and with a competence to express chemistry orally. Regarding their chemistry level, the students had read as much chemistry as possible in the Swedish school system before entering university level.

Interview procedure

In preparation for each interview, the interviewer (first author) preselected two of the five topics to assure that all topics were equally covered. To start the actual interview, students were asked to read the three corresponding tasks (covering all three contextual settings for each topic) and then to decide which of the three contextual settings they wanted to solve. Since the three contexts requested similar responses, the students could choose the tasks most motivating for them. This procedure was repeated for the second preselected topic.

Students were then asked to solve the two tasks according to think-aloud techniques and both the students’ responses and the interviewer’s prompts were analysed according to the Model of Hierarchical Complexity in Chemistry (MHC-C, cf. next paragraph). The full interviews lasted 40–60 min and consisted of a warm-up phase where students’ ideas about their chemistry studies and their future educational choices were elaborated and thereafter 5–10 min were used to solve each task.

Scaffolding

During the problem-solving process, the interviewer gave predefined hints to the students when they provided incorrect or incomplete responses. The hints from the interviewer, also an experienced upper secondary chemistry teacher, were given in a preplanned manner, partly from experiences from analysing students’ responses to context-based problems in a previous study (Broman, Bernholt, & Parchmann, 2015). Based on the approach by Fach et al. (2007) for stepped supporting tools, the underlying idea was to trigger students’ reasoning to the highest possible level of complexity, thus reaching higher levels than students were able to perform by themselves (assuming that students were willing to provide their best possible answer in the interview).

In order to link the analysis of the students’ responses to the design of the scaffolds with regard to their complexity (and thus, consequently, to the affordances of a particular level), scaffolding was based on the MHC-C-operators (i.e. name, describe, explain). Operators are signal words that can be encountered in any given problem and which reflect the affordances and activities that are attached to the task at hand. These three operators were derived by re-analysing tasks from previous studies on the MHC-C with regard to the frequency of all different operators on each level of complexity (Bernholt & Parchmann, 2011; Podschuweit, Bernholt, & Brückmann, 2016). Accordingly, the operators ‘name’ (level 2), ‘describe’ (level 3), and ‘explain’ (levels 4 & 5) were seen to represent the affordances of each specific level of complexity. Level 1 (everyday experiences) was left out of this study as no increase in the complexity of students’ reasoning was expected from scaffolding with the lowest level. Only one operator represented levels 4 and 5 because performance on level 5 was rarely observed by any student in all studies carried out so far. To facilitate the interpretation of results, the three operators and the corresponding levels of complexity were numbered in line with the original numbering of the MHC-C from 2 to 4 (cf. Table 1 and Appendix).

Table 1. Overview about the operators used as specific prompts and their corresponding level of complexity.

Level	Operator	Level of complexity (MHC-C)	Example
2	Name …	Facts	‘Could you name any functional groups in the molecules?’
3	Describe …	Description of Processes	‘Can you describe how these functional groups relate to solubility?’
4	Explain …	Uni- and Multivariate Empirical and Logical Causality	‘Can you explain how solubility is related to both intra- and intermolecular bonding?’

The application of a specific operator was based on a student’s performance during the interview, as perceived by the interviewer. Hence, the interviewer monitored the progress of the interview by evaluating both the accuracy (in order to include clarification questions) and the level of complexity of the student’s utterances (in order to include the appropriate operator as a scaffold). In general, scaffolds were used to trigger students’ reasoning on the next higher level of complexity. If a student was able to solve the task on the third MHC-C level on his own, i.e. description of processes, the explain-operator was used to reach higher levels of thinking, whereas if a student only could state facts, the interviewer first gave the describe-operator and thereafter the explain-operator. If a student solved the problem on his/her own, only clarifying questions were given from the interviewer. Besides the operator, the scaffold was also aligned to the content aspect a student just addressed in his response. An example of an interview procedure is given in Figure 2.

Figure 2. An exemplary interview transcript showing the three different MHC-C-operators; name, describe, explain.

Analysis

Each utterance by a student was assigned to one level of complexity according to specific indicators (cf. Table 1 and Appendix); the interviewer’s hints were assigned based on the three MHC-C operators; name, describe and explain (cf. Table 1). In both cases (students’ responses and interviewer’s hints), the coding unit was the full contribution by one of the persons, i.e. all sentences by this person until the next change of speaker. Hence, the contributions could range from single words to several sentences but each of these contributions was assigned to one code as long as it was a consecutive statement by one person.

In total, 611 student utterances were transcribed and analysed. The first author analysed all 611 student statements three times, 6 months between the first and second categorisation, and 9 months between the second and third categorisation. The intra-rater reliability was 97% and 99% respectively. Thereafter, the co-authors categorised 111 statements (i.e. 18%) applying the MHC-C model and inter-rater reliability was 93%. The final categorisation of the statements was made through adjusting categorisation that was differing and thereafter patterns were analysed. The high intra- and inter-rater agreement indicates that the MHC-C provide a suitable framework to reliably categorise the complexity of students’ approaches when solving context-based problems.

Results and discussion

The results will be presented in two steps related to the research questions: In the first section, students’ own approaches towards the problems without any scaffolding will be discussed with a specific focus on how they make use of the contextual setting of the tasks. In the second part, the scaffolding is scrutinised to understand how the MHC-C operators impact students’ problem-solving processes. In both sections, qualitative (with regard to students’ uptake of the problem context) as well as quantitative results (with regard to analysing the complexity of students’ answers to the problem based on the MHC-C) are presented. At the end, both sections are contrasted in order to negotiate how the scaffolding applied in this study can be used by teachers when helping students to develop their own problem-solving competencies.

Students’ (initial) approaches to open-ended context-based problems

Before solving the task, students could choose the context of the task they solved whereas the topic was decided by the interviewer to get results from all topics. Out of the 40 responses, 30 chose a personal context, five a societal and another five a professional contextual setting (cf. Table 2). Students’ evident appreciation of the personal setting is in line with previous research where a clear connection to the students’ own life is preferred (Stuckey, Hofstein, Mamlok-Naaman, & Eilks, 2013). The students’  answers to those tasks with a personal context included context-related information in 27 out of 30 responses (cf. Table 2).

Table 2. The use of context information in students’ responses to the context-based chemistry problems (cf. Broman & Parchmann, 2014); number of responses and (in parentheses) the percentage of responses including context-related terms and examples.

Topic	Context
	Personal	Societal	Professional
Medical drugs	5 (100%)	2 (50%)	1 (0%)
Energy drinks	6 (100%)	0 (n/a)	2 (0%)
Fats	6 (83%)	1 (0%)	1 (0%)
Fuels	6 (50%)	1 (100%)	1 (0%)
Soap and detergents	7 (100%)	1 (100%)	0 (n/a)

The students picked up the context by using specific words or examples provided in the task (cf. Broman & Parchmann, 2014). For instance, students who chose to solve the personal context in the topic of energy drinks mentioned the uptake of polar molecules (taurine and caffeine) in the intestines and through the blood–brain barrier to reach the brain to discuss solubility (cf. Table 8). The following quote presents an example of a student connecting the contextual setting of the task to the chemistry content, moreover the student shows connections to both everyday life as well as to other science subjects.

My friends drink RedBull all the time and I have read in the papers that some have died from it. I recognise caffeine from coffee; it’s interesting to see the molecular structure of something you know from outside of class. From our biology course I know that dopamine makes you happy, like when you run the body releases dopamine. That is perhaps why it feels good to drink coffee or RedBull. I know that taurine is the ‘bad’ molecule since there are warnings on the can, this might be related to being a signal substance in the nervous system. When I see the structure, I see sulphur, I think this is a bad compound. And also, it must be something incorrect with the sulphur, it has too many bonds. [translated excerpt from transcript]

Such a use of the context information from the task in the response is not limited to the personal context. A student who chose to solve the professional context of the same topic discussed how analytical methods as NMR can be used to both explore the molecular structure of the active ingredients (taurine and caffeine) as well as deciding the concentration of the molecules in the energy drinks:

I look at the table of contents quite often to check since I’m allergic to lactose and I sometimes think about how they measure these values, like milli grammes and so on. Here I learnt that they use NMR to do this. I’m not sure what NMR is, but it must be some kind of analytical instrument. We read analytic chemistry in the end of the chemistry course, but only for a few weeks before summer. I remember that it was possible both to check the amount of a compound as well as which compound. This is good. [translated excerpt from transcript]

As the study incorporated a series of problems with a systematic variation of the contextual framing for different topics, it was possible to analyse how students combined different words and examples from each problem into their answers. The results showed that students used (or were triggered by) the contextualisation of the problem differently, i.e. that students ‘extract’ (cf. diSessa et al., 2016) different information depending on the contextual setting at hand. Consequently, the contextualisation impacts a student’s approach to construct a scientific explanation to the problem by providing them with different starting points for their answer. Relating to the notion of ‘fluid transitions’ (cf. Bellocchi et al., 2016), students applied the contextual settings from the problems in their responses and thereby connected context with content (King & Ritchie, 2013). Due to the small amount of students who decided to work on problems in a societal or professional context, it was difficult to compare the ratio of responses including contextual terms across the three different contextualizations.

When assessing the accuracy and complexity of the 40 final student solutions as a whole and when analysing the highest MHC-C level students reached during the interview before the interviewer provided any scaffolds, it was found that the majority of students’ answers (28 out of 40) were either only repeating facts (MHC-C level 1&2) or providing rather simple descriptions of processes and sequences of events (MHC-C level 3; cf. Table 4) without any causal elements.

The effect of model-based scaffolds on students’ problem-solving processes

The second step of analysis provided insights into effects of the scaffolds. Therefore, the analysis of the interviews now focuses on the second phase of the interviews, i.e. on student statements’ complexity after the first scaffold was provided by the interviewer (Tables 3 and 4). First, it is important to take into account that six students solved the problems by themselves with only clarifying questions and without any scaffolds from the interviewer. These students could on their own solve the problem on the highest level of the MHC-C, i.e. by combining several explanations and content areas in the solution. They were labelled as independent problem-solvers. An example is given in Table 5 to highlight that hints were not needed from the interviewer or teacher to reach higher levels of the MHC-C as the student solved the problem without scaffolding.

Table 3. Distribution of student statements across the four levels of complexity before and after the interviewer provided at least one scaffold.

Level of complexity	Number of student statements prior to any hints	Number of final student statements (after all hints were provided)
5 (Multivariate Causality)	18	44
4 (Uni-variate Causality)	47	193
3 (Description of Processes)	33	107
1&2 (Facts and Everyday Experiences)	56	59

Table 4. Distribution of student complete answers across the four levels of complexity before and after the interviewer provided at least one scaffold.

Level of complexity	Number of student responses prior to any scaffolds	Number of final student responses (after all scaffolds were provided)
5 (Multivariate Causality)	9	15
4 (Uni-variate Causality)	3	16
3 (Description of Processes)	18	3
1&2 (Facts and Everyday Experiences)	10	6

Table 5. Exemplary excerpt from an interview transcript by one student labelled as independent problem-solver.

Speaker	Transcript	Classification of interviewer contributions	Classification of student answer
Interviewer	How would you start solving this energy drink task?
Student	OK, if taurine and caffeine from RedBull is going to react in the body, it has to go from the stomach to the blood from the intestines and then follow the blood to the brain. If they’re going to be solved in the blood … blood is mostly water … the molecules have to be water-soluble. So I have to define polar parts in the molecules … Let’s see … if I start with caffeine, it has methyl groups that aren’t polar, C-O-groups that might be polar I think, and then it’s the nitrogen … So caffeine is perhaps not so polar, but on the other hand, caffeine is in coffee and coffee is water-soluble, so caffeine must be polar … Then the taurine molecule, that’s tricky, is it drawn correctly?		level 4
Interviewer	Yes, why?	clarifying question
Student	The sulphur has so many bonds … But if you say it’s correct, it must be … Then, water-soluble … yes it’s water-soluble since it has an OH-group, that’s the definition: OH equals hydrogen bonding. So taurine is water-soluble. And it’s also a small molecule, easier to solve in water, since water is very small …		level 5
Interviewer	Is it only the OH that makes in water-soluble?	clarifying question
Student	Since you ask, perhaps not … I don’t know the electronegativity value for S, but perhaps the amino group … it was N, O, F connected to H that was hydrogen bonding …		level 4

In this excerpt, the interviewer only needed to give clarifying questions to get a broad, exhaustive and complete response. Moreover, the response focused both on context as well as content areas; the student mentions both how the energy drink is affecting the person drinking it through chemical reactions in the body, as well as the solubility of taurine and caffeine molecules.

In contrast to these independent solutions, six solutions (provided by three individual students) remained on the level of only repeating facts (MHC-C level 1&2) even when scaffolds were provided. These three students were therefore clustered in a second group, labelled as insufficient problem-solvers. In Table 6, an insufficient problem-solver only gave responses on the lowest level according to the MHC-C, i.e. only stating facts. Hints from the interviewer were of no help, the scaffolding had no impact on the students’ answer quality. The student did not go further in his/her problem-solving process, did not make use of the scaffolding from the interviewer, and focused on a recall of factual knowledge when pointing out functional groups.

The remaining 22 responses reached higher levels of the MHC-C after scaffolding from the interviewer. The 11 students who could make use of the hints from the interviewer to develop their responses further (as illustrated in the examples provided in Figure 2 and Table 7) were named progressing problem-solvers.

Table 6. Exemplary excerpt from an interview transcript by one student labelled as insufficient problem-solver.

Speaker	Transcript	Classification of interviewer contributions	Classification of student answer
Interviewer	How would you solve the item on energy drinks?
Student	Caffeine and taurine, it’s not molecules I know so much about …		level 1&2
Interviewer	Any idea where to start?	clarifying question
Student	Caffeine has two rings, carbon and nitrogen, and also hydrogen and oxygen … Taurine has the same atoms but also S … I don’t know much more …		level 1&2
Interviewer	Can you name any important functional groups?	hint (name)
Student	OH in taurine, that one I recognise …		level 1&2
Interviewer	Anything else?	clarifying question
Student	Double bonding to O in caffeine, but I don’t know what they do …		level 1&2
Interviewer	Can you describe the molecules, what chemical properties do they have?	hint (describe)
Student	I don’t know …
Interviewer	How do you think they react in your body?	hint (describe)
Student	I don’t know … perhaps that they make you awake?		level 1&2
Interviewer	How?	clarifying question
Student	I don’t know …
Interviewer	Do you know any part in the body in charge of these reactions?	hint (name)
Student	The brain?		level 1&2
Interviewer	Yes, and can you explain how can the molecules get to the brain?	hint (explain)
Student	No, not really …

Table 7. Exemplary excerpt from an interview transcript by one student labelled as progressing problem-solver.

Speaker	Transcript	Classification of interviewer contributions	Classification of student answer
Interviewer	How would you solve the item, how can caffeine and taurine react?
Student	Well, the solvent must have the same chemical properties as the compound to solve, since it’s ‘like dissolves like’ … it’s always this like dissolves like in chemistry …		level 1&2
Interviewer	What does that mean?	clarifying question
Student	That the molecules have to be alike to solve … this is just something I know, I have read it in the chemistry textbook and my teacher uses this expression all the time, it’s something we have heard since we started reading chemistry many years ago …		level 1&2
Interviewer	Can you elaborate your ideas about like dissolves like?	clarifying question
Student	It’s just that both substances that are going to be mixed have to look the same …		level 1&2
Interviewer	What does it mean to look the same, what happens when for example taurine or caffeine mix in the blood?	hint (describe)
Student	If they look the same, I mean look the same in regard to chemistry, the blood and the energy drink with the taurine and caffeine, there is a solution, they will be solved with each other …		level 3
Interviewer	Can you try to describe this chemical process when something is solved in something else?	hint (describe)
Student	So … I check for polar and unpolar parts … caffeine has many polar groups so this might be solved into … perhaps ethyl acetate … but of course ethyl acetate is not found in the body, it’s just the solvent my teacher often mentions …		level 4
Interviewer	Where do you see polar groups in caffeine?	clarifying question
Student	It’s oxygen in C = O … perhaps also N, I don’t really know about this, but also, it doesn’t have many long C-chains … But the other one, it has … taurine has O and N, but it also has other atoms too … so … but I think it’s more polar than unpolar …		level 4
Interviewer	If you compare the two, which one do you think is more polar?	hint (explain)
Student	… oh … I would say caffeine … since it feels as if it has more polar groups … even though a bigger molecule, it has many more … these are soluble in water		level 5

Overall, the student answers were more complex after scaffolds were introduced by the interviewer, with 31 out of 40 solutions assigned to the two highest levels of the model (uni-variate and multivariate causality), i.e. exhibiting essential features of an explanation (cf. Taber & Watts, 2000).

As students in all three groups worked on problems from all five different topics (e.g. medical drugs, energy drinks, see Table 2), no relationships can be inferred between topic and difficulties in providing successful solutions. However, all 20 students from all three groups of problem-solvers demonstrated the same problem-solving strategy in both problems, i.e. not dependent on the topic or context of the problem; both problems solved by each student were approached and solved in a very similar way (cf. Table 8). This seems possibly suggestive that students may be consistent in their problem-solving approach in relation to scaffolding regardless of topic or context but since the study only had 20 students, this result is impossible to generalise.

Table 8. Students’ final solutions to the two problems they solved.

Problem 1	Insufficient problem-solvers	Independent problem-solvers	Progressing problem-solvers
Problem 2
Insufficient	3	0	0
Independent	0	6	0
Progressing	0	0	11

To graphically illustrate the three types of problem-solvers, the different statements’ MHC-C levels over time are presented, see Figure 3. Here, differences between the problem-solving patterns become more visible. The solid line shows the insufficient problem-solvers who did not make use of the scaffolds, they only recalled facts. The dashed line represents students who solved the problems by themselves with only clarifying questions; they gave responses on the highest level of complexity without any scaffolding from the interviewer, i.e. an independent problem-solver. These students applied higher-order thinking without external help. A little over half of the students were of the third problem-solver type, the progressing problem-solver (dotted line), reacted to the interviewer’s hints and developed responses towards higher levels of complexity.

Figure 3. The categorisation of the students’ statements according to the MHC-C framework. The figure presents three interviews to exemplify each problem-solver type.

With regard to the three contextual settings of the problems (personal, societal, and professional), 29 out of 30 responses to problems framed in a personal context were categorised as independent or progressing problem-solving (Table 9). While the assignment of students’ answers to one of the three contextual settings was based on the students’ decisions which of the three settings they wanted to solve (after having read the tasks to all three contextual settings), the assignment to one of the three problem-solving types was based on the students’ performance on the task they had chosen (see Table 9).

Table 9. Distribution of the categorised problem-solver types across the three contextualizations of problems

Problem-solver type	Personal context	Societal context	Professional context
Insufficient	1	2	3
Independent	10	1	1
Progressing	19	2	1
Total	30	5	5

It is difficult to identify a causal mechanism behind the apparent relationship between choice of personal context and successful problem solving whether independently or with scaffolds. Is it that a contextualisation that address the students’ personal life makes it easier for students to make connections between context and content, resulting in a higher chance of solving these problems? Or is it a preference for the personal context by those students who are better problem-solvers? Both assumptions are partly impugned by the aspect that nearly all students performed consistently in problem-solving type across the two tasks (cf. Table 8), albeit some students opted for different contextual settings in both rounds. Hence, with regard to the results presented above, the type of contextualisation (personal, societal, professional) seems to play an important role while the topic (medical drugs, fats etc.) does not seem to make a difference for students’ problem-solving (cf. Table 9). Still, further research involving larger cohorts and experimental variations of all contextual settings would be needed to support this hypothesis.

Further results

No clear relation between final examination grades and the three different types of students (insufficient, independent, and progressing) was found; all 20 students had high grades on their final written exam. Nevertheless, there was a difference in how the students expressed their own competence to discuss chemistry problems orally. In the discussion after the problem-solving process, the students argued about their experiences of problem solving in chemistry. Almost all students claimed that it is common to be satisfied after giving one answer, an exemplary quote from one of the students was: In chemistry, I look for the correct answer, reasoning is more social science whereas another student said that the focus is on giving responses that the teacher ‘wants’. The involvement of scaffolds influenced this perception in some cases. One student, for example, explicitly claimed after solving the problems: It’s interesting to see that you can explain solubility with many things, it’s not just one correct answer. The three insufficient problem-solvers, not responding to the scaffolds, all said that they were not at all used to discussing or thinking about chemistry in any other way than stating the correct response.

All students in this study claimed that these tasks were very worthwhile and they all expressed ideas that problems like these would be good to use in class to learn from. They were not as positive to have them as test tasks, mainly because they were not used to solving problems requiring different responses. An eloquent utterance is from a student ending the interview with: At first when you showed me the RedBull task, I had no idea where to start. I couldn’t come up with the correct solution at once, but then when you helped me to unravel the problem, I think I have learnt so much during this half-an-hour and it would be great if we could work with tasks like this in class, where our teacher could walk us through the problem.

Summary and conclusions

Context-based open-ended problems are considered to provide students the opportunity to further develop their problem-solving skills. According to Smith (2011), CBL is an adequate way to teach students higher-order thinking when the subject-matter knowledge is embedded in different contexts, requiring the learner to identify the chemical aspects within a complex problem (cf. Nentwig & Waddington, 2005; Zoller & Dori, 2002). However, these complex problems make high demands on students’ abilities (e.g. reading skills) and problem-solving strategies. Hence, students often perceive context-based tasks as less prestructured and more complex (Salta & Tzougraki, 2011). In addition, students often are inexperienced with problem-solving of open conceptual tasks but are used to searching for the one single correct answer. They struggle with transferring content knowledge from one task to another and often stick to a recall of facts when solving chemistry problems. As school learning still focuses mainly on lower-order cognitive skills such as rote memorisation, recall, and algorithmic teaching with ‘one correct answer’ (Bennett, 2008; Leou et al., 2006; Sevian & Talanquer, 2014), scaffolds supporting the problem-solving of more complex context-based tasks seem to be highly important for practice. Such scaffolds need to provide guidance with regard to the choice of a certain concept in a particular context when the particular example has not been discussed before (Wagner, 2010). Students need to be trained in identifying relevant information from the context, using this information to identify relevant concepts, and to make connections between the contextual information and their conceptual knowledge (Bellocchi et al., 2016).

In the present study, both these specific affordances of context-based problems as well as the unfamiliarity with this type of questions was completely reflected in the students’ answers. Almost all students claimed that it is common to be satisfied after giving one answer or that the focus in class is on giving responses that the teacher wants. As context-based learning is not a common approach in teaching chemistry in Sweden, all students stated that the type of tasks used in this study were rather unusual when compared to the tasks they normally work on. Consequently, the tasks were evaluated as less positive with regard to their usage in tests due to the students’ unfamiliarity with the format and its open-endedness.

With regard to the first research question, three aspects can be emphasised: (1) almost all students extract multiple information from the context each problem was embedded in, but (2) not all students were able to integrate this information and their own knowledge to develop an explanation of the problem at hand, and (3) this result (i.e. students’ success in developing an explanation) did not differ between the tasks they decided to work upon. It became obvious that not all students were able to solve the different context-based tasks successfully, even though the sample comprises only students that were medium- to high-achievers (according to the evaluation of their teachers) at the end of their secondary school. As stated above, CBL approaches set high demands on students’ higher-order thinking, even though these concerns are not limited only to CBL (King & Ritchie, 2012).

Despite their unfamiliarity with this type of problem, students picked up the context by using specific words or examples provided in the task, i.e. the contextualisation influenced how students ‘extract’ information. Hence, students’ answers were in most cases specifically related to the contextualisation of the problem. The context thus provided different starting points for the students to address the problem. However, students’ responses also shared common features across contexts. One of the outstanding aspects is again related to the demands students are usually confronted with in class. Accordingly, students’ problem-solving approach can mostly be characterised as mere recall of memorised facts and a strong tendency to provide the one single expected answer (cf. Broman & Parchmann, 2014; Bennett, 2008; Leou et al., 2006; Overton et al., 2013; Sevian & Talanquer, 2014).

In summary, students’ responses to the context-based problems seemed to be influenced by the contextualisation (mainly in a positive way by providing students with possible starting points, not in a confusing way, as sometimes reported in the literature; cf. Parchmann et al., 2006; Wickman, 2014). Moreover, the structural features of their answers were quite coherent (in terms of a strong focus on facts, so that students rarely provided an extended explanation structure or a complete problem-solving process without being triggered). While most answers were initially assigned to the lowest level of the MHC-C, higher levels were reached without scaffolds only by few students and by most students with scaffolds.

Besides these descriptive results, the second research question of the present study intended to evaluate the extent to which scaffolding according to the MHC-C supports students in working on the context-based problems by challenging students’ responses with a set of predefined prompts. The results for about half of the students (11 out of 20), who were progressing problem-solvers indicated that the provided scaffolds tended to result in student answers that were increasingly complex. These students seemed to have been highly influenced by the MHC-C-operators and could further complexify their problem-solving processes by applying higher-order thinking after the hints from the interviewer. In other words, providing students with these rather simply designed scaffolds seemed to make an enormous difference with regard to the quality of students’ responses. When analysing students’ responses in reaction to specific prompts by the interviewer, it became obvious that the MHC-C-operators provided students a clear focus they could use to elaborate on their answer given so far. For two smaller subgroups of students (insufficient and independent problem solvers), however, these prompts seemed to have been only of little use. On the one hand, some students (6 out of 20, independent) were able to come to an accurate solution on their own. On the other hand, some students (3 out of 20, insufficient) had extensive problems in coming up only with an approach to solving the task so that the hints from the interviewer provided little help for them.

Implications

The results support the assumption that a teacher can intentionally manipulate a students’ level of complexity, e.g. by fostering students to respond on specific levels or by using a specific sequence of complexity levels. The MHC-C operators were helpful for about half of the students to develop their own problem-solving skills and move towards higher-order thinking in CBL problems. The operators in themselves (i.e. name, describe, explain) are simple to use for teachers to support their students. Besides the present study, first results from other studies support this assumption as well. For instance, Knobloch, Sumfleth, and Walpuski (2012) included specifically designed instructions to foster students’ content-related communication in chemistry classes. According to their study, supporting students in gradually establishing links between different content aspects led to both a higher rate of high-quality content-related statements in the group work and to a higher learning achievement in the test. Consequently, the explicit incorporation of complexity training into instructional settings may directly impact students’ learning and achievement, although open questions remain regarding the role of multiple factors, ranging from the role of teacher characteristics (PCK, enthusiasm, beliefs, etc.) over content influences, to student characteristics (interest, attitudes, learning goals etc.).

Both the tasks as well as the scaffolds challenged students to develop their reasoning and problem solving during the interviews, regardless of the specific contextualisation of the problem at hand, and were considered by the students to be very worthwhile and conducive to learning. This suggests that the regular implementation of context-based tasks and learning settings might provide opportunities to students to develop their problem-solving skills. In this study, students’ problem solving seems to be influenced predominantly by the presence of a context, and not by the particular contextual setting (personal, societal, professional) or the specific topic (fuels, energy drinks, etc.), as the structural features of their answers were quite coherent across these conditions. With regard to the second research question, an implication from this study is that students benefit from practising the solving of more open tasks where they can apply their chemistry content knowledge, rather than only remembering factual knowledge. Practising to solve context-based tasks is also a prerequisite to including this type of problems on exams, as students perform better on tests with questions that are similar to those they have previously experienced during chemistry lessons (Bennett et al., 2007).

In general, the results provide insights into an approach of how teachers can successfully support students in solving open-ended context-based problems. The sample of this study, however, sets some limitations for the generalisation of the obtained results, as it was only 20 students working on the problems and these 20 students were medium- to high-achievers in chemistry. Accordingly, the group of students where the set of prompts used in this study gave little help to their problem-solving approach (i.e. insufficient problem-solvers) might in fact be larger when also including low-achieving students. Here, further studies including a more representative sample of students over the full range of abilities and prior knowledge might provide more insights into the efficacy of the model-based prompts in this study.

Disclosure statement

No potential conflict of interest was reported by the authors.

Appendix

Table A1. Illustration of the complexity coding scheme (cf. Podschuweit et al., 2016).

Table

Level of complexity and indicators	Indicators	Examples
(5) Multivariate Causality   - describing and justifying arrays of causes   - taking multiple factors into consideration	Justifications beyond linear relations; handling of several variables, their interplay and interdependent cause-and-effect chains. In case of several linear chains, their interplay and interference has to be taken into consideration.	Caffeine is polar due to carbonyl groups with resonance and nitrogen in the rings, so it’s water soluble. Caffeine is also much bigger than water, I mean, the molecules. So many more water molecules probably can come closer to caffeine which probably makes it more water-soluble because of more hydrogen bond possibilities.
(4) Uni-variate Causality   - giving reasons for linear chains of causes and effects or reaction schemes   - justifying results or steps within a process   - justifying factors of influence   - deductive reasoning   - calculations with multiple steps	Registration and description of linear cause-and-effect chains. A justification of the relation is of great importance, not only the description of if-then-relations.	The molecules in RedBull must be soluble in water since we drink energy drinks. The caffeine has nitrogen atoms, and that’s one of the atoms that can participate in hydrogen bonding, you know, NOF, so caffeine can do hydrogen bonding with water … Both water and caffeine are polar molecules.
(3) Description of Processes   - describing processes or reactions   - finding or describing commonalities and differences   - drawings (unguided)	Description of processes which show a chronological or sequential progress; with or without the use of (mental) models; not only simple before-after-descriptions, but a focus on the actual process, e.g. a description of a mechanism; simple relations and dependencies without reasoning;	The taurine molecule is strange; the sulphur has six bonds … I remember sulphur as different from for example carbon which always has four bonds. It has to do with electron structure, not being as simple as we often read in books, but I don’t know how it can have six bonds.
(1 & 2) Facts and Everyday Experiences   - definitions   - stating laws, principles, terms, models, symbols   - simple calculations (routines)   - drawings (guided)   - examples   - knowledge from everyday life	Nomination of isolated terms, laws, principles without explication, argumentation or comparisons; bits of isolated factual knowledge; Pieces of knowledge that stem from students’ everyday life^a; statements without explanatory base; direct observations or sensory impressions; statements without technical terms;	The caffeine molecule has two rings, lots of nitrogen atoms in the rings and also these carbon – oxygen double bonds. The taurine molecule is trickier, the S in the middle looks strange …

^aThe reference to everyday life is not to be understood as context, e.g. when students try to connect their knowledge learned in classroom to everyday natural and societal phenomena. In this meaning, it would be possible to find aspects on each level of complexity. Rather the label stresses that knowledge on this level is seen as being predominantly isolated and unrelated to the subject structure.