Using formal and informal formative assessment to support bilingual argument mapping in university bilingual science courses

ABSTRACT

The rise in university bilingual science courses is explained by a certain phenomenon, the internationalization of higher education. Bilingual Argument Mapping (BAM) – the ability to construct argument maps using two languages – is a key aspect of bilingual scientific literacy. An argument map is a visual representation of argument structure. The problem is that little is known about how to foster BAM in this type of course. The goal of this study was to explore the possibility of using formal and informal formative assessment (FIFA) – a type of assessment for learning in which feedback is both preplanned and instantaneous –, to support undergraduates’ BAM. The data consist of the bilingual argument maps constructed by forty-four students (27 females and 17 males, 18–23 years old) in Colombia during a university Spanish-English bilingual science course. Results indicate that FIFA helped the participants’ creation of valid and coherent argument maps in Spanish, in English and in a hybrid version using code-switching as response to argumentative questions related to the scientific topics covered in the course. The potential contributions of FIFA and its implications are discussed in light of research and theory of bilingual science education.

KEYWORDS:

• Argumentation
• formal and informal formative assessment
• bilingual argument mapping
• university bilingual science courses

Introduction

Bi/multilingual science education is an umbrella term that subsumes multiple forms. In the current article, this term is used to refer to the use of two (or more) languages for science teaching and learning (Archila 2013). The discussion on the issue of effective bi/multilingual science education is gaining attention at the university level (Archila, Molina, and Truscott de Mejía 2021a). One point of concern in the discussion is about how to cultivate bilingual scientific literacy (BSL) – scientific literacy in two languages (Airey and Linder 2019). Airey and Linder (2011) stress that BSL is critical for students to participate appropriately in the written and verbal communicative practices of science in two languages.

With this point of concern in mind, the study described in this article is based on the premise that ‘formative assessment, bilingualism, and argumentation when combined can enrich bilingual scientific literacy’ (Archila, Molina, and Truscott de Mejía 2018, 1669). Additionally, we acknowledge that bilingual argument mapping (BAM) – the ability to construct argument maps using two languages – is a key aspect of BSL. BAM occurs when students are able to (1) create two maps in two different languages and/or (2) use two languages in the same argument map (e.g. code-switching). An argument map, also called an argument diagram, is a visual representation of argument structure to allow for easy communication of core statements and relations (Dwyer, Hogan, and Stewart 2013). A burgeoning literature illustrates the utility of argument maps in several domains such as, for example, CO2 capture and storage (van Egmond and Hekkert 2012, 153), nuclear power (Davies, Barnett, and van Gelder 2021, 116), and vaccination (van Amelsvoort and Schilperoord 2018, 58). The problem is that very little is known about how to support BAM in university first language-English bilingual science courses – a type of course that is growing because of the internationalization of higher education in countries where English is the second or foreign language (Archila and Truscott de Mejía 2020a, 2020b; Archila et al. 2021).

Based upon the above arguments, the purpose of the present study was to explore the possibility of using both formal and informal formative assessment (FIFA) – a type of assessment for learning in which feedback is both preplanned and instantaneous –, to support undergraduate BAM. To this end, we implemented a FIFA-based pedagogical strategy in a one-semester-long university Spanish-English bilingual science course by providing students with opportunities to reflect and construct argument maps in Spanish and in English related to the scientific topics covered in the course. The central question addressed in this article is: To what extent does the FIFA-based pedagogical strategy support the development of university students’ BAM?

Main features of BAM

In order to represent argument structure, an argument map often adopts a two-dimensional representation of a box-and-arrows (or lines) diagram which resembles a tree. It is multimodal in that it combines text with visual features, such as arrows, color, lines, and/or spatial lay-out. In other words, text boxes contain sentences that communicate claims and reason and/or evidence-based arguments, counterarguments, and rebuttals, and lines or arrows represent inferential relationships between them. Moreover, colors (optional) indicate the relations (argument, counterargument or rebuttal) among the elements (Uçar and Demiraslan Çevik 2021). At first glance, producing an argument map means producing a box-and-arrows diagram. Nonetheless, Twardy (2004) presents us with the following reflection: for some reason, no one believes that argument mapping requires time and practice until they try to map an argumentative paragraph. In addition, Schwarz and Glassner (2003) point out ‘that there is no such thing as the correct AM [argument map]’ (241). They expressly recommend that science instructors take advantage of this plurality and use it to initiate a discussion on maps constructed by students who can be invited to present and justify their argument diagrams, and to criticize or evaluate the maps of the others (peer assessment).

‘Nowadays, argument and debate are virtually absent from university science education’ (Archila, Molina, and Truscott de Mejía 2020b, 647). Consequently, in his book, Improving How Universities Teach Science, Nobel laureate Carl Wieman (2017) insists that it is vital to take concrete and research-based actions to make the preparation of scientifically literate arguers a reality rather than mere rhetoric. This is why a science course is a legitimate and desirable scenario for exploring the effectiveness of FIFA in supporting undergraduate BAM. As mentioned earlier, BAM implies the ability to construct argument maps using two languages. Recently, opportunities to foster bilingual written scientific argumentation in university bilingual science courses have been explored (Archila et al. 2020a, 2021a). However, very little is known about how to support BAM. The reason for this is that the phenomenon, namely, the internationalization of higher education, is only now influencing research in university bilingual science courses and, hence, there is still much work to be done in this respect (Pavón Vázquez 2021). With this in mind, in this article we claim that one possible way of supporting BAM can be to start by training students about how to produce argument maps in one language (e.g. Spanish), and once they have developed an acceptable level of ability in the construction of argument maps in this language, a second step can be to provide them with explicit opportunities to construct argument diagrams in another language (e.g. English). To be precise, our claim is based on Cummins’s (2017) interdependence theory which states that competencies and knowledge developed in one language (e.g. Spanish) can be available in another (e.g. English). It should be mentioned that this theory has been a pillar of previous educational interventions aimed at giving undergraduates opportunities to write Spanish-English bilingual argumentative paragraphs (see Archila et al. 2020, 2021a, 2021b, 2021c, for a fuller discussion). How interdependence theory can be articulated to FIFA to support BAM is explored in the next section.

Benefits of including FIFA in university bilingual science courses

Dori and Avargil (2015) note that mapping (e.g. argument mapping) is a concrete activity instructors can implement to engage students in formative assessment practice. At the heart of formative assessment is useful feedback (Song and Sparks 2019; Song, Sparks, and Guzman-Orth 2021), which can be preplanned – formal formative assessment (Grob, Holmeier, and Labudde 2021) – or which can take place during the course of events – informal formative assessment (Rached and Grangeat 2021). Cowie (2012) emphasizes that (1) planned tasks, and (2) instructor-student and student-student interaction are key aspects of formative assessment. In particular, a number of studies have demonstrated that instructor-student verbal interaction is a desirable scenario for the instructor to provide useful feedback while student-student verbal interaction is a favorable platform for the students to become engaged in peer assessment (also known as peer critique) (Archila, Molina, and Truscott de Mejía 2018, 2020, 2021a, 2021b, 2021c).

We decided to integrate formal and informal formative assessment for three reasons. The first is that it is rational and reasonable to assume that this combination extends the variety of possibilities students would find to become engaged in formative assessment practices. The second is the fact that previous studies have reported the use of formal (Archila et al. 2022) and informal (Archila, Molina, and Truscott de Mejía 2018) formative assessment separately to promote bilingual written scientific argumentation. Thus, to the best of our knowledge, this study is the first to include instantaneous feedback, preplanned feedback, and peer critique as a way to support student BAM. Furthermore, the importance of our study is more tangible once it is recognized that until now intervention studies do not even include at least one of these three elements to enrich BAM.

And the third reason is that this combination increases instructors’ opportunities to harness assessment information in order to monitor students’ progression and redirect educational practice where necessary. Arguably, the importance of using FIFA to support BAM is that students receive feedback from the instructor immediately (informal) in class as well as in a preplanned (formal) way. This clearly is beneficial for the honing of students’ skills to create argument maps using two languages. Moreover, FIFA contributes to engage students in authentic educational practices since Duss (2020) and Jönsson and Eriksson (2019) observe that formative assessment seems either non-existent or only insufficiently implemented in the teaching and learning structure of today’s universities.

For all the reasons just mentioned, it makes sense to give students explicit opportunities to not only construct argument maps in one language and then in another, as suggested by interdependence theory (Cummins 2017), but also to (1) receive useful preplanned feedback as well as to (2) interact with their instructor to receive useful instantaneous feedback and (3) to practice their peer critique skills during student-student interaction as part of a FIFA-based pedagogical strategy. Hence, we consider that Cummins’s (2017) interdependence theory can be articulated with FIFA to support BAM, whether or not FIFA practice is carried out when students produce argument maps in one language as well as in another. In this way, FIFA data can enable the instructor to better understand and respond to students’ BAM difficulties.

Methods

Context

The intervention study of FIFA took place in a university Spanish-English bilingual science course at a prestigious mid-sized university in Bogotá, Colombia. This course is called: Food Microbiology. It is taught over a 16-week period and consists of two lectures per week (75 min each) and one practical assignment per week (120 min). The intervention was limited to the lecture sections only. Each semester 40–60 students are enrolled in this course which is usually taken by undergraduates who have different English levels, and are of different ages and studying various majors, such as chemical engineering, food engineering, and microbiology. The instructor adopts a common bilingual science teaching practice (Mazak and Herbas-Donoso 2015) in relation to the use of Spanish (the students’ first language) and English (a foreign language) in the course. She uses slides that mix text in English and Spanish, and discusses them in Spanish. Most of the academic material assigned for reading is in English, and instructor-student and student-student interactions take place in Spanish.

Participants

Before the start-up of the intervention in the Food Microbiology course, clearance was secured from the University’s Research Ethics Committee. A total of 56 students volunteered to participate in this study. Nonetheless, only participants who constructed at least 6 of the 7 argument maps asked for throughout the intervention were considered in the analysis. Thus, in this article we report only the results of 44 students (27 females and 17 males). Ages ranged from 18 to 23 (M = 20.2, SD = 1.17). Of the 44 participants, 17, 19, and 2, graduated from monolingual (Spanish), bilingual (Spanish-English), and trilingual (e.g. Spanish-English-French) schools, respectively (six participants did not specify type of school). To ensure confidentiality during the data collection and analysis process, participants were assigned codes, for example U17 means Undergraduate Number 17.

Intervention

This study follows the formative assessment-based intervention model for bilingual courses proposed by Archila, Molina, and Truscott de Mejía (2018, 2020, 2021a, 2021b, 2021c, 2022). We decided this for three reasons. First, this model is based on Cummins’s (2017) interdependence theory. Second, this has been validated in various university Spanish-English bilingual science courses. And third, although the model has been created to promote bilingual written scientific argumentation, we considered that this could be used, for the first time, to foster BAM because of the formative assessment nature of this model. How this model of intervention was adopted is represented in Figure 1 and explained in Table 1.

Figure 1. Main features of our eight-week FIFA intervention. AM Argument mapping.

Table 1. Description of our FIFA intervention per week.

Week	Activity	Description
1	Instruction in argument mapping	Undergraduates received instruction on the basics of constructing argument maps and why this type of map is relevant in scientific communication. Special emphasis was placed on the importance of formulating concisely and clearly the four basic elements of argument diagrams, namely: (1) claim; (2) arguments – data and/or facts that support the claim; (3) counterarguments – data and/or facts that refute the claim and the arguments; and (4) rebuttals – data and/or facts that refute the counterarguments. This instruction is important to help participants become aware of what argument mapping entails, as well as how to identify and evaluate what counts as a high-quality argument map.
2–4	Construction of argument maps in Spanish	In the first lecture of each one of these three weeks, the instructor presented students with three argumentative questions (Questions 1–3 in Appendix 1) and asked them to build an argument map in Spanish (in total three maps) that presented solid scientific concepts, at least two valid and coherent arguments, at least two valid and coherent counterarguments, and at least one valid and coherent rebuttal. As recommended by Harrell (2008): each student was free to use any kind of medium (e.g. argument mapping software, paper and pencil, presentation software, word processor) with which s/he preferred to construct her/his argument maps, thus making the FIFA intervention more about engaging in argument mapping and less about computer software mastery
	Space for peer critique	In the second lecture of each one of these three weeks, participants were presented with some argument maps of those created in the first lecture and previously selected by the instructor. She selected those which evidenced difficulties (e.g. arguments based on invalid data) and those which did not – the selected maps were seen as evaluation information that were then used to involve participants in peer assessment and useful feedback. Then, students were asked to individually review the maps in order to critically identify and assess the arguments, counterarguments, and rebuttal(s). Finally, participants were organized in small groups so as to discuss and exchange points of view about the review of the maps that each had created individually (peer critique)
	Space for informal formative assessment	After the small-groups discussion, the instructor listened to those undergraduates who voluntarily socialized their opinions and criticisms about the maps to the whole class. This socialization scenario was particularly valuable for the instructor as a means of using the students’ participation to provide instantaneous and useful feedback to the class as a whole (informal formative assessment). Furthermore, the instructor played a supportive role in guiding whole-class discussion by emphasizing the validity of arguments, counterarguments, rebuttals and the coherence of their interrelations rather than the form (appearance) of argument maps
	Space for formal formative assessment	A few days after each ‘second lecture’, every student received her/his instructor’s written feedback about the argument map s/he had constructed in the ‘first lecture’ (formal formative assessment). Such written feedback included comments on specific parts of the map (claim, argument, counterargument, rebuttal) in order to provide clearer personalized feedback. In accordance with Floyd’s (2011) recommendation, the instructor avoided concentrating her feedback on correcting grammar, and instead emphasized the importance of criticality in students’ BAM. This preplanned feedback was a key opportunity to give participants useful and personalized comments to improve BAM, according to the strengths and weaknesses of the argument map produced by each student.
5–8	Creation of argument diagrams in English	In the first lecture of each one of these four weeks, the participating students were asked to construct four argument maps in English as a response to argumentative questions 4–7 (Appendix 1). Students were also allowed to use Spanish-English code-switching as a way of helping them grow in self-confidence
	Space for peer critique	In the second lecture of each one of these four weeks, undergraduates individually reviewed some diagrams previously selected by the instructor. They then discussed in small groups about the critique of the maps that each had carried out individually
	Space for informal formative assessment	The instructor took advantage of the students’ oral participation to provided instantaneous and useful feedback to the class as a whole,
	Space for formal formative assessment	A few days after each ‘second lecture’, every student received written feedback about the argument maps they had constructed in English

The lecture sessions of the Food Microbiology course were intervened for eight weeks (Figure 1). According to Cummins (2017), languages have an underlying cognitive/academic proficiency that is common to them. ‘This common underlying proficiency makes possible the transfer of cognitive/academic or literacy-related proficiency from one language to another’ (106). Previous intervention studies conducted by Archila, Molina, and Truscott de Mejía (2018, 2020, 2021a, 2021b, 2021c, 2022) which draw on Cummins’ (2017) theory of the interdependence of literacy-related skills and knowledge across languages, corroborate the view that developing students’ competence in the basic argumentation structure in their first language can help in the production of scientific argumentation in a foreign language. This is why in our eight-week FIFA intervention, from the second to the fourth week, the instructor followed a first-second lecture methodology to assist in the creation of argument diagrams in Spanish. This methodology was replicated from the fifth to the eighth week. In this period, students created argument maps in English (Table 1).

Research design

Data collection

As stated earlier, the purpose of this empirical study was to explore the possibility of using FIFA to support student BAM. Written responses served as the primary source of data for this article. The written responses were obtained by means of the seven argument maps created by each participating student to provide an appropriate answer to the seven argumentative questions (Appendix 1). Given that constructing an argument map is a time-consuming activity (Twardy 2004), in each of the seven 75-min lectures (in total 525 min), participants were given 40–50 min (in total 280–370 min) to construct each of the seven argument maps. They were also given the opportunity to use part of their ‘free time’ after each lecture to finish the maps and to upload them to an ‘Argument maps’ file created by the instructor in Blackboard Learn^™ – a learning management system.

Archila, Molina, and Truscott de Mejía (2018, 2020, 2021a, 2021b, 2021c) recommend assuming educational intervention as a continuous process of refinement. Also, they note that a pedagogical intervention is more likely to be functional and realistic when it leaves room for hearing student voice, and thus receiving feedback for future improvements. With this in mind, in the last session of the intervention, students were asked to voluntarily answer an anonymous online fifteen-item survey (Appendix 2). 40 out of the 44 participants answered the survey. The survey allowed us to collect information about respondents’: (1) previous instruction in argument mapping (Question 1 in Appendix 2); (2) opinions about the pedagogical intervention (Questions 2–7 in Appendix 2); and (3) impressions about the use of FIFA as an authentic practice (Questions 8–15 in Appendix 2). Because the construction of argument maps is a key purpose of our FIFA-based pedagogical strategy, we decided to include questions about time and practice as aspects required to create this type of diagrams. Arguably, at the end of our eight-week FIFA intervention students will have developed an opinion about whether the construction of an argument map requires (or not) time and/or practice.

It is important to clarify two points here. First, the fifteen items were adapted from feedback surveys developed in pedagogical interventions related to argument diagrams (Salminen, Marttunen, and Laurinen 2010) and bilingual written scientific argumentation (Archila, Molina, and Truscott de Mejía 2018, 2020, 2021a, 2021b, 2021c). This adaptation was solely nominal and did not change the purpose of each item. For example, the question, ‘Apart from the Biology of Organisms course, have you ever had received instruction in argumentation?’ (Archila, Molina, and Truscott de Mejía 2018, 1695), was adapted to ‘Apart from the Food Microbiology course, have you ever received instruction in the construction of argument maps?’

(Question 1 Appendix 2). Second, the whole instrument was presented to the students in Spanish and they were free to use the language(s) (Spanish, English or code-switching) they preferred in order to answer the open-ended questions (Questions 3–5 in Appendix 2).

Data analysis

In response to the research question, ‘To what extent does the FIFA-based pedagogical strategy support university students’ BAM?’ the argument diagrams (in total 291 argument maps) created by each participant were analyzed using frequency counts and percentages to determine the number of words written in Spanish and the number of words written in English. The analysis of each argument map continued by coding (scoring) them according to four criteria (codes). These criteria are presented in Table 2. For example, a score of 6 (the maximum score) was given to the argument map in Figure 2 for three reasons. The first is that U30 correctly used the scientific concepts: temperature and species (V. parahaemolyticus) (1 mark). The second is that the two arguments produced by this student are valid (it is a fact that that temperature had to be changed to ⩽10°C. It is also true that a temperature of 10°C is better than 12°C to control the growth of this pathogenic Vibrio species) as well as being coherent (both arguments effectively support her/his claim) (2 marks). The third is that the two counterarguments are not only valid (it has been demonstrated that V. Parahaemolyticus survives at temperatures above 5°C. Moreover, it is a reality that not all the restaurants are able to ensure a cold chain of their seafood products), but also coherent (the two counterarguments effectively oppose the claim and/or the argument) (2 marks). And the third reason is that the rebuttal produced by U30 is both valid (it is a fact that seafood is commonly consumed even while fresh) and coherent (this rebuttal is successful in objecting to the counterargument) (1 mark).

Figure 2. An argument map constructed by U30 as response to the Argumentative question 5 (Appendix 1).

Table 2. Rubric for scoring participants’ argument maps.

Component	Score	Description
Use of scientific concepts	0	A wrong use of scientific concepts
	1 (Maximum)	A correct use of scientific concepts
Arguments	0	There are no valid and coherent arguments
	1	There is one valid and coherent argument
	2 (Maximum)	There are at least two valid and coherent arguments
Counterarguments	0	There are no valid and coherent counterarguments
	1	There is one valid and coherent counterargument
	2 (Maximum)	There are at least two valid and coherent counterarguments
Rebuttal	0	There are no valid and coherent rebuttals
	1 (Maximum)	There is at least one valid and coherent rebuttal
Maximum total score	6

Note: In the components ‘Arguments’ and ‘Counterarguments’ the maximum score was two because participants were expected to produce at least two valid and coherent (counter) arguments.

The information displayed in Table 3 gives an idea of the importance of the data corpus on which this study was based as we examined 1756 boxes in total. Thus, for instance, the diagram in Figure 2 has 2, 2, and 1, argument boxes, counterargument boxes, and rebuttal boxes, respectively. To ensure the validity and reliability of our coding, we proceeded as follows. First, all the data were coded by the first author. Second, we adopted a multiple coders approach (Neuendorf 2009). Much of the reason for this is that this allowed us to not only ‘reduc[e] the volume of material that must be reviewed by each individual’, but to ‘enhanc[e] confidence in the replicability of the data’ (Neuendorf 2009, 70). To be clear, 85% of the data was coded independently by six coders. This process resulted in an inter-coder agreement of 93.18% for Argument map 1; 92.85% for Argument map 2; 88.63% for Argument map 3; 80.95% for Argument map 4; 90.47% for Argument map 5; 93.02% for Argument map 6; and 91.17% for Argument map 7. Discrepancies which emerged among coders were resolved after further reexamination of the corpus. We consider that our inter-coder agreement is acceptable since ‘the 80–90% range seems a minimal benchmark to those most concerned with an evidentiary statistic’ (Saldaña 2013, 35).

Table 3. Details of the 1756 boxes of the 291 argument maps coded.

Type of box	AM 1 (n = 44)	AM 2 (n = 42)	AM 3 (n = 44)	AM 4 (n = 42)	AM 5 (n = 42)	AM 6 (n = 43)	AM 7 (n = 34)	Total (N = 291)
AB	95	91	107	106	94	106	75	674
CB	108	103	107	88	104	98	75	683
RB	78	63	61	53	54	50	40	399
Total	281	257	275	247	252	254	190	1756

AM Argument map, AB Argument boxes, CB Counterargument boxes, RB Rebuttal boxes.

Moreover, to evaluate the magnitude of undergraduates’ gain throughout the eight-week FIFA intervention and as recommended by Archila et al. (2022), we calculated effect size using participants’ first argument map (week 2) as the control condition and treating diagrams created after this (weeks 3–8) as the experimental condition. Our results were interpreted following the benchmarks (small (d = 0.2), medium (d = 0.5), and large (d = 0.8)) proposed by Cohen (1988, 25–26).

Finally, the undergraduates’ responses to the 15-item feedback survey were analyzed using frequency counts (Questions 1, 2, 6, and 7 in Appendix 2). In the next section, some answers to open-ended questions (Questions 3–5 in Appendix 2) are commented on. The last stage of the analysis involved placing responses to Questions 8–15 (Appendix 2) on a rating scale range of frequency from Strongly Disagree (1) to Strongly Agree (5), and then calculating the Cronbach’s alpha coefficient of these questions using the Statistical Package for the Social Sciences (SPSS®).

We assumed that ‘Chronbach’s alpha is designed as a measure of internal consistency; that is, do all items within the instrument measure the same thing?’ (George and Mallery 2020, 236). In our case, the internal consistency reliability of these items was considered to be ‘acceptable’ (α = 0.71) (244). Despite this positive value, we should acknowledge that ‘the closer the alpha is to 1.00, the greater the internal consistency of items in the instrument being assessed’ (236).

Results

In summary, during the eight weeks of the FIFA intervention, participating students created seven argument maps to provide an appropriate answer to each of the seven argumentative questions (Appendix 1). In order to provide a more complete context for the results presented in this section, the students’ responses to the 15-item feedback survey were included. Table 4 presents the results of the argument maps produced by the students. Six was the maximum possible and the desirable total score. Our outcomes indicate that the maximum average score reached was 5.86 (Argument map 6; Week 7), showing a large effect size (d = 0.88). This is a positive result that demonstrates the importance of our FIFA intervention once several contextual factors are taken into consideration. First, only 14 out of the 40 participants who answered the survey had received instruction in the construction of argument maps before taking the Food Microbiology course (Question 1 in Appendix 2). Unfortunately, it is impossible to make associations between these results and the performance of each student because of the anonymous nature of the 15-item feedback survey. In other words, we do not know which are these 14 students. But it has never been the intention of our study to make this kind of association.

Table 4. Means, standard deviations, and total scores of the argument maps constructed per week (N = 44).

	n	USC		Arguments		Counter-arguments		Rebuttal		Total
		Mean (SD)	d	Mean (SD)	d	Mean (SD)	d	Mean (SD)	d	Mean (SD)	d
AM in Spanish
1 (Week 2)	44	0.97 (0.15)		1.65 (0.52)		1.72 (0.45)		0.88 (0.32)		5.25 (0.89)
2 (Week 3)	42	0.97 (0.15)	0	1.88 (0.32)	0.53	1.90 (0.29)	0.47	0.83 (0.37)	0.14	5.59 (0.66)	0.43
3 (Week 4)	44	0.95 (0.21)	0.10	1.81 (0.39)	0.34	1.88 (0.32)	0.40	0.79 (0.40)	0.24	5.45 (0.72)	0.24
AM in English
4 (Week 5)	42	0.95 (0.21)	0.10	1.76 (0.53)	0.20	1.59 (0.58)	0.25	0.90 (0.29)	0.06	5.21 (1.07)	0.04
5 (Week 6)	42	0.92 (0.26)	0.23	1.69 (0.56)	0.07	1.80 (0.45)	0.17	0.95 (0.21)	0.25	5.38 (0.93)	0.14
6 (Week 7)	43	0.97 (0.15)	0	1.93 (0.25)	0.68	1.95 (0.21)	0.65	1.00 (0)	0.53	5.86 (0.41)	0.88
7 (Week 8)	34	0.97 (0.17)	0	1.88 (0.40)	0.49	1.82 (0.38)	0.24	0.91 (0.28)	0.09	5.58 (0.70)	0.41

AM Argument map, USC Use of scientific concepts

Second, 35 out of the 40 respondents never (24/40) or infrequently (11/40) had had the opportunity to construct argument diagrams in Spanish in other university courses (Question 6 in Appendix 2). The importance of our intervention is even more evident as nearly all the students (39/40) never (31/40) or infrequently (8/40) had had the opportunity to produce this type of diagram in English in other university courses (Question 7 in Appendix 2).

As part of our FIFA-based pedagogical strategy (Figure 1), students were provided with instantaneous feedback. Consider the following excerpt (see Supplementary file 1) as an example of a whole-class discussion about an argument map (Figure 3) that has been previously critiqued by the students individually and in small groups. This excerpt illustrates how the instructor assumed the role of a facilitator when provided instantaneous feedback. There are three key points to comment on in this example. First, the instructor made it clear to the students that there are things she does not know (lines 9 and 13). This demonstrates that our strategy contributed to the creation of a student-centered learning environment since a typical characteristic of this type of environment is that instructors ‘gladly acknowledge that they do not know everything and look forward to learning along with students’ (Jacobs, Renandya, and Power 2016, p. xiv). This is particularly valuable as at university level, unfortunately, most of the instructors like to be seen as those who know everything (Wieman 2017). Second, the instructor took advantage of the students’ participation to provide feedback to the class as a whole about an important aspect that has been expressly noted by several scholars in the construction of argument maps: clarity in communicating ideas (Rapanta and Walton 2016; Salminen, Marttunen, and Laurinen 2010; van Amelsvoort and Schilperoord 2018). And third, this excerpt shows that U19 (line 4), U32 (line 14), and U17 (line 19) gave constructive criticism which is recognized as a fundamental condition for successful peer critique in argument mapping (Uçar and Demiraslan Çevik 2021).

Figure 3. An Argument map constructed by U5 as response to the Argumentative question 5 (Appendix 1).

Another key outcome is that a considerable number of the students who answered the survey (32/40) mentioned that the small-group discussion sessions about argument diagrams previously selected by the instructor were useful for them (Question 3 in Appendix 2). Some of their reasons include the following: ‘This helped me to realize that others can see the same map in a different way, and thus I could enhance my own perspective with that of others’ and ‘to hear different points of view was useful to improve my understanding’. Likewise, all the students (40/40) considered that the instantaneous oral feedback provided by the instructor was useful for them (Question 4 in Appendix 2). Some responses include: ‘It was useful that the instructor suggested ways of improving our maps’, ‘it helped me to better understand the scientific content’, and ‘it was useful to become aware of the most common mistakes and how to improve the next argument map’. Furthermore, the great majority of the students (38/40) felt that the personalized written feedback provided by the instructor was helpful for them (Question 5 in Appendix 2). Some of the reasons they gave include the following: ‘Due to the personalized manner of this feedback, I could find out about the specific aspects I had to improve’, ‘it helped me to improve map to map’ and ‘I was able to realize that there were aspects that I may have overlooked when constructing the maps’.

As part of our eighth-week FIFA intervention, students were asked to create argument maps in Spanish for argumentative questions 1–3 (weeks 2–4) and in English for questions 4–7 (weeks 5–8) (Appendix 1). In questions 4–7, they were also allowed to map in a hybrid version using code-switching. Figure 4 shows the number of students who constructed their argument maps in Spanish, in English and in a hybrid version. Although only 23 out of the 34 students who constructed the last argument map of the intervention (AM7 in Figure 4) did it fully in English, we believe that this is a promising result for two reasons. First, ‘writing, for nonnative EFL [English as foreign language] learners, is the most difficult skill to be learned’ (Shirazi and Mousavi Nadoushani 2017, 5). Second, science courses in non-Anglophone higher education institutions provide students with very few explicit, purposeful, and systematic opportunities to practice their English writing skills (Archila and Truscott de Mejía 2020a, 2020b). With this in mind, an essential question can be asked: Which type of difficulties did participants come up against in constructing argument maps in English? The results of the survey reveal that the top three difficulties were grammar (16/40)≥scientific vocabulary (16/40)>mastery of scientific knowledge (13/40) (Question 2 in Appendix 2).

Figure 4. Number of students who created their maps in Spanish, in English or in Spanish-English code-switching. AM Argument map.

Finally, Table 5 displays the students’ average scores along with the standard deviations on questions from 8 to 15 of the feedback survey. Five was the maximum possible average score for each item. The average scores varied between 4.05 and 4.60 with a mean of 4.30. This value corresponds to the ‘agree’ (Bringula et al. 2012, 1073) choice. Thus, it is plausible to claim that the undergraduates had positive impressions about the implementation of FIFA practices to support BAM in the Food Microbiology course. Importantly, participants scored the item ‘Argument mapping should continue to be promoted in the Food Microbiology course’ (Item 14 in Table 5) the highest (4.60). This result reinforces the idea that the promotion of argument mapping in university science courses should be an issue worthy of our serious attention (Rapanta and Walton 2016).

Table 5. Descriptive statistics of survey questions 8–15.

Question		Mean	SD
8.	The construction of an argument map requires time	4.45	0.55
9.	The construction of an argument map requires practice	4.12	0.93
10.	I liked to revise in class argument maps previously selected by the instructor	4.05	0.90
11.	The instantaneous oral feedback helped me to enrich my argument mapping skills	4.17	0.74
12.	The personalized written feedback helped me to enrich my argument mapping skills	4.25	0.74
13.	Constructing argument maps in English was an opportunity to practice my English	4.37	0.66
14.	Argument mapping should continue to be promoted in the Food Microbiology course	4.60	0.63
15.	Written tasks in English should be promoted in other university courses as well	4.40	1.00

Discussion and educational implications

Several academics agree that argument diagrams are a valuable means to communicate scientific argumentation (Davies, Barnett, and van Gelder 2021; van Amelsvoort and Schilperoord 2018; van Egmond and Hekkert 2012). Constructing an argument map, however, is a challenging task as it requires the coherent visual representation of inferential relationships between valid arguments, counterarguments, and rebuttals (Uçar and Demiraslan Çevik 2021). Since the use of two languages in this process constitutes an extra challenge, and there is no research investigating how to promote BAM in university bilingual science courses, it is important to address this gap in the literature. Consequently, in this study, we sought to explore the possibility of using FIFA to support student BAM. Our research question focused on the potential of a FIFA-based pedagogical strategy to support students’ BAM. Overall, findings indicate that there are various potential benefits of this strategy which are discussed in this section.

First, our results add evidence to Dori and Avargil’s (2015) idea that asking students to construct argument maps is a concrete activity that science instructors can use to involve students in formative assessment practice. This is a contribution to recent literature that alerts to the need to counter the long-established hegemony of summative assessment in higher education through the implementation of research-based formative assessment strategies (Duss 2020; Jönsson and Eriksson 2019; Wieman 2017). Moreover, the initial evidence provided in this study shows that our FIFA-based pedagogical strategy is unique in bringing together three key elements of desirable assessment, namely, (1) formal formative assessment, (2) informal formative assessment, and (3) peer critique. It is especially noteworthy that these elements are essential to successfully move from instructor-centered learning to student-centered learning (Chang, Hill, and Hannafin 2021).

Second, the FIFA-based pedagogical strategy provided each participant with seven explicit opportunities to create argument diagrams throughout the whole academic semester. Our results show that, in general, participating students used scientific concepts correctly (Table 4). These findings are consistent with the literature, where engaging students in argumentation practices is an effective way to make visible their conceptual understanding (Archila 2013). Importantly, this article extends earlier work by engaging participants in an under-researched argumentation practice in university bilingual science education, namely BAM. Another interesting finding concerns the regression-progression students shown in the seven argument maps they created. This confirms the claim that the promotion of argumentation is not necessarily achieved after an absolute linear progression. Archila, Molina, and Truscott de Mejía (2018, 2020, 2021a, 2021b, 2021c, 2022) explain that regression-progression in argumentation activities commonly occurs when students are asked to consider multiple elements in the same activity (in our case four: (1) scientific concepts, (2) arguments, (3) counterarguments, and (4) rebuttal). Thus, students tend to focus their attention on some elements, neglecting others. This is a plausible explanation of why the participants of this study regressed in the production of rebuttals in the first weeks of the intervention (weeks 3 and 4 in Table 4), while progressing in the development of (counter) arguments during these weeks.

Third, as part of the FIFA-based intervention (Figure 1), students were asked to construct argument maps in Spanish as well as in English. Cummins (2017, 106) insists that ‘there is extensive empirical research that supports the interdependence of literacy-related skills and knowledge across languages (see reviews by Baker 2001; Cummins 2001; Genesee et al. 2006)’. Having this in mind, our results corroborate this (Figure 4). In particular, we demonstrate that students’ ability to construct argument maps in Spanish can be transferred to English. Besides, the findings of this study extend those reported by Archila et al. (2020a, 2021a, 2021b, 2021c), who confirmed that participating university students were able to produce written scientific argumentative paragraphs in English (a foreign language) as response to argumentative questions when previous training and opportunities were provided in their first language (Spanish). An implication here is that students enrolled in bilingual science courses in similar contexts would greatly benefit from interdependence theory. This assertion is based on the fact that in the context in which our intervention took place, the participants’ first language (Spanish) was recognized as a valuable resource to facilitate argument mapping in a foreign language (English), and thus strengthen their Spanish-English BAM skills.

Fourth, in the feedback survey, participants commented that argument mapping requires time and practice (Table 5). According to Davies (2009, 2011) and Twardy (2004), these are two conditions that influence the quality of argument mapping. It may be obvious to point out that we obtained positive results (Table 4) because all the participating students: (1) received instruction on the basics of constructing argument maps in the first week of the intervention; and (2) were given sufficient time and various explicit opportunities to practice their BAM skills along one academic semester. Nevertheless, it is very unlikely that students would be able to continue enhancing these skills if they do not find these conditions in other university courses. Also, students commented that they found the instantaneous oral feedback (informal formative assessment), as well as the personalized written feedback (formal formative assessment), useful. Although this is an encouraging result, we confirmed that, as Grob, Holmeier, and Labudde (2021) note, formal formative assessment is an incredibly time-consuming process when compared to informal formative assessment. Hence, an important implication here is that the time-consuming nature of formative assessment should be seriously considered by administrators, organizers and policymakers when planning the implementation of a formative assessment-based curriculum.

Limitations and scope for future research

The limitations of our study are primarily related to its exploratory nature. The major weakness is that most of the results reported in this article came from an exhaustive coding of a large data corpus (1756 boxes, Table 3), focusing solely on the argument maps produced by the students during the eight-week FIFA intervention. Clearly, reporting not only the product (argument diagrams), but also the process (e.g. quality of the feedback students received) may provide a better understanding of the effect of the intervention, although this limitation applies to nearly every study interested in examining the effect of formative assessment practices on argumentation skills (Archila et al. 2022; Song and Sparks 2019; Song, Sparks, and Guzman-Orth 2021; Uçar and Demiraslan Çevik 2021). Moreover, despite the fact that participants considered that the instantaneous oral feedback as well as the personalized written feedback given by the instructor were useful for them, our analyses do not permit us to report how much each type of feedback effectively supported BAM. It may be obvious to point out that in a student-centered learning environment, students’ impressions about the FIFA intervention can be assumed as a crucial indicator of its effectiveness (Song, Sparks, and Guzman-Orth 2021); however, we recognize that it would be interesting to determine how specifically students benefited from each type of feedback. Another limitation is that we did not collected data from students’ argument mapping skills before the pedagogical intervention. This situation makes it difficult to know whether the good performance students showed in the first argument map (week two in Table 4) can with certainty be attributed to the instruction on the basics of constructing argument maps participants received in the first week of the intervention (Figure 1). This attribution is not entirely unsupported if we take into consideration the fact that only 14 out of the 40 undergraduates who answered the survey had received instruction in argument mapping before taking part in the intervention (Question 1 in Appendix 2).

Finally, our outcomes provide initial evidence that FIFA can be used to support BAM. This leads to very interesting directions in research. Future research is warranted, for example, to explain how BAM influences deep science learning, and vice-versa. At the same time, due to the rise in university first language-English bilingual science courses in countries where English is the second or foreign language, there is a need for research related to the use of FIFA to monitor not only conceptual learning, but also students’ argumentation and bilingual skills. Another future line of research could be to elucidate whether and in what ways BAM skills developed in a university bilingual science course help undergraduates to face the complex problems they will find in their professional context after graduating.

Acknowledgement

The authors would like to thank all the university students involved in the study for the time and effort they invested in the construction of Spanish-English bilingual argument maps and for their permission to use their maps and survey responses for the analysis reported in this article.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was financially supported by the Vice-Presidency of Research and Creation, Universidad de los Andes, Bogotá, Colombia.

Notes on contributors

Pablo Antonio Archila

Pablo Antonio Archila is a researcher in Argumentation, Bilingualism, and Science Education at the Vice-Presidency of Research and Creation, Universidad de los Andes, Bogotá, Colombia. He holds a Ph.D. in Educational Sciences from Université Lumière Lyon 2, Lyon, France, and a Ph.D. in Education from Universidad Distrital Francisco José de Caldas, Bogotá, Colombia. He conducted postdoctoral research at Universidad de los Andes, Bogotá, Colombia, and Université Paris Diderot, Paris 7-Sorbonne Paris Cité, Paris, France.

Gissel Gravier

Gissel Gravier is a student of Chemical and Food engineering at Universidad de los Andes, Bogotá, Colombia. She also provides support to the teaching process in the field of Food Microbiology.

Laura Levy

Laura Levy is a microbiologist from Universidad de los Andes, Bogotá, Colombia. She supported the development of the Food Microbiology class. She was part of the Tropical Microbiology and Parasitology Research Center (CIMPAT) where she developed her degree work.

Brigithe Tatiana Ortiz

Brigithe Tatiana Ortiz is a microbiologist, a student of the Master's degree course in Chemical Engineering, a graduate teaching assistant and researcher at the Universidad de los Andes in the city of Bogotá. She is also a student of the Master's degree course in Education at the Universidad Pontificia Bolivariana in the city of Medellin, Colombia. She carried out a research internship in Phytopathology at Cornell University. Her research interests include foodborne diseases and pathogens, phytopathology, medical microbiology, education in both face-to-face and virtual settings.

Alejandra Rodríguez

Alejandra Rodríguez holds a Bachelor's degree in Biology from Universidad de los Andes, Bogotá, Colombia and a Bachelor's degree in Microbiology from the same university.

Luciana Wilches

Luciana Wilches is recognized as one of the first people to hold a degree in Food Engineering from Universidad de los Andes, Bogotá, Colombia.

Anne-Marie Truscott de Mejía

Anne-Marie Truscott de Mejía has worked at the School of Education at Universidad de los Andes, in Bogotá, Colombia as the director of the Ph.D. Programme in Education and the research group “Education for bilingualism and multilingualism”. She holds a Ph.D. in Linguistics in the area of Bilingual Education from Lancaster University, U.K. Her research interests include teacher empowerment, interculturality, language and education policy, and bilingual teacher development.

Silvia Restrepo

Silvia Restrepo is Full Professor and Vice-President for Research and Creation at Universidad de los Andes, Bogotá, Colombia. She holds a Ph.D. in Phytopathology from Université Paris VI / Université Pierre et Marie Curie, Paris, France. She carried out postdoctoral research at Cornell University. Recently, she has cooperated in multiple university Spanish-English bilingual science education innovation projects.