Using formal and informal formative assessment to support bilingual argument mapping in university bilingual science courses

Figures & data

Figure 1. Main features of our eight-week FIFA intervention. AM Argument mapping.

Table 1. Description of our FIFA intervention per week.

Week	Activity	Description
1	Instruction in argument mapping	Undergraduates received instruction on the basics of constructing argument maps and why this type of map is relevant in scientific communication. Special emphasis was placed on the importance of formulating concisely and clearly the four basic elements of argument diagrams, namely: (1) claim; (2) arguments – data and/or facts that support the claim; (3) counterarguments – data and/or facts that refute the claim and the arguments; and (4) rebuttals – data and/or facts that refute the counterarguments. This instruction is important to help participants become aware of what argument mapping entails, as well as how to identify and evaluate what counts as a high-quality argument map.
2–4	Construction of argument maps in Spanish	In the first lecture of each one of these three weeks, the instructor presented students with three argumentative questions (Questions 1–3 in Appendix 1) and asked them to build an argument map in Spanish (in total three maps) that presented solid scientific concepts, at least two valid and coherent arguments, at least two valid and coherent counterarguments, and at least one valid and coherent rebuttal. As recommended by Harrell (2008): each student was free to use any kind of medium (e.g. argument mapping software, paper and pencil, presentation software, word processor) with which s/he preferred to construct her/his argument maps, thus making the FIFA intervention more about engaging in argument mapping and less about computer software mastery
	Space for peer critique	In the second lecture of each one of these three weeks, participants were presented with some argument maps of those created in the first lecture and previously selected by the instructor. She selected those which evidenced difficulties (e.g. arguments based on invalid data) and those which did not – the selected maps were seen as evaluation information that were then used to involve participants in peer assessment and useful feedback. Then, students were asked to individually review the maps in order to critically identify and assess the arguments, counterarguments, and rebuttal(s). Finally, participants were organized in small groups so as to discuss and exchange points of view about the review of the maps that each had created individually (peer critique)
	Space for informal formative assessment	After the small-groups discussion, the instructor listened to those undergraduates who voluntarily socialized their opinions and criticisms about the maps to the whole class. This socialization scenario was particularly valuable for the instructor as a means of using the students’ participation to provide instantaneous and useful feedback to the class as a whole (informal formative assessment). Furthermore, the instructor played a supportive role in guiding whole-class discussion by emphasizing the validity of arguments, counterarguments, rebuttals and the coherence of their interrelations rather than the form (appearance) of argument maps
	Space for formal formative assessment	A few days after each ‘second lecture’, every student received her/his instructor’s written feedback about the argument map s/he had constructed in the ‘first lecture’ (formal formative assessment). Such written feedback included comments on specific parts of the map (claim, argument, counterargument, rebuttal) in order to provide clearer personalized feedback. In accordance with Floyd’s (2011) recommendation, the instructor avoided concentrating her feedback on correcting grammar, and instead emphasized the importance of criticality in students’ BAM. This preplanned feedback was a key opportunity to give participants useful and personalized comments to improve BAM, according to the strengths and weaknesses of the argument map produced by each student.
5–8	Creation of argument diagrams in English	In the first lecture of each one of these four weeks, the participating students were asked to construct four argument maps in English as a response to argumentative questions 4–7 (Appendix 1). Students were also allowed to use Spanish-English code-switching as a way of helping them grow in self-confidence
	Space for peer critique	In the second lecture of each one of these four weeks, undergraduates individually reviewed some diagrams previously selected by the instructor. They then discussed in small groups about the critique of the maps that each had carried out individually
	Space for informal formative assessment	The instructor took advantage of the students’ oral participation to provided instantaneous and useful feedback to the class as a whole,
	Space for formal formative assessment	A few days after each ‘second lecture’, every student received written feedback about the argument maps they had constructed in English

Figure 2. An argument map constructed by U30 as response to the Argumentative question 5 (Appendix 1).

Table 2. Rubric for scoring participants’ argument maps.

Component	Score	Description
Use of scientific concepts	0	A wrong use of scientific concepts
	1 (Maximum)	A correct use of scientific concepts
Arguments	0	There are no valid and coherent arguments
	1	There is one valid and coherent argument
	2 (Maximum)	There are at least two valid and coherent arguments
Counterarguments	0	There are no valid and coherent counterarguments
	1	There is one valid and coherent counterargument
	2 (Maximum)	There are at least two valid and coherent counterarguments
Rebuttal	0	There are no valid and coherent rebuttals
	1 (Maximum)	There is at least one valid and coherent rebuttal
Maximum total score	6

Note: In the components ‘Arguments’ and ‘Counterarguments’ the maximum score was two because participants were expected to produce at least two valid and coherent (counter) arguments.

Table 3. Details of the 1756 boxes of the 291 argument maps coded.

Type of box	AM 1 (n = 44)	AM 2 (n = 42)	AM 3 (n = 44)	AM 4 (n = 42)	AM 5 (n = 42)	AM 6 (n = 43)	AM 7 (n = 34)	Total (N = 291)
AB	95	91	107	106	94	106	75	674
CB	108	103	107	88	104	98	75	683
RB	78	63	61	53	54	50	40	399
Total	281	257	275	247	252	254	190	1756

AM Argument map, AB Argument boxes, CB Counterargument boxes, RB Rebuttal boxes.

Table 4. Means, standard deviations, and total scores of the argument maps constructed per week (N = 44).

	n	USC		Arguments		Counter-arguments		Rebuttal		Total
		Mean (SD)	d	Mean (SD)	d	Mean (SD)	d	Mean (SD)	d	Mean (SD)	d
AM in Spanish
1 (Week 2)	44	0.97 (0.15)		1.65 (0.52)		1.72 (0.45)		0.88 (0.32)		5.25 (0.89)
2 (Week 3)	42	0.97 (0.15)	0	1.88 (0.32)	0.53	1.90 (0.29)	0.47	0.83 (0.37)	0.14	5.59 (0.66)	0.43
3 (Week 4)	44	0.95 (0.21)	0.10	1.81 (0.39)	0.34	1.88 (0.32)	0.40	0.79 (0.40)	0.24	5.45 (0.72)	0.24
AM in English
4 (Week 5)	42	0.95 (0.21)	0.10	1.76 (0.53)	0.20	1.59 (0.58)	0.25	0.90 (0.29)	0.06	5.21 (1.07)	0.04
5 (Week 6)	42	0.92 (0.26)	0.23	1.69 (0.56)	0.07	1.80 (0.45)	0.17	0.95 (0.21)	0.25	5.38 (0.93)	0.14
6 (Week 7)	43	0.97 (0.15)	0	1.93 (0.25)	0.68	1.95 (0.21)	0.65	1.00 (0)	0.53	5.86 (0.41)	0.88
7 (Week 8)	34	0.97 (0.17)	0	1.88 (0.40)	0.49	1.82 (0.38)	0.24	0.91 (0.28)	0.09	5.58 (0.70)	0.41

AM Argument map, USC Use of scientific concepts

Figure 3. An Argument map constructed by U5 as response to the Argumentative question 5 (Appendix 1).

Figure 4. Number of students who created their maps in Spanish, in English or in Spanish-English code-switching. AM Argument map.

Table 5. Descriptive statistics of survey questions 8–15.

Question		Mean	SD
8.	The construction of an argument map requires time	4.45	0.55
9.	The construction of an argument map requires practice	4.12	0.93
10.	I liked to revise in class argument maps previously selected by the instructor	4.05	0.90
11.	The instantaneous oral feedback helped me to enrich my argument mapping skills	4.17	0.74
12.	The personalized written feedback helped me to enrich my argument mapping skills	4.25	0.74
13.	Constructing argument maps in English was an opportunity to practice my English	4.37	0.66
14.	Argument mapping should continue to be promoted in the Food Microbiology course	4.60	0.63
15.	Written tasks in English should be promoted in other university courses as well	4.40	1.00

Harrell, M. 2008. “No Computer Program Required: Even Pencil-and-Paper Argument Mapping Improves Critical-Thinking Skills.” Teaching Philosophy 31 (4): 351–374.

 Web of Science ®Google Scholar

Floyd, C. B. 2011. “Critical Thinking in a Second Language.” Higher Education Research & Development 30 (3): 289–302.

 Web of Science ®Google Scholar