First- and second-order scaffolding of argumentation competence and domain-specific knowledge acquisition: a systematic review

ABSTRACT

Results of research on intentions and effects of first- and second-order argument scaffolding of computer-supported collaborative argumentation competence development and domain-specific knowledge acquisition are ambivalent. A systematic review of research in secondary and higher education (SE and HE) has been conducted to clarify and synthesise these intentions and effects, thereby differentiating between communication type (synchronous–asynchronous) and group size. Empirical research with pre-post-test designs was included only. Using specific search terms, 527 articles were found; 19 of these met pre-set selection criteria. Results indicate that HE studies intended to foster argumentation knowledge and domain-specific knowledge acquisition (i.e. knowledge construction), and reported significant effects for both types of knowledge. SE studies, however, intended to foster argumentation behaviour and domain specific knowledge acquisition (i.e. learning by doing), and showed significant effects regarding the latter only. HE studies predominantly used asynchronous, and SE studies synchronous communication. Choice of group size was not explicitly justified.

KEYWORDS:

• Argumentation
• scaffolding
• computer-supported collaborative learning
• computer-supported argumentation
• effect

1 Introduction

Diverse argumentation-scaffolds, like visual representations and scripts, have been designed and embedded in web-based systems, including social networking sites, to facilitate, coordinate and orchestrate diverse roles, interaction patterns and activities of students (Kirschner, Buckingham Shum, & Carr, 2003; Noroozi, Weinberger, Biemans, Mulder, & Chizari, 2012; Scheuer, Loll, Pinkwart, & McLaren, 2010; Tsovaltzi, Greenhow, & Asterhan, 2015). Such scaffolds could have been designed as first-order scaffolds, to acquire domain-specific knowledge, or as second-order scaffolds, to acquire argumentation competence (i.e. students’ argumentation knowledge, argumentation behaviour and attitude towards argumentation). Nevertheless, empirical research on computer-supported collaborative argumentation (CSCA) presents a lack of clarity with respect to the intention and effects (whether they were found or not) of first- and second-order argument scaffolding on argumentation competence and domain-specific knowledge. This review not only aims to clarify and synthesise such intentions and effects (whether they were found or not) in terms of the educational level of the participants (higher education [HE] and secondary education [SE]), but also reports on the communication form (synchronous or asynchronous), and group size used in the studies.

1.1 Argumentation

Argumentation is a key competence across domains and in different aspects of daily life. In the particular context of education, students are typically encouraged to work together and solve tasks in teams with partners holding various perspectives and knowledge conceptions about an issue (Noroozi et al., 2012). In such scenarios, students need to build upon, relate to and refer to what has been said by their peers to learn and co-construct knowledge (Noroozi, Kirschner, Biemans, & Mulder, 2018; Noroozi et al., 2012). Argumentation facilitates the comprehension of differing meanings, the acceptance, consideration and integration of others’ perspectives and opinions of the problem at stake, and reflection (Toulmin, 1958; van Bruggen & Kirschner, 2003). Despite the importance of argumentation competence and the attempts to offer argumentation courses to students, argumentation competence is regularly developed indirectly and informally in the classroom (Driver, Newton, & Osborne, 2000; Osborne, 2010). When argumentation is considered in the classroom, a teacher can effectively provide individualised support, supervision and tutoring to one student or a small group of students (Bloom, 1984). However, this type of support falls short if the number of students increases, since the teacher will not be able to thoroughly supervise and tutor the argumentative activities of all students during peak times (Loll, Scheuer, McLaren, & Pinkwart, 2010). Similarly, students struggle to argue in a reasoned way in academic settings (Noroozi, Teasley, Biemans, Weinberger, & Mulder, 2013) due to different factors. Students struggle, among others factors, with the intricate, non-linear and ill-structured character of argumentation (Lynch, Ashley, Pinkwart, & Aleven, 2009; Scheuer et al., 2010), to generate, analyse and evaluate arguments based on rules of logic (Kuhn, 1991) and to deal with different interpretations of ‘facts’ (Scheuer, McLaren, Weinberger, & Niebuhr, 2013). The latter makes argumentation difficult to teach, learn and follow its rules regarding the construction of arguments and counter-arguments (Toulmin, 1958), and to engage in sequential discourse (Leitão, 2000).

1.2 Argumentation competence

This study considers that argumentation competence is comprised of students’ argumentation knowledge, argumentation behaviour and attitude towards argumentation, since these components are related and thus influence the learning outcome of the discourse (Noroozi et al., 2018). Moreover, argumentation competence is not only considered as the capacity to argue, think critically and reason logically to explain one’s informed opinions, positions and decisions in contrast to other’s viewpoints and opinions, but also as the capacity to handle equivalent tasks and continue learning in the future. In contrast, there is no homogenous definition of argumentation competence among researchers (Rapanta, Garcia-Mila, & Gilabert, 2013). Scientific evidence shows that researchers tend to measure argumentation competence by focusing mainly on the skills that individuals manifest during discourse (Rapanta et al., 2013), or by measuring students’ knowledge on argumentation prior to and after collaborative discourse activities (Noroozi, Weinberger, Biemans, Mulder, & Chizari, 2013). This is striking, since in many situations students’ actual argumentation knowledge is not reflected in their argumentation behaviour during discourse activities. For example, in several studies by Stegmann, Weinberger, and Fischer (2007), Stegmann, Wecker, Weinberger, and Fischer (2012), Kollar, Fischer, and Slotta (2007), as well as Noroozi, Weinberger, et al. (2013), although individual students showed they had knowledge for construction of the formal quality of single arguments, they were not able to put their knowledge in practice either during discourse or in a similar argumentation task. Therefore, one should not only rely on students’ argumentation knowledge but also their behaviour during actual discourse (see also Andrew & McMullen, 2000). Furthermore, students’ psychological, emotional, motivational and social barriers may also affect their argumentative discourse activities. For instance, some individuals might experience emotions like nervousness or anxiety while providing a claim or receiving a question (Gilbert, 2004), or may perceive peer feedback as critiques and personal attacks (Rourke & Kanuka, 2007). Also, students emotionally attached to the topic of discussion can make argumentation unfruitful, complicated or even impossible (Baumeister & Scher, 1988; Leith & Baumeister, 1996). Therefore, next to students’ knowledge and behaviour, their attitude toward argumentation (e.g. psychological, emotional, motivational and social barriers) should also be considered. Moreover, being competent not only implies the capacity to apply a given competence in new situations possibly taking place in a different context, but also learning from the given problem and further developing the competence (Mulder, 2014).

One way to foster the acquisition of argumentation competence and domain-specific knowledge is to use computer-based learning systems and instructional scaffolds.

1.3 CSCA, scaffolding and its effects

Previous research has found that CSCA can facilitate constructing, representing and sharing arguments in diverse formats (Noroozi et al., 2012; Scheuer et al., 2010). Similarly, CSCA environments are considered important instructional tools to scaffold and structure students’ argumentative learning (Jeong & Lee, 2008), promote in-depth discussions (Andriessen, Baker, & Suthers, 2003), and in consequence facilitate in-depth understanding and the construction of productive arguments (Buckingham-Shum, 2003). In addition, CSCA systems make possible the scaffolding of important discourse and argumentation processes (Jeong & Lee, 2008).

To support learners in focusing on specific content, argumentation must be framed, scaffolded and guided by external representations (e.g. Belland, Glazewski, & Richardson, 2008; Muller Mirza, Tartas, Perret-Clermont, & de Pietro, 2007). Many studies have shown the benefits and advantages of argumentation-based computer-supported collaborative learning (ABCSCL) in terms of constructing knowledge, gaining a comprehensive understanding, cognitive development and solving complex problems (e.g. Andriessen et al., 2003; Kirschner et al., 2003). In addition, CSCA systems make possible the scaffolding of important discourse and argumentation processes (Jeong & Lee, 2008). Scaffolding can be defined as any kind of support that facilitates students’ participation or acquisition of skills or knowledge during a task or activity which, otherwise, they could not have completed or acquired on their own (Belland, 2010; Hannafin, Land, & Oliver, 1999; Wood, Bruner, & Ross, 1976). Therefore, the design of scaffolds is based on the identification of problematic areas that impede learners from performing a given task independently (Lepper, Drake, & O’Donnell-Johnson, 1997).

In CSCA, many instructional scaffolds have been designed and integrated in web-based systems using graphical representations in the form of diagrams formed by nodes and links, tables and visualisations, or in a more text-based representation in the form of hints, prompts or scripts. Such scaffolds are designed to facilitate and orchestrate diverse roles, interaction patterns and activities of students at the individual and group level (Kirschner et al., 2003; Noroozi et al., 2012; Scheuer et al., 2010) and could have been designed as first-order or second-order scaffolding. In the first case, the scaffolds are designed to stimulate students’ argumentative discourse activities for acquiring domain-specific knowledge within a specific domain (e.g. Dutch labour law), or learning complex skills (e.g. collaborative learning) within the domain being taught (e.g. patient care for professional medical practice). In the second case, the scaffolds are designed for acquiring argumentation competence such that students are able to handle equivalent tasks themselves and continue learning in the future (van Merriënboer & Kirschner, 2012) in the same or similar domain. Nevertheless, it is not clear what the effects of first-order and second-order scaffolding on argumentation competence and domain-specific knowledge acquisition are, or how acquiring argumentation competence influences the acquisition of domain-specific knowledge.

1.4 Educational level, communication form and group size

Scientific research indicated that different variables such as educational level of the participants, communication form used and group size (Noroozi et al., 2012; Rapanta et al., 2013) could influence the outcomes of CSCA. However, such variables were not the main interest in those studies even though they may play a role and thus influence the learning outcomes of CSCA.

Regarding the educational level, we focused on HE and SE as our interest lies in these levels.

1.4.1 Educational level (HE and SE)

There is no simple definition of higher education. The Association des États Généraux des Étudiants de l’Europe (AEGEE) indicates that the international definition of HE, tertiary (post-school) education, divides HE into two parts, namely Type A (Higher Education) and Type B (Further Education). The definition provided by the AEGEE is as follows:

It will have a theoretical underpinning, it will be at a level which would qualify someone to work in a professional field and it will usually be taught in an environment which also includes advanced research activity. Shortly, higher education mainly and generally means university level education … Further education generally includes post graduate studies in where you can gain your Master and Doctorate degrees.¹

Moreover, HE is more abstract, theoretical, and demands analytical skills and asking questions. Students are expected to take learning decisions, and carry out significant unsupervised work on which they receive less substantial feedback (Macdonald, 2000). Thus, HE is more about the ‘why’.

Based on the International Standard Classification of Education (ISCED),² SE can be defined as education typically designed to prepare students for tertiary education, or provide skills relevant to employment, or both. Instruction is more varied, specialised and in-depth than programmes at ISCED level 2. Programmes are more differentiated, with an increased range of options and streams available. In contrast to HE, SE tends to be more concrete and practical, learning decisions are barely left to the students, the work is mainly supervised and students receive more substantial feedback (Macdonald, 2000). Thus, SE typically focuses on the ‘how’. The latter suggests that students in HE and SE not only differ substantially in the way they perform self-regulated learning and construct knowledge, but also in the level of complexity and cognitive workload required for their respective tasks. Therefore, scaffolds should consider the educational level of the target audience in their design such that they provide task support to students rather than cognitive overload.

1.4.2 Communication form

Regarding the communication form, asynchronous communication provides time to reflect and better analyse information (Veerman, Andriessen, & Kanselaar, 2000); time to read assignments and to prepare for deliberations that is necessary to generate complex discussions (Dysthe, 2002; Salmon, 2002). Yet, asynchronous communication presents non-serial messages, time lag between messages, and requires participants to be aware of the thread (Khine, Yeap, & Chin Lok, 2003). In contrast, synchronous communication allows work on a common shared artefact which facilitates a higher degree of elaboration and construction of arguments (de Vries, Lund, & Baker, 2002; Janssen, Erkens, & Kanselaar, 2006), facilitates higher-order thinking and discussion (Ravenscroft, McAlister, & Baur, 2006) and stimulates conceptual development (Ravenscroft, Wegerif, & Hartley, 2007). Hence, the design of scaffolds should take into account the characteristics of the task at stake as they may affect the learning outcomes.

1.4.3 Group size

Finally, regarding group size, the review of Noroozi et al. (2012) indicates that students are typically grouped in dyads, triads and larger groups, yet the reasoning behind the group size setting and the effects it entails are unclear. According to previous research, students in groups learn more than individuals (Dochy, Segers, Van den Bossche, & Gijbels, 2003). In contrast, working in groups may reduce team performance due to socio-psychological effects such as social loafing, e.g. free-riding and the sucker effect (Salomon & Globerson, 1989). Therefore, the size of groups may improve or reduce the learning outcomes.

1.5 Research questions

The aforementioned paucity in the literature drives this review in the form of the following research questions (RQ):

1. What are the effects of first-order and second-order argument-scaffolding on the elements of argumentation competence and domain-specific knowledge acquisition in HE and SE, and how does one way of scaffolding influence the other?

2. Which argumentation competence components (students’ knowledge, behaviour and attitude toward argumentation) have been considered for the provision of first-order and second-order argument-scaffolding in HE and SE?

3. What is the communication form used during the provision of first-order and second-order argument-scaffolding in HE and SE?

4. What is the group size used for the provision of argument-scaffolding in HE and SE?

2 Methodology

2.1 Development of a search strategy

To identify relevant literature a systematic search strategy was executed between December 2014 and February 2015 in the bibliographic databases Web of Science, Scopus, PsycINFO and ERIC. Inclusion criteria were defined to limit the scope and ensure the quality of the literature. First, a set of concepts related to CSCA was defined, namely learning, argumentation, collaboration and computer support (Noroozi et al., 2012), which was complemented with the concepts scaffolding and empirical study as the interest of this study is in empirical research on CSCA scaffolding. To increase the inclusion of relevant articles a list of similar terms was created for each of the concepts (see Table 1). Second, only articles written in the English language were considered since research on CSCA is commonly published in international journals written in English. Third, only articles from peer-reviewed journals were considered to guarantee a high level of quality. No time frame was defined. Table 1 shows the final search strategy.

Table 1. Search strategy.

Concept	Search term
Argumentation	argument* OR e-argumentation
	AND
Scaffolding	scaffold* OR support* OR moderat* OR script* OR guid* OR facilitat* OR affordance* OR peer-scaffolding
	AND
Computer	computer* OR virtual-environment OR online-environment* OR online OR e-discussion* OR web-based OR hypermedia OR technology-mediated OR technological-setting OR technology-enhanced
	AND
Collaboration	collaborat* OR cooperat* OR team* OR group* OR CSCL OR CABLE OR ABCSL
	AND
Learning	learn* OR practic* OR construct OR refle*
	AND
Empirical Study	empirical OR study OR experiment* OR observation* OR behavioral-assessment OR qualitative* OR quantitative* OR practical
	AND
Other	Document Type = peer-reviewed article AND Language = english

*Wildcard: represents zero or more characters.

2.2 Identification of relevant articles

Relevant articles were identified using a systematic set of steps. First, the titles and abstracts of articles matching the search criteria were read and checked against pre-determined criteria for eligibility and relevance. Articles had to focus on computer-supported/assisted/based argumentation, address educational purposes, investigate argument-scaffolding, should not be focused on mere collaborative learning (i.e. argumentation was not used to resolve differences of opinion collectively) and were not of a conceptual or review nature. In case of doubt the article was carried forward to the next step. The inter-rater agreement of two coders (i.e. the first and second author) was calculated by randomly selecting 10% of the articles. To assure reliability of the coding process, coding rubrics were created and the second author was trained on the rubrics and the process. Then, the first author and the coder independently coded 10% of the data. Discrepancies were resolved through discussion until agreement was reached on how to resolve them. Afterwards, the first author coded the remaining data.

Then, the methodology section of the articles matching the criteria was read and labelled as experimental, quasi-experimental, non-experimental or other. In this review, an experimental study has a pre- and post-test design, a control group and at least one treatment group, random assignment of study participants to groups (i.e. comparable groups) and random assignment of treatment to groups. A quasi-experimental study has a pre- and post-test design and two or more comparable groups, or assurances are provided to guarantee that the groups are comparable. A non-experimental study lacks one or more design elements of a quasi-experimental study, is a qualitative study or a study where the researcher starts from the effect/outcome of an observed phenomenon and attempts to determine what caused it (Kumar, 2011). The coding procedure was similar to the one used before for coding for the relevance of articles but was conducted by the main author. To assure reliability of the coding process the following actions were conducted: creating coding rubrics, defining coding process, coding 10% of randomly selected data, adjusting rubrics with further coding criteria after consultation of the co-authors and adding examples to facilitate the resolution of discrepancies. Afterwards, the first author coded the remaining data. Next, low internal-validity articles were discarded (i.e. non-experimental and other). Similarly, non-topic-related articles identified during this step were labelled as other and were discarded as well. Afterwards, the relevance of the articles was re-checked by reading their full text. This step was performed by one researcher. Then, articles not conducted in HE or SE were discarded. Elementary school was not considered in this review as the learning environment differs substantially from HE and SE. In addition, our particular interest lies in HE and SE. Finally, a mapping comprised of multiple codes related to the relevant variables was defined (see Table 2). The study’s intention was obtained from the research questions, the educational goals, the research goals or from the article’s text. The data extraction and coding were conducted using coding rubrics, with review guidelines containing definitions and hints for applying the codes. Articles not considering any of the variables were discarded. To assure reliability of the data extraction and coding process, the extraction and coding were conducted by the main author following the same coding procedure used for the coding of the study design. First creating coding rubrics, defining coding process, coding four randomly selected articles, adjusting rubrics with further coding criteria after consultation of the co-authors and adding examples to facilitate the resolution of discrepancies. Afterwards, the first author coded the remaining data. The outcome of the process was a systematic map.

Table 2. Independent variables.

Name	Description	Example
1st order scaffolding	The argumentation-scaffolds are designed to stimulate students’ argumentative discourse activities for learning complex skills, or acquiring domain-specific knowledge within a specific domain.	‘… to guide and support students’ argumentative interactions … with collaboration scripts to foster students’ meaningful science learning and retention’
2nd order scaffolding	The argumentation-scaffolds are designed for acquiring argumentation competence such that students are able to handle equivalent tasks and continue learning in the future in the same and similar domains.	‘… the system’s impact on their (students’) argumentation ability’
1st & 2nd order scaffolding	The argumentation-scaffolds are designed to stimulate students’ argumentative discourse activities for learning complex skills, acquiring domain-specific knowledge and for learning argumentation competence.	‘… explore the extent to which … micro-script influences domain-specific and domain-general knowledge’

2.3 Results of the systematic search

The total number of hits was 527, published in the years 1982–2015, including duplicates (214) and book chapters or books (3), or 310 relevant records. Table 3 shows the hits per database, while Table 4 shows the overlap between databases.

Table 3. Number of hits per database.

Database name	Number of hits
Web of Science	149
PsycINFO	104
Scopus	162
ERIC	112
	527

Table 4. Overlap between databases.

	WebOfScience	PsycINFO	Scopus	ERIC
WebOfScience	-	54	90	46
PsycInfo	-	-	52	41
Scopus	-	-	-	48
ERIC	-	-	-	-

Screening based on titles and abstracts resulted in a set of 84 relevant articles and a set of 58 articles which could not be identified as being relevant or not and thus they were carried forward to the next step. The inter-rater agreement on the relevance of articles, considering titles and abstracts, was substantial (Cohen’s Kappa = 0.731) according to Landis and Koch (1977), while the overall percentage agreement was 0.87. Discarded articles fell in the categories different topic (159), conceptual (4) and reviews (5). Next, the main author labelled and screened the articles based on their study design, namely experimental (10), quasi-experimental (18), non-experimental (77) and other (37). In cases of doubt, the second author was consulted. After this, the full text of articles was read by the main author, and articles were coded as relevant (20), not investigating argument-scaffolding (3), focusing on mere collaborative learning (3), elementary school (1) and pre- and in-service teachers (1). Finally, one article not considering any of the dependent variables was discarded. The final number of relevant articles is 19, published in the years 2005–14. Finally, the 19 articles were coded on the student design and variables by the main author supported by the other authors as described in the previous section. The outcome of the process was a systematic map.

3 Results

In this section the research questions are addressed.

3.1 R Q1 – What are the effects of first-order and second-order argument-scaffolding on the elements of argumentation competence and domain-specific knowledge acquisition in HE and SE, and how does one way of scaffolding influence the other?

The following numbers consider the multiple conditions that some studies had in HE (13) and SE (10). In HE 38% of the studies reported significant effects in the acquisition of domain-specific knowledge, 53% of the studies found significant effects on acquisition of argumentation knowledge, and 15% of the studies reported significant effects facilitating argumentation behaviour. Meanwhile, attitude towards argumentation was not considered at all (see Table 5). Successful argumentation-scaffolds regarding the acquisition of domain-specific knowledge are a collaborative argumentation script and a concept map (Marée, van Bruggen, & Jochems, 2013), group awareness and an argumentation script to annotate general argument types (ontology) (Tsovaltzi, Puhl, Judele, & Weinberger, 2014; Weinberger, Stegmann, & Fischer, 2010). Regarding acquisition of argumentation knowledge, Stegmann et al. (2007) reported that either a script for the construction of single arguments, a script for the construction of argumentation sequences, or both (additive effect) facilitated argumentation knowledge specific to the scaffold intention. The effect of the script for the construction of single arguments was later confirmed in another study (Stegmann et al., 2012). Meanwhile, Bouyias and Demetriadis (2012), Noroozi, Weinberger, et al. (2013) and Weinberger et al. (2010) reported significant effects on both argumentation knowledge and domain-specific knowledge by using peer-monitoring and a script for the construction of single arguments, a transactive discussion script and a script for the construction of single arguments in combination with the learning arrangement respectively. In HE, the results indicate that argumentation-scaffolds have been mostly successful in facilitating the acquisition of argumentation knowledge and domain-specific knowledge.

Table 5. Scaffold, order, intention, measures, effects, communication and group size of HE studies.

			Intention				Measures (pre- & post-test)				Effects (pre- & post-test)
Higher education			Arg				Arg				Arg
Reference	Scaffold	Order	K	B	A	DK	K	B	A	DK	K	B	A	DK	Com	Group size
Marée, van Bruggen, and Jochems (2013)	Collaborative argumentation script and concept map (ESCoM)	1st				1				1				1	sync	2
Weinberger, Marttunen, Laurinen, and Stegmann (2013)	Peer-critique collaboration script (conflict script)	1st				1				1					async	3
Bouyias and Demetriadis (2012)	Script for the construction of single arguments continuous (control)	both	1	1		1	1	1		1
	Script for the construction of single arguments fading		1	1		1	1	1		1		1			async	2
	Peer-monitoring w/script for the construction of single arguments continuous		1	1		1	1	1		1	1	1		1
Loll, Scheuer, McLaren, and Pinkwart (2010)	Argument representations (different ontologies) and collaborative vs. individual use	both	1	1		1	1			1					sync	3
Noroozi, Weinberger, Biemans, Mulder, and Chizari (2013)	Transactive discussion script	both	1	1		1	1	1		1	1			1	async
Stegmann, Weinberger, and Fischer (2007)	Script for the construction of single arguments	both	1			1	1			1	1
	Script for the construction of argumentation sequences		1			1	1			1	1				async	3
	Both		1			1	1			1	1
Stegmann, Wecker, Weinberger, and Fischer (2012)	Script for the construction of single arguments	both	1			1	1			1	1				async	3
Tsovaltzi, Puhl, Judele, and Weinberger (2014)	Group awareness and argumentation script to annotate general argument types (ontology)	both	1	1		1				1				1	async
Weinberger, Stegmann, and Fischer (2010)	Script for the construction of single arguments and the learning arrangement (individual and collaborative)	both	1			1	1			1	1			1	async	3 vs 1

With respect to SE, 50% of the studies reported significant effects in the acquisition of domain-specific knowledge. Significant effects on acquisition of argumentation knowledge were reported by 30% of the studies, while an additional 20% of the studies reported partial effects in only one of multiple indicators of argumentation knowledge measured, or within a specific subgroup of a treatment group. Successful argumentation-scaffolds in terms of domain-specific knowledge are the ‘conflict schema’ script and personally seeded discussions (Clark, D’Angelo, & Menekse, 2009), the structuredness of scripts, for the construction of single arguments and argumentative sequences (Kollar et al., 2007), and the use of external representations (i.e. argumentative diagram, argument list and matrix) (van Drie, van Boxtel, Jaspers, & Kanselaar, 2005). In terms of argumentation knowledge, successful-argumentation scaffolds are the scripts for the construction of evidence-based arguments (Belland, Glazewski, & Richardson, 2011). Moreover, Yeh and She (2010) and Chen and She (2012) reported significant effects on both argumentation knowledge and domain-specific knowledge by using a script to annotate general argument types using an ontology, and sentence openers. Effects on attitude towards argumentation were not reported at all. Table 6 shows that research on argumentation-scaffolds in SE has been mostly successful in facilitating the acquisition of domain-specific knowledge. Two studies, Kollar et al. (2007), and Yeh and She (2010), supported the acquisition of domain-specific knowledge and argumentation knowledge by: (1) formally explaining to students argumentation theory, e.g. Toulmin’s model of argumentation and/or Leitão’s argumentative sequences; (2) supporting the construction of arguments; and (3) facilitating argumentative discourse. In contrast, almost all the rest of the studies supported the acquisition of argumentation knowledge by (1) supporting the construction of arguments without providing argumentation theory (it was not reported), and (2) facilitating argumentative discourse. The exceptions were Weinberger, Marttunen, Laurinen, and Stegmann (2013), Weinberger and Fischer (2006), and Clark et al. (2009). Finally, Slof, Erkens, and Kirschner (2012) used ‘representational tools’ to facilitate the construction and adjustment of students’ representations.

Table 6. Scaffold, order, intention, measures, effects, communication and group size of SE studies.

			Intention				Measures (pre- & post-test)				Effects (pre- & post-test)
Secondary education			Arg				Arg				Arg
Reference	Scaffold	Order	K	B	A	DK	K	B	A	DK	K	B	A	DK	Com	Group size
Belland (2010)	scripts for construction of evidence-based arguments in PBL	2^nd	1				1				p				async	2-3
Belland, Glazewski, and Richardson (2011)	scripts for construction of evidence-based arguments in PBL	2^nd	1	1			1				1				async	2-3
Slof, Erkens, and Kirschner (2012)	representational tools	1^st				1				1					sync	3
Chen and She (2012)	script to annotate general argument types (ontology) and sentence openers	both	1			1	1			1	1			1	sync	ca. 6
Clark, D’Angelo, and Menekse (2009)	‘conflict schema’ script and personally seeded discussions	both		1	1	1				1				1	sync	3-5
Kollar, Fischer, and Slotta (2007)	structuredness of scripts - construction of single arguments and argumentative sequences	both	1	1		1	1	1		1				1	sync	2
Lund, Molinari, Sejourne, and Baker (2007)	argumentation diagram - to debate or to represent a debate	both		1		1	1				p				sync	2
van Drie, van Boxtel, Jaspers, and Kanselaar (2005)	external representations - argumentative diagram, argument list and matrix	both		1		1				1				1	sync	2
van Drie, van Boxtel, Erkens, and Kanselaar (2005)	external representations - argumentative diagram and argument list	both		1		1				1					sync	2
Yeh and She (2010)	script to annotate general argument types (ontology) and sentence openers	both	1	1		1	1			1	1			1	sync	ca. 5
Intention	of the study	Com	Communication
Arg	Argumentation	Async	Asynchronous
K	Knowledge	Sync	Synchronous
B	Behaviour	p	Partial result. Significant only in one element out of multiple measured
A	Argumentation attitude	1	Existence of
DK	Domain-specific knowledge

3.2 RQ2 – Which argumentation competence components (students’ knowledge, behaviour and attitude towards argumentation) have been considered for the provision of first-order and second-order argument-scaffolding in HE and SE?

The following numbers consider the multiple conditions that some studies had in HE (13) and SE (10). Pre- and post-test measurements on the components of argumentation competence were considered 21 times (HE = 14, SE = 7), more specifically: argumentation knowledge 16 times (HE = 10, SE = 6), argumentation behaviour 5 times (HE = 4, SE = 1) and attitude towards argumentation 0 times. In line with this, pre- and post-test measures on a single component of argumentation competence were exclusively focused on argumentation knowledge (HE = 6, SE = 5). Two components, argumentation knowledge and argumentation behaviour, were measured only five times (HE = 4, SE = 1).

3.3 RQ3 – What is the communication form used during the provision of first-order and second-order argument-scaffolding in HE and SE?

Table 5 shows that HE studies were typically conducted using asynchronous communication; this is clear when we consider that 77% of the studies employed this communication form. In contrast, as shown in Table 6, SE studies were commonly conducted using synchronous communication, that is, 80% of the studies used such a communication form. The aforementioned results show that HE (asynchronous) and SE (synchronous) studies differ substantially in the communication form they used.

3.4 RQ4 – What is the group size used for the provision of argument-scaffolding in HE and SE?

None of the studies explicitly provided a reason for using a given group size. Nevertheless, we present the group sizes found. Roughly half of the studies present an homogeneous group size in the form of dyads (6) and triads (5), while others present an heterogeneous group size (e.g. groups with different sizes, a combination of dyads and triads or groups ranging from three to six students). In general, HE and SE studies considered grouping students in dyads or triads. Nevertheless, studies in HE are stricter in the group size as they tended to enforce only a specific number of participants. In contrast, studies in SE presented more flexibility as the group size could vary among groups (see Table 5 and Table 6).

4 Discussion

Our findings contribute in at least two ways to the field of computer-supported collaborative argumentation. First, the results lead to a clearer idea of the effects (whether they were found or not) of first-order and second-order argument scaffolding in HE and SE. Second, they offer guidance to practitioners and researchers in the field of CSCA in terms of successful approaches of argument scaffolding and communication form in HE and SE.

The findings regarding the argument scaffold, intention, measures and effects are diverse, yet interesting patterns were found. An unanticipated finding was the general lack of consideration of attitude towards argumentation. Such a finding is not only inconsistent with our definition of argumentation competence, but also with previous research where it was demonstrated that students’ psychological, emotional, motivational and social barriers may affect argumentative discourse activities (Baumeister & Scher, 1988; Gilbert, 2004; Leith & Baumeister, 1996; Rourke & Kanuka, 2007). Similarly, studies rarely measured argumentation behaviour. The latter contrasts with previous research where individuals holding argumentation knowledge were not able to put their knowledge in practice during discourse (dialogical) or in a similar argumentation task (monological) (Kollar et al., 2007; Noroozi, Weinberger, et al., 2013; Stegmann et al., 2007, 2012). The aforementioned results suggest that argumentation competence has not been considered as a composite of diverse elements, such as students’ argumentation knowledge, argumentation behaviour and attitude towards argumentation, but rather as a single element of either argumentation knowledge or argumentation behaviour. Therefore, argumentation competence has been mostly considered as skills that individuals manifest during discourse (Rapanta et al., 2013), or as the knowledge on argumentation that students have prior to and after collaborative discourse activities (Noroozi, Weinberger, et al., 2013).

It was also found that studies in both HE and SE aim to obtain first- and second-order scaffolding effects. Therefore, such studies strive to develop both argumentation competence and domain-specific knowledge. In line with this, it was found that a couple of studies, Kollar et al. (2007) and Yeh and She (2010), supported the acquisition of argumentation knowledge and domain-specific knowledge by providing students with argumentation theory before engaging in argumentative discourse activities. According to research on the field, students constructing arguments in interaction with their learning partners acquire argumentation knowledge and domain-specific knowledge (Andriessen et al., 2003). Moreover, argumentative knowledge construction assumes that knowledge acquisition is related to the frequency with which students engage in specific discourse activities (Noroozi, Weinberger, et al., 2013; Stegmann et al., 2007, 2012; Weinberger & Fischer, 2006). Argumentative knowledge construction suggests that if students lack the theoretical knowledge underpinning the construction of arguments (Toulmin, 1958), the construction of argumentation sequences (Leitão, 2000) or the ability to ‘reason operating on the reasoning of the other’ (transactivity) (Teasley, 1997), then students may acquire such knowledge by ‘learning by doing’ in an scaffolded environment, i.e. arguing-to-learn (Andriessen et al., 2003; Jiménez-Aleixandre, 2002; von Aufschnaiter, Erduran, Osborne, & Simon, 2008; Zohar & Nemet, 2002). The latter suggests that students’ learning would be more mechanical, concrete and practical, thus it would be focused on the ‘how’. In contrast, students receiving argumentation theory before engaging in CSCA would internalise better the theory by practising. Such students would be aware of how to successfully construct knowledge individually (i.e. constructing single arguments), and how to co-construct knowledge collaboratively (i.e. constructing argumentation sequences and operating in a transactive way). Moreover, such practice may trigger the application of theoretical concepts in the problem space and the construction and internalisation of relations between the two (Palincsar, Anderson, & David, 1993). The latter suggests that students may be able to transfer and apply this knowledge to future problem cases in the same or similar contexts (Vygotsky, 1978). The second approach strives not only to foster conceptual understanding and learning, i.e. arguing-to-learn, but also foster learning of argumentation, i.e. learning-to-argue (Kelly, Druker, & Chen, 1998; Kuhn, 2005; Osborne, Erduran, & Simon, 2004; Reznitskaya et al., 2001). This learning seems to be more abstract, theoretical, analytical, and about knowledge construction, more about the ‘why’.

It was also hypothesised that if second-order scaffolding has first-order effects as well, research on argument-scaffolding should be centred on second-order scaffolding approaches. Nevertheless, this hypothesis cannot be confirmed nor refuted as most of the studies, in both HE and SE, have the intention to achieve both first- and second-order scaffolding.

Regarding the communication form, it was found that HE studies (asynchronous) differ substantially from SE studies (synchronous). The difference in communication can be explained if we consider the complexity and cognitive workload required for the tasks in each level. Asynchronous communication is a good approach if we consider that the complexity and cognitive workload of the task in question is high. Asynchronous communication provides time to reflect and better analyse information (Veerman et al., 2000), to read assignments (Dysthe, 2002; Salmon, 2002), to construct well-conceived and complex arguments, and it also allows equitable participation (Schellens & Valcke, 2006), and can also generate critical dimensions of learning and higher cognitive levels of knowledge construction (Andresen, 2009; Schellens & Valcke, 2006). Nevertheless, asynchronous communication presents some drawbacks such as non-serial messages and time lag between messages, and requires participants to be aware of the thread (Khine et al., 2003). In contrast, synchronous communication can deliver a higher degree of elaboration and construction of arguments as students can work in a shared workspace (de Vries et al., 2002; Janssen et al., 2006). Additionally, previous research indicated that synchronous communication supports higher-order thinking and discussion (Ravenscroft et al., 2006), and conceptual development (Ravenscroft et al., 2007). The previous arguments would imply that the design of scaffolds takes into account the context where they are to be used, and thus be tailored to such context.

Finally, the articles reviewed did not present the reason behind their choice of group size, typically dyads or triads. Previous research suggests that students learn more in groups than individually (Dochy et al., 2003), and that learning partners may be also beneficial for motivation and social skills (Johnson & Johnson, 1994). Yet, group size choice may affect collaboration and learning. Thus, a choice of small-size groups may not only avoid free-riding and the sucker influence, but also may facilitate participation, turn taking, discussion, common ground and consensus.

5 Conclusions and future research area

Our article’s main contribution is shedding light on the intention and presence, or not, of effects of first- and second-order argumentation-scaffolds in terms of argumentation knowledge, argumentation behaviour, attitude towards argumentation and domain-specific knowledge (presented in Table 5 and Table 6) by means of a systematic approach to select, code and cluster the studies, and their effects. The findings serve as guidelines for future researchers and practitioners that want to achieve specific effects with argumentation-scaffolds. The criteria to only consider articles with a (quasi-) experimental design substantially reduced the number of articles under consideration, yet such design provides certainty on the effects of argumentation-scaffolds in educational settings. Finally regarding future research, we suggest to broaden the spectrum of the dependent variables and to take all elements of argumentation competence, as well as domain-specific knowledge, into account. Also, future research should explore the extent to which the provision of theoretical knowledge on argumentation before engaging students in CSCA affects the acquisition of argumentation knowledge and domain-specific knowledge. Furthermore, future research should investigate if second-order scaffolding has first-order effects as well, since this hypothesis could neither be confirmed nor rejected in our study. Last but not least, the design of argumentation-scaffolds should consider the identification of problematic areas that impede learners from performing a given task independently, as well as the context where they are to be used, and thus be tailored to such contexts.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Notes on contributors

Anahuac Valero Haro

Anahuac Valero Haro is a PhD student at the Education and Learning Sciences (ELS) Chair Group, Wageningen University, the Netherlands. His research interests include e-learning and distance education, computer-supported collaborative learning and computer-supported collaborative argumentation to facilitate acquisition of domain-specific knowledge and argumentation competence.

Omid Noroozi

Omid Noroozi is an Assistant Professor at the Education and Learning Sciences (ELS) Chair Group, Wageningen University, the Netherlands. His research interests include collaborative learning, e-learning and distance education, computer-supported collaborative learning (CSCL), argumentative knowledge construction in CSCL, argumentation-based CSCL, CSCL scripts and transactivity.

Harm Biemans

Harm Biemans is an Associate Professor at the Education and Learning Sciences (ELS) Chair Group, Wageningen University, the Netherlands. His research interests concern competence development, competence-based education, educational psychology, educational development and evaluation, and computer-supported collaborative learning.

Martin Mulder

Martin Mulder is a Professor at the Education and Learning Sciences (ELS) Chair Group, Wageningen University, The Netherlands. His research interests include competence theory and research, human resource development, computer-supported collaborative learning, competence of entrepreneurs, competence of open innovation professionals, and interdisciplinary learning in the field of food quality management education.

Notes

1. Higher Education: What does it mean? Retrieved from https://www.wg.aegee.org/ewg/higheredu.htm

2. Statistical framework for organising information on education maintained by UNESCO (UNESCO Institute for Statistics, 2012).