A meta-analysis on the effect of formal peer learning approaches on course performance in higher education

Abstract

Peer learning is a distinct category of student support gaining popularity globally and in South Africa, although these are typically informal approaches which have not been sufficiently researched to examine their influence and impact. Some scholars have criticised the available research as patchy, inconsistent, and methodologically weak. Underpinned by the author’s interest in peer learning in South Africa, this meta-analysis examines the outcomes of formal peer learning approaches on-course performance. The authors identified 1645 articles from four search engines which were published between 2010 and 2021. Of these, only 37 articles written in English, with a control and comparison group, reported effect sizes, were included in the analysis as they met the specified inclusion criteria. The analysis was conducted using R (version 4.0.0) and the metafor package (version 2.4.0). Effect sizes and their variances were computed within each study to allow comparability across studies using the common effect size metric, Hedges’s g statistic. Our findings suggest a moderately strong relationship between peer learning approaches and course performance; however, its significance cannot be ignored. As such, universities should consider whether such moderate effects justify the money, time and effort invested in peer learning approaches. While this meta-analysis provides external validity, more experimental studies are required in South Africa to provide a more contextually robust evidence base to inform policy.

Keywords:

• peer learning
• supplemental instruction
• meta-analysis
• course performance

1. Introduction

Student success is arguably one of the most widely deliberated topics in higher education, with several studies showing that it remains a significant problem globally. Why is student progression proving to be such an intractable challenge? The problem is multifarious, but increasingly, studies have identified articulation gaps and student under-preparedness as contributing factors (see, Dhunpath & Subbaye, 2018; Dhunpath & Vithal, 2012; Lavhelani et al., 2020). In response, several higher education institutions have invested significant resources in student support programmes, which include tutoring, mentoring, supplemental instruction, peer-wellness and peer-learning (Dhunpath & Subbaye, 2018). In this article, given South Africa’s unsustainably low progression rates, we use a meta-analytic method to appraise the value and efficacy of peer-learning approaches and consider whether significant resource investments are warranted in relation to the available empirical evidence.

The problem of student success is widespread and prevalent even in developed countries. For instance, it is estimated that the average dropout in the Organisation for Economic Co-operation and Development (OECD) countries is approximately 30%, with the status quo being worse in Italy, the United States of America and New Zealand, where the rate is almost twice the regional average (European, 2015). Similar trends have also been observed in the BRICS countries (Brazil, Russia, India, China and South Africa), which constitute a grouping of emerging world economies (daCosta, Bispo, & Pereira, 2018; Elena, 2018). For instance, daCosta et al. (2018) report that Brazil’s average dropout rate is between 40–50%. Russia, which has one of the lowest dropout rates in both OECD and BRICS, on average, 22% of the students leave without completing a degree (Elena, 2018).

In South Africa, university progression is perhaps the most analysed topic in higher education, against a backdrop of low participation and degree completion (Council on Higher Education, & CHE, 2013), as depicted in Figure 1 below:

Figure 1. Throughput from Grade 1 to university, 2008 matric cohort. ref. CHE, Briefly Speaking, October 2017 (Figure redrawn from: Spaull (2016), ‘Important research inputs on #FeesMustFall’).

STUDENT SUCCESS IN THE SOUTH AFRICAN EDUCATION SYSTEM

Of those students who manage to enter higher education in South Africa, an estimated 50% leave before graduating (CHE, 2013). A broad range of factors that impact student success have been identified. These include demographic characteristics, previous academic achievement, financial aid, self-efficacy, and learning styles. Some scholars have suggested that most students leave university due to adjustment or environmental factors rather than intellectual difficulties (Tinto, 1993). The clash between home languages and mediums of instruction is equally influential in determining progression rates (van Rooy & Coetzee Van Rooy, 2015). While some of these factors are beyond the control of HEIs, Tinto (1993) has argued that academic support, especially for students from disadvantaged backgrounds, might be the key to closing the achievement gap. Tinto (2017) conceptualised the link between the academic environment and student persistence using three concepts: institutional experiences, academic integration, and social integration. Ever since the notion of social integration has gained traction, universities increasingly recognise the significance of “a process in which students develop a social sense of the university environment that results from positive peer interaction, friendship formation and student participation (Tinto, 1993) Consequently, more student-centric learning methods are being promoted to improve student retention. One such method is peer learning, an educational approach rooted in active learning theories (Ten Cate & Durning, 2007)

Researchers also trace peer learning to Vygotsky’s social development theory which emphasises the importance of social interaction for learning. In Vygotsky’s view, learning is a sociogenetic process where individuals gain mastery through interacting with adults or more capable peers (Nardo, 2021; Vygotsky, 1980). He argued that learning is leveraged through the Zone of proximal development (ZPD), i.e., the distance between what an individual can do on their own and what they can do with the help of others. At the same time, the presence of a more capable peer in a learning group can catalyse long-term cognitive growth. While working on their ZPD enables individuals to internalise new knowledge (so that they do not remain reliant on the support), they can also externalise on their own in future instances (Roth & Radford, 2010). It has been suggested that students can identify with peer leaders given the latter’s social proximity or cognitive congruence to their experiences (Schmidt & Moust, 1995). This is because peer leaders “share the same knowledge framework and familiar language”, enabling them to explain concepts at a level that students can easily understand (Loda et al., 2020). This thinking has influenced formal peer learning approaches such as supplemental instruction, peer-assisted learning, peer-assisted study sessions and peer-led team learning which have been used globally to improve course grades (D. R. Arendale, 2014; Dawson et al., 2014). In these programmes, the role of the peer leader is pivotal in modelling both learning behaviours and course content mastery, allowing students to experiment with these until they reach mastery (D. R. Arendale, 2014).

While peer learning interventions have a distinct role in higher education, their effects on student performance remain unclear (Dawson et al., 2014)^, especially in South Africa. This is largely because research on peer learning interventions is scarce and mostly restricted to descriptive and advocacy accounts (Paideya & Sookrajh, 2014) Consequently, the evidence that supports these interventions tends to be inadequately defined. This meta-analysis investigates the effect of peer learning interventions on course performance -during the last few years (2010–2021). The findings from the meta-analysis are then contextualised, given the authors’ interest in peer learning interventions in South Africa. While peer learning interventions are not necessarily new, they are now a distinct category of student support in South Africa. To this end, a review of research studies that have adopted evidence-based designs is needed to establish whether the value ascribed to these interventions is justified and warrants the time, effort, and money invested in this form of student support.

1.1. Characteristics of peer learning

Peer learning encompasses various approaches that range from the “traditional proctor model, where senior students provide tutoring for junior students, to models in which students at the same level of study assist each other with both course content and personal concerns” (Boud, 2001). Ten Cate and Durning (2007) identify three broad manifestations of peer learning characterised by the distance between the tutor and tutee, the group’s size and the formality of the process. With regard to distance, tutors and tutees may be at the same level of training, also called reciprocal teaching, or at different levels. One example of the latter is when postgraduate students tutor undergraduate students, which is called cross-level tutoring. The process is referred to as near-peer teaching when both the tutor and tutee are at the same level but differ by one or two years (e.g. when a final-year undergraduate student tutors students at the first-year level). The second distinction relates to the size of the group. Peer learning can happen in one-to-one groups or be applied to small groups of more than three participants. Lastly, peer learning can occur as part of a formal learning programme or informally, as peers work together to prepare for class assessments.

Different terminologies have been used in the literature, but perhaps the most well-known model is Supplemental Instruction (SI), which was developed in the United States of America in the 1970s through the work of Deanne Martin (D. R. Arendale, 2014; Dawson et al., 2014). After observing a sudden change in the student body in the USA and a concurrent sudden rise in attrition rates (D. Arendale, 2002), a need was identified for a programme that would enhance learning. SI focuses on high-risk courses and not ‘high-risk students; hence, no remedial stigma is attached to it. Variants of the SI model have been developed in other countries, and the most popular is Peer Assisted Study Sessions (PASS), a name that was adopted in many Australian universities as a form of SI (Dawson et al., 2014). Other types of peer learning include peer-led team learning (PLTL), which originated in a Chemistry Course at the City College campus in New York in the 1990s. PLTL began as the “Workshop Chemistry Project”, which drew on peer support in problem-solving for introductory chemistry courses (D. A. I. K. A. I. Gosser et al., 2001; D. K. Gosser, 2009). This programme has been supported through a grant from the American National Science Foundation, leading to its adoption in more than 100 American institutions (D. R. Arendale, 2014). Like SI, PLTL involves peer leaders who have successfully completed the course and are model students. Although peer assisted learning is a generic term that has been used to refer to different forms of peer learning approaches (D. R. Arendale, 2014), the term has been mainly used within medical education, and unlike SI, PASS and PLTL use both same level and near peers as facilitators. Nevertheless, despite differences in the origins of the different peer-learning approaches, they all share a similar description of collaborative learning (Vygotsky, 1980) and are believed to be somewhat homogeneous internationally regarding the distance, size of the group, and formality of the process (Dawson et al., 2014).

1.2. History of peer learning programmes in South Africa

Although peer learning models are not new, they started gaining attention in South Africa in the early 1990s. The impetus for this attention was the high attrition rates that scourged the country, mainly affecting black South African students (Zerger et al., 2006). To ameliorate the problem, the SI model was first piloted at the University of Port Elizabeth in 1993, followed by Rhodes University in 1995. This shift coincided with changes in the political front. There was a need to redress social inequalities, which had led to the systemic exclusion of the black populace under colonialism and apartheid (Badat, 2019) The increasing number of non-traditional students admitted to higher education forced many institutions to embark on educational reforms to improve retention and throughput. These students, most of whom were first-generation, needed support with the transition and enculturation process (Scott, 2017). Consequently, universities were looking for innovative ways of supporting these students.

In most cases, academic support initiatives developed organically in response to contextual needs in the different HEIs (Dhunpath & Subbaye, 2018). However, formal peer learning models were slow to catch up (Zerger et al., 2006). A review of the university websites in South Africa indicates that not many universities were offering these before 2000- although a supplemental instruction national office had been established at the National Mandela University.

In the early 2000s, scholars in academic development began to question the type and quality of support given to students. Several qualitative and conceptual studies published during this period indicate a growing concern with peer tutoring (B. J. Page et al., 2005; Fouche, 2007). However, these studies mainly provided historical accounts of peer tutoring interventions (B. J. Page et al., 2005), motivational studies or studies of students’ experiences (Thomen & Barnes, 2005) or descriptive accounts of the efficacy of these interventions. During the same period, student engagement became a buzzword in South Africa, while Vincent Tinto’s model of student persistence gained paradigmatic status in the academic development literature (Grayson, 2014). This led to the Council for Higher Education (CHE) inviting Tinto to a symposium in South Africa, where he offered several lectures on his theory. Ever since, peer learning and student engagement models have enjoyed a resurgence, with many universities adopting peer-mediated support policies (Cupido et al., 2022. This resurgence is also evidenced by the biennial conferences first hosted by Nelson Mandela University in 2015. Currently, South Africa is the only country in the world with two national centres for supplemental instruction, indicative of the increasing focus on peer learning interventions in South Africa. However, while there is a strong theoretical and epistemological base, this meta-analysis investigates whether the empirical evidence has matched this shift.

1.3. Empirical studies

As shown in Figure 2, more than 1645 studies have been conducted on formal peer learning programmes between 2010 and 2021, of which 63% (n = 1034) were done in the second half of the decade. Most of the studies on peer-assisted learning were conducted in medical disciplines, while supplemental instruction was more common in Engineering and peer-led team learning in Chemistry.

Figure 2. Number of research articles returned by year using the following search terms “supplemental instruction” “peer-assisted learning”, “peer-assisted study sessions” or “peer led team learning”.

Despite the optimism surrounding peer learning models, little is known regarding their impact on academic performance, at least in South Africa. This is because studies that measure the impact of these programmes continue to be limited, and where these are done, they often lack rigour. In addition, most of this research has been criticised for being patchy and inconclusive (Dawson et al., 2014; Zha et al., 2019), with relatively few control studies that evaluate the effect of peer learning models on academic performance. To illustrate this, a recent meta-analysis of research conducted between 1993 and 2017 on peer-led learning found only 28 studies that had used quasi-experimental designs to estimate the impact of different student support interventions (Zha et al., 2019). Another systematic review of peer-assisted study sessions and supplemental instruction found that only 29 articles had evaluated the impact of these peer learning approaches on academic performance (Dawson et al., 2014). As such, there is concern about whether short-term “experiments” can adequately inform policy and practice. This is particularly important when considering the time, effort and funding invested in these programmes.

2. Methods

This study uses a meta-analytic method to examine the impact of peer learning on student performance. A meta-analysis is a statistical technique used to combine and analyse the results of multiple studies on a particular topic (Borenstein et al., 2009). It involves quantitatively synthesising data from different studies to estimate the overall effect size of a particular intervention or treatment using a common metric (Rosenthal, 1991). In addition, the results of individual studies are weighted based on sample size, and statistical techniques are used to estimate the overall effect size and the degree of variation among the studies (Borenstein et al., 2009; Field & Gillett, 2010). In this way, studies included in a meta-analysis can be compared using a similar construct (Borenstein et al., 2009). This helps to enhance accuracy and affords authors the opportunity to settle controversies arising from conflicting studies.

Support programmes such as peer learning are now a staple in higher education. However, a key question that many university administrators have is whether these will result in tangible benefits to both students and the institutions at large. While several metrics can be used to measure effectiveness, we focus on-course performance in this study. Our review of studies published between 2010 and 2020 is guided by the following question.

• What is the effect of formal peer learning programmes on course performance?

We defined course performance as the i) final grade pass grade or mark the students received b in a particular course or ii) the difference in mean marks between the control and experimental groups. Given our interest in peer learning in South Africa, we also contextualise the emerging evidence for potential use in decision-making academic development programmes in South Africa

2.1. Study selection

We conducted a comprehensive search on different search engines such as ERIC, EBSCO, Google Scholar and Digital Dissertations. Our search terms included words such as “peer learning” OR “supplemental instruction” OR “peer-assisted learning” OR “peer learning team learning” OR “effect of peer instruction” OR “effect of supplemental instruction.” We also searched for existing systematic reviews on the topic and scrutinised the references in the resulting studies. This helped us locate some studies that we might have missed in our search using keywords.

2.2. Inclusion criteria

To be eligible for inclusion, studies had to be evaluative and focused on assessing the effect/impact of peer learning on-course performance. These studies also had to have a control or comparison group that did not receive the treatment and reported effect sizes. We also included studies written in the English language and focused on peer learning at the undergraduate level. Finally, we considered studies that were conducted between 2010 and 2021. The research studies were selected following the PRISMA statement (Page et al. 2021).

Using a four-stage screening process(Figure 3), we initially identified 1645 studies published in English between 2010 and 2021. Two of the investigators then examined whether these articles met the agreed-upon inclusion criteria. After de-duplication, 295 records were excluded, and the remaining 1350 were then screened using the inclusion criteria. This led to the exclusion of 1275 studies i) did not focus on higher education, ii) were qualitative studies, iii) did not have an English translation, or iv) focused on irrelevant topics. We then proceeded to read the full texts and concluded that 75 of these were potentially usable. However, thirty-eight (38) of the studies did not provide adequate information to compute the effect sizes. After the screening process, 37 studies from twelve countries were retained in the analysis (Table 1).

Table 1. Frequency distribution of study characteristics

		Freq	%
Year	2010–2015	24	64.9
	2016–2021	13	35.1
Publication type	Journals	30	81.1
	Dissertation	5	13.5
	Conference	2	5.4
Discipline	STEM	28	75.7
	Management	4	10.8
	Others	5	13.5
Country^a	Australia	2	5.4
	Canada	1	2.7
	Germany	1	2.7
	Hong Kong	1	2.7
	Iran	1	2.7
	Israel
	Pakistan	1	2.7
	South Africa	2	5.4
	South Korea	1	2.7
	Sweden	1	2.7
	UK	1	2.7
	USA	26	70.2
Tutor Grade ^a	Near peer	25	67.6
	Matched	3	8.1
	Postgraduate/Instructor	7	18.9
	Missing	2	5.4
Length of intervention ^a	>60 minutes	22	47.8
	≤60 minutes	10	52.2
	Missing	5	13.5
Duration of training	2-day workshop	21	56.8
	Extended	5	13.5
	Course credit	4	10.8
	None/not mentioned	7	18.9

Note: ^a missing value are present

Figure 3. Flow diagram for the search strategy using the PRISMA approach. .

2.3. Coding of studies

We designed a coding form to extract effect sizes and code features from the 37 studies. We selected five variables; publication type, sample size, length of intervention, country of origin, and discipline). To ensure accuracy, the selected studies were double-coded by two researchers. Inconsistencies in coding were reviewed until an agreement was reached. Inter-rater reliability was high in that the average percentage of agreement was 95.0%, and kappa coefficient was 0.82.

The majority (24, 64.9%) of the studies included in this meta-analysis were published between 2010 and 2015, with 81% being published in peer-reviewed journals, while 5 were dissertations. Only two studies were published in conference proceedings. Most of the studies were also in STEM disciplines (28, 75.7%). As expected, supplemental instruction (21, 56.8%) was the most common approach, followed by peer-led team learning (11, 29.7%)) and peer-assisted learning sessions (5, 13.5%). The sample sizes for these studies ranged from n = 56 to n = 11809.

Most peer-led sessions were≤60 minutes long, while SI and PLTL mainly used undergraduate near-peer tutors. PAL studies, on the other hand, used mainly matched peers. Training was compulsory for both PLTL and SI/PASS, with SI training sessions being mainly two days long, while for PLTL, it ranged from a few hours to extended training (weekly sessions) or credit-bearing courses.

Regarding the geographical distribution of the studies included in the meta-analysis, most were from North America, particularly the United States of America 25(67.6%). Australia and South Africa, which had only two studies each (5.4%), Asia (Iran, Israel and Hong Kong), and Europe (United Kingdom, Germany and Sweden) had three studies each (Figure 4).

Figure 4. Global study distribution.

3. Analysis

3.1. Effect size calculation and heterogeneity

Effect sizes and their variances were computed within each study to allow comparability across studies. We utilised a common effect size metric, Hedges’s g statistic. A positive effect size indicates comparatively better academic performance in the intervention group than in the control group. Put differently, an effect size of 0.1 reflects 1 out of 10 improvements in the standard deviation for students in the experimental group relative to the control group. Only one effect size was reported for each study. However, when there were multiple outcome measures or subgroups within a study, composite effect size and variance were calculated for each study (Borenstein et al., 2009) by aggregating the effect sizes and corresponding variances.

An initial statistical assessment of the effect sizes flagged two studies (g = 2.45 and 2.02) as outliers and were removed. We fitted a random-effects model and estimated the amount of heterogeneity ${\tau }^2$ using the restricted maximum-likelihood estimator (Viechtbauer, 2005). We also report the $Q$-test for heterogeneity (Cochran, 1954) and the $I^2$ statistic (Higgins and Thompson 2002). To assess the funnel plot symmetry, we used the a regression, ‘which uses the standard error of the observed outcomes as a predictor” test (Sterne & Egger, 2005). The analysis was carried out using R (version 4.0.0) and the metafor package (version 2.4.0).

4. Results

We included $k=37$ studies in the final analysis, of which 21 focused on supplemental instruction, 11 peer-led team learning, and four (4), peer-assisted learning. Given the differences in the nature of these programmes, we present the findings for each programme individually.

4.1. Supplemental instruction

We included 21 studies on SI/PASS (Figure 5), which all used quasi-experimental designs. Given their similarities, we chose to combine the analysis for these two interventions (Dawson et al., 2014). Out of these, 14 were from the USA, two (2) South Africa, two (2) were from Australia, while Canada, Hong Kong and Sweden had one (1) study each. The sample sizes ranged from 70 to 11,738, which was also the largest sample among all studies included in the meta-analysis. A key characteristic of the studies on SI was that all the peer leaders underwent a training programme over two days covering pedagogical, communication and facilitation skills. The peer leaders were also carefully chosen based on their excellent knowledge of the subject matter, successful completion of the SI course with an above-average grade (Malm et al., 2016; Terrion & Daoust, 2011). Successful implementation of the SI programme required a team consisting of a trained SI supervisor, the academic/instructor teaching on the course, the peer leader and the students. While the common format is to use undergraduate cross-year peer leaders, some studies showed that the peer leaders were postgraduate students (Harding et al., 2011) or faculty (Bryan, 2013).

Figure 5. Forest plot showing the observed effect sizes and the random-effects model estimates (SI/PASS).

Although most of the studies were in STEM disciplines, there were also a few that focused on psychology (Guarcello et al., 2017), business and accounting (Kwan-Ning & Downing, 2010 Mangold et al., 2021; Mitra et al., 2018; Price et al., 2012), College reading (Dalton, 2011) or multiple subjects (Bryan, 2013). Seven studies were published between 2010 and 2015, while the remaining nine (9) were published between 2016 and 2021.

The observed mean effect sizes all positive and ranged from $0.09$ to$1.09$ The estimated overall effect based on the random-effects model was moderate positive ($\mathrm{g}=0.40$; 95% CI: $0.30$ -$0.50$). The overall effect differed significantly from zero ($z=7.614$,$p<0.0001$). We also estimated effect sizes for studies that reported regular attendance at SI sessions. The overall effect for studies that reported regular attendance was ($\mathrm{g}=0.43$; 95% CI: $0.19$ -$0.67$), while for regular attendance it was ($\mathrm{g}=0.41$; 95% CI: $0.28$ -$0.54$). There was also a high level of heterogeneity (Q = 94.47, p < 0.0001, ${\hat{\tau }}^2=0.04$ I² = 81.97.

4.2. Peer led team learning (PLTL)

For PLTL, we included 11 studies (Figure 6) which were all conducted between 2010 and 2018. All these studies were from the USA and focused on STEM subjects, particularly Chemistry and Engineering. PLTL sessions were generally run in workshop form either as a lecture replacement (Lewis, 2011) or as an added part of the course (Snyder et al., 2015; Liou-Mark, Ghosh-Dastidar, Samaroo & Villatoro, 2018). Thus, in all the studies, PLTL was incorporated into the course, with some studies making it compulsory or giving students a penalty for non-attendance. Like the SI model, the PLTL programmes were highly prescriptive, requiring training of peer leaders as either credit-bearing courses (Lewis, 2010; Mitchell et al., 2012) or weekly meetings (Bénéteau et al., 2016; Loui et al., 2013). The peer leaders were chosen based on having successfully completed the course and their problem-solving, communication, interpersonal and facilitation skills. The average treatment effect for PLTL was ($\mathrm{g}=0.42$; 95% CI: $0.29$ -$0.56$) and ranged from 0.18 to 0.90. Heterogeneity statistics also revealed large variability across ES values (Q = 34.37, p < 0.0001, ${\hat{\tau }}^2=0.04$ I² = 80.37.

Figure 6. Forest plot showing the observed effect sizes and the random-effects model estimates (PLTL).

4.3. Peer-assisted learning (PAL)

Five studies which estimated the effect of PAL on-course performance were included. All these studies were in the medical field and were conducted between 2010 and 2015 in countries such as the United Kingdom (Batchelder et al., 2010), Iran (Turk et al., 2015), USA (Hughes, 2011), South Korea (Han et al., 2015) and Germany (Cremerius et al., 2021). PAL sessions were all conducted by peers at the same level and taking the same course. Unlike SI and PLTL, the PAL programmes were less prescriptive in nature and did not emphasise the training of PAL leaders but rather relied on the cognitive congruency (Han et al., 2015) between matched or same-level peers rather than near-year peers. Given that PAL sessions were conducted during lecture time, attendance was compulsory. PAL sessions had an average positive effect on course performance ($\mathrm{g}=0.35$; 95% CI: $0.05$ -$0.64$). There was also a high level of variability in the studies ($Q\left(4\right)=12.85$ $p<0.0001$, ${\hat{\tau }}^2=0.08$, $I^2=70.85$).

An initial assessment of the funnel plot and Egger’s regression test indicated asymmetry (results not shown). We thus used the “trim and fill method”, which imputes the missing values on one side of the funnel plot until there is symmetry around the new effect size. Both the funnel plot (in Figure 7) and regression test did not indicate funnel plot asymmetry ($p=0.2810$). Therefore, there was no evidence of publication bias.

Figure 7. Forest plot showing the observed effect sizes and the random-effects model estimates (PAL).

4.4. Moderator analysis

We further explored heterogeneity using study-level independent variables or moderators via meta-regression.³² Homogeneity tests were used to examine whether the variance in the true effect size was accounted for by sampling error alone or by one or more cross-study characteristics. Both Q statistics and an ${\tau }^2$ index were adopted to assess the extent of heterogeneity, as used for the overall effect in the earlier Section.

To explain the extra-heterogeneity in a meta-regression, we use the moderators from the data, including the year of publication, publication type, length of intervention, tutor grade, duration of the training, sample size and country (Table 2). For SI/PASS. the moderator analyses yielded statistically significant effect sizes for country, length of intervention and subject (STEM). For PLTL the results were statistically significant for length of intervention and tutor grade only. The fitted model significantly reduced the residual heterogeneity (${\hat{\tau }}^2$ = 0.016, p < 0.0001) and accounts for 40.7% gains in the amount of heterogeneity explained. The type of intervention (supplemental instruction vs peer-assisted; $p<0.01$) and (peer-led vs peer-assisted; $p<0.05$) is significantly related to the effectiveness of peer tutoring in improving academic performance. Since the residual heterogeneity test is still statistically significant (Q(30) = 74.56, $p<0.0001$), it suggests that there are still unknown, uncontrolled moderators that impact the effectiveness of the intervention.

Table 2. Meta-regression of the moderating effect of type of intervention

	SI/PASS			PLTL
	Estimate	SE	P-value	Estimate	SE	P-value
Country (USA)	0.4230	0.1767	0.0167*
Length>60 mins	−0.4326	0.2132	0.0425*	−0.3455	0.1005	0.0006**
Subject (STEM)	0.4077	0.1304	0.0018**
Tutor grade (graduate/instructor)				0.4221	0.1715	0.0138*

Note: *p < 0.05; **p < 0.01

4.5. Publication bias

Publication bias can occur when studies which have non-significant are not reported or due to location bias where studies from certain countries only are reported (Brender, 2006). Most of the studies reported in our meta-analysis were journal articles, followed by dissertations and then conference proceedings. Most of these studies were also from the United States of America, with all studies on PLTL coming from this country- which suggests that we have to deal with publication bias. In terms of statistical significance, 4 out 21 SI studies of the studies reported non-significant results. Only one out of eleven PLTL studies reported non-significant results, whilst 2 out of the five PAL studies had non-significant results.

To statistically assess the possibility of publication bias, we use the funnel plot (of our total sample of 37 studies, (See Figure 8). An initial assessment of the funnel plot and the Egger’s regression test indicated asymmetry (results not shown). We thus used the “trim and fill method”, which imputes the missing values on one side of the funnel plot until there is symmetry around the new effect size. Both the funnel plot (in Figure 7) and regression test indicated funnel plot symmetry (p = 0.7497). Therefore, there was no evidence of publication bias.

Figure 8. Funnel plot assessing evidence for publication bias.

5. Discussion

This study set out to examine the effect of peer learning on-course performance. Despite the optimism surrounding peer learning in the past few years and the increase in the number of studies, very few control studies have evaluated its effect on academic performance during this past decade. For instance, our findings indicate that only 37 studies employed experimental designs or reported sufficient information which could be used to calculate effect sizes. Moreover, most of the studies were conducted in the USA, with supplemental instruction being the most studied programme.

Often, meta-analyses on peer learning have focused on individual forms of peer learning models (Zhang & Maconochie, 2022) or combined the different forms to get a single effect size (Zha et al., 2019). While both approaches have their merits, they also have limitations. For instance, although the different peer learning models share a similar description of collaborative learning, our findings revealed that they are not homogeneous regarding the distance, size of the group, and formality of the process. For example, while SI (e.g. Altomare and Moreno-Gongora (2018; Guarcello et al., 2017) and PASS (Dancer et al., 2015; Price et al. (2012) often use near peer leaders, most of the PAL studies (e.g. Han et al., 2015; Turk et al. (2015) used matched peers as leaders. The group size for PAL sessions ranged from 6–12 students, while in some cases, for SI, the groups had as many as 60 students. Another difference was that while most SI and PASS leaders were trained for two days, the PLTL leaders were trained for longer periods and in some cases in a credit-bearing course. The training was not a significant component of the PAL sessions. These differences, therefore, justify the separate analyses of the different programmes.

All the 37 studies included in our metanalysis had positive effect sizes, indicating that the students exposed to peer learning had better academic outcomes relative to those exposed to the traditional lecture and/or tutorial methods. Specifically, the weighted effect size of peer-learning against no peer-learning was 0.40 for SI/PASS, 0.42 for PLTL and 0.39 PAL. These results are aligned with findings in other meta-analysis studies. For example, Zha et al. (2019), found a weighted effect size 0.36 of peer-led learning on academic achievement, taking into consideration the duration and task. In Zhang and Maconochie’s study, the effect of PAL on examination performance from 13 studies published up to July 2020 was 0.38. Our analysis, therefore, supports the view that peer learning approaches, in general, positively affect student learning. They are a valuable form of collaborative learning and an alternative teaching and learning mode which can potentially enhance the student learning experiences (Dawson et al., 2014; Loda et al., 2020). Moreover, augmenting peer learning approaches with traditional teaching methods will help instructors cater for students with different learning preferences (Zhang & Maconochie ., 2022). Hence, although the magnitude of the effect size is considered moderate, this effect is significant enough not to be ignored.

In general, most of these studies were in STEM courses, although SI studies had some variability covering subjects such as business management (Kwan-Ning & Downing, 2010), psychology (Guarcello et al., 2017) and accounting (Mangold et al., 2021). The PLTL studies were predominantly Chemistry and or engineering, most likely due to the initiative originating in a Chemistry course (D. K. Gosser, 2009). The PAL studies covered medical related subjects such as Anatomy (Han et al., 2015; Cremerius et al., 2019) or Physiology (Hughes, 2011). A review of the literature on peer learning in general also reveals that peer learning models have been used mainly in STEM courses. These courses require analytical thinking (D. K. Gosser, 2009) and traditionally have low completion rates in most institutions (Bengesai & Paideya, 2018; D. R. Arendale, 2014; Dawson et al., 2014). Noting the concept of “cognitive and social congruence”, first investigated by (Schmidt & Moust, 1995), we, therefore, advance the proposition peer learning might be more efficacious in particular disciplines such as those in the STEM disciplines. The question that warrants further investigation is whether active learning approaches, e.g. peer learning, can benefit students across different subjects. If so, HEIs have a social justice imperative to reform undergraduate education, ensuring that the curriculum accommodates this mode of support more prominently.

Moderator analyses also indicated that the duration of intervention influenced the differences in effect sizes among the studies for both SI/PASS and PLTL. Other significant moderators for SI/PASS include country and subject, while for PLTL, it was tutor grade. Although the duration of an intervention is considered an important characteristic of student engagement (Zha et al., 2019), there is little empirical evidence regarding how long peer learning sessions should be. Even in the studies reviewed in this meta-analysis, no justification was given for the duration of the intervention, although some, especially the PLTL (e.g. Reisel et al., 2014; Snyder et al., 2015) studies indicated that they followed the approved recommendations for the model (D. K. Gosser, 2009). Despite this lack of justification for the duration of intervention, our meta-analysis findings suggest that peer learning interventions that are longer than 60 minutes are more likely to be more effective than those shorter. This is most likely because shorter sessions limit the time for protracted problem-solving, especially in STEM subjects (D. K. Gosser, 2009; Tan et al., 2022). Further, peer leaders might not only fail to attend to all students’ needs in a shorter session but also fail to keep up with the full reinforcement of the lecture material. Of course, additional studies are needed to test specifically the length of intervention based on Peer Instruction and its effect on learning.

Regarding tutor grade, the finding that peer learning sessions facilitated by senior students (postgraduate) or course instructors are more effective than those facilitated by undergraduate cross-year or matched peers is contrary to the principles guiding SI/PASS (Arendale, 2014) as well as PLTL (D. K. Gosser, 2009) which favour the later. A plausible explanation is that postgraduate students and/or instructors are more experienced in facilitation and might also have deeper content knowledge. Thus, although the common practice remains to use cross-year and matched peers owing to the social and cognitive congruence (Loda et al., 2020; Schmidt & Moust, 1995), there might be a need for institutions which offer peer learning programmes to seriously consider the level and quality of professional development and training support offered to peer leaders (Clarence, 2016). This is especially important considering that the training that peer leaders get is often varied, with some models such as SI and PASS offering training over two days, while for PLTL, training was done weekly. Thus, there is a possibility that, in some cases, peer leaders might be under-supported (Clarence, 2016).

5.1. Contextualising the research evidence

Although our primary aim was to examine the impact of peer learning models on academic performance, we also sought to contextualise the findings to South Africa for several reasons. High attrition rates continue to plague the country, and universities consistently try to develop initiatives to improve persistence (Council on Higher Education, & CHE, 2013; Pocock, 2012). Peer learning models are currently seen as solutions to the problem of unsatisfactory persistence (Bengesai & Paideya, 2018; Sneyers & De Witte, 2018). However, there has been little empirical evidence to confirm their effectiveness. While the results from independent small-scale studies can provide valuable insights, some scepticism is required when interpreting the results. For instance, the results could be a chance occurrence and not necessarily generalisable to a broader population (Borenstein et al., 2009). However, pooled results from multiple studies can provide a generalised view of the state of knowledge, provided we interpret this evidence acknowledging the influence of contextual factors.

There is an apparent lack of impact studies on peer learning models from Africa in general and South Africa specifically. For a country with an established academic development history (McKenna, 2012; Scott, 2017) and two SI Centres, one would have expected more research that assesses the impact of these programmes. What could explain the status quo? As Scott (2017) notes: academic development initiatives such as peer-assisted learning or supplemental instruction remain peripheral to a university’s core teaching and research activities in South Africa. The marginal nature of academic development is reinforced by the fact that the scholarship of teaching and learning is underrated and not considered a legitimate area of scholarship (Leibowitz & Bozalek, 2016). This marginalised nature is well reflected in ad hoc staffing arrangements, with most academic support staff on short-term contracts (Paideya & Dhunpath, 2018). Moreover, the political and transformative agenda that influenced South Africa’s academic development resulted in an activist rather than conventional academic ethos (Scott, 2017). Consequently, most innovative work has not been researched or published, with much of the evidence informing academic development policy coming from institutional documents and conference proceedings (Paideya & Bengesai, 2017) These two factors perhaps explain the limited research on peer learning in South Africa, despite much commentary on its efficacy.

This raises the question of whether universities in South Africa should continue investing time and resources into these peer learning approaches, which continue to exist as interventions rather than core components of a transformed student-centric curriculum (Scott, 2017).²¹ Theoretically, peer learning approaches have potential in South Africa, given that they signal a shift from deficit approaches designed to address perceived “pathologies” in students through remediation to approaches that value students’ schemas (Coleman, 2016; Dhunpath & Vithal, 2012). Given the discriminatory nature of apartheid education, from which higher education is yet to recover (Scott, 2018) typical academic development interventions targeting at-risk students have tended to be racialised (CHE, 2013) – targeting the mainly Black South African students (Mayet, 2016). However, peer-based learning shifts the emphasis from high-risk students to identifying high-risk courses, thus removing the potential stigma attached to academic development courses.

6. Limitations

This study is not without limitations. First, we acknowledge the possibility of publication bias which has the potential to influence our findings. Although we used the funnel plot to detect publication bias, the possibility that studies with positive effect sizes are more likely to be published cannot be overlooked. Publication bias can result in the overestimation of the effect of the meta-analysis. Another second limitation relates to the search strategy. While we attempted to use various keywords related to peer learning, we might have missed some articles that might not have been indexed using our choice of keywords. Third, given that a meta-analytic method combines different studies, the pooled effect might conceal significant differences between and among studies. We acknowledge that this may potentially obscure true effects. Despite these limitations, the pooled results from our meta-analysis provide external validity of the efficacy of formal peer learning models on-course performance. We also recommend reading our findings as main effects and a quantitative summary of previous research rather than descriptions of individual studies.

7. Concluding comments

Several implications for South African higher education arise from this meta-analysis. First, the evidence from this meta-analysis suggests that peer learning interventions can improve course performance in high-risk courses. Although the effect size is moderate, it is not trivial. Many factors should be considered to determine whether an effect size should be considered large or small (Glass et al., 1981). For educational interventions, this would include implementation costs, time invested, feasibility, and the effect sizes from comparable interventions (Sneyers & De Witte, 2018) Thus, the question this meta-analysis raises for South African university administrators is whether a 39–42% (pooled results) or a 35 % improvement (based on the South African studies (Harding et al., 2011; Varghese et al., 2020) in course performance is worth the money, time and effort invested in peer learning interventions. This might require further investigations linking peer learning interventions to student retention and throughput. Such analyses will require longitudinal approaches exploring associations between peer learning and graduation outcomes. In general, longitudinal studies are limited not only in our sample but in the field as a whole, as the focus has largely been on cross-sectional explorations of immediate outcomes such as course grades.

A second implication relates to building research capacity among academic development staff involved in peer learning interventions. One strategy is to develop academic development research units that monitor and evaluate academic development interventions and support emerging practitioners. At the time of writing this article, the University of Free State had launched ELETSA, the Academic Advising Association of South Africa. It is hoped that formations such as these will contribute to the professionalisation of student support.

Thirdly, and from a theoretical perspective, our findings support the usefulness of collaborative learning (Vygotsky, 1980) and the notion of social congruence (Schmidt & Moust, 1995) in the teaching and learning environment. Thus, even in courses without formal peer learning support, course instructors can adopt some peer learning principles to optimise student learning experiences.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors

Notes on contributors

Annah Vimbai Bengesai

Annah Vimbai Bengesai, is the Head of Teaching and Learning at the University of KwaZulu-Natal. Her research interests are in social statistics, institutional research on student access and success, and the demography of education.

Lateef Babatunde Amusa

Lateef Babatunde Amusa, is a lecturer and researcher at the University of Ilorin. His research interests are in Biostatistics, public health, and machine learning.

Rubby Dhunpath

Rubby Dhunpath is the Director of Teaching and Learning at the University of KwaZulu-Natal