Does implementation of competence-based education mediate the impact of team learning on student satisfaction?

ABSTRACT

Competence-based education (CBE) is an innovation in (vocational) education aimed at improving students’ competences. Little is known, however, about the processes leading to successful implementation of CBE and about its outcomes. This study investigates the effects that the level of CBE implementation has on student satisfaction (regarding the quality of education, guidance, and the development of interpersonal and general vocational skills) and to what extent CBE implementation mediates the relationship between teacher team learning activities and student satisfaction. To this end, data was gathered from 662 teachers belonging to 46 teacher teams in senior secondary vocational education in the Netherlands, and their students. Multilevel structural equation modelling revealed that teacher team learning was positively associated with te implementation of CBE. Furthermore, CBE had a positive effect on student satisfaction with quality of education, guidance, and development of interpersonal skills; however, no significant effect was found on student satisfaction with the development of general vocational skills. These results indicate that implementation of CBE has, to some degree, fulfilled its promise of better preparing students for their future workplace and that teacher team learning can support the further implementation of CBE.

Abbreviation: CBE – Competence-based education

KEYWORDS:

• Teacher teams
• vocational education
• student satisfaction
• team learning
• competence-based education (CBE)

Competence-based education (CBE) is an educational innovation that has been introduced in various countries around the world and which aims to improve the transition from the school environment to the workplace that students will encounter after graduation (Brockmann et al. 2008; Mulder, Weigel, and Collins 2007). The need for CBE arose because it was found that graduates often possessed sufficient knowledge but lacked the skills and attitudes needed to function properly in a workplace (Biemans, Nieuwenhuis, Poell, Mulder, and Wesselink, 2004). CBE aims to improve the transition to the workplace by focusing on the integration of knowledge, skills, and attitudes in education and by creating authentic learning environments. This should in turn lead to improved quality of education, more satisfied students, and lower dropout rates (Biemans et al. 2004). As an educational innovation, CBE calls for changes in assessment and the role of the teacher (Sturing et al. 2011).

Most of the research on CBE so far has focused on how CBE has been implemented and on the difficulties that teachers have encountered in doing so (Koenen, Dochy, and Berghmans 2015). The implementation of CBE has not gone smoothly and remains challenging for teachers due to the many changes in education that CBE calls for and CBE implementation differs strongly between educational programs (e.g. de Bruijn and Leeman 2011; Struyven and De Meyst 2010). However, despite its widespread adoption in European countries and beyond, there is still little knowledge about the extent to which CBE succeeds in achieving teaching students more skills and whether CBE leads to greater student satisfaction (Lassnigg 2017; Wesselink, Biemans, Gulikers, and Mulder 2017).

CBE calls for changes not only at the level of single lessons, which are typically taught by one teacher, but involves whole educational programs for which teams of teachers are responsible. This means that the contents of courses need to be aligned with one another and that teaching of vocational competences needs to be integrated in all courses. For instance, in a competence-based educational program, English classes would include learning the vocabulary needed for functioning in a particular vocation. A student learning to become a receptionist would thus learn the English words necessary to deal with customers.

Wijnia, Kunst, van Woerkom, and Poell (2016) established that team learning activities among teachers are associated with greater implementation of CBE, as rated by teachers providing the education. However, it was not investigated whether greater implementation of the principles of CBE would in turn lead to greater satisfaction among students. The aim of the current study is to unravel the relationships between teacher team learning, implementation of CBE, and student satisfaction. To this end, we used the data collected by Wijnia et al. (2016) on teacher team learning and CBE implementation and combined this data with data from a student questionnaire so that we were able to add student satisfaction scores regarding quality of education, guidance and skill development for each participating teacher team.

Theoretical framework

Competence-based education

Despite their widespread use in educational literature, confusion still exists concerning the precise meaning of both the terms competence and competence-based education (Cowan, Norman, and Coopamah 2005; Frank et al. 2010; Mulder, Weigel, and Collins 2007; Sturing et al. 2011; van der Klink, Boon, and Schlusmans 2007; Velde 1999; Westera 2001). Competences can generally be described as the skills that enable a person to successfully perform specific tasks in a work environment (Le Deist and Winterton 2005). However, the word skill does not fully cover the meaning of the term competence and contemporary interpretations of the concept of competence also include practical and theoretical knowledge, attitudes, and personal and social skills (Mulder, Weigel, and Collins 2007; van der Klink, Boon, and Schlusmans 2007).

The most important characteristic of CBE is that it takes these competences as the starting point for the development of an educational program. This means that in CBE, competences and vocational core problems, problems that can be encountered in the workplace, must be identified and included in the curriculum. CBE also calls for authenticity in education, meaning that the environment in which students learn competences must strongly resemble a real working environment. CBE has taken on several additional characteristics. CBE also calls for more personalised education (with a teacher functioning as an expert, coach, or mentor), for greater flexibility in educational programs, for self-assessment, and for more formative assessment. Students are also required to reflect on their own learning process and increasingly steer their own learning. Sturing et al. (2011) compiled a list of 10 principles for the design of CBE based on group discussions with Dutch vocational education teachers (Table 1).

Table 1. Design principles for competence-based education

1	Competences must be defined and these competences must form the basis of the educational program.
2	Vocational core problems, i.e. professional situations that students will encounter in the workplace, should be the organizing units for the design of the educational program.
3	Students should learn individually as well as in teams in concrete practical settings.
4	Knowledge, skills, and attitudes should be integrated in both learning and assessment.
5	Assessment must take place before, during, and after the learning process and must be both summative and formative.
6	Students must be challenged to reflect on their own learning.
7	The educational program must be designed in such a way that students increasingly direct their own learning.
8	The educational program must be flexible.
9	The teacher must adjust guidance to the learning needs of the students and has to function, at different points in time, as an expert, coach, or mentor.
10	In the educational program, attention must be paid to learning, career, and citizenship competences.

Due to the many adjustments that need to be made to teaching and educational programs, implementation of CBE is not all-or-nothing. Education can be less or more competence-based in nature depending on the extent to which the various design principles behind CBE have been implemented. For instance, the use of authentic settings can range from including no authentic situations, to including some classroom exercises related to practice, to a situation in which all education takes place in authentic settings (Wesselink, Biemans, Mulder, and Elsen 2007).

CBE and student outcomes

CBE has received ample academic interest. However, most studies on CBE have dealt with the problem of defining CBE, or with the challenges teachers encounter when implementing CBE (Biemans et al. 2009; Koenen, Dochy, and Berghmans 2015). For a large-scale educational innovation, that was implemented in the entirety of senior secondary vocational education in the Netherlands (as well as in many other countries), relatively few large-scale studies have been conducted on the antecedents and outcomes of CBE implementation. Moreover, only few studies have investigated whether students actually become more competent or more satisfied due to CBE (Lassnigg 2017). In the Netherlands, several large surveys on CBE were carried out but these surveys primarily focused on whether the new educational programs were in line with government guidelines (Wesselink et al. 2017). van den Berg and de Bruijn (2009) conducted four group discussions with school staff and found anecdotal evidence that implementation of CBE leads to more satisfied students, which was attributed to earlier exposure to workplace practice in the renewed educational programs. Furthermore, it was observed that students had improved ‘soft’ skills such as social skills that are needed to interact with colleagues and customers and the ability to plan work tasks ahead. However, these findings were based upon the impressions of teachers and no more rigorous scientific studies have been conducted to confirm these conclusions. van der Meijden, van den Berg, and Román (2013) evaluated the outcomes of CBE in the Netherlands by comparing the period before and after its introduction with the help of national data on graduation rates and several large-scale surveys that were conducted among students, recent graduates of vocational education, and teachers. They found that in the years after specific educational programs had introduced CBE, teachers as well as employers offering internships were more satisfied about the education that was offered. In contrast, student satisfaction remained stable over this period. However, the authors based their study on a comparison between the period before and after the official introduction of CBE and did not include a measurement of the degree to which CBE had been implemented in different educational programs. A study conducted in Indonesia found that intrinsic motivation among students was higher in schools that offered a higher level of CBE compared to schools that offered a lower level of CBE (Misbah et al. 2015) but no effects of CBE on satisfaction or obtained competences were measured. A study by van Dinther et al. (2014) revealed a correlation between student self-efficacy and the level of CBE, as measured by student perception of authenticity and teacher evaluation of obtained competences, quality of feedback, and the degree to which assessment was perceived to reflect real-life situations. Although these studies indicate that CBE might be associated with several positive outcomes concerning education, the evidence for these associations is still quite weak and no evidence has yet been found for an association with student satisfaction.

Team learning and CBE

Team learning in teacher teams has been found to facilitate the continuing professionalization of teachers as well as the creation and adaptation of new teaching methods and curricula (Bakkenes, Vermunt, and Wubbels 2010; Runhaar et al. 2013; Vangrieken et al. 2017). Teacher team learning is necessary to accelerate the implementation and adjustment of large educational innovations that transcend individual courses and impact a curriculum in its entirety. This is especially important for teacher teams in Vocational Education and Training (VET) that need to implement CBE, an educational innovation that requires changes at the level of the curriculum and alignment of the content of different courses (Oude Groote Beverborg, Sleegers, and van Veen 2015).

Team learning refers to a combination of processes within a team that generate changes or improvements for the team, its members, and the larger organization in which the team operates (Decuyper, Dochy, and Van den Bossche 2010). Two activities that are part of team learning have been included in the present study: information processing and information storage and retrieval. Information processing refers to the distribution of new information among team members, and the interpretation and discussion of this new information among team members (van den Bossche et al. 2011; van Woerkom and Croon, 2009). Information processing involves multiple team members communicating about the meaning they attach to information, negotiating different interpretations, and coming to a shared understanding of the material and how it will impact teaching (Havnes 2009; Uline, Tschannen-Moran, and Perez 2003; van Den Bossche et al. 2011). Information storage and retrieval refers to storing and retrieving the gathered information as well as the notes of the meetings in which the information was discussed (Decuyper, Dochy, and Van den Bossche 2010). Storage and retrieval are necessary for the effects of learning to persist over time as it allows teachers to go back to previously made agreements and decisions (Wilson, Goodman, and Cronin 2007).

Hypothesis

Based on the reasoning outlined above, we expect that team learning in teacher teams will contribute to the implementation of CBE because team learning enables teams to process, store and retrieve new information regarding the new teaching methods and curricula and can lead to increased shared understanding about CBE. We expect that these effects will take place both at the level of the teacher team and at the level of individual teachers. Moreover, based on the original intentions behind the introduction of CBE (Brockmann et al. 2008) and the evidence that has been provided so far concerning the outcomes of CBE (van den Berg and de Bruijn 2009; van der Meijden, van den Berg, and Román 2013), we expect a positive relationship between CBE implementation and the three student satisfaction constructs that are included in our research. As CBE is intended to improve the development of vocational skills in education, we expect that higher levels of CBE will be positively associated with greater student satisfaction with skill development. The introduction of CBE in the Netherlands has always had two goals, not only improving students’ vocational skills but also making education more appealing and thereby decreasing school dropouts (van der Meijden et al. 2009). We thus expect that CBE implementation will be associated with greater satisfaction with the quality of education. Furthermore, because CBE calls for the adjustment of guidance to the needs of students by making the educational program more flexible and giving teachers the roles of mentor and coach, we also expect that it will be associated with greater student satisfaction with guidance. As we expect that team learning increases CBE implementation and that CBE implementation in turn increases student satisfaction, we hypothesize that CBE mediates the relationship between team learning and student satisfaction. However, we expect a partial and not a full mediation (Figure 1). There are a number of ways in which team learning could improve education and student satisfaction that are unrelated to the principles behind CBE. It has been found that teams that engage in team learning improve their performance in various ways by becoming more effective, efficient or innovative (Chan, Lim, and Keasberry 2003; Chan, Pearson, and Entrekin 2003; van Woerkom and Croon 2009). It is also possible that team learning will directly increase satisfaction through changes in education that are unrelated to CBE such as providing more appealing lectures or greater use of multimedia in the classroom. Our full hypothesis is therefore as follows:

Hypothesis: Teacher team learning activities will have a positive relationship with student satisfaction concerning quality of education, guidance, and skills developed in education, and the implementation of CBE will partly mediate this relationship.

Figure 1. Conceptual model and hypothesized associations between team learning, CBE and student satisfaction at the team and individual level

Methods

Context

This study was conducted among teachers in senior secondary vocational education and training (SSVET) in the Netherlands. Students in this type of education are typically between 16 and 22 years old and enter this type of education after completing junior secondary education. Educational programs in SSVET last between one to four years (with a majority of students attending three or four-year programs) and educate students for a wide range of vocational jobs, from hairdresser to IT worker.

In the Netherlands, CBE was discussed in the 1990s and many educational innovations from that time period share characteristics with CBE and can be seen as predecessors. What is referred to as CBE, by both the government and schools, had a phased introduction since the mid-2000s. The Dutch government gradually introduced new guidelines for educational programs and schools could choose in which schoolyear they would adopt these new guidelines (van der Meijden, van den Berg, and Román 2013). CBE only became obligatory in 2012, two years before the start of this study.

The introduction of CBE can be characterized as top-down as well as bottom-up. The guidelines from the Dutch Ministry of Education clearly specified what should be learned by listing the competences that educational programs should address. However, no exact guidelines were given on how CBE should be implemented (Wesselink et al. 2017). Schools were given considerable leeway in deciding which curricular changes were to be made. School and teams of teachers could thus implement CBE in very different ways, for instance by choosing to teach competences mostly during internships or by focussing on integrating theory and practice in the curriculum (Struyven and De Meyst 2010).

Research design and participants

Two different data sources were combined in this study, a teacher survey and a national student survey. In 2014, managers and teachers in VET institutions in the Netherlands were approached to participate in a study on team learning and CBE in teacher teams. In total, 1650 teachers from 104 teams in 23 VET institutions received the invitation to participate in an online survey. Teams were made up of teachers that were responsible for one or more (related) educational programs. Of those 1650 teachers, 1147 participated in the questionnaire study (response rate = 69.51%).

For student satisfaction, data of the biennial national student satisfaction questionnaire, the so-called JOB-monitor (Jongeren Organisatie Beroepsonderwijs or Dutch Union of Vocational Students) (Wartenbergh-Cras, Bendig-Jacobs, and Brukx 2014) was used. This biennial survey is sent to all students participating in VET education in the Netherlands and is administered by the Dutch Union of Vocational Students and financed by the Dutch Ministry of Education. In 2014, the survey had a net response rate of 51% among all students in senior secondary vocational education (Crul et al. 2014). All teacher teams that participated in our survey were asked to share the results from the 2014 JOB survey in the form of aggregated student data. We obtained the mean scores per question for all the students involved in one of the educational programs a team is responsible for and who completed the questionnaire. Data on student satisfaction was obtained for 46 teacher teams (response rate = 44.23%).

The two datasets were matched by adding mean student scores to the teacher data for the teams for which student data was available. The remaining sample had 662 teachers from 46 teacher teams. On average, the teachers were 47.42 years old (SD = 10.81) which is slightly higher than the average age of 44 for teachers in 2014 (Statistics Netherlands 2017a). 324 Women (48.0%) and 318 men participated (20 participants did not report their gender), compared to 56.2% women in the population (Statistiek ArbeidsMarkt OnderwijsSectoren 2017). The majority of teachers in the sample completed at least higher professional education (76.0%) which made the sample representative for the whole Dutch teacher population in SSVET (Statistics Netherlands 2017b). The teacher teams represented all sectors of SSVET and the sample distribution between sectors was roughly representative of the national distribution between sectors (28.5% commerce and administration sector; 26.9% services and health-care sector; 36.6% technical sector, and 8.0% of the teams represented educational programs labelled as cross-sectoral).

Measures

Teacher team learning

Teacher team learning was measured with 15 items which were part of two existing scales: information processing and information storage and retrieval (Wijnia et al. 2016). A 5-point Likert scale ranging from 1 (never) to 5 (always) was used to assess all items. Information processing was measured using a scale with 10 items. An example item for this scale is: ‘In my team, we challenge each other to take new perspectives concerning our work’. This scale had a Cronbach’s alpha of .92. Information storage and retrieval was measured using five items. An example item for this scale is: ‘In my team, we refer to previous events or agreements and make use of stored information regarding these events or agreements’. This scale had a Cronbach’s alpha of .83. Results of the two-factor confirmatory factor analysis indicated an acceptable fit of the data: χ²(89) = 508.55, p < .001, RMSEA = .09, 90% CI [.080 – .096], TLI = .90, CFI = .92, SRMR = .07. While the value for RMSEA is slightly too high, all other fit indices are acceptable (L-t and Bentler 1999).

Implementation of competence-based education

The implementation of competence-based education was measured with 13 items using the scale developed by Wijnia et al. (2016). All items were measured on a 5-point scale (1 = never and 5 = always). Two example items are: ‘Learning activities take place in different, concrete, meaningful vocational situations’ and ‘Education is based on core tasks, working processes, and competences from the qualification profile’. Results of the confirmatory factor analysis indicated an acceptable fit of the data: χ²(65) = 309.40, p < .001, RMSEA = .08, 90% CI [.074 – .093], TLI = .91, CFI = .92, SRMR = .04. This scale had a Cronbach’s alpha of .91.

Student satisfaction

Student satisfaction was measured using three indicators: satisfaction with quality of education, satisfaction with guidance, and satisfaction with the development of vocational skills (Wartenbergh-Cras, Bendig-Jacobs, and Brukx 2014). For each teacher team, we received mean student satisfaction scores for all the students that were taught by the team and that had completed the satisfaction survey.

Satisfaction with quality of education was measured with 4 items. An example item for this scale is: ‘Are you satisfied with the mix of working independently and working in groups?’ Different 5-point Likert scales were used for these items (Very unsatisfied – very satisfied; Not at all – very much so; Very bad – very good). Principal component analysis revealed this scale to be one dimensional, explaining 55.38% of the variance. This scale had a Cronbach’s alpha of .89.

Student satisfaction with guidance was measured with four items, which made use of different 5-point Likert scales (Not at all – very much so; Very bad – very good; Very little – more than enough). An example item for this scale is: ‘Do you think you are offered sufficient guidance when you have studying problems?’ Principal component analysis revealed this scale to be one dimensional, explaining 75.15% of the total variance of student satisfaction with guidance. Cronbach’s alpha for this scale was .85.

Students’ satisfaction with the development of vocational skills was measured with 9 items (Table 2). An example item for this scale is: ‘In the educational program that you follow, do you learn enough about communicating?’ All items employed the same 5-point Likert scale (Far too little – more than enough). A principal component analysis on the aggregated data (N_teams = 46) revealed the presence of two factors, explaining 75.99% of total variance (Table 2). We therefore continued our analyses with two different constructs for satisfaction with learning skills. Two items had high cross-loadings (> .4) on both factors and were therefore excluded. The first factor refers to satisfaction with the development of interpersonal skills (communication, teamwork, problem solving, planning). Cronbach’s alpha for this scale was .90. The second factor refers to satisfaction with the development of general vocational skills (working independently, vocational preparation, evaluating one’s work). Cronbach’s alpha for this scale was .76.

Table 2. Results for principal component analyses for student satisfaction with development of vocational skills

	Factor Loadings^1
Item	1	2
In the educational program that you follow, do you learn enough about collaborating with others?	.994	−.220
In the educational program that you follow, do you learn enough about communicating?	.864	.106
In the educational program that you follow, do you learn enough about solving problems?	.807	.114
In the educational program that you follow, do you learn enough about planning and organizing?	.752	.241
In the educational program that you follow, do you learn enough for the job that you want to have later in life?	−.187	.904
In the educational program that you follow, do you learn to work independently?	.084	.800
Do you think that you learn enough at school?	.214	.646
In the educational program that you follow, do you learn to judge yourself and your own work?^2	.442	.635
In the educational program that you follow, do you learn to work according to plan? ^2	.492	.567

N = 46; ¹Oblimin rotation was used; ²Items were removed in the final factor;

Statistical analysis

We tested our hypotheses with Multilevel Structural Equation Modelling (MSEM) using Mplus 7.3 (Muthén and Muthén 2012) because our team-level constructs implementation of CBE and team learning were measured at the level of individual teachers who were nested within teacher teams. ICC(1), ICC(2), and Rwg*(J) values for information processing, storage and retrieval, and implementation of CBE were evaluated (LeBreton and Senter 2008; Molleman 2005). We used two indicators to evaluate the reliability across teams, ICC(1) and ICC(2) and one indicator to evaluate the reliability within teams, R*wg(J)). ICC(1) refers to the individual level variance that can be explained based on team membership and ICC(2) refers to the reliability of the group mean (Woehr et al. 2015). In general, ICC(1) values above .10 indicate that the team level should be taken into account and aggregation is advised when values are above .20 (Woehr et al. 2015). The ICC(1) values that we found ranged between .06 (implementation of competence-based education) and .083 (information processing). For ICC(2), values above .60 are acceptable but values above .80 are preferred (Woehr et al. 2015). The ICC(2) values that we found ranged between .605 (implementation of competence-based education) and .68 (information processing). For within-group reliability, R*wg(J) was calculated (LeBreton and Senter 2008). RwgJ* values above .51 demonstrate moderate agreement, and values above .71 demonstrate strong agreement. In our study, RwgJ* values ranged between .55 (information storage and retrieval) and .74 (information processing). Even though the values for ICC(1) were lower compared to the defined norm of .10, ICC(2) and Rwg*(J) were sufficiently high to justify aggregation of the teacher team data. The constructs that were measured by the student questionnaire were made available to us, only in the form of average scores per educational program for which a particular teacher team was responsible.

To evaluate model fit, multiple fit-indices were used. For the comparative fit index (CFI) (Bentler 1990) and Tucker-Lewis index (TLI) (Bentler 1990), values above .95 were considered good, and values above .90 were considered as acceptable. For the Root-mean-square error of approximation (RMSEA) (Steiger 1990) and Standardized Root-Mean-Square Residual (SRMR) values below .08 were interpreted as acceptable and below .05 were considered good (Kline 2010). Team size was used as a control variable in multilevel structural equation modelling, as team effectiveness has been found to depend on team size, with an optimal size around six members (Kayes, Kayes, and Kolb 2005) and both smaller and larger teams performing worse (Cohen and Bailey 1997). Very small teams are prone to problems with groupthink or exclusion of members (Kayes, Kayes, and Kolb 2005) whereas large groups become unmanageable and tend to split up into subgroups (Cohen and Bailey 1997).

Results

Descriptive statistics

Table 3 presents the means, standard deviations, correlations, and interrater reliability scores for all variables included in this study. Variables from one dataset (teacher survey or student survey) were mostly correlated with other variables from that same dataset and only occasionally with variables from the other dataset. A strong correlation was found between information processing and information storage and retrieval (r =.71, p < .01). Between data sets, implementation of CBE was positively associated with student satisfaction with quality of education (r =.13, p < .01), guidance (r =.16, p < .01), development of interpersonal skills (r =.14, p < .01), and development of general vocational skills (r =.12, p < .01). Team size, added as a control variable in this study, was not significantly associated with any of the team learning activities but was negatively associated with satisfaction with quality of education (r = −.14, p < .01) and satisfaction with guidance (r = −.08, p < .05).

Table 3. Descriptive statistics, correlations and interrater reliability indicators for all variables used in this study

	M	SD	1.	2.	3.	4.	5.	6.	7.	8.
1. Information processing	3.02	.67	1.00
2. Information storage and retrieval	3.12	.72	.71**	1.00
3. Implementation of Competence Based Education	3.37	.64	.57**	.46**	1.00
4. Student Satisfaction – Quality of Education^1	3.63	.18	.03	−.03	.13**	1.00
5. Student Satisfaction – Guidance^1	3.64	.17	.04	−.00	.16**	.85**	1.00
6. Student Satisfaction – Interpersonal Skills^1	3.74	.21	−.01	−.06	.14**	.69**	.73**	1.00
7. Student Satisfaction – General Vocational Skills^1	3.74	.22	.03	−.07	.12**	.64**	.73**	.66**	1.00
8. Team size	23.9	8.3	−.04	−.03	−.05	−.14**	−.08*	−.05	.053	1.00
ICC(1)			.083	.069	.060	-	-	-	-	-
ICC(2)			.683	.639	.605	-	-	-	-	-
Mean Rwg*(J)			.74	.55	.73	-	-	-	-	-

N = 662; **p < .01, *p < .05; ¹We were able to retrieve the mean scores for student satisfaction per team

Multilevel structural equation modelling

The conceptual model demonstrated a good fit of the data: χ² (7) = 14.21, p = .05. RMSEA = .039, TLI = .95, CFI = .99, SRMR_within = .001, SRMR_between = .086. Figure 2 shows the unstandardized estimates for each path. At the individual level, our results show that both information processing (B = .413, p < .001) and storage and retrieval (B = .135, p < .01) were positively associated with CBE implementation. At the team level, in which aggregated team scores for information processing, storage and retrieval, and CBE implementation were used, a significant association was found between information processing and CBE implementation (B = .894, p < .01) but not between storage and retrieval and CBE implementation. A positive association was found between information processing and storage and retrieval (B = . 294, p < .001). In addition, we found positive relationships between implementation of CBE and satisfaction with quality of education (B = .942, p < .05), and satisfaction with guidance (B = .992, p < .05). A positive association was found between implementation of CBE and satisfaction with the development of interpersonal skills (B = 1.462, p < .05), but not between implementation of CBE and satisfaction with the development of general vocational skills. We found a significant negative association between information processing and satisfaction with developing interpersonal skills (B = 1.091, p < .05). All other direct effects between team learning activities and student satisfaction were also negative but not significant.

Figure 2. Empirical model reporting unstandardized effects. *** p < .001, ** p < .01, * p < .05 All effects between storage and retrieval and student satisfaction measures were not significant and are omitted for clarity

None of the CBE-mediated associations between team learning activities and student satisfaction were significant, although the indirect associations between information processing and satisfaction with guidance (B = .862, p = .051) and between information processing and satisfaction with the development of interpersonal skills (B = 1.306, p = .053) came close. Although we found several positive associations between CBE and student satisfaction, we can therefore not confirm our hypothesis that CBE plays a mediating role in the relationship between team learning activities and student satisfaction.

Conclusions and discussion

This study investigated to what extent the level of implementation of CBE mediates the relationship between teacher team learning and student satisfaction. To this end, survey data about team learning and CBE implementation were gathered among teachers and survey data about satisfaction with quality of education, guidance and skill development were obtained from the students who were taught by these teachers. We found that the team learning activities information processing and information storage and retrieval were positively associated with the implementation of CBE both at the individual and team level. Moreover, implementation of CBE (as perceived by the teachers) was positively related to student satisfaction regarding their development of interpersonal skills and to student satisfaction with quality of education and guidance, but unrelated to student satisfaction with developing general vocational skills. A direct significant negative association between information processing and satisfaction with the development of interpersonal skills was found. However, no mediating effect of CBE implementation on the relationship between team learning and student satisfaction was established. We therefore have to reject the main hypothesis of this study. Although it is unfortunate that we cannot conclusively state that teacher team learning leads to greater student satisfaction via CBE implementation, our model still produced many significant positive effects.

This study is, to our knowledge, one of the first empirical studies to reveal a positive association between the level of implementation of CBE and student satisfaction. Most notably, our study found that students are more satisfied with how they develop interpersonal skills, such as communicating and collaborating with co-workers, as their education becomes more competency-based in nature. These findings are in line with more anecdotal evidence indicating that CBE enhances the so-called ‘soft’ skills of students (van den Berg and de Bruijn 2009). The effect of implementing CBE on satisfaction with development of general vocational skills was not significant. This is surprising, as one of the original intentions behind CBE was to improve these skills in order to smoothen the transition to the labour market. A possible explanation is that, even before the introduction of CBE, Dutch senior secondary vocational education had classes in which students would learn vocational skills and all students would have to do internships during their studies; the connection between education and the labour market was already quite well developed (Onstenk 2005; Wesselink, Dekker-Groen, Biemans, and Mulder 2010). CBE was meant to further improve those skills but as these skills were already addressed to a large extent, the impact of CBE might have been too small to show a significant increase. Interpersonal skills such as communicating and collaborating were found to be lacking before the introduction of CBE (Biemans et al. 2004). The strong focus of CBE on learning these skills, which was absent in previous curricula likely explains why this effect is significantly positive.

The level of CBE implementation was also positively associated with student satisfaction with quality of education and guidance. Education that is more competence-based in nature is thus rated more positively by students in terms of quality and guidance the students receive. One characteristic of CBE is its focus on the different forms of guidance a teacher must offer which make education more personalized (design principle 9 in Table 1). The association between CBE implementation and greater satisfaction with quality of education is also an indication that CBE succeeds in making education more attractive to students. This has not been reported to date and these findings contrast those of van der Meijden, van den Berg, and Román (2013), who found no difference in satisfaction with education before and after the introduction of CBE.

As reported before in a study based on the same teacher data that we use in the current study (Wijnia et al. 2016), information processing was more strongly associated with CBE implementation than storage and retrieval of information, both at the individual and team level. It is understandable that information processing was most strongly associated with CBE implementation, as information processing encompasses all activities in which the acquired data is interpreted and discussed among team members, and these activities are more directly related to CBE implementation than storage and retrieval is (which is mainly used to establish agreements and to avoid possible future conflicts).

A negative direct association was found between information processing and satisfaction with development of interpersonal skills. Between information processing and the other student satisfaction measures, there were also negative, although non-significant, associations. Due to the cross-sectional nature of the current study, it is not possible to establish the direction of effects. A possible explanation for the negative associations can be found by interpreting the effect in the opposite way, low student evaluations (and bad course evaluations) cause teachers to invest more time in team learning activities. This could especially be the case for satisfaction with the development of interpersonal skills as these are likely to be more associated with a failure to implement CBE than satisfaction with quality or guidance. On the other hand, the effect between satisfaction with the development of interpersonal skills and information processing could be the only significant effect because its effect size is larger than those of the other effects.

Limitations and future research

There are several limitations to this study that could be addressed in future research. First, our sample of 46 teacher teams is relatively small and a larger number of teacher teams would give more power to our multilevel analyses. Two mediating effects that were found in this study approached significance and a larger sample may establish whether implementation of CBE really functions as a mediator in the relationship between team learning and student satisfaction with guidance and the development of interpersonal skills. Furthermore, the values we obtained for ICC(1), ICC(2) and Rwg*(J) were high enough for aggregation but still on the low side. In addition, the cross-sectional nature of our design makes it impossible to ascertain causal effects over time, for instance if teacher team learning at one point in time leads to more competence-based education and more satisfied students at a later point in time. Moreover, we received aggregated student data for this study and non-aggregated data would have enabled us to do additional studies.

Second, to measure our dependent variable regarding student satisfaction we made use of a student survey that was developed by a third party and that did not include items about all aspects of CBE as described in the framework of Sturing et al. (2011). For instance, the student survey did not include questions about citizenship competences or self-reflection. Inclusion of questions on satisfaction with all aspects of CBE would give a better picture of the impact of this educational innovation.

A third limitation is the possibility of social desirability in the answers of teachers concerning the implementation of CBE. It could be the case the teachers overestimated the level of CBE implementation. Apart from social desirability, the question remains whether teachers are the best judges of the education that they provide. By measuring CBE implementation via a student questionnaire, an alternative measure of the level of CBE implementation could be obtained that can be used to contrast or complement the teacher measure.

A fourth limitation is that the student questionnaire was distributed to students still actively participating in their studies. This means that some of the respondents will be in the first year of their studies. If some skills are developed only in later years, these students may not be fully aware of this yet. This study suggests that education that is more competence-based in nature will lead to more student satisfaction regarding their development of interpersonal skills. However, we cannot be sure that this increased satisfaction will truly smoothen the transition from school to work. To investigate this, it would be necessary to gather data from graduates who have recently entered the workforce and from their employers.

Implications for practice

CBE has been a much-discussed educational innovation and remains a challenge for many teachers. Although in the Netherlands, the implementation of CBE in vocational education and training became mandatory a few years ago, not all educational programs have implemented the CBE principles to the same degree (Wesselink et al. 2007). Based on the results of our study, we can recommend that VET institutions strive for education that is more competence-based in nature, as this is associated with greater student satisfaction with quality of education, guidance, and the development of interpersonal skills. As we found that team learning is associated with the implementation of CBE, it would be wise to facilitate team learning among teachers. Because teachers are often pressed for time for professional development (Poell, Valk, and van der Krogt 2014), reserving time for team learning activities during team meetings should be stimulated.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This research was supported by a grant from the Netherlands Organization for Scientific Research (NWO) (411-12-070).