Goal Orientation, Deep Learning, and Sustainable Feedback in Higher Business Education

Abstract

Relations between and changeability of goal orientation and learning behavior have been studied in several domains and contexts. To alter the adopted goal orientation into a mastery orientation and increase a concomitant deep learning in international business students, a sustainable feedback intervention study was carried out. Sustainable feedback implies acknowledgment of students’ need to be actively involved in their own feedback process. First, relations between and changeability of the concepts found in previous research were validated. Second, the effects of the sustainable feedback intervention were analyzed. Although sustainable feedback helped mastery-oriented learners maintain deep learning, it did not directly influence their goal orientations.

Keywords:

• Deep and surface learning
• Goal orientation
• Sustainable feedback
• Problem-based learning
• Higher business education

1. Introduction

Students in higher education live and study in a rapidly changing and globalizing world that affects the content they study and the ways they study. Although the primary goal of higher education is the “production” of experts who are masters in their field (Fryer & Elliot, 2007), students often choose the path of least resistance. The production of experts is known as a mastery orientation with respect to learning and has been associated with deep learning, while the “least resistance” path is known as a performance orientation associated with more surface learning (Elliot & McGregor, 2001). In the field of international business, students need to make use of deep learning behavior to optimally develop problem-solving and critical thinking skills (Paul & Mukhopadhyay, 2005). Furthermore, the specific international business context characterized by cross-cultural aspects such as social and individual values and norms influencing business decisions, requires students to be able to develop new mindsets and ways of critical thinking (Aggarwal & Goodell, 2015). Ideally, higher education in international business should stimulate students to develop and maintain a mastery orientation to learning and to use a deep learning behavior or to change their existing performance orientation to one of mastery to stimulate deep learning.

The concept of goal orientation is based on a social-cognitive theory of achievement motivation that specifies the kinds of goals that direct achievement-related behaviors (Maehr & Zusho, 2009). Goal orientation research has its origin in a dichotomous framework distinguishing mastery and performance goals (Dweck & Leggett, 1988). When the orientation is toward truly understanding or mastering what is being taught or at least getting better at something, one speaks of a mastery orientation. An orientation toward simply obtaining a grade and/or outperforming others is a performance orientation (Elliott & Dweck, 1988). Elliot (1994) expanded the dichotomous framework by adding approach and avoidance motivations to the performance orientation. In other words, performance-approach orientations refer to demonstrating competence relative to others (i.e., being competent enough to pass a test), and performance-avoidance orientations refer to avoiding a demonstration of a lack of competence relative to others (i.e., not doing worse than classmates). A 2 × 2 framework was thus established by also adding approach and avoidance motivations to the mastery orientation (Elliot, 1999; Elliot & McGregor, 2001). Mastery-approach oriented students are interested in truly mastering an academic task; in contrast, mastery-avoidance oriented students are interested in avoiding a misunderstanding of the task.

In previous research, the different frameworks were all used, leading to difficulties in comparing the obtained results. Huang (2012) conducted a meta-analysis (i.e., N = 52,986 participants analyzed in 151 studies) to examine the discriminant and criterion-related validity of the achievement goal models (i.e., dichotomous, trichotomous, and 2 × 2 frameworks) in predicting academic achievement. He concluded that the 4-factor achievement goal model (i.e., 2 × 2) is the best model to gain an understanding of learning outcomes.

Research on goal orientation addresses questions about why students engage in academic work the way they do (Huang, 2012). According to DeShon and Gillespie (2005), studies of goal orientation focus on the choice of behavior in achievement situations. Performance orientation involves an interpersonal desire to demonstrate competence and/or to outperform others, while mastery orientation is an intrapersonal desire to enhance competence (Senko, Hulleman, & Harackiewicz, 2011). The underlying performance orientation related desires of competence demonstration or outperforming others did not exclude each other in previous research. However, in reality, students are often led by the desire to pass their exams instead of the desire to outperform others as a way of demonstrating their competence. It should be noted that students oriented toward mastery-avoidance are driven by an intrapersonal desire to master a task, but they are also afraid of failing. If one of the goals of higher education is to influence these choices in a particular direction, then it is imperative that the factors influencing students’ choices are understood. However, not only the influencing factors are of interest: The time period in which goal orientations can change is of interest as well. As it is known that the choice of behavior (i.e., goal orientation) is related to achievement situations, it is worthwhile knowing if and how goal orientations change over the course of carrying out a substantial task. Ideally, from a learning perspective, teachers prefer working with students with the aim of the students’ mastering knowledge and skills. But this ideal is not always shared by what some refer to as “calculating” students and teachers (Van Bijsterveldt, 2011), because they are often directed to gaining credits and achieving high graduation rates in a short time. In reality, many factors such as time and efficiency might cause a more performance-oriented approach.

From an international business education perspective the connection between goal orientation theory and cultural diversity is relevant. International business cooperation often takes places in culturally diverse teams. Nederveen Pieterse, van Knippenberg, and van Dierendonck (2013) hypothesized and confirmed that cultural diversity is more positively related to team performance with members with a higher mastery approach orientation. Furthermore, they found that cultural diversity is more positively related to team performance for teams with members lower in performance-avoidance. They suggested that inducing a high mastery-approach orientation and preventing a performance-avoidance orientation might help teams with cultural diversity.

Elliot and McGregor (2001) found that goal orientations led to different patterns of learning behavior. A distinction in such behaviors is found between surface learning and deep learning (Biggs, Kember, & Leung, 2001; Entwistle, 1991). The latter is characterized by strategies such as elaborating on ideas, thinking critically, and linking or integrating one concept with another. Deep learning directs a student’s attention toward comprehending what the author wants to say. It is associated with a willingness to understand and be engaged in meaningful learning. Surface learning, on the other hand, is characterized by strategies such as rote learning and reproduction of the learning materials and is associated with an economic way of being engaged in learning (Aharony, 2006; Biggs, 1987; Vanthournout, Coertjens, Gijbels, Donche, & Van Petegem, 2013). Deep learning behavior matches the need for international business students to become problem-solving, creative, life-long learners, so that they will be able to meet the demands of working in a rapidly changing, globalizing business environment. Critical competencies for business undergraduates entering the business working field are, for example, critical and analytical thinking and the ability to see the bigger picture (Azevedo, Apfelthaler, & Hurst, 2012). In addition to these specific business related competencies, international business students have to develop intercultural competencies and learn how to reflect on intercultural and international differences in order to be able to act in new cultural contexts (Aggarwal & Goodell, 2015). Deep learning is therefore essential because this behavior helps students to transfer constructed knowledge to new, culturally different, contexts.

Mastery goal orientations have been shown to trigger a deep-level, strategic processing of information, while performance approaches have been shown to trigger superficial, rote-level processing (Elliot, McGregor, & Gable, 1999; Covington, 2000; Elliot & McGregor, 2001). Just like the mastery orientation, deep learning is favored by educators because of the willingness of students to really understand the learning material (Aharony, 2006).

The learning behavior that students adopt can be influenced by both the learning context and the task (Biggs et al., 2001). This means that the discipline of study, such as international business, might affect the learning behavior that students’ display. It can be questioned whether the specific learning environment within a specific domain of study affects the adopted learning behavior or that students with a specific, preferred learning behavior enroll in a study program that suits them best (Smith & Miller, 2005). A study on the relation between discipline and learning behavior found that business students favor a surface learning approach for their study (Smith & Miller, 2005). This was debated by Pang, Ho, and Man (2009), because of the presented dichotomous view of deep and surface learning in the study of Smith and Miller. Pang et al. found that business students switched and transferred between learning behaviors, presumably due to the practical orientation of business education aimed at preparing students for commercial functions. Knowledge of both learning behaviors and goal orientations sheds light on what students are trying to achieve but also on why they are trying to do so (Cano & Berbén, 2014). As a mastery orientation and concomitant deep learning are desirable in education from a learning point of view, it is important for educators to know how to influence the goal orientation of students toward this orientation and approach. Specifically, 1st-year students have been found to display a “novice” learning profile, meaning low scores on both deep and surface learning (Gijbels, van de Watering, Dochy, & van den Bossche, 2005). This “novice” profile has been associated with low study success (Lindblom-Ylänne, 2003) and therefore it is worthwhile to investigate how 1st-year students can be guided to display a learning profile in accordance with the demands of their future working environment.

DeShon and Gillespie (2005) conceptualized goal orientation as “a label used to describe the pattern of cognition and action that results from pursuing a goal at a particular point in time in a specific achievement situation” (p. 1114). This could be interpreted to mean that a person is able to switch goal orientations over the course of working on a task, within for example, business education. This implies that goal orientations are not fixed but can change. Changes in goal orientation have been found by Winne, Muis, and Jamieson-Noel (2003), Muis and Edwards (2009), and Fryer and Elliot (2007). However, their findings did not always point in the same direction or the same interrelation between the goal orientations. Nor was the amount of variation the same for all types of goal orientation. Muis and Edwards (2009) found that mastery-approach orientations displayed the most variation, followed by the performance-approach, with performance-avoidance being most stable over the course of a task.

The causes of these changes and differences are not really clear. The researchers did not find solid evidence for the task itself being the cause of either change or stability, but there is some evidence that feedback might cause the variation. For example, in Winne et al. (2003), positive feedback resulted in a decrease of performance-avoidance orientation, and negative feedback resulted in a decrease of performance-approach orientation. Although most of the studies report group means, Fryer and Elliot (2007) examined changes in goal orientation at both the sample and the individual levels. At the sample level, they found a decrease in the mastery-approach and an increase in performance-avoidance between two exams, whereas performance-approach and mastery-avoidance goals remained stable. At the individual level, they found increases and decreases, respectively, in performance-approach and mastery-avoidance that canceled each other out when combined at the sample level. They did not, however, study the causes of these differences.

Senko et al. (2011) conceptualized that students might be able to switch during an academic period, starting with a mastery orientation and then switching to a performance orientation before starting to prepare for exams, when focusing on outperforming their peers. It is useful to gain more knowledge on the changeability of goal orientation during a relatively short period, because this knowledge might help educators to better guide their students toward exhibiting a mastery orientation. However, the switch Senko et al. conceptualized might not really be a switch, as Pintrich (2000) found that it is possible to simultaneously adopt multiple goals. Maehr and Zusho (2009) discussed that students with high levels of both mastery and performance goal orientations might be most successful because of the opportunity to select the most suitable approach in an achievement situation. In other words, the ideally valued mastery orientation is combined with the often observed performance orientation. Research into the switch between a goal approach and a goal avoidance behavior and the corresponding predictors is limited. Table 1 shows an overview of the characteristics of the four goal orientations. More research is necessary to better inform teachers why these orientations occur and how they as teachers can play a role in influencing the goal orientation to enhance deep learning.

TABLE 1 Goal Orientations and Differential Effects on Learning

	Approach	Avoidance
Performance	• Goal is to demonstrate ability compared to others	• Goal is to demonstrate ability compared to others
	• Extrinsic goal value/normative standards	• Extrinsic goal value/normative standards
	• Interested in competition, demonstrating their competence, and outperforming others	• Interested in competition, demonstrating their competence, and outperforming others
	• Use other students as points of comparison, rather than themselves	• Use other students as points of comparison, rather than themselves
	• Focus on outcome	• Focus on outcome
	• Interested in demonstrating higher competence than other students	• Interested in avoiding appearing incompetent or stupid
	• Seeking flattering information	• Avoiding unflattering judgments
	• Surface approach to learning	• Surface approach to learning
Mastery	• Goal is to truly understand or master the task at hand	• Goal is to truly understand or master the task at hand
	• Intrinsic goal value/personal standards	• Intrinsic goal value/personal standards
	• Interested in self-improvement	• Interested in self-improvement
	• Compare their current level of achievement to their own prior achievement	• Compare their current level of achievement to their own prior achievement
	• Interested in truly mastering an academic task	• Interested in avoiding misunderstanding the task
	• Seeking accurate information about ability	• Avoiding unflattering judgments
	• Focus on process	• Shifting from process to outcome
	• Deep approach to learning	• Deep approach to learning

1.1. Feedback as a Tool to Influence the Adoption of a Specific Goal Orientation and Learning Approach

Feedback is a powerful instrument to improve learning (Hattie & Timperley, 2007; Kluger & DeNisi, 1996). Hattie’s (2013) meta-study on the effect of feedback on learning shows an effect size of .75. The effect of feedback extends further than “just” learning for a task; it might also affect students’ development, reflection, and improvement of future work (Blair, Wybum-Powell, Goodwin, & Shields, 2014). Feedback directed at students’ engagement in the learning process—through dialogues between teachers and students or among peers—might increase their satisfaction with feedback and their ability to learn from and understand feedback. How students use feedback might be related to their goal orientation; Evans (2013) stresses interest in investigating the relation between goal orientations and the way feedback is used and interpreted. Winne et al. (2003) found a relation between feedback and goal orientations, but they did not find an increase of mastery-approach orientation. A possible reason might be in the kind of feedback provided in that study; students received what could be called unidirectional feedback on a task from their teacher. According to Boud and Molloy (2013), the most powerful use of feedback is not in the way Winne et al. implemented it; to be as powerful as possible, it is important to shift the focus of feedback from telling or providing feedback to sustainable feedback. Sustainable feedback means a shift from information merely transmitted to students to the acknowledgment of the need for students to be actively involved in their own learning and to be agents of their own change. In concrete terms, this means students asking for and seeking feedback. Students have to give meaning to feedback, for example, through discussions, before they can use it. According to Boud and Molloy, sustainable feedback focuses on the purpose of the feedback and not only on the learning outcomes. It stimulates students to seek and solicit feedback with who, what, where, when, and how questions, and it asks tutors and peers to provide performance information to the learner. It encourages students to articulate judgments (self-evaluation), and it has students compare internal and external judgments and decide how to meaningfully interpret these messages; the comparison of both types of judgment in relation to the standards has to be used to generate a plan for improved work. Also, sought and solicited feedback and the evaluative comparison should not lead to a formal judgment (summative assessment).

In feedback processes, it is important to take into account the position of the student in relation to other students. Dialogue between peers is an important feature of feedback (Nicol & Macfarlane-Dick, 2006). As collaboration between peers is part of the learning process in groups, the sustainable feedback should fit the learning objectives of the group. Advantages of feedback by peers is that peers have equal status and training, so their relationship is not disturbed by a hierarchical relationship, and the feedback is often more timely and immediate (Finn & Garner, 2011). Group work is commonly used as an educational approach in higher education to prepare students for their future working environment. The aims of utilizing group work as an educational approach are to construct knowledge as a result of collaboration between peers and to develop skills such as argument, conflict handling, and analysis. Tutors guiding this group work play a role in the process of constructing knowledge and skills (Chng, Yew, & Schmidt, 2011). Chng et al. (2011) found that the interpersonal qualities of the tutor have a significant influence on the learning process.

While the findings of Winne et al. (2003) made clear that feedback could be used to alter goal orientations, the suggested approach of Boud and Molloy (2013) might be more effective in stimulating a mastery-approach. The assumption is that when students actively seek feedback, instead of unidirectionally receiving tutor feedback, they themselves are in control and can give meaning to the feedback and discuss it on an equal level with their peers. They learn not to experience feedback as a threat or as an embarrassment but as a natural part of the learning process.

The question is whether sustainable feedback alters the adopted goal orientation of 1st-year bachelor students from a performance to a mastery orientation and thus also alters the learning behavior from surface learning to deep learning. To investigate this, first a replication study is carried out to validate previous findings in this specific Business Administration context with the following research questions:

• RQ1. What is the relation between goal orientation and learning behavior? It is expected that students reporting a mastery orientation also show deep learning, and students reporting a performance orientation also show surface learning. In this study these findings will be validated.

• RQ2. Do goal orientation and learning behavior indeed change over time, and if so, in what direction? Previous findings support the changeability of goal orientation and learning behavior; in this study, these findings will be validated.

Secondly, the effect of sustainable feedback on goal orientation and learning behavior is investigated:

• RQ3. What are the effects of sustainable feedback from peers and tutors on goal orientation and learning behavior? It is expected that a mastery goal orientation and deep learning will manifest itself.

2. Method

2.1. Context

This experiment was conducted in a 1st-year higher education Bachelor of Business Administration course in the Netherlands. The academic year is divided into four periods of 8 weeks, and the experiment was carried out in the third period of the students’ 1st year. The students worked together in problem-based learning (PBL) groups consisting of 12 students, solving practical domain-specific problems, meeting twice a week. The learning environment is composed of group work and individual work. The scheduled workload in the third period is 15 European Credits (EC; 1 EC = 28.35 hours of study), of which 3 ECs are for PBL group work and 4 × 3 EC are for courses on subjects related to a practical problem. The students already gained PBL experience in the two previous teaching periods (i.e., in total one semester).

Each group was subsequently split into two smaller groups during the elaboration of the tasks. In the day-to-day execution of PBL, the instructions for tutors and students with guidelines for feedback were described in a PBL manual.

2.2. Participants

Participants were 105 1st-year students in Marketing (N = 105, 54 male, 51 female; M_age = 20.29; SD = 2.37; range: 17–30 years) divided among 12 PBL groups guided by seven tutors.

An overview of tutors, groups, and conditions is given in Table 2.

TABLE 2 The Distribution of Tutor Group Over Tutors and Experimental and Control Condition

Tutor	Experimental Condition	Control Condition
1	Groups 7 and 8
2	Groups 3 and 4
3	Groups 1 and 2
4		Groups 5 and 6
5	Groups 9 and 10
6		Group 12
7		Group 11

2.3. Design

To investigate the effect of sustainable feedback on goal orientations and learning behavior, an experimental pretest-posttest nonequivalent group design intervention study (Experimental: N = 62; Control group: N = 43) was carried out. Existing groups were randomly assigned to the conditions, taking into account that different groups were in the same condition (see Table 2). As a consequence, the numbers of students in the conditions are not equal.

2.4. Instruments

Two questionnaires were used, both having been used in previous research among undergraduate students.

• Goal orientation was measured using a validated translated version of the Achievement Goals Questionnaire (Elliot & McGregor, 2001), a 12-item measure that assesses learners’ orientation on a 7-point Likert scale. The questionnaire represents the 2 × 2 model (Huang, 2012).

• Learning behavior was measured using a validated translated version of the Revised Two-Factor Study Process Questionnaire (R-SPQ-2F; Biggs et al., 2001); a 20-item, 5-point Likert scale. This is one of the most commonly used questionnaires in higher education, in which learning behavior is described using the two factors, deep and surface learning (Baeten, Kyndt, Struyven, & Dochy, 2010).

Information about sex and age was registered at the beginning of the experiment.

2.5. Procedure

The procedure of the feedback intervention is presented in Table 3.

TABLE 3 Procedure of the Feedback Intervention

	Procedure of Feedback Intervention
Week 0	Tutors of the experimental PBL groups were given instructions on feedback.
Week 1: First meeting of the week	Students filled out questionnaires measuring self-efficacy and learning behavior.
	Students received a bundle of feedback forms.
	Students wrote down individual learning points based on their experience in PBL groups in Periods 1 and 2.
	Learning points were formulated as a question and related to aspects of collaboration in PBL groups.
Week 1: Second meeting of the week	Members of each group shared their feedback questions with their peers and tutor.
	Everyone was in possession of all the feedback questions.
Weeks 2, 3, and 4	The tutor stimulated and guided the students to seek feedback on their own feedback questions.
	All students wrote down their own judgments (evaluation) of their performance on the specific learning points (i.e., feedback questions).
	At the end of every meeting, the students had to write down the feedback they sought and the feedback message they received.
Week 4: Second meeting	An evaluative moment in which all students had to compare their own judgements with the external judgments.
	If a student was satisfied with the feedback, then it was possible to rephrase the feedback questions to be able to ask and seek feedback for other learning points from that point on.
Week 5: First meeting	All (rephrased) feedback questions were shared amongst the students.
Weeks 5, 6, 7, and 8	The tutor stimulated and guided the students to seek feedback on their own feedback questions.
	Students wrote down their own judgments (evaluation) of their performance on the specific learning points (i.e., feedback questions).
	At the end of every meeting, the students had to write down the feedback they sought and the feedback message they received.
Week 8: Second meeting	The feedback questions were evaluated within the PBL group.
	Students filled out the questionnaires on self-efficacy and learning behavior.
	There were no formal (summative) judgments of the feedback process.
Week 9	Submission of PBL assignment and examination of the supporting courses (written tests).

2.6. Data Analysis

Data analysis started with descriptive analyses to summarize the data. To check whether goal orientation and learning behavior were comparable among experimental and control conditions, an independent t-test was conducted at pretest. To check differences between PBL groups at pretest, a one-way ANOVA Bonferroni post hoc analysis was conducted. To answer RQ1 (i.e., relation between goal orientation and learning associations), a correlation analysis was conducted. A mixed-model analysis was used to assess the associations and directions between goal orientations and learning behavior over time. The mixed-model analysis is an expansion of a regression analysis; to take the two measurements per student into account, the data set was transformed into a vertical structure. RQ2 (i.e., changes in goal orientation and learning behavior) was answered conducting a paired t-test to investigate changes over time within the experimental and control groups. Additional to these mean-level analyses, an individual-level analysis was conducted by calculating the reliable change index (RCI; Zahra & Hedge, 2010). The RCI is calculated by dividing the difference in pretest and posttest scores by the standard error of the difference score. Individuals are characterized as showing either a significant increase, a significant decrease, or no significant change from pretest to posttest. RCI values smaller than −1.96 or larger than +1.96 are unlikely to occur by chance (Fryer & Elliot, 2007). The RCI calculator of Zahra (2010) was used. For RQ3 (i.e., the effect of the intervention on goal orientation and on learning approaches), a multiple regression analysis was conducted to determine the overall fit (variance explained) of the model, and the relative contribution of each of the predictors of the total variance was explained.

3. Results

Reliabilities of the scales (Cronbach’s alpha) are presented in Table 4. Overall, the Cronbach’s alphas (Nunnally, 1978) were questionable to good. One item of the performance-avoidance scale was removed to improve the internal consistency of the scale (i.e., “My fear of performing poorly in this class is often what motivates me”).

TABLE 4 Cronbach’s Alpha at Pretest and Posttest

Measurement	Pretest	Posttest	Items
Goal orientation
Performance-approach	.78	.87	3
Performance-avoidance	.67	.61	2
Mastery-approach	.78	.71	3
Mastery-avoidance	.63	.65	3
Learning approach
Deep learning	.79	.83	10
Surface learning	.68	.76	10

Mean scores, standard deviations for the group as a whole, and the experimental and control conditions are presented in Table 5.

TABLE 5 Means, Standard Deviations at Pretest and Posttest, Control, Experimental, and Total Groups

	Control		Experimental		Total
	M	SD	M	SD	M	SD
PAp Pre	3.85	0.97	3.41	1.60	3.59	1.39
PAp Post	4.21	1.18	3.68	1.47	3.90	1.38
PAv Pre	4.34	1.39	5.12	1.37	4.80	1.42
PAv Post	4.69	1.21	5.07	1.21	4.91	1.21
MAp Pre	4.91	1.14	5.28	1.12	5.13	1.14
MAp Post	4.98	0.85	4.99	1.06	4.99	0.98
MAv Pre	3.53	1.08	3.80	1.21	3.69	1.16
MAv Post	3.74	1.09	3.71	1.17	3.72	1.13
DL Pre	2.72	0.49	2.97	0.67	2.87	0.61
DL Post	2.70	0.57	3.00	0.59	2.88	0.60
SL Pre	2.57	0.44	2.73	0.62	2.67	0.59
SL Post	2.75	0.58	2.81	0.61	2.79	0.59

Note. Pre = pretest; Post = posttest; PAp = performance-approach; PAv = performance-avoidance; Map = mastery-approach; MAv = mastery-avoidance; DL = deep learning; SL = surface learning.

Preliminary, independent t-test analyses showed a significant difference on performance-avoidance (t = −2.867; df = 102, p < .05) and deep learning (t = −2.058; df = 101, p < .05). No significant differences between groups were found; however, the one-way ANOVA showed significant differences between tutors on performance-avoidance at pretest F(6, 97) = 2.293, p = .041, and Bonferroni post hoc analysis showed a significant difference between the highest scoring in the groups with Tutor 2 (M = 5.37; SD = 1.141) and the lowest scoring in the groups with Tutor 6 (M = 3.69; SD = 1.614; p = .037). These differences are taken into account in further analyses.

3.1. RQ1. What Is the Relation Between Goal Orientation and Learning Behavior?

To gain insight into the relations between goal orientation and learning behavior, correlations were calculated over the total group (Table 6). At pretest, all goal orientations, except performance-avoidance, were significant positively correlated with deep learning. A performance-approach had a significant negative correlation with surface learning. At posttest, a positive significant relation was found between a mastery-approach and deep learning, and a negative significant relation was found between a mastery-approach and surface learning.

TABLE 6 Correlations Between Goal Orientations and Learning Approaches at Pretest and Posttest

	Perfor-mance Approach Pretest	Perfor-mance-Avoidance Pretest	Mastery Approach Pretest	Mastery Avoidance Pretest	Perfor-mance Approach Posttest	Perfor-mance Avoidance Posttest	Mastery Approach Posttest	Mastery Avoidance Posttest
Deep learning pretest	.41**	.13	.50**	.32**
Surface learning pretest	−.25*	.17	−.06	.09
Deep learning posttest					.15	.16	.42**	.13
Surface learning posttest					−.09	−.05	−.29**	−.05

Note. *Correlation is significant at the .05 level (2-tailed). **Correlation is significant at the .01 level (2-tailed).

In the experimental condition, the mixed-model analysis showed that both a mastery-approach and a performance-approach related significantly positively with deep learning over the course of time. A mastery-approach and mastery-avoidance related significantly negatively with surface learning (see Table 7). In other words, while both a mastery-approach and a performance-approach orientation related positively to deep learning, both mastery orientations related negatively to surface learning.

TABLE 7 Significant Associations Based on Mixed-Model Analyses, Experimental Condition

		df	t
Mastery-approach	→Deep learning	120.993	6.141**
Performance-approach	→Deep learning	119.953	4.896**
Mastery-avoidance	→Surface learning	88.974	−2.785**
Mastery-approach	→Surface learning	105.415	−2.152*

Note. *Significant at the .05 level. **Significant at the .01 level.

In the control group, mastery-avoidance related significantly positively with a surface learning approach (t = 2.71, df = 71.297, p = .009).

3.2. RQ2. Do Goal Orientation and Learning Behavior Indeed Change Over Time, and If So, in What Direction?

Changes over time in goal orientation and learning approach are presented in Table 8.

TABLE 8 Changes Over Time of Goal Orientations and Learning Approaches in the Group as a Whole, the Experimental Group, and Control Group

Sample	Orientation or Approach	Change	Cohen’s d
Total group
	Performance-approach	.305	.29**
	Surface learning	.134	.29**
Experimental group
	Mastery-approach	−.290	.26*
	Performance-approach	.269	.25***
Control group
	Performance-approach	.367	.35*
	Surface learning	.190	.38*

Note. *Significant at the .05 level. **Significant at the .01 level. ***Significant at the .1 level.

A performance-approach increased in both groups, whereas a mastery-approach decreased in the experimental group and surface learning only increased in the control group.

In other words, all students became more performance-approach oriented; the students lacking the feedback intervention increased in surface learning, and the students in the experimental condition decreased in a mastery-approach, with a concomitant increase in surface learning.

Mean-level change was complemented with an individual-level change by calculating the RCI; see Table 9.

TABLE 9 Reliable Change in Self-Efficacy and Learning Approach for the Group as a Whole, Experimental Group, and Control Group

	Pretest-Posttest Group as a Whole		Pretest-Posttest Experimental Group		Pretest-Posttest Control Group
	% Decrease	% Increase	% Decrease	% Increase	% Decrease	% Increase
Performance-approach	1.0	2.9	1.6	1.6	4.7	9.3
Performance-avoidance	1.9	4.8	3.3	3.3	0.0	7
Mastery-approach	2.9	2.9	4.8	1.6	0.0	4.7
Mastery-avoidance	2.9	4.8	4.8	1.6	0.0	9.3
Deep learning	21.4	17.5	0.0	1.6	21.4	14.3
Surface learning	14.4	28.8	16.1	27.4	11.9	31

In the group as a whole all goal orientations remained stable, deep learning showed more students decreasing than increasing (increase 17.5%, decrease 21.4%, stable 61.1%) and surface learning showed more students increasing than decreasing (increase 28.8%, decrease 14.4%, stable 56.8%).

In the experimental group all goal orientations also remained stable on an individual level, and deep learning also remained stable. Surface learning showed a pattern of more students increase significantly different from what would be expected if change were random (increase 27.4%, decrease 16.1%, stable 56.5%). In other words, the students maintained their deep learning behavior but also reported an increase of surface learning.

In the control group, on an individual level more significant changes were found, deep learning showed more students decreasing than increasing (increase 14.3%, decrease 21.4%, stable 64.3%) whereas surface learning showed more students increasing than decreasing (increase 31.0%, decrease 11.9%, stable 57.1%). In other words, students who did not receive the feedback intervention decreased in their deep learning and reported the largest increase in surface learning.

3.3. RQ3. The Effect of Sustainable Feedback on Goal Orientations and Learning Behavior

Standard multiple regression analyses were performed to examine the influence of the intervention on the goal orientations and learning behavior. The assumptions of linearity, independence of errors, homoscedasticity, unusual points, and normality of residuals were met. The outcomes were controlled for differences in tutor group and the pretest scores. The students’ performance-approach at pretest (B = .698, p < .0005), the tutor group (B = −.135, p < .047), and the feedback intervention (B = −.568, p < .03) predicted the posttest performance-approach, F(3, 101) = 37.03, p < .0005, R² = .524. Beta weights and their associated t-values are presented in Table 10. A significant negative effect for the intervention was found only for the performance-approach. No significant effects of the intervention were found for the other goal orientations or learning behavior.

TABLE 10 Multiple Regression Performance-Approach, Intervention, Tutor Group

					95% Confidence Interval for B
	B	SE	Standardized Beta	t	Lower Bound	Upper Bound
Constant	2.257	.462	.703	4.88**	1.341	3.174
Intervention	−0.568	.258	−.204	−2.198*	−1.081	−0.055
Tutor group	−0.135	.067	−.186	−2.007*	−0.269	−0.002
PAp-1	0.698	.069	.703	10.089**	0.560	0.835

Note. Dependent variable: PAp-2. PAp-1 = performance-approach, pretest; PAp-2 = performance-approach, posttest.

*Significant at the .05 level. **Significant at the .01 level.

4. Conclusion and Discussion

Overall, it can be concluded that significant relations were found between goal orientations and learning behavior. Deep learning related significant positively to mastery-approach, mastery-avoidance, and performance-approach. Surface learning related significant negatively to mastery-approach and performance-approach. Changeability occurred for performance-approach in the group as a whole as well as in the experimental and control groups separately (i.e., increase), mastery-approach in the experimental condition (i.e., decrease) and surface learning in the group as a whole and in the control condition. (i.e., increase). Deep learning remained stable within the experimental context. As these findings are contrary to our expectation, analyses on an individual-level were carried out. These analyses showed an increase in surface learning in the experimental and control group and a decrease in deep learning in the control group. Deep learning did not change (i.e., remained stable) in the experimental group. Also, this was not what we expected and will be discussed in the following parts of this section. The effect of the intervention was only, negatively, found for performance-approach. The main findings in terms of relations are summarized in Table 11.

TABLE 11 Overview of Significant Relations Between Goal Orientations and Learning Behaviors

		Relations
		All	Exp	Con
Mastery-approach (MAP)		DL+	DL+
		SL−	SL−
Mastery-avoidance (MAV)		DL+	SL−	SL+
Performance-approach (PAP)		DL+	DL+
		SL−
Performance-avoidance (PAV)

Note. All = total of all of the groups; Exp = experimental group; Con = control group;

These findings have implications for the educational program of international business students. The aim of the intervention was to enhance a mastery-approach and deep learning because of the assumed positive effect on successful working in an international and intercultural context. The relation found between a mastery-approach and deep learning among international business students is worthwhile, and the stability of deep learning within the experimental groups is therefore a promising result. However, further adjustments to the sustainable feedback approach are necessary in order to achieve an increase in a mastery-approach orientation and deep learning. Aspects such as the initial learning profile of international business students might be relevant for international business education, but the alignment within the learning environment needs more attention as well. These aspects will be discussed in this section.

Now, the main findings are discussed and subsequently, methodological and practical implications are given.

4.1. Discussion

The main findings of our study are discussed in terms of relations and changeability, and the contribution of the sustainable feedback intervention within the domain of international business.

4.2. Relations and Changes Over the Course of Time

The main findings, as presented, mainly confirmed previous research on the relation between mastery-approach and deep learning (Elliot et al., 1999; Covington, 2000; Elliot & McGregor, 2001) and changeability of mastery-approach and performance-approach (Muis & Edwards, 2009). However, some results in our study seemed to be specific for our research context and intervention. Specific relations and changes over time will be discussed.

The replicated significant relation between mastery-approach and deep learning means that students who strive for mastering knowledge and skills are, not surprisingly, positively related to deep learning as well. Both mastery orientations (i.e., approach and avoidance) in our experimental group appeared to be negatively related to surface learning; that is to say, independent of approach or avoidance motivation, students who learned with a mastery orientation in the experimental group shied away from surface learning. These results were also found in previous research such as Maehr and Zusho (2009) and Midgley, Kaplan, and Middleton (2001). The feedback provided in the experimental group, might have mediated the relation between mastery orientations and learning behavior. As the aim of the intervention was to increase mastery orientations and a concomitant deep learning, the effect of feedback on the relation between mastery orientations and surface learning is not investigated. However, this is interesting for further research. Within our control group, mastery-avoidance related positively to surface learning. An explanation might be that mastery-avoidance orientations led to both positive (i.e., deep learning) and negative (i.e., surface learning) outcomes because of the avoidance component and the mastery component, each leading to different outcomes (Maehr & Zusho, 2009). In other words, a mastery orientation is associated with deep learning, and an avoidance orientation might lead to surface learning, two opposing effects resulting from a mastery-avoidance orientation. Fryer and Elliot (2007) denote this effect as a mixed antecedent profile, grounded in a positive need (e.g., need for achievement), a negative need (e.g., fear of failure), or in both antecedents. Overall, the expected relation between mastery orientation and deep learning was found and is in line with previous research (Covington, 2000). However, the expected positive relation between performance orientation and surface learning was not found.

In terms of changeability it can be concluded that the mastery-approach and performance-approach and deep and surface learning changed over the course of time. Muis and Edwards (2009) found that mastery-approach displayed the most variation, followed by performance-approach, and performance-avoidance being the most stable over time. In our study, performance-approach scores increased most and in all groups. It is noteworthy that scores on performance-approach orientation significantly increased in both groups over time (i.e., within a teaching period of 8 weeks). Students seemed to focus more on demonstrating competence and/or on outperforming their peers and therefore on meeting the assessment requirements. This increase in an orientation of performing might be a valuable combination with a mastery orientation. There is some evidence that performance-approach goals may be valuable in conjunction with high mastery orientations (Bouffard, Boisvert, Vezeau, & Larouche, 1995; Pintrich, 2000). Senko et al. (2011) suggested that performance-approach oriented learners might be more aware of the teacher’s agenda; that is, the specific knowledge topics and skills the teacher considers important and is likely to assess. This focus might help performance-approach oriented students pass the course, whereas mastery oriented students might be directed to the topics they themselves are most interested in. The learning environment, including the assessment procedures, might have directed students toward a performance approach. From this perspective, the significant association found between both a mastery-approach orientation, a performance-approach orientation and deep learning in the experimental condition is valuable. For international business students to be able to meet the requirements of the learning environment, but also to meet the demands of their future working environment this combination might be valuable. Specifically, in the experimental condition sustainable feedback might have contributed to the maintainance of the deep learning behavior, while it was lacking in the control condition where deep learning on the individual level decreased.

4.3. The Contribution of the Sustainable Feedback Intervention Within the Domain of International Business

The aim of this study was to alter adopted goal orientations into a mastery-approach and surface learning into deep learning. The sustainable feedback intervention is, as far as we have found in published research, a relatively new approach within the domain of international business. The limited effect of the intervention might be caused by the way the learning environment was organized. It might be that the constructive alignment within the learning environment worked against the intended effect; it should be noted that the intervention was solely implemented within the PBL group work. Besides this, four supportive lecture-based courses were part of the period and were assessed individually via examinations. These supportive lecture courses, the assessment forms (i.e., written and oral tests), and the workload at the end of the teaching period might have affected the students in choosing for an “economical” way of studying, meaning that students adjusted their learning behavior to the demands of the learning environment and specifically to the assessment structure. In other words, as the deadline nears, students chose an economical behavior such that requirements were met and were directed to a performance-approach and surface learning (Senko et al., 2011).

Another explanation for the limited effects of the intervention might be that the domain of study (i.e., international business and the practical approach of the PBL assignments) and the preferred learning profile of the students limited the effect of the sustainable feedback intervention. It might be worthwhile to gain more knowledge on the preferred learning profile of business students, in our study this was beyond our scope of research.

The intervention had a significantly negative effect on performance-approach. There was also a significant difference in the performance-approach caused by differences between the tutor groups (negatively) and the previous performance-approach scores (positively). In other words, the intervention and the tutor group negatively influenced the score on performance-approach at posttest; this might imply that the sustainable feedback and the collaboration in the tutor group limited the increase in using a performance-approach (i.e., achieving passing grades). Previous findings show an effect of tutor behavior, as part of the total group, on learning processes and student achievement (Chng et al., 2011). The tutor behavior in the experimental groups was influenced by the specific instruction they had to follow to guide the sustainable feedback process within the groups. An explanation might be that their guidance directed the students away from a performance-approach. From previous research it is known that competitive environments and emotional corrective feedback directs students toward a performance-approach (Jiang, Song, Lee, and Bong, 2014). The intervention (i.e., sustainable feedback) and the tutor behavior prevented the students of a much stronger increase in performance-approach.

However, two promising results should give direction for future research on goal orientation, deep learning, and sustainable feedback. First, the students within the experimental condition maintained their initial deep learning. Second, the intervention negatively influenced the performance-approach of the students, meaning that without the sustainable feedback their performance-approach would have increased even more.

Overall, it can be concluded that sustainable feedback helped mastery-oriented learners maintain their deep learning behavior, but it did not directly change the goal orientations. Students lacking sustainable feedback showed a decrease of deep learning on an individual level.

5. Methodological limitations and future research

A limitation of this study was that while it was a real-life learning situation, the length of the intervention period was limited. The aim of sustainable feedback was to make students think for themselves about what they wanted to learn and to be active in this learning process. In the day-to-day execution of the study program, the students and the tutors were not familiar with this approach. Only the tutors received training; students were not prepared in advance for this specific feedback method. Students should gain more prolonged experience on sustainable feedback to become more confident with this approach.

It has been taken into consideration that the operationalization of learning behavior differs among studies. Elliot and McGregor (2001) used the students’ study strategy questionnaire (Elliot et al., 1999), in which the learning behavior is operationalized in deep processing, surface processing, and disorganization. Deviating results among studies might be attributed to these differing operationalizations of learning behaviors.

The research questions underlying this study addressed a replication of previous research to study the relation between goal orientation and learning behavior, and the changeability of these concepts within the domain of international business. Replication studies imply the use of the same methods. While we are replicating several studies with different methodologies, we have used analyses on mean-level and individual-level.

The feedback intervention was executed within the PBL groups, but this educational approach cannot be seen as an isolated approach in the entire learning environment. Besides the PBL group work, this environment is composed of supporting courses and differential forms of assessment. The perception of students and tutors might influence the adopted goal orientations and learning behavior. The students and tutors’ perceptions of the learning environment should be investigated in future research to provide additional insights into the reported goal orientations and learning approach.

Sustainable feedback seems to be a promising approach within the context of international business education and problem-based learning. However, more research is needed on the learning profile of international business students, because the initial learning profile of students seems to be of influence on the results gained in this intervention study (e.g., the pretest performance-approach positively influenced the posttest performance-approach). Along with the learning profile the learning environment—in terms of assessment forms and guidance by tutors—should be fully aligned with the sustainable feedback and—from an educational point of view—the preferred mastery orientation and concomitant deep learning.

Sustainable feedback as a tool to enhance mastery orientations and deep learning seems to be promising if more aligned with the learning environment and the accompanying perceptions of both students and tutors. Overall, it can be concluded that sustainable feedback helped students to maintain their deep learning.

Additional information

Notes on contributors

Gerry Geitz

Gerry Geitz is Academic Dean of the School of Commerce and associate professor of problem-based learning at Stenden University of Applied Sciences. She is the chair for the National Platform for the Bachelor of International Business and Languages. Her research focuses on problem-based learning, self-efficacy, goal orientation, learning behavior, and feedback in the context of higher education.

Desirée Joosten-ten Brinke

Desirée Joosten-ten Brinke is associate professor at the Welten Institute at the Open University of the Netherlands and associate professor of quality of testing and assessment at Fontys University of Applied Sciences. She is head editor of a Dutch journal on assessment (Examens, tijdschrift voor de toetspraktijk), teacher of assessment courses, and project manager of national assessment projects.

Paul A. Kirschner

Paul A. Kirschner is university professor at the Open University of the Netherlands and Professor of Education at the University of Oulu. He is the author of the highly acclaimed book Ten Steps to Complex Learning, member of the Scientific Technical Council of SURF, and past president of the International Society of the Learning Sciences.