Refining success and dropout in massive open online courses based on the intention–behavior gap

Abstract

In this paper we present an alternative typology for determining success and dropout in massive open online courses (MOOCs). This typology takes the perspectives of MOOC-takers into account and is based on the their intentions and subsequent behaviour. An explorative study using two MOOCs was carried out to test the applicability of the typology. Following the traditional approach based on course completion to identify educational success, success rates were 6.5 and 5.6%. The success rates from the perspectives of the MOOC-taker were 59 and 70%. These findings demonstrate that merely looking at course completion as a measure for success does not suffice in the context of MOOCs. This change in addressing MOOC success and dropout provides an alternative view and demonstrates the importance of MOOC-takers’ perspectives.

Keywords:

• Open education
• MOOCs
• online learning
• success
• dropout
• intention

Introduction

Massive open online courses (MOOCs) as a novel form of open education were initially received with great enthusiasm. Hundreds of thousands of learners and even more enrolled in MOOCs (Jordan, 2014). However, after a short time, this first excitement was followed by frustration. Despite its popularity, the number of MOOC-takers who actually completed a MOOC after enrollment was reported to be very low with dropout rates between 98 and 90% (Jordan, 2014, 2015; Koller, Ng, Do, & Chen, 2013; Liyanagunawardena, Parslow, & Williams, 2014; Reich, 2014). Many researchers agreed that the learning circumstances in MOOCs are exceptional (Huin, Bergheaud, Caron, Codina, & Disson, 2016; Koller et al., 2013; Liyanagunawardena et al., 2014). In contrast to traditional face-to-face education and also distance education – where students often have to meet certain admission requirements and primarily follow full educational programmes – a MOOC is a relatively short course (generally 5–12 weeks) which is accessible to anyone, anywhere, at any time in disposal of an Internet connection. It is therefore recognized that it should not be compared to the traditional learning context with respect to completion and dropout (Huin et al., 2016; Walji, Deacon, Small, & Czerniewicz, 2016).

Academic research on dropout has a long tradition. Tinto (1975) differentiated between two levels of perspectives for defining dropout in his seminal work on college droput: On the one hand, there is the level of the educational institution dealing with students who leave without receiving an end qualification; on the other hand there is the state- or country-wide perspective of students who attend one or more educational institutions, but never receive an end qualification from any of these institutions. Tinto (1975) proposed a model for explaining student dropout that includes a combination of individual and organizational variables that influence dropout. Sweet (1986) used this theoretical model and applied it in a study situated in a distance education context. Results of this study partially confirm the relations as proposed by Tinto (1975). Motivation, measured in the form of locus of control, had a direct and indirect effect on persistence. Further, Garrison (1987) argued that research on dropout in distance education was too much focused on understanding and predicting without actually taking into account the nature of distance education. Garrison recommended focusing on the student’s perspectives and developing situation-specific models and theories before trying to generalize. This is also in line with recommendations by Tinto (1975): ‘A […] more important limitation […] is the tendency to ignore the perspective of the individual’ (p. 5). In addition Peters (1971) argued that new criteria for analyses are necessary when analyzing different forms of education.

We assume that Tinto’s (1975) model and the variables he described are important for the context of open education and MOOCs. Yet, due to the variety of possible goals in this context, we expect that individual differences in goal commitment will play a prominent part in the understanding of dropout in MOOCs compared to the distance education context. We expect that the non-formal nature of learning in open education requires a situation-specific approach to understand success and failure as signified by dropout. Framing success from a certificate- and completion-centric view will nurture a false understanding of success and dropout in MOOCs, which may subsequently lead to unnecessary interventions and unjustified negative reviews. The benefit of MOOCs is that they afford individuals the opportunity to follow their own learning paths. It seems, therefore, legitimate to take the intention of the individual MOOC-taker as a starting point for measuring and interpreting success and subsequently dropout. These intentions may vary from simply browsing through a MOOC to – indeed – getting a certificate.

In this paper we present a typology based on the individual intentions of MOOC-takers. As the point of departure is the MOOC-takers’ intention, it is necessary to consider theories about intention formation and how individuals comply with this intention and ultimately translate this intention into actual behaviour. In our study we used Fishbein and Ajzen’s (2010) reasoned action approach (RAA) as described in Kalz et al. (2015) and built on the intention–behavior patterns as defined by Sheeran (2002) to reinterpret success and dropout in MOOCs.

The paper is structured as follows: First we discuss the theoretical background and related work. Via the integration of several socio-psychological perspectives on the connection between intention and behavior we introduce the new typology. Next, data from two MOOCs is presented and analysed in line with the theory and new typology. Lastly, we discuss the results and implications for using the typology in empirical studies, and consider limitations of the approach.

Related work on dropout and success in MOOCs

In his extensive research on the process of dropping out in higher education, Tinto (1975) defined dropout as students who leave the educational institution at which they are registered without an end qualification. This dropout rate is fairly easy to calculate (i.e., the number of students without an end qualification divided by total number of registered students) and is widely used as a measurement for institutional success and quality of education (Eisenberg & Dowsett, 1990; Peters, 1992; Tinto, 1975).

When assessing success in MOOCs, this definition of dropout (i.e., dropout equals not receiving a certificate) is often used. A study by Breslow et al. (2013) determined the success rate by calculating the percentage of students who earned a certificate for completion. This resulted in a success rate of 5%, hence a dropout rate of 95%. Likewise, Belanger and Thornton (2013) analysed Duke University’s first MOOC and found a success rate of 2% and a dropout rate of 98%. A study by Jordan (2014) further illustrated that success assessment of MOOCs was primarily directed at earning an end qualification in line with the approach discussed in the introduction. She found that on average 6.5% of the students who enrolled in a MOOC met the certificate-earning criteria of the course. In a later study she found that the success rate reached a mean value of 12.6%, which entailed an average dropout rate of 87.4% (Jordan, 2015).

This approach to success assessment of MOOCs resulted in very low success rates and, subsequently, extremely high dropout rates. It also ignored the viewpoint of the individual student. Following Tinto (1975), the perspectives of individual learners in the assessment of dropout brings a new point of view to the discussion. This new viewpoint adds the origin of the leaving behaviour to the discussion. Tinto (1982) classified academic dismissal and voluntary withdrawal as two distinct types of leaving behavior. Academic dismissal, a type of dropout, which will not occur in the context of a MOOC, is initiated by the educational institution. A reason for this could be insufficient performance. Voluntary withdrawal, on the other hand, is initiated by the individual and can be caused by multiple factors in the dynamic between the individual, peers, and the institution (Eisenberg & Dowsett, 1990; Tinto, 1975).

Voluntary withdrawal could, however, also be retraced to the individual intentions of the student. Students entering education might have intentions other than receiving an end qualification; for example, they may intend to complete only some courses to develop specific skills and knowledge (Roberts, 1984; Tinto, 1982). Also, intentions may change over time. For example, the initial intention of a student may be to receive the end qualification, yet over time this intention may change and the student leaves the educational institution and/or system voluntarily completely satisfied with the accomplishment at hand (Tinto, 1982). As was pointed out in the introduction, MOOCs provide an exceptional learning context. The open and accessible character of MOOCs affords individuals to follow their own personal learning paths, which are likely to be based on a variety of individual intentions and not merely on receiving an end qualification.

Recent work on dropout in MOOCs has shifted from an outcome-related perspective to a more individual perspective. Liyanagunawardena et al. (2014) argue that the way dropout is measured fails to identify various forms of dropout such as academic failure and voluntary withdrawal. Categorizing participants who do not complete a course as dropouts leads to ambiguous conclusions regarding course success. The main conclusion of their study is that factors like start date and intentions should be considered when it comes to defining dropout.

Koller et al. (2013) have provided the first peer-reviewed article in which retention is considered in the context of student intent. They purport that ‘observing how students participate in online classes can reveal student intent’ (p. 2). By using log data to reveal behavioral patterns they distinguished four categories of learners: browsers, passive participants (limited course engagement), active participants (full course engagement), and community contributors (course engagement specifically aimed at generating new content). Yet, even though they acknowledged the fact that success measurements of MOOCs should be interpreted with individual intention in mind, they nevertheless focused on studying student intention to complete a MOOC. MOOC completion, therefore, still implied ultimate study success. Reich (2014) also explored the issue of MOOC completion and retention in the context of student intent. He found that students with the intention to complete were most likely to earn a certificate (22%). In contrast to the research by Koller et al. (2013), student intention was based on intentions reported by the students in a pre-course survey rather than derived from student log data. Their choice of intentions was limited to four options: unsure, browse, audit, complete. Similar to the research by Koller et al. (2013), Reich (2014) regarded solely the intention to complete as the preeminent success measurement. Neither research by Koller et al. (2013) nor by Reich (2014) took into account related theories and empirical research on intentions.

Huin et al. (2016) propose a learner-centered model for measuring completion and dropout in MOOCs. This model can be regarded as the first attempt to refine the view on learner success and failure in MOOCs. Their proposed typology is structured along three theoretical key concepts: intention, commitment, and behavior. These concepts are based on related research on intention and motivation, namely the integrative model of motivation (Fenouillet, 2012), Fishbein and Ajzen’s (1975) theory of reasoned action, and Deci and Ryan’s (2002) self-determination theory. However, it remains unclear what precise aspects Huin et al. (2016) used from these theories and how they were reflected in their model. In their study learners could indicate whether they chose to follow the learning objectives provided by the instructional design or their personal learning objectives. This intention and subsequently the actual behavior was then inferred from log data in the MOOC.

To summarize, studies to date illustrate the growing awareness that the individual intention of MOOC-takers should indeed be taken into account to avoid misinterpretations of success and dropout in MOOCs, as well as individual success and failure. These studies, however, merely based the intention of the MOOC-takers on log data, focused only on the intention to earn a certificate or did not reflect findings based on the theoretical model they used for research with regard to understanding success and dropout in the context of MOOCs.

Theoretical foundation

As indicated in the introduction, the RAA (Fishbein & Ajzen, 2010) serves as our theoretical framework. This framework contains two levels: the formation of an intention to reach certains goals and the translation of this intention into actual behavior, which may or may not lead to an intention–behavior gap. Our focus in this study is on the latter level. Even though the RAA was originally developed to explain and predict behavior in the field of health science, it has been widely adopted by numerous other fields to gain insight into intention–behavior relations (Kreijns, Vermeulen, Kirschner, Buuren, & Acker, 2013). According to Fishbein and Ajzen (2010), intention is determined by three main factors: an individual’s attitude towards the behavior, perceived norm (an individual’s perceived social pressure to perform or not to perform the intended behavior), and perceived behavioral control (an individual’s perception of whether a person is capable or has control over the performance of the intended behavior). Furthermore, intention is expected to be a predictor for behavior.

A study by Sutton (1998) on how well intentions predict behavior found an average correlation of .48 (equivalent to explaining 24% of the variance). Sheeran (2002) conducted a meta-analysis of ten meta-analyses on how well intentions actually predict behavior and found an average correlation of .53 (equivalent to explaining 28% of the variance), which approximately matches Sutton’s (1998) findings. According to Cohen’s (1992) power primer, these findings can be regarded as a large effect size (r = 0.10 is ‘small’, r = 0.30 is ‘medium’, and r = 0.50 is ‘large’), yet these results are biased due to the fact that negative intentions (i.e., the intention to not engage in something) are more often translated into actual behavior than positive intentions (i.e., the intention to engage in something) (Fishbein & Ajzen, 2010, p. 59). To illustrate this, a study by Sheeran and Orbell (2000) found that individuals who form a negative intention (i.e., they were not willing to exercise) indeed did not exercise (97%). Of the individuals who formed the positive intention (i.e., willing to exercise) only 46% actually did so. In general, most intention–behavior studies supported this finding, which indicates that there is a substantial gap between (mainly) positive intentions and actual behavior.

Fishbein and Ajzen (2010) describe two possible reasons why certain behavior is not performed and thus the possibility of the intention–behavior gap arises:

(1)	The intention to perform specific behavior has not been formed.
(2)	The intention is formed, but cannot be performed due to certain barriers which impede the performance.

McBroom and Reed (1992) and Sheeran (2002) described four different intention–behavior patterns which can be distinguished:

(1)	inclined actors: individuals who formed a certain intention and did act according to this intention
(2)	inclined abstainers: individuals who formed a certain intention but fail to act according to this intention
(3)	disinclined actors: individuals who formed no intentions but acted anyway
(4)	disinclined abstainers: individuals who formed no intentions and accordingly did not act.

In the context of a MOOC many individual intentions are possible. We adapted the initial definitions of Sheeran (2002) and subsume in the group of disinclined actors also individuals who did form initial intentions and acted out behavior that went beyond these initial intentions. The group of disinclined abstainers is included in the context of MOOCs, for the reason that this group will never start a MOOC in the first place.

Intentions in MOOCs may vary from the intention to finish only the first three modules or completing the course and getting the certificate, to expanding one’s network (or any other intention an individual might have). Following the discussed theories, MOOC-takers who formed the intention to finish the first three modules of a MOOC and actually succeed in doing so achieved their respective goal and can be defined as inclined actors and are considered successful MOOC-takers. MOOC-takers who only planned to browse through the course or download some interesting materials and who eventually finish three modules are also considered successful. These ‘disinclined actors’ did more than intended, which can be regarded as a positive outcome. This reasoning does not (yet) take into account the weight of the effort of various intentions. For example, the intention to browse requires less effort to translate to actual behavior than the intention to complete all modules. Consequentially, the effort to change the initial intention from browsing into a behavior in which a MOOC-taker participates in a single learning activity is a smaller step compared to MOOC-takers who intended to participate in some learning activities and in the end finish the course with a certificate.

Thus, to refine the assessment of success and dropout in MOOCs, the individual intention should be taken as a starting point. The following subsection will explain the (theoretical) scope of intentions when considering a MOOC environment and present a typology based on the translation of individual intention into actual behavior.

The MOOC-taker typology

Taking individual intention as a starting point for the discussion about dropout and success in MOOCs leads to various intention–behavior patterns that can be identified. The composition of these intention–behavior patterns, contrasted by the goals set by the MOOC provider, lead to the identification of different types of MOOC-takers. We use Venn diagrams (Figure 1) to visually illustrate the variety of intention–behavior patterns with respect to goal achievement. In these diagrams the black dots represent all possible goals that can be formulated as an intention, when following a MOOC. Generally, a MOOC provider defines a certain set of goals that must be achieved in order to obtain a certificate; in Figure 1 the grey ellipse with the dotted outline represents this MOOC provider’s set of goals (i.e., the minimal set of requirements that must be satisfied in order to earn a certificate). However, a MOOC-taker may formulate a different set of goals; this individual set of intended goals is represented in the Venn diagrams by a circle with a solid outline. As can be seen from Figure 1(a), the individual set of intended goals is, in this case, a subset of the MOOC provider’s set of goals; apparently, the MOOC-taker is not planning to obtain a certificate. But any other individual set of intended goals is possible. Individual sets of intended goals generally may or may not overlap, match, or even encompass the MOOC provider’s set of goals. The circle with the dotted outline represents the set of intended goals that a MOOC-taker actually has achieved (i.e., actual behavior). The difference between the set of intended goals and the set of achieved goals identifies the type of MOOC-taker. From the Venn diagrams we can see that the MOOC-taker may have achieved more, less, or other goals than initially intended. Three types of MOOC-takers were identified:

(1)	Inclined actors: These MOOC-takers fully achieved their individual set of intended goals and are considered successful according to our perspective. (Figure 1(a)).
(2)	Inclined abstainers: These MOOC-takers achieved none or less than their individual set of intended goals or decided to quit the MOOC, and are considered not successful according to our perspective (dropouts; Figure 1(b)).
(3)	Disinclined actors: These MOOC-takers achieved more than their individual set of intended goals and are considered successful according to our perspective. (Figure 1(c)).

For each of these types if their achieved set of goals encompasses the MOOC provider’s set of goals, the MOOC-takers may obtain a certificate. Only then they will be considered successful according to the traditional success perspective.

Figure 1. Venn diagrams illustrating intention–behavior relations that identify MOOC-takers.

This set of Venn diagrams, as depicted in Figure 1, is a non-exhaustive overview of possible intention–behavior combinations. For instance, an inclined actor’s individual set of intended goals may also equal the MOOC provider’s set of goals. MOOC-takers then merely follow the pre-defined set of goals of the MOOC provider to obtain a certificate. Or this MOOC-taker’s individual set of intended goals may not overlap the pre-defined set of goals by the MOOC provider at all, as is depicted in Figure 1(a). If MOOC-takers achieve their individual sets of goals, they are considered successful according to our perspective. Thus, in every possible scenario, the MOOC-taker’s individual intention is the starting point for measuring success or failure. To explore the applicability of the typology for assessing MOOC success and dropout, we have conducted an explorative study which is described in the next section.

Method

Participants

Participants took part in two MOOCs. Both MOOCs were designed by respective teams at the Open University of the Netherlands in cooperation with external parties. None of the authors was involved in the design of the courses; one of the authors supported the technical implementation of the courses.

The first MOOC (MOOC-I) was a MOOC about marine litter which ran from October until December 2015, covering eight modules for 8 weeks. MOOC-takers who completed all study tasks, including the final assignment, obtained a certificate of participation free of cost. The study load was estimated at 4 h per week. A pre-course questionnaire was completed by 689 MOOC-takers (487 women, 202 men, M_age = 35.6, age range: 17–73 years). The post-course questionnaire was completed by 163 MOOC-takers (109 women, 54 men, M_age = 38.9, age range: 17– 71 years). In total 65 MOOC-takers completed both questionnaires (49 women, 16 men, M_age = 40.3, age range: 21–66 years).

The second MOOC (MOOC-II), ‘The Adolescent Brain,’ was in Dutch, and ran from April until June 2016, covering seven modules for 7 weeks. MOOC-takers who participated in all learning activities could request a certificate free of charge. The weekly study load was estimated at 3 to 5 h per week. The pre-course questionnaire was completed by 821 MOOC-takers (664 women, 157 men, M_age = 45.1, age range: 18–74 years). The post-course questionnaire was completed by 126 MOOC-takers (unfortunately, participant demographics were not available). In total 101 MOOC-takers completed both questionnaires (90 women, 11 men, M_age = 37 age range: 18–54 years).

Materials

To measure the initial intention of the individual MOOC-takers a self-constructed set of items was used which were aligned with the design of the respective MOOCs. Items covered increasing intentions from browsing, partial participation in one or more modules, up to participating in all learning activities and receiving a certificate (see Appendix 1). These items were included in the pre- and post-course questionnaires of both MOOCs. In the post-course questionnaire MOOC-takers were asked to indicate their actual behavior on the same set of items used in the pre-course questionnaire taking into account the methodological issues of scale correspondence (Sutton, 1998) and the principle of compatibility (Fishbein & Ajzen, 2010). Scale correspondence refers to using corresponding magnitudes, frequencies, or response formats when measuring intention and behavior. The principle of compatibility requires that when measuring intention–behavior relations, both intention and behavior should be measured at the same level of specificity or generality. If even one is defined on another level, it will not be possible to find a reliable correlation between intention and behavior.

Procedure

In the first week of both MOOCs, all the registered MOOC-takers received an invitation to participate in the pre-course questionnaire. At the end of the last week of the MOOCs all the registered MOOC-takers received an invitation to participate in the post-course questionnaire. Participation was voluntary, and informed consent was obtained from participants following ethical guidelines of the providing institution.

Results

Traditional success and dropout measurement of MOOCs

The analyses focused on the success and dropout rates of the two MOOCs following the traditional dropout calculation (Peters, 1992; Tinto, 1975): number of certificates earned by the MOOC-takers divided by the total number of registered MOOC-takers (Figure 2).

Figure 2. Certificate-oriented success measurement of MOOC-I and MOOC-II.

MOOC-I had 6452 registered MOOC-takers, of whom 422 earned a certificate. This results in a success rate of 6.5%and consequently a dropout rate of 93.5% (Figure 2(a)). MOOC-II had 1763 registered MOOC-takers of whom 98 earned a certificate. This results in a success rate of 5.6% and a dropout rate of 94.4% (Figure 2(b)).

Intention–oriented success and dropout measurement of MOOCs

The second analysis focused on identifying success and dropout rates taking the intention of the MOOC-taker as a starting point. In this analysis the data allowed us to identify the three types of our proposed typology. In MOOC-I, 65 participants completed both the pre-course and post-course questionnaires (Figure 3(a)).

Figure 3. Intention–behaviour relations in MOOC-I and MOOC-II.

Of these 65 MOOC-takers, 42% can be regarded as inclined actors, their actual behavior being equal to their intention. A further 17% can be regarded as disinclined actors, their actual behavior exceeding their intention, and 41% of the MOOC-takers are inclined abstainers since their intention exceeded their actual behavior. This results in an overall success rate of 59% and a dropout rate of 41%.

In MOOC-II, a total of 101 participants completed both the pre- and the post-course questionnaires (Figure 3(b)). Of these 101 MOOC-takers, 49% are inclined actors as their behavior equaled their intention. Disinclined actors represent 21% of the MOOC-takers as their actual behavior exceeded their intention, and 30% of the MOOC-takers can be regarded as inclined abstainers as their intention exceeded their actual behavior. This results in an overall success rate of 70% and a dropout rate of 30%.

Discussion

In this paper, we have presented an alternative typology to refine the measurement of success and dropout in MOOCs. This typology is based on the initial intentions of individual MOOC-takers and their subsequent behavior, which in the end results in a number of achieved goals. The RAA by Fishbein and Ajzen (2010), which centers around the formation of an intention to achieve certain goals and the translation of this intention into actual behavior, as well as the intention–behavior patterns as defined by Sheeran (2002), served as a theoretical framework for our typology. Furthermore, a first explorative study was carried out to test the applicability of the typology for assessing success and dropout in MOOCs and to compare it to the currently used approach to identify educational success.

One of the implications of our proposed typology is that all MOOC-takers who actually do as they intended (inclined actors) or do more than they intended (disinclined actors) are considered successful. Only MOOC-takers who quit during the runtime of the MOOC or who end up doing less than they intended are regarded as dropouts (inclined abstainers). This way of calculating success and dropout in MOOCs leads to a completely different picture. To illustrate this, we used data of two MOOCs; following the currently used approach to identify educational success, success rates of the MOOCs were between 5.6 and 6.5%. The success rates from the perspectives of the MOOC-takers were between 59 and 70%. These findings demonstrate that merely looking at course completion as a measure for MOOC and individual success does not suffice. This small change in the way we look at assessing MOOC success and dropout may have a large impact on future research on MOOCs. This approach represents a situation-specific approach that should build the foundation for future studies on dropout in the context of MOOCs.

In both MOOCs, most MOOC-takers were identified as inclined actors (42 and 49%). This group did what they intended to do in the MOOC, whether it was completing only some modules, justing watch all the videos, or earning a certificate. It can therefore be expected that these MOOC-takers are content with their achievement. However, this does not necessarily imply that they were satisfied with issues such as MOOC content, design, or learning experience. Future research should aim to analyse their learner profile and their activities in more detail.

A substantial group in both MOOCs was classified as disinclined actors (17 and 21%). At some point during the runtime of the MOOC they found themselves exceeding their intentions. Reasons for this could be that they might have set low targets for themselves (just browse or do some learning activities), or the course content might have unexpectedly interested them more than they anticipated. Further research is necessary to understand reasons behind this behavior.

The last group comprised the inclined abstainers (41 and 30%). In both MOOCs, this was the second largest group. These MOOC-takers formed certain goal intentions but were not able to or did not transform these intentions into actual behavior. Did they set the highest targets? Were they first time MOOC-users and therefore not familiar with this learning environment? Were they dissatisfied with the course design or content? Future research should aim to map the complex and dynamic process of intention–behavior and provide some insight into possible reasons that can cause the intention–behavior gap.

The typology, based on individual intentions versus actual behavior of the MOOC-takers, provides a more nuanced insight into individual learner success, hence MOOC success. It gives an indication of which group of participants is responsible for the intention–behavior gap. Sheeran (2002) found, in the context of health science, that it was mainly the group of inclined abstainers who were responsible for the intention–behavior gap. However, according to his theory the group of disinclined actors can also add to the intention–behavior gap, as their intention does not reflect their actual behavior either (Sheeran, 2002). Future studies should explore whether this is indeed the case in the context of MOOCs and consider dividing the intention–behavior gap into a positive gap – caused by disinclined actors – and a negative gap – caused by inclined abstainers – as these respective groups have a very different impact on establishing MOOC success or dropout and subsequently MOOC (re)design. Furthermore, the impact of variables defined in the model by Tinto (1975) and confirmed in a distance education context need to be evaluated with regard to their impact on the three different types of MOOC learners we have proposed in this paper.

A limitation which needs to be taken into consideration is the weight or effort of the intention in comparison to the actual behavior. A MOOC-taker who intended to download materials but ends up finishing one module is regarded as a disinclined actor. A MOOC-taker who intended to download materials but ends up completing the course is also regarded as a disinclined actor. Yet, the weight of the difference in behavior is substantial; the step from downloading to finishing one module requires less effort than the step from downloading to completing the course. Future research should take this into account by, for instance, applying weighted factors to intention–behavior data.

Furthermore, several methodological issues regarding the measurement of intention–behavior should be taken into account (Fishbein & Ajzen, 2010; Hassan, Shiu, & Shaw, 2014; Sutton, 1998). For our follow-up studies we regard these methodological issues as guidelines for the development of our intention–behavior scales. One of the issues concerns when to measure intention and behavior, because the timing can be of great influence on the correlation between intention and subsequent behavior as intentions may change due to various reasons (Sutton, 1998; Tinto, 1982). Consequently, ‘the more distal the behaviour is when intention is measured, the less likely the intention will provide an accurate prediction of the then intended behavioural enactment’ (Hassan et al., 2014, p. 7). This indicates that the longer the time between measuring the formed intention and measuring the subsequent behavior, the more likely it is that they don’t match. Also, scale correspondence (Sutton, 1998, p. 1328) and the principle of compatibility (Fishbein & Ajzen, 2010, p. 44) should be considered, as described in the Method section.

Lastly, some general issues should be noted. The MOOC-takers who participated in both questionnaires are likely to belong to the group of MOOC-takers with higher intentions. This leads to survival bias (mostly MOOC-takers who ‘survive’ until the end of the MOOC participate in both questionnaires), a form of selection bias that can occur in MOOCs (Reich, 2014) and should be taken into consideration when interpreting the results. Also, the samples are relatively small, especially the matched intention–behavior data from the pre- and post-course questionnaires. Future studies should strive to increase the number of MOOC-takers who complete both pre- and post-course questionnaires. Yet when interpreting results, using self-reporting for measuring intention and behavior might not be as accurate as independent observation. In the context of MOOCs, however, independent observation will not suffice for establishing individual intentions and possible re-formulation of intentions.

In conclusion, with our proposed MOOC-taker typology we aim to underline the importance of individual perspectives when assessing MOOC success and dropout, thus taking into consideration that individual goal achievement does not necessarily matches goal achievement from the institutional perspective. This does not mean that it should replace the institutional perspective, but rather complement it. For MOOC-takers who want to gain institutional credit in the form of a certificate and thus need to demonstrate performance in line with certain institutional criteria that were set, the institutional perspective is valuable and necessary. Although further research needs to validate the practical applicability of the typology, it is a first step towards more profound and theoretically grounded research into dropout in MOOCs.

Funding

This work was supported by the Netherlands Initiative for Education Research (NRO), The Netherlands Organisation for Scientific Research (NWO), and the Dutch Ministry of Education, Culture and Science [grant number 405-15-705].

Notes on contributors

Maartje A. Henderikx is a PhD candidate at the WELTEN Institute of the Open University of the Netherlands. Her PhD research, which focusses on motivation and intention as key to dropout in MOOCs, is part of the SOONER project, a five-year project on open online education in the Netherlands.

Karel Kreijns is associate professor at the Welten Institute of the Open University of the Netherlands. His research interests are social aspects of computer-supported collaborative learning (CSCL) and networked learning, teachers’ use of technology/open educational resources, and BIE-coaching (bug-in-ear technology) to improve the quality of teaching.

Marco Kalz is full professor and UNESCO chair of open education at the Faculty for Management, Science and Technology and the Welten Institute of the Open University of the Netherlands. His research interest lies in the use of open education, pervasive technologies, and formative assessment to support self-directed lifelong learning.

Disclosure statement

No potential conflict of interest was reported by the authors.

Appendix 1

These are the general standard questions, which will be aligned with the design of the respective MOOC before use.

Pre-course questionnaire

Table

In this MOOC, I intend to …
	• Browse • Browse and download learning materials • Participate in some learning activities of … * ... modules and optionally browse and download learning materials* • Participate in most learning activities of … * ... modules and optionally browse and download learning materials* • Participate in all learning activities of … * … modules and optionally browse and download learning materials* • Participate in all learning activities and strive for a certificate of participation (pass at least 75% of the course) • Participate in all learning activities and strive for a certificate of accomplishment (pass 100% of the course)	*
		Only one module
		Two modules
		Three modules
		Four modules
		All modules

Note: Single response. The Asterisks signifies the number of modules that a MOOC-taker intents to participate in.

Post-course questionnaire

Table

In this MOOC, I have …
	• Browsed • Browsed and downloaded learning materials • Participated in some learning activities of … * … .. modules and optionally browsed and downloaded learning materials* • Participated in most learning activities of … * ... modules and optionally browsed and downloaded learning materials* • Participated in all learning activities of … * ….. modules and optionally browsed and downloaded learning materials* • Participated in all learning activities and earned a certificate of participation (pass at least 75% of the course) • Participated in all learning activities and earned a certificate of accomplishment (pass 100% of the course)	*
		Only one module
		Two modules
		Three modules
		Four modules
		All modules

Note: Single response. The Asterisks signifies the number of modules that a MOOC-taker intents to participate in.