3,570
Views
21
CrossRef citations to date
0
Altmetric
Articles

A set of patterns for the structured design of MOOCs

&

Abstract

A design pattern approach, in the form of participatory pattern workshops, has been used to explore the design approaches that experts in the field of online learning have used to develop and deliver Massive Open Online Courses (MOOCs). Over the course of 3 intensive workshops a total of 20 design patterns were developed from shared narratives of successful practice. These patterns describe solutions to problems that are contextualised to six design dimensions: structure; orientation; participation; learning; community and management. The validity of these patterns has been tested against novel design challenges. In this paper, we present the 20 design patterns as a scaffold for both novice and expert developers to build a MOOC and suggest that integrating design patterns into a simple iterative design cycle can provide a powerful course development approach.

Introduction

The rapid rise in massive open online courses (MOOCs) has led to a series of different approaches to their delivery, pedagogy, functionalities and support mechanisms. Some of these have been successful and others not so successful, for example, we have witnessed high variability in the documented retention rates across different MOOC offerings (Jordan, Citation2014). There have been a small number of systematic reviews of the MOOC landscape. This includes Liyanagunawardena, Adams, and Williams Citation2013, who looked across the MOOC literature, classifying the publications on MOOCs during a six-year period from 2008 onwards. In a more recent study by Margaryan, Bianco, and Littlejohn (Citation2015), a selection of 76 MOOC offerings were analysed for their quality of instructional design. They found that the majority of the MOOCs examined scored poorly against a set of predefined design principles.

The expansion in the MOOC literature space has reflected the appetite of academics and commenters in interrogating the nature and form of the MOOC phenomenon, with both positive and negative debate evident. Baggaley (Citation2013, Citation2014) has written several commentaries that have plotted the to-and-fro, from initial hyperbole to a more measured detachment and the breaking up of the MOOC design landscape into more granular forms that have included the smaller private online courses. These clearly resemble a more traditional distance education model, albeit with a stronger technology base. The exposure of MOOCs in publications such as the UK Government Department for Business Industry and Skills research papers series (Haggard, Citation2013), which covers much of the current literature, is reflective of the breadth of impact they have already had on the education sector. The MOOC research area itself has also been well represented through projects and associated work that has been brought together at the MOOC research hub (http://www.moocresearch.com/). The statistical analyses of MOOC participation rates and entry and exit velocity are plentiful with the major MOOC provider platforms capturing large amounts of data to be fed into analytic algorithms. These kinds of reports have been valuable but as yet have had limited impact on MOOC development, particularly in the area of enhancing MOOC designs and helping novice MOOC designers benefit from the expertise of others.

It is within this design territory that this paper situates itself. The goal of the MOOC design patterns project has been to explore, define and articulate the emerging design principles and patterns that underpin the development and delivery of massive open online courses and to demonstrate them by their application to the design of new MOOCS. The project has been ambitious in adopting a multidimensional approach that has incorporated input from diverse but complementary perspectives. These have included designers, deliverers, researchers, learners and tutors who have been engaged in the area of MOOCs and Open and Distance Learning development more broadly.

The project has been driven by a set of key questions that revolved around a desire to understand the design processes and mechanisms by which we come to create and deliver open and distance learning at scale and by extension how we can formulate this into sharable design solutions that can be applied by others. This is particularly pertinent where we are observing differentiation and varying degrees of success in the current MOOC landscape, reflected in:

choice of delivery mode and platform;

style of MOOC (‘x’ vs. ‘c’ vs. ‘p’ and so on);

reported experiences from learners;

reported experiences from tutors;

increased use of motivational schemes such as micro-certification and badging; and

reported retention figures.

The particular methodological approach in this project draws on previous work in the field of design patterns and pattern languages. The design patterns paradigm (Alexander, Ishikawa, & Silverstein, Citation1977) was developed as a form of design language within architecture. Several studies (Ronen-Fuhrmann, Kali, & Hoadley, Citation2008; Voogt et al., Citation2011) have demonstrated the value of engaging in design for educators and extensive research over the last decade highlights the complexity of learning design and the design of learning technologies (Beetham & Sharpe, Citation2007; Mor & Winters, Citation2007). It has been argued that this complexity calls for novel approaches to the articulation, validation, sharing and application of design knowledge and here, design patterns can be viewed both as ‘solutions to problems’ and also developed as a way to support theory-praxis conversations (Goodyear, de Laat, & Lally, Citation2006). As defined by Alexander et al. (Citation1977), design patterns represent themselves as ‘… a formula that describes a problem which occurs over and over again, and then the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice’. Here a scaffolded approach to developing, refining and testing patterns has been adopted using the Participatory Patterns Workshops (PPW) methodology (Mor, Citation2013; Mor, Warburton, & Winters, Citation2012). The work here builds on and extends previous projects where the PPW methodology has been successfully used, including: The Pattern Language Network (Planet) project (funded by JISC) produced over 100 design narratives, close to 30 design patterns and 13 scenarios; the formative e-assessment project FEASST (funded by JISC) produced 10 patterns (Mor, Mellar, Pachler, & Daly, Citation2011); the MUVEnation project (funded by the EU) produced 28 design patterns, over 80 design narratives and more that 20 design scenarios on the use of immersive virtual worlds for learning and teaching (Warburton, Citation2009).

Methodology

The SNaP! (Scenarios, Narratives and Patterns) methodology has been implemented in the form of PPW. The Participatory Methodology for Practical Design Patterns (Mor, Citation2013; Mor et al., Citation2012) is a process by which communities of practitioners and experts collaboratively reflect on the challenges they face and the methods for addressing them (Figure ). Participants share accounts of their experiences prior to the workshop series, which are formulated as design narratives and then collaboratively extract design patterns from these over the course of three workshop sessions. The workshop approach is one where participants move from:

(1)

sharing case story narratives of successful practice grounded in their experience in the domain of interest;

(2)

to abstractions of the problem space, solution and context of these shared narratives in the form of design patterns; and

(3)

to validating the design patterns against novel design scenarios, in other words, new and challenging problem spaces.

Figure 1. Overview of the participatory pattern workshop methodology with auxiliary support toolkit (Mor et al., Citation2012).

Figure 1. Overview of the participatory pattern workshop methodology with auxiliary support toolkit (Mor et al., Citation2012).

One of the strengths of the PPW methodology is that it is itself written in the form of a pattern language and therefore possesses an inherent flexibility in the style and format of the workshops. In this project, the configuration of the workshop series was adjusted to promote a more rapid pathway from pattern extraction to testing patterns against novel design challenges. The workshops (with attendee numbers) were deployed as follows:

Workshop 1 (25–30 participants): sharing of design narratives using the ‘Three Hats’ activity, followed by the development of proto-design patterns (based as far as possible) on the ‘Rule of Three’ i.e. each design pattern is supported by three design narratives.

Workshop 2 (20–25 participants): reviewing, refactoring and iteration of the design patterns; aggregating and mapping of patterns; rapid design activity testing the design patterns against novel design scenarios.

Workshop 3 (8 participants): the final session was run with selected design patterns in the form of a writer’s workshop (Gabriel, Citation2002). The aim being to finalise these chosen design patterns for release to publication and to adapt elements to a set of learning design principles for future open online course development.

To provide continuity before during and after each workshop a suite of online tools were deployed to support collaborative and sharing processes:

Googlesites (http://www.moocdesign.cde.london.ac.uk/) provided a flexible web presence to organise and engage participants in their journey through the PPW process and the activities that were conducted during each of the workshops.

The ‘Learning Design Grid’ (http://www.ld-grid.org/) provided a resource of tools and instructions to the approach being used.

The ‘Integrated Learning Design Environment’ (http://ilde.upf.edu/moocs/) was used as an online tool to support the recording, sharing and editing of the design narratives, patterns and scenarios. This has acted as a living repository for the outputs of the MOOC design patterns project.

Results

The workshops were conducted over a period of several months and the ILDE online tool was used to gather and share participant outputs in the form of design narratives, patterns and scenarios. Examples of each of these three output areas are illustrated below.

Design narratives

The MOOC design narratives were gathered from participants in advance of the first workshop sessions and followed a STARR (Situation, Task, Actions, Results and Reflection) template format. Participants were asked to submit these as narratives. This process of ‘storytelling’ was valuable in helping to structure practices (good or bad) in an emerging field, where perceptions of these practices vary widely amongst practitioners. It also resists a temptation for over-analysis and over-theorisation that may interfere with the description of practices. Thus, storytelling in the form of design narratives was used here as a method to synthesise and articulate personal experiences, communicate feelings and construct meaning. A total of 25 design narratives were collected. Two of the design narratives from the project are shown here:

Design Narrative 1

Name: Success Criteria

(1) Situation (Describe the context, including social, technological and organisational setting/s)

An open online taster course for Systems Biology, four weeks in length and delivered at Masters (MSc) level. It was designed to give potential Postgraduate Taught students a taste of the subject area i.e. disciplinary background, methodologies, tools, analysis, types of output, research, and potential employment prospects if you studied the full MSc course.

(2) Task (What were the aims, the problem you were trying to solve or the intended effect/outcome?)

Define a set of success criteria for the course that could be agreed by all stakeholders, be meaningful to the senior management team and provide a basis for later evaluation of the course.

(3) Actions (What happened? What was done to accomplish the task?)

The course management team put together a set of goals and success criteria for the running of the course and then circulated these for comment by all stakeholders. These comments were then integrated into the criteria. These success criteria were listed and then further defined according to three different categories:

(1)

Minimum expectation (what we want at a base level).

(2)

Normal expectation (what we would like, and generally be happy with).

(3)

Bonus (what would be a bonus).

(4) Results (Was the aim achieved? If not, why? Did you make any adjustments?)

After some debate, an agreed set of success criteria were documented against which we developed and then ran the programme. One of the key items on this list was a realistic expected completion rate, bearing in mind the evidence from other MOOCs on the high drop-off rate that was likely to be experienced.

(5) Reflection (What did you learn from the process?)

Having a clear set of goals is vital when you have multiple stakeholders involved in a project. What may be seen as success in the management team’s eyes may be seen entirely differently by the marketing team and likewise by the learning technologists and designers. What made this a valuable exercise was that the agreed goals kept everyone focused and the team did not become distracted by making changes mid-stream. For example, it stopped any over reaction to the drop off curve and the decline in numbers was accepted as normal for a MOOC. Therefore in our agreed success criteria, the potential high drop off rate was already accounted for.

Design Narrative 2

Name: Moodle Glossary:

(1) Situation

This was a small scale professional development MOOC for teaching staff across the five Bloomsbury Colleges of the University of London. The aim was to provide an experience of online learning whilst imparting Moodle skills to support online and/or blended learning. Participants were time-poor, so we wanted both to provide easy ways into engaging with others online, whilst simultaneously modelling effective Moodle learning designs that teachers could use in their own courses.

(2) Task

Add an entry to a course glossary

(3) Actions

Entries are added by students to a course glossary that are auto-linked to concepts used in the class. Participants are invited to contribute an entry to the glossary. They follow the entry template to add a definition of a concept (including weblink and picture). Concepts used in the course are highlighted as they are linked to the glossary. Clicking on a highlighted link brings up the definition provided by a course participant. There is an opportunity to discuss the definitions by the students though the commenting function in the glossary itself. The co-constructed resource develops over the life of the course as participants add entries, and more and more concepts are auto-linked to definitions in the glossary.

(4) Results

Students co-construct the shared resource. Students feel ownership of the course and are able to see their growing body of shared knowledge. The glossary can be used to provide a focus for discussion between participants. It is easy for participants to take part in the collaboration since it is an individual task but one that results in a shared output.

(5) Reflection

This was effective in engaging participants to add entries and to view each other’s entries (the glossary activity was the 4th most frequently viewed activity in the course, including the introduction forum). Students of all types can be reluctant to engage in some collaborative tasks, such as editing each other’s contributions. However, here participants were not required to edit each other’s work and were not dependent on others to complete the task. Nevertheless, they were still contributing to a jointly produced resource, and they were able to feel a sense of communal contribution. It also offered something to participants who did not contribute a definition, since they could benefit from seeing others’ contributions. There is an element of risk for a teacher wishing to use this learning design, however, since the lack of tutor-moderation meant that students could get a definition wrong, which is then circulated to the rest of the class getting things wrong. However, the glossary relies on the same principles of crowd sourced knowledge as in projects like Wikipedia, where the group moderates itself, reducing the risk of errors. This could be further facilitated by allowing multiple entries for the same concept, and commenting or voting/rating to highlight particularly good definitions. This would have the added benefit of foregrounding the partiality of apparent statements of truth and the possibility of different perspectives on the topic, to develop critical and independent thinking.

Design patterns

Twenty MOOC design patterns were developed from discussion and interrogation of the submitted design narratives over the course of two workshop sessions. These range from alpha to beta (iterated) state and foreground particular areas of interest such as design principles for participation, delivery mechanisms and the creation of ‘participation pathways’ adapted or adaptable to specific learner profiles. The full list of design patterns is organised in Table and indicates the pattern title, a brief description of the problem space that the pattern addresses and the brief solution statement. The full patterns are publically available from the MOOC ILDE site (http://ilde.upf.edu/moocs/).

Table 1. List of MOOC design patterns.

At the end of the phase, a mapping activity was used to organise the MOOC design patterns into key design decision domains, with one meta-design pattern sitting outside these core elements (Figure ). These key decision areas are integral to the majority of MOOC development projects (Table ).

Figure 2. Design pattern mapping showing the overlapping design domains.

Figure 2. Design pattern mapping showing the overlapping design domains.

Table 2. MOOC design patterns organised by design domain.

Design scenarios

The design scenarios represented the final stage of the design pattern development activity. A total of five novel design scenarios were explored. These were used as design challenges against which the design patterns were tested.

Design Scenario example from the MOOC design project:

(1) Name

‘New to online learning’

(2) Context

A MOOC in India on open practices and Open Educational Resources, aimed at teacher educators.

(3) Challenge

Teacher educators in the identified cohort have good educational backgrounds in practice, but little to no experience in online learning. How do we ensure that they get maximum benefit from following the MOOC?

(4) Patterns used

Know Your Audiences

Induction

Drumbeat

Knowing the Story

Crowd bonding

Chatflow

Fishbowl

(5) Proposed Solution

Create a series of ‘Induction’ materials to help participant orientation, and target these materials by applying ‘Know your Audiences’. Then set up a regular ‘Drumbeat’ communication mechanism, and make sure that they can navigate the course through ‘Knowing the Story’. Use ‘Crowd Bonding’ and ‘Chatflow’ to support activity in the discussion forums and converge the participants together at pre-defined intervals for ‘FishBowl’ sessions.

Discussion

The outputs of this project have included a set of design patterns and a prospective pattern language to support the continued development of MOOCs in relation to the particular design challenges this form of open and distance learning presents. The PPW methodology itself has demonstrated how rich it can be as a source for generating material. In the project, a total 25 design narratives, 20 design patterns and 5 design scenarios were produced. The MOOC design space is still nascent and therefore it could be argued that the quality of patterns derived within this domain remain to be tested as expertise here grows. Yet the MOOC phenomenon is built upon a long history of innovation in distance education. There is a substantial body of literature surrounding distance education that spans, for example, early work on transactional distance and the emerging impact of new technologies (Bates, Citation1995; Moore & Kearsley, Citation1996) through to modern approaches that acknowledge more open access to learning and the impact of the Internet on modes of study. Innovations in pedagogy are also represented in broad overviews such as Moore and Anderson (Citation2003) and the range of academic journals in the field of distance learning. The number and range of patterns that have been developed in this project also reflects the underlying stability and duration of expertise in the field of distance education that underpins much of the MOOC design activity and analysis. It is this expertise that by extension has fed into the rich pool of design knowledge in this area. The mapping activity does reveal some gaps, particularly in the area of online assessment, which remains under-represented. Meaningful formative assessment at scale is an area of interest for all MOOC providers (Ferguson & Sharples, Citation2014) with current practice strongly moving towards peer review (Nicol & Macfarlane‐Dick, Citation2006) style approaches.

The preliminary grouping of patterns is a first step towards establishing a coherent pattern language for MOOC development. A design language is where patterns come into their own, in other words, by being connected together. A pattern language being ‘a collection of such patterns organised in a meaningful way’ (Dearden & Finlay, Citation2006). The language structure underpins the pattern approach to design as it allows the practitioner to work through a series of related patterns to generate a complete design that could be a piece of software or in this case a learning experience in the form of a MOOC.

There is some overlap between the pattern elements and this suggests some refactoring is still needed. This suggests a direction for future work which would include linking in adjacent and complementary pattern languages. One meta-pattern did appear in the area of developing a strong and well-balanced MOOC development team and this requirement is echoed in other publications that have explored collaborative online course development (Xu & Morris, Citation2007). The design scenario workshop is the process that was used to test these patterns as valid design scaffolds in a form that resonates with participatory design mechanisms (Dearden, Finlay, Allgar, & McManus, Citation2002). This is a critical part of the PPW methodology. It is the oscillation between the pattern and the designed output (via design scenarios) that forms an important iterative move in validating and refining the patterns themselves. This process opens the design patterns and language to scrutiny as a design object.

Conclusions

The PPW approach is a powerful methodology for coalescing and then abstracting expert knowledge, but this comes at a price in terms of time, effort and organisation. This to some extent is mitigated by the use of an online tool set to support inter-workshop activity and processes. The advantage of organising face-to-face sessions is the eventual quality of the patterns that can be developed through an intensive process of shared scrutiny. The work presented here creates a starting point for further activity in the domain of MOOC design and development using design patterns. We suggest the use of meta-design patterns or an approach based on a simple design cycle to scaffold the MOOC design process and provide an entry point where patterns can be deployed (Figure ).

Figure 3. A six-step design model indicating an entry point for design pattern use.

Figure 3. A six-step design model indicating an entry point for design pattern use.

In this model, the move from design challenge to research of the problem space can be cyclical and highlights the importance of questioning and refining the initial design. Critically, new design perspectives are likely to be gained through the prototyping and evaluation stages and these design insights can be fed back into the design patterns themselves, adding evidence and refinements that future designers can benefit from. Efforts in this direction have been explored through the double loop design (Warburton & Mor, Citation2015) approach where the PPW methodology has been linked with the Learning Design Studio (Mor & Mogilevsky, Citation2013) via the pivotal point of the design scenario. This emphasises the importance of documenting the future application of these patterns as applied against novel design scenarios. It is this feedback that will enhance the patterns themselves and ultimately impact on the success of using a design pattern approach to support the structured design of MOOCs and open online courses more generally.

Notes on contributors

Steven Warburton is the head of department of Technology Enhanced Learning at the University of Surrey. He is interested in educational design research, and is currently working in the area of active learning spaces.

Yishay Mor is the educational design scientist at PAU education. He is interested in design practices and design research in education, and the role of narratives and design patterns in connecting theory and practice and sharing knowledge between professionals.

Acknowledgements

The project team thank all participants who have attended the Participatory Pattern Workshops held in London during 2014/15.

Disclosure statement

No potential conflict of interest was reported by the authors.

Funding

University of London International Academy under the Centre for Distance Education’s Teaching and Research Award Scheme.

References

  • Alexander, C., Ishikawa, S., & Silverstein, M. (1977). A pattern language: Towns, buildings construction. New York, NY: Oxford University Press.
  • Baggaley, J. (2013). MOOC rampant. Distance Education, 34, 253–263.
  • Baggaley, J. (2014). MOOC postscript. Distance Education, 35, 126–132.10.1080/01587919.2013.876142
  • Bates, A. (1995). Technology, open learning and distance education. London: Routledge.
  • Beetham, H., & Sharpe, R. (Eds.). (2013). Rethinking pedagogy for a digital age: Designing for 21st century learning (2nd ed.). Abingdon: Routledge.
  • Dearden, A., & Finlay, J. (2006). Pattern languages in HCI: A critical review. Human-Computer Interaction, 21, 49–102.10.1207/s15327051hci2101_3
  • Dearden, A., Finlay, J., Allgar, E., & McManus, B. (2002). Evaluating pattern languages in participatory design. In L. Terveen & D. Wixon (Eds.), CHI 2002, Conference extended abstracts on human factors in computer systems (pp. 664–665). Minneapolis, MN: ACM Press.
  • Ferguson, R., & Sharples, M. (2014, September 16–19). Innovative pedagogy at massive scale: Teaching and learning in MOOCs. 9th European conference on technology enhanced learning (EC-TEL 2014): Open learning and teaching in educational communities, (pp. 98–111). Graz: Springer.
  • Gabriel, R. P. (2002). Writer’s workshops and the work of making things. Boston, MA: Addison-Wesley Longman.
  • Goodyear, P., de Laat, M., & Lally, V. (2006). Using pattern languages to mediate theory-praxis conversations in design for networked learning. ALT-J: Research in Learning Technology, 14, 211–223.10.1080/09687760600836977
  • Haggard, S. (2013). The maturing of the MOOC: Literature review of massive open online courses and other forms of distance learning. Retrieved February 15, 2015, from https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/240193/13-1173-maturing-of-the-mooc.pdf
  • Jordan, K. (2014). Initial trends in enrolment and completion of massive open online courses. The International Review of Research in Open and Distributed Learning, 15, 133–160.
  • Liyanagunawardena, T. R., Adams, A. A., & Williams, S. A. (2013). MOOCs: A systematic study of the published literature 2008–2012. The International Review of Research in Open and Distributed Learning, 14, 202–227.
  • Margaryan, A., Bianco, M., & Littlejohn, A. (2015). Instructional quality of massive open online courses (MOOCs). Computers & Education, 80, 77–83.
  • Moore, M. G., & Anderson, W. G. (2003). Handbook of distance learning. Mahwah, NJ: Lawrence Erlbaum Associates.
  • Moore, M. G., & Kearsley, G. (1996). Distance education: A systems view. Belmont, CA: Wadsworth.
  • Mor, Y. (2013). SNaP! Re-using, sharing and communicating designs and design knowledge using scenarios, narratives and patterns. In R. Luckin, S. Puntambekar, P. Goodyear, B. L. Grabowski, J. Underwood, & N. Winters (Eds.), Handbook of design in educational technology (pp. 189–200). London: Routledge.
  • Mor, Y., Mellar, H., Pachler, N., & Daly, C.(2011). Formative e-Assessment: Case stories, design patterns and future scenarios. In C. Kohls & J. Wedekind (Eds.), Investigations of e-learning patterns: Context factors, problems and solutions (pp. 199–219). Hershey, PA: Information Science. 10.4018/978-1-60960-144-7
  • Mor, Y., & Mogilevsky, O. (2013). The learning design studio: Collaborative design inquiry as teachers’ professional development. Research in Learning Technology, 21, 22054. Retrieved from http://dx.doi.org/10.3402/rlt.v21i0.22054.
  • Mor, Y., Warburton, S.,& Winters, N. (2012). Participatory pattern workshops: A methodology for open learning design inquiry. Research in Learning Technology, 20, 19197. Retrieved from http://dx.doi.org/10.3402/rlt.v20i0.19197.
  • Mor, Y., & Winters, N. (2007). Design approaches in technology enhanced learning. Journal of Interactive Learning Environments, 15, 61–75.10.1080/10494820601044236
  • Nicol, D., & Macfarlane‐Dick, D. (2006). Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31, 199–218.10.1080/03075070600572090
  • Ronen-Fuhrmann, T., Kali, Y., & Hoadley, C. (2008). Helping education students understand learning through designing. Educational Technology, 48, 26–33.
  • Voogt, J., Westbroek, H., Handelzalts, A., Walraven, A., McKenney, S., Pieters, J., & de Vries, B. (2011). Teacher learning in collaborative curriculum design. Teaching and Teacher Education, 27, 1235–1244.10.1016/j.tate.2011.07.003
  • Warburton, S. (2009). WP4c report on design patterns for virtual worlds. Retrieved February 17, 2015, from http://dl.dropbox.com/u/4424450/KCL_v6_MUVENATIONDesignPatternsWP4.pdf
  • Warburton, S., & Mor, Y. (2015). Double loop design: Configuring narratives, patterns and scenarios in the design of technology enhanced learning. In Y. Mor, M. Maina, & B. Craft (Eds.), The art and science of learning design (pp. 93–104). Rotterdam: Sense.10.1007/978-94-6300-103-8
  • Xu, H., & Morris, L. (2007). Collaborative course development for online courses. Innovation in Higher Education, 32, 35–47.10.1007/s10755-006-9033-5