Speculative method in digital education research

ABSTRACT

The question of ‘what works’ is currently dominating educational research, often to the exclusion of other kinds of inquiries and without enough recognition of its limitations. At the same time, digital education practice, policy and research over-emphasises control, efficiency and enhancement, neglecting the ‘not-yetness’ of technologies and practices which are uncertain and risky. As a result, digital education researchers require many more kinds of questions, and methods, in order to engage appropriately with the rapidly shifting terrain of digital education, to aim beyond determining ‘what works’ and to participate in ‘intelligent problem solving’ [Biesta, G. J. J. 2010, “Why ‘What Works’ Still Won’t Work: From Evidence-Based Education to Value-Based Education.” Studies in Philosophy and Education 29 (5): 491–503] and ‘inventive problem-making’ [Michael, M. 2012, “‘What Are We Busy Doing?’ Engaging the Idiot.” Science, Technology & Human Values 37 (5): 528–554]. This paper introduces speculative methods as they are currently used in a range of social science and art and design disciplines, and argues for the relevance of these approaches to digital education. It synthesises critiques of education’s over-reliance on evidence-based research, and explores speculative methods in terms of epistemology, temporality and audience. Practice-based examples of the ‘teacherbot’, ‘artcasting’ and the ‘tweeting book’ illustrate speculative method in action, and highlight some of the tensions such approaches can generate, as well as their value and importance in the current educational research climate.

KEYWORDS:

• Digital education research
• methodology
• speculative method
• inventive approaches
• not-yetness

1. Introduction

Emerging technologies in education, as defined by Veletsianos (2010), are those which are ‘not yet fully understood’ and ‘not yet fully researched, or researched in a mature way’ (15). It is not only technologies, but also practices, subjectivities and pedagogies involved with them which are marked by this ‘not-yetness’ (Collier and Ross, forthcoming). At the same time, the relationship of educators, institutions and educational researchers to technology is one that has often been characterised by attempts at control, efficiency and enhancement (Bayne 2015a), underplaying more ‘disruptive, disturbing and generative dimensions’ (7). Working with the not-yetness of digital education means engaging with complexity, uncertainty and risk, not as factors to be minimised or resolved, but as necessary dimensions of technologies and practices which are unknown and in flux, or what Barnett and Hallam (1999) call ‘conditions of radical and enduring uncertainty, unpredictability, challengeability and contestability’ (142). Not-yetness offers a conceptual handle for digital education approaches that ‘help us stay open to what may be genuinely surprising about what happens when online learning and teaching meets emerging technologies’ (Collier and Ross, forthcoming). This paper argues that in such a sphere of not-yetness, we must work with approaches to research which aim beyond determining ‘what works’, to engage in ‘intelligent problem solving’ (Biesta 2010) and ‘inventive problem-making’ (Michael 2012). These approaches can produce valuable insights and contribute to a flourishing ecosystem of knowledge practices that can respond flexibly to not-yetness.

Hybridity, interdisciplinarity and boundary crossings are cast here as appropriate and necessary responses to ‘not-yetness’, especially in a climate of educational research in which the value of evidence-based practice is enshrined in policy and in funding priorities (Biesta 2010), and where uncertainty and risk are seen as unwelcome, or as flaws in research design (Gough 2010). In practice, the question of ‘what works’ is typically answered with large-scale experimental or quasi-experimental studies designed to be replicable and generalisable and to focus on identifying whether interventions, necessarily incrementally and narrowly defined, produce measurable increases in student achievement. This question, and the approaches it generates, is limited in its ability to investigate the complexity of ‘inputs’ and ‘outputs’ in education:

we should understand that educational processes ought to be characterised by gaps between ‘inputs’ (policy, curriculum, pedagogy) and ‘outputs’ (learning) … what we have previously imagined to be ‘outcomes’ or ‘products’ … emerge in and through educational processes in unique and unpredictable ways. (Gough 2010, 26)

We require many more kinds of questions, and methods, in order to engage imaginatively with the rapidly shifting terrain of digital education, and to respond creatively and critically to claims of ‘disruption’ and the ‘narratives of promise and threat’ (Hand 2008) that accompany social and technological change.

This paper brings ‘speculative method’ concepts of critical design, speculative design and design fiction from the social sciences and from fields of art, design and human computer interaction to bear on digital education research. Speculative or ‘inventive’ research: ‘is explicitly oriented towards an investigation of the open-endedness of the social world. … the happening of the social world – its ongoingness, relationality, contingency and sensuousness’ (Lury and Wakeford 2012, 2, emphasis in original). Such speculative approaches are aimed at envisioning or crafting futures or conditions which may not yet currently exist, to provoke new ways of thinking and to bring particular ideas or issues into focus.

Digital education research at present is overly focused on the dominant ‘what works’ agenda of educational research (Selwyn 2012), and consequently lacks the imaginative resources to stake a strong position at the edges of educational change, where it is urgently needed. Speculative approaches can bring new practices and ideas into being while maintaining space for curiosity, critique, doubt, unintended consequences and emergent properties of technologies in use. They have a quality of participant-observation, interweaving insider and outsider perspectives, and they are by necessity ‘baroque’ methods, as described by MacLure: ‘resist[ing] clarity, mastery and the single point of view [and] be[ing] radically uncertain about scale, boundaries and coherence’ (2006, 731). Speculative methods in education may tend to blur boundaries between research, design and teaching, and therefore to provoke questions about how best to understand them as methods, and about the nature of the researcher’s responsibilities when adopting such approaches: questions which will be discussed in what follows.

This article begins with a critique of an over-emphasis on evidence-based research in digital education, and argues that a response to the narrowness of research horizons in digital education is needed. One fruitful approach is to critically explore and generate a greater range of methods, including those which have been described as ‘inventive’ or ‘speculative’. It moves on to define and explore speculative methods in terms of epistemology, temporality and audience, before discussing three examples of such methods at work in digital education: the ‘teacherbot’, ‘artcasting’ and the ‘tweeting book’. It closes with some reflections on ways forward for speculative methods in education research.

2. Evidence, complexity and the limits of ‘what works’

Educational researchers are currently inundated by calls for evidence-based practice, and what counts as ‘evidence-based’ is increasingly circumscribed. For example, in the UK, randomised controlled trials, where research participants are randomly allocated to either a baseline ‘control’ group or one or more ‘intervention’ groups, have recently been recommended by the Department for Education in England (Goldacre and Plant 2013) as the ‘gold standard’ for educational research. MacLure (2006) describes this move as ‘animated by the desire for certainty, willing to sacrifice complexity and diversity for “harder” evidence and the global tournament of standards’ (730).

In digital education, one of the most influential strands of thought about evidence-based practice currently comes in the form of what is called design-based research, which emerged as a specific response to claims that educational research is too divorced from practice and needs to be made more valuable by having more direct impacts. This research approach is explicitly about ‘mov[ing] beyond a particular design exemplar to … generate evidence-based claims about learning that address contemporary theoretical issues’ (Barab and Squire 2004, 6). Anderson and Shattuck (2012) characterise design-based research as aiming to ‘increase the impact, transfer, and translation of education research into improved practice’ (16). In their 2012 review of the literature on design-based research, Anderson and Shattuck found that ‘the majority (68%) of interventions involved the use of online and mobile technologies’ (23). Hoadley (2007) describes design-based research as an empiricist method belonging to the ‘learning sciences’ (147), particularly suitable to e-learning research, which ‘aims to provide theories, tools, activities, and design models’ (152). However, Walker (2011) points out that this method achieves some of its reputation for utility at the expense of meaningful engagement with epistemological issues (53–54).

Biesta’s (2007, 2010) papers on evidence-based practice highlight some epistemological difficulties inherent in making claims of generalisability (‘what works’) for experiment-based educational research. He identifies three areas of insurmountable deficit in evidence-based practice – in knowledge, efficacy and application – and argues that these deficits ought to lead us to a much more critical position in relation to evidence than is usually seen (Biesta 2010).

The ‘knowledge deficit’ refers to the inability of experiment-based evidence to provide ‘rules for action’ because such evidence is based on ‘transactional epistemology’ – where the world ‘changes as a result of our interventions’. We are ‘participants in an ever-evolving universe’, and the future can therefore not be predicted with certainty (2010, 495–6), nor can action be prescribed purely on the basis of previous experiments. An ‘efficacy deficit’ comes about because education is an ‘open recursive semiotic system’ – interactions within the system are probabilistic, feed back into themselves, and are based on meaning-making between actors. The relationships between interventions and results or effects are therefore non-linear, so making sense of educational realities involves making choices about what to pay attention to (‘complexity reduction’), which is inevitably a political act (496–497). Finally, the ‘application deficit’ describes the particular kind of knowledge experiment-based methods produce in the ‘lab’, which allows for effective interventions in the world only to the extent that the world is changed to accommodate it. Biesta, drawing on Latour, calls this ‘the work that is needed to transform the outside world so that knowledge becomes applicable’ (499). As with the efficacy deficit, the work taken to bridge the application deficit is political, and therefore can never leave aside questions of values and priorities – otherwise ‘educational practice would be entirely without direction’ (500).

Biesta argues that research can only give an ‘understanding of possibilities’ and of ‘what the problem might be’, not tell us what to do. He invites us to think about research of all kinds as a support for the ‘intelligent selection of possible lines of action’ (2007, 16). For this reason, the question of ‘what works’, of what educational techniques are effective, is a problem because it obscures its own contestability:

forgetting, among other things, that what counts as ‘effective’ crucially depends on judgments about what is educationally desirable. … The focus on ‘what works’ makes it difficult if not impossible to ask the questions of what it should work for and who should have a say in determining the latter. (Biesta 2007, 5)

The politics of evidence-based research and the oversimplification of education that comes with it have been further highlighted by Gough (2012), who observes that researchers, policy-makers and others pursue ‘complexity reduction’ while neglecting to ask ‘how [it] is achieved and, perhaps more importantly, who is reducing complexity for whom and in whose interests’ (47).

While design-based research is distinct from randomised controlled trials in its focus on ‘real life’ interventions, whether designed by the researchers or by others, Engeström (2011) characterises it as ‘captive to the linear view of interventions typical of the ‘gold standard’ [of randomised controlled trials]’ (600):

the process of design research is depicted in a linear fashion, starting with researchers determining the principles and goals and leading to completion or perfection. This view ignores the agency of practitioners, students, and users. It seems blind to the crucial difference between finished mass products and open-ended social innovations, as well as between designer-led and user-led models of innovation process. (602)

Experimental and quasi-experimental research requires well-established methodological techniques, but it also requires questions and hypotheses to be framed in particular, limited ways. This can be a strength, but to believe that this, or any, research method will deliver simple answers or clear guides for action in education is to misunderstand what research does. If we accept Biesta’s claim that ‘what works won’t work’ in any straightforward fashion, the door is open for many more kinds of research, and research questions, to help us engage with problems intelligently. As Denzin (2009) puts it, ‘we must resist the pressures for a single gold standard, even as we endorse conversations about evidence, inquiry and empirically warranted conclusions’ (152).

Lather’s work on educating educational researchers has drawn attention to the problem of privileging ‘resurgent positivism’ as the gold standard and inevitable end point of educational policy and practice, when educational research has been and continues to be caught up in ‘paradigm proliferation’, which works ‘against a linear sense of development toward ‘one best way’ and ‘consensus’ approaches [and] deliberately holds together necessary incompatibilities’ (2006, 36). Acknowledging paradigm proliferation helps researchers to foster their ability to value and work with a whole range of methodologies, including those which challenge our ‘limits of intelligibility’ (41).

All educational researchers need the conceptual resources to engage with the epistemological and ontological debates that pervade the field. Digital education researchers, educators and technologists within education communities have a particular responsibility to be ready and able to engage productively as research users and creators with varied epistemological and methodological approaches. For one thing, digital education research works with ideas and methods from fields, including cultural studies, informatics and design, as well as from more traditional educational research disciplines such as psychology and sociology, and such a variety of influences and sources of knowledge inevitably will lead to the sorts of fractures and tensions that the question of ‘what works’ attempts to write out. To write them back in to enable working in interdisciplinary teams means refusing such erasure so that we can ‘engender new kinds of analytical orientations and tackle different questions’ (Enriquez 2013, 324). We must do so without succumbing to the temptation to present a tidied or oversimplified version of concepts we ‘borrow’ (Gough 2012), or attempting to resolve or dissolve them into a sanitised and ultimately unsatisfying vision of ‘best practice’. We must also foreground the volatility and ‘not-yetness’ that comes with working with emerging technologies in education. As Selwyn (2012) puts it:

There is rarely (if ever) any pre-determined outcome to the development and implementation of technologies in education – despite all the rhetoric to the contrary. Instead technologies are subjected continually to complex interactions and negotiations with the social, economic, political and cultural contexts into which they are situated. (214–215)

More nuanced approaches to evidence and practice are available, for example in shifts towards ‘evidence-informed’ rather than ‘evidence-based’ practice (Hargreaves 1999, 246), but attention is needed to the types of knowledge that ‘count’, not just how they should be counted (Biesta 2007, 5).

Finally, digital education as a field is strongly influenced by visions of the future put forward by a range of stakeholders, including political, corporate and media interests. Teachers and researchers are required to respond to the highs and lows of the hype cycle and often must defend their practices and matters of concern against a relentless orientation to the ‘next big thing’. To work with technology is to face constant evangelisation, whether of the transformative power of a digital platform or of the dehumanising danger of whatever the kids are into these days. ¹ Digital education needs researchers who can offer counter-visions which address, for example, issues of equality, diversity and social justice. Maintaining a creatively critical stance towards digital futures for education involves navigating multiple and often competing visions without succumbing to cynicism, a narrowing of perspective, or a turn away from not-yetness.

As Barnett (2013) says of the university, the field needs ‘a proliferation of ideas … if only to begin to demonstrate that things could be other than they are’ (5). Digital education research needs ‘political teeth’, and one way to accomplish this is through an ‘ontology of becoming(s)’ which, as Martin and Kamberelis (2013) describe it: ‘enables (even urges) us to see things differently – in terms of what they might become’ (670).

Having made the case for the value of complexity and a nuanced approach to evidence in digital education research, I now go on to suggest ‘speculative method’ as a family of approaches which have not yet been widely adopted in digital education, but which offer us a highly generative set of methodological considerations in relation to epistemology, temporality and audience.

3. Inventive problem making and speculative method: researching not-yetness

Gough’s (2010) principles for futures study include several features which point the way towards speculative method for digital education: creative imagination as part of an ‘eclectic approach’ to methods (15), critical use of science fiction (16) and the concept of ‘rehearsal of surprise’ (17). Speculative method encompasses a range of approaches in the social sciences (Lury and Wakeford 2012), and art and design (DiSalvo 2012), and has been described by Michael (2012) as ‘“inventive problem making” in which the parameters of the issue are reconfigured’ (536).

In the social sciences, speculative method is a response to some of the epistemological issues raised in the previous section – linearity, fixity and the tendency of research to conceal the extent to which it is involved in creating the realities it uncovers. Researchers in design fields are concerned with the nature of knowledge, too, but also draw particular attention to issues of temporality and engagement: DiSalvo (2012) defines speculative design as ‘the use of designerly means to express foresight in compelling, often provocative ways, which are intended to engage audiences in considerations of what might be’ (109).

In defining inventive methods, Lury and Wakeford (2012) highlight the following key points which are of particular relevance to digital education research, namely that they:

• are designed to investigate ‘the open-endedness of the social world’ (2);

• require critical reflection on epistemology and the ‘value, significance and status’ of knowledge (3);

• both investigate and engage the social world (6);

• address specific problems and are ‘adapted in use in relation to that specificity’ (11);

• are always oriented to ‘making a difference’ (11).

This section explores three areas of focus – epistemology, temporality and the future, and the nature of performativity and engagement – and examines their implications for researching ‘not-yetness’ in digital education.

3.1 Epistemology and speculative method

Inventiveness, for Lury and Wakefield, means that ‘answerability’ of the problem at hand is introduced by crafting a method specifically to address that problem. Furthermore, ‘if methods are to be inventive, they should not leave that problem untouched’ (3). They summarise inventive approaches as

methods or means by which the social world is not only investigated, but may also be engaged … the knowledge of change they permit need not be limited to ascertaining what is going on now or predicting what will go on soon, but may rather be a matter of configuring what comes next. (6)

Sociologists Wilkie, Michael, and Plummer-Fernandez (2015) describe a speculative method involving the creation of a series of ‘Twitter-bots’ to participate in exchanges about environmental issues, and they characterise these bots as ‘methodological interventions that are overtly constitutive of the material that is gathered, but in ways that are open, ambiguous or troublesome. … the aim is to access new and emergent formulations of the “issues at stake”’ (80). They frame methodology itself as ‘a process of asking inventive, that is, more provocative questions’ (4). Like Lury and Wakefield, Wilkie et al. identify the active creation of the issues through the intervention. They also emphasise the intervention’s ambiguous nature and consequences.

Indeed, understanding methods as ‘inventive’ means accepting a degree of openness in relation to methodological boundaries and constraints – as Lury and Wakeford put it, ‘the inventiveness of a method is to do with its ability to generate its own boundary conditions … to organise itself – to self-organise – in a (changing) relation to a (changing) context’ (2012, 13). In practice, this means that the legitimacy of a method as inventive is closely tied to its ability to engage with and affect the problem it addresses. It takes what Haraway (1991) once described as ‘pleasure in the confusion of boundaries and … responsibility in their construction’ (150).

Such self-organisation of boundaries is challenging in a methodological context, where stability and replicability are valued. Offering a challenge to the authors of a recent special issue on ‘post-qualitative research’, Greene (2013) enquires of their insistence on the ‘dynamic, fluid, indefinite, unfolding’ nature of such research:

Without a shared understanding of how some knowledge or insight or understanding about the world is obtained … how will such knowledge or understanding be warranted? And how will it be of any meaningful or constructive consequence in the world? (753)

Here the importance of the ‘object of study’, or the ‘problem’, as Lury and Wakeford put it, comes to the fore in inventive or speculative methods. MacLure (2006) describes ‘baroque method’, which can respect the ‘recalcitrance of the object of study – not only its complexity but also its capacity for resisting social explanation and for unsettling the composure of researchers’ (734). Ambiguity and recalcitrance are important because paying attention to them helps resist the simplistically relativist idea that, in working inventively with methods, ‘anything goes’. Law (2004) argues that ‘if realities may be built … it is difficult to do so’ (13). Following Deleuze and Guattari’s (1988) concept of assemblage, what Law explains as a ‘process of bundling, of assembling, of better and recursive self-assembling’ (2004, 42), ² he describes the work of building a reality that can ‘cope with a hinterland of pre-existing social and material realities’ as a ‘method assemblage’ (2004, 13). He calls for method assemblages that are generous and uncertain, and suggests multiplicity, imaginaries, indefiniteness and re-enchantment, among others, as important issues informing a methodological practice that can look beyond reality-as-destiny (152–154). There is no single reality, he argues, but realities are not conjured from nothing:

the practices of method assemblage craft out-thereness by condensing particular patterns and repetitions whilst ignoring others … they manifest realities/signals on the one hand, and generate non-realities/silences and Otherness on the other. (113)

In working with not-yetness as a principle for digital education research, we must allow for the ambiguities it creates, and be prepared to craft and adapt our methods to take account of these. One frequent source of ambiguity is researching practices, such as blogging, which are also, or primarily, used outside formal education settings (Collier and Ross, forthcoming). The use of such technologies ‘inevitably brings the “outside” in’ (Collier and Ross, forthcoming), and one of the challenges that confronts digital education researchers is continually drawing and redrawing the conceptual and pedagogical boundaries of what we mean by education. This challenge is of course not limited to digital education, but is always present when technologies, and the subjectivities and practices associated with them, are at play, because they are so underdetermined, allowing ‘subjects and objects increasingly to appear in configurations of space and time, mind and body, human and machine, which disaggregate the real/actual into constellations of indeterminate – not amorphous – complexity’ (Poster 1999, 17). Speculative methods can help us visualise and critique the possible nature and consequences of particular kinds of complexity and boundaries. The next section of this paper will demonstrate how this can play out in practice.

3.2 Temporality and speculative method

Speculative method in technology design disciplines such as human computer interaction, design informatics and ubiquitous computing has an uneasy relationship with time and temporality. While the practices of design fiction, for example, take aim at the future (proximate, alternative or distant), theorists in these fields emphasise the troublesomeness of such a target. For example, Dourish and Bell (2011) discuss the ‘centrality of [ubiquitous computing’s] proximate future’, and argue that ‘the framing of ubicomp as something yet to be achieved allows researchers and technologists to absolve themselves of responsibilities for the present’ (22). Auger (2013) is optimistic that thinking through these issues can lead to generative approaches: ‘contemplation on speculative design is not only to encourage the technological future but can also provide a system for analysing, critiquing and re-thinking contemporary technology’ (12). The politics of ‘futurology’, however, come under scrutiny by Gonzatto et al. (2013), who describe design fictions as ‘not just an uncommitted exercise of creativity; they come from the interest of someone who acts on the present social order’ (40). Design fictions are not ‘mere speculation’ (Gonzatto et al. 2013) because visions of the future generate effects in the present. The temporalities of speculative method and design are therefore unstable and interwoven. Hales (2013) refers to design fiction’s temporalities as both ‘estranged futurities-to-come’; and a ‘distributed accumulation of past or otherwise temporary futures’:

design fictions form part of the genre of an estranged futurity-to-come; they form a part of the contemporary technological prospective. … Sterling signals the paleo-future: the reserve of historical ideas, visions and projections of the future – a historical futurity of that prospective. The paleo-futuristic, as objects, as images, form a distributed accumulation of past or otherwise temporary futures. (7)

He is hopeful that this complex of temporalities ‘opens design to theoretical and artistic methodologies that can be used to excavate past, present and future media’ (4), but seems to acknowledge that such excavations can never be frictionless.

The not-yetness of digital education, too, carries multiple overlapping temporalities, and is never simply about the future. The pedagogical futures we envisage inform us about what matters now in this field, what issues and problems we have inherited and what debates define what can and cannot currently be thought about or imagined. Not-yetness may also imply ‘someday’ – there is a definiteness here which, I would argue, is also of its time. In a context where visions of the future of learning are presented with apparent certainty by those who stand to benefit in various ways from being visionaries, even the most speculative approaches need to promise something. There is relatively little critical attention paid in this field to the consequences of ‘never-happened-ness’³ – to predicted change which does not play out as envisaged – and the probable tensions between ‘not-yetness’ and ‘never-happened-ness’ are a fruitful area for further consideration.

3.3 Engagement, performativity and speculative method

A third key element of speculative method is its interactive and performative properties. Speculative methods are performative in the sense discussed earlier that they are ‘constitutive of the object they “study”’ (Wilkie, Michael, and Plummer-Fernandez 2015, 79). In practice this means that, as forms of communication, they act to create the futures they portray. In part they do this through their focus on engaging publics.

Auger (2013) focuses on the centrality of the audience towards whom a given speculative design is directed, and explores the sorts of issues that can arise when the design and its intended audience are not well-matched – principally, that ‘the audience will not relate to the proposal resulting in a lack of engagement or connection’ (12). Educational research faces similar imperatives to engage with research users, and indeed one reason given for the rise of design-based research is the alienation of practitioner and policy-maker audiences from more abstract or theoretical research contributions. Auger’s point is that designers require ‘conceptual bridges’ for their design fictions, which create a connection ‘between the audience’s perception of their world and the fictional element of the concept’ (12). These techniques include consideration of context (ecological approach); provocation (uncanny approach); verisimilitude, familiarity, specificity, attention to detail and going to extremes (observational comedy approach); and the use of counterfactuals. Such techniques might offer insights to researchers in digital education, as they work to engage stakeholders with concepts and findings, especially those concepts which may be counter-intuitive or otherwise challenging.

The notion of ‘audience’ may be problematic, however. The value and impact of academic research has been reconfigured in recent years away from ‘dissemination’ towards increasing emphasis on practices of public engagement, knowledge exchange and participation (Bannister and O’Sullivan 2013). As those who attempt public engagement with non-academic audiences discover, though, participants have a tendency to ‘“misbehave” in various ways – they “overspill” the parameters of the engagement event’ (Michael 2012, 529). How speculative method ‘audiences’ are framed, in other words, requires careful consideration of possible roles of research informants, participants, users, stakeholders, critical friends and so on. As researchers we also have to be careful about not erasing the overspills, succumbing to ‘a tacit process of sanitization whereby the engagement event is cleaned up so that the existing methodological, conceptual, and institutional frames of the engagement event remain unchallenged’ (Michael 2012 , 529). Talking of the metaphorical figure of the ‘idiot’, which transforms events by resisting consensus and insists on ‘something more important’ (Stengers, in Michael 2012, 535), Michael explains the balancing act involved in heeding, but not attempting to tame, such interventions:

As soon as we think we have ‘deployed’ the idiot, slowed our thinking, and invented novel problems, we have also tamed it, and the process of querying our assumptions has become compromised. The idiot reminds us that we must never get too comfortable with ‘what we are busy doing’ – we should be open to creative or inventive problem making. (536–537)

In digital education, our publics can comprise students, teachers, families, learning technologists, administrators, employers, policy-makers and commercial interests. Furthermore, at various times and in various places, almost anyone who has been educated themselves will have a view and some beliefs about what sorts of educational futures are desirable. Often unquestioned assumptions about the importance of the ‘human touch’ in education, for example, have informed debate about the sufficiency of online learning for years (Dreyfus 2001). Speculative method can expose and work with such assumptions in truly novel ways, and interventions can be crafted to appeal to and engage with a variety of publics, as the stories of the teacherbot and artcasting will show. There may also be times, however, when an inventive approach includes silences and gaps as part of its design, as in the case of the tweeting book, and these approaches, too, have something to tell us about the things that speculative design can make happen.

In the next section, I attempt to bring these considerations and tensions to life by introducing three examples of speculative method in digital education, and to explore engagement, temporality and epistemology as they play out in practice.

4. Teacherbot, artcasting and the tweeting book: three examples of speculative method in digital education

This part of the paper moves from the conceptual dimensions of speculative method to three examples of practice generated from the Digital Education group at the University of Edinburgh between 2014 and 16. The first, the ‘teacherbot’ project, was designed to probe the possibilities of automation in relation to teaching at scale, specifically in the E-learning and Digital Cultures Massive Open Online Course (EDCMOOC) (Bayne 2015b). The second, the ‘artcasting’ digital platform, was part of an AHRC-funded project⁴ exploring new forms of arts engagement and evaluation through a mobilities perspective (Ross et al. 2015). And the final example was a ‘tweeting book’ prototype created to problematise the emphasis in learning analytics on human activity and data (Knox 2014). All three of these projects worked with the epistemological not-yetness of matters of concern: MOOC pedagogy, arts evaluation and learning analytics, respectively; they suggested visions of the future of learning which were distinctively other than those being offered at the time; and they prompted exploration of issues raised by those visions.

4.1 Teacherbot

Teacherbot was an automated agent which participated in the Twitter conversations surrounding the third instance of EDCMOOC, a five-week open course aimed at educators and learning technologists and exploring the significance of digital cultures on digital education practice, and which had approximately 12,000 people registered. ⁵ The teacherbot (nicknamed ‘Botty’ by some of the course participants) was a series of pre-prepared statements, questions and provocations which would be triggered by particular keywords as they appeared in tweets related to the course (anything using the #edcmooc hashtag). Teacherbot’s responses were written by the course team based on course content, along with questions and issues which had arisen in previous instances of the course. The human course teachers could also add new trigger keywords and responses on the fly, using the graphical user interface developed for the project (Figure 1).

Figure 1. Teacherbot’s graphical user interface.

The purpose of teacherbot was to work creatively with

ways of theorising and practising digital education and automated teaching which are driven neither by technical-rational efficiency models, nor by equally instrumentally focused social models which assume a position of humanistic opposition to, or appropriation of, digital technology. (Bayne 2015b, 460)

To move against both efficiency discourses and an uncritically deterministic view of human agency in relation to technology, teacherbot was positioned as playful exploration of how ‘human and non-human teachers might work together in a teaching “assemblage” which refuses ontological hierarchy’ (Bayne 2015b, 460). MOOC participants were invited to respond to the bot itself and to discuss it with the wider group, which some did with enthusiasm. Teacherbot became a point of shared experience for the group, as well as intervening in conversations (sometimes sensibly, other times not), answering questions (again, sometimes sensibly) and offering ideas for further reading. Its interventions produced a new set of relations within the course which provoked new questions and perspectives for both participants and teachers as researchers.

Teacherbot’s creation as an artefact specifically designed to intervene in EDCMOOC, and to explore possible futures for teacher presence, makes it a powerful example of a speculative method for digital education. It was conceived and created at a moment of significant interest in, and debate about, automation in education. Automation is a site of multiple ‘not-yetness’: it is thought to be necessary to cope with the increasing need for scaled-up, ‘on-demand’ teaching; the vast and complex technical and pedagogical challenges associated with automating teaching, feedback and assessment have not yet been solved; and the implications of reshaping education to fit the capabilities of a partially automated system have not yet been conceptually or empirically understood.

4.2 Artcasting

Artcasting is a digital platform which experiments with inventive approaches to the evaluation of gallery and museum learning and engagement. It was designed to address the question of whether evaluation of engagement could be more engaging, and more meaningful, if it was more inventive. The focus for the project was the ARTIST ROOMS On Tour exhibition, a collection of more than 1600 works of international contemporary art, jointly owned and managed by Tate & National Galleries of Scotland, and shared throughout the UK in a programme of exhibitions organised in collaboration with local associate galleries of all sizes, aiming particularly to ensure the collection engages new, young audiences.

ARTIST ROOMS On Tour provided a context in which to explore the use of a mobilities framework for museum and gallery evaluation. It puts internationally important contemporary artworks in many locations that do not routinely have access to such works and puts the task of making them relevant to young audiences in the hands of local galleries and users. Mobilities perspectives have been increasingly adopted in the social sciences in recent years, because they offer new ways of understanding social phenomena (Sheller and Urry 2006) and artcasting aimed to understand museum and gallery learning from a theoretical perspective that took into account social, spatial and technological mobilities. The project explored the idea that ROOMS are in tension between stability, expressed through the focus on an individual artist, and movement, expressed through the touring model on which the exhibition is built – with ever-changing locations and character. Travelling ROOMS are put continually into dialogue with new gallery spaces, localities, communities and permanent collections.

In practice, artcasting involved gallery visitors in digitally re-locating artworks in other places and times, and gave visitors opportunities to re-encounter and respond to artworks from beyond the gallery space. These locations, encounters and responses, in turn, were used to challenge the galleries and researchers to develop new approaches to evaluating visitor engagement. Artcasting was designed to challenge dominant approaches that separate engagement from evaluation (Belfiore and Bennett 2010) by simultaneously encouraging visitors to make connections and reflect on what they have experienced, and capturing those connections for sharing, analysis and evaluation. It aimed to both reframe and reconfigure evaluation – it is inventive in the way that Lury and Wakeford describe:

the inventiveness of methods is to be found in the relation between two moments: the addressing of a method – an anecdote, a probe, a category – to a specific problem, and the capacity of what emerges in the use of that method to change the problem. (2012, 7)

Artcasting was steeped in not-yetness from its inception, because it was so difficult to explain to gallery partners and other collaborators exactly what the vision was (Figure 2). To get to the stage of being able to make artcasting, a large number of people (culminating with the funders – the Arts and Humanities Research Council in the UK) had to be ‘sold’ on a highly speculative version of what evaluation could be. The vision was inviting because it encompassed two elements which are absolutely ‘of the moment’ – evaluation, and digital engagement – and it was likely this combination of currency and not-yetness which gained it support. Before the project had even begun, a public had sprung up around it – ARTIST ROOMS research group members, associate gallery educators, colleagues in the research offices of the University, and the anonymous reviewers who supported and championed the project. Engaging with such publics and persuading them to help and support the development of a speculative digital education project is a form of engagement and performance which may sometimes be overlooked in accounts of speculative method and its opportunities and challenges.

Figure 2. Part of the visual evidence provided with the artcasting project bid. Illustration by Kevin Allen.

4.3 The ‘tweeting book’

The tweeting book was a prototype sensor system developed as a design experiment by Knox (2014) during his doctoral studies. Intended primarily to help him explore some conceptual questions emerging from his research and to ask ‘how object agency might be incorporated into a research strategy’ (73), the materiality of the prototype has provided a useful prompt for thinking through possible implications of learning analytics. The tweeting book system: ‘made use of RFID sensor technology to “give voice” to books by allowing them to contribute to the social media service Twitter’ (Knox 2014). When placed on the stand (a proxy for being read), the book would tweet a 140-character extract from itself (Figure 3).

Figure 3. Tweets sent by the tweeting book system.

Here we encounter a challenge to the notion of audience discussed in the previous section. The book, by participating in a tiny way amongst the deluge of a social media environment such as Twitter, conceptually foregrounds the ‘stuff’ of education which is silent in representations of ‘online learning’ – the books, the materials, the physical artefacts that make up learning environments beyond what can be captured by analytics. The tweeting book raises a question about how speculative method can function when silences and gaps are the ‘noisiest’ part of its design. In digital education, we require space for experiments which cannot engage, which are by nature antisocial, because all our present-day theories so strongly emphasise collaboration, connection, participation and sociability. In Wilkie, Michael, and Plummer-Fernandez’s (2015) conception, the tweeting book could be a ‘parasite’ in such a setting: ‘an uninvited, and initially disruptive, guest at the dinner table … untrustworthy, and the connections and communications [it] mediates might entail mistranslations and mischief – that is, inject disorder into apparent order’ (88). But its impact may never be seen directly because it was primarily intended to generate new ideas about post-humanist approaches to analytics, not to test or disseminate these. The tweeting book explored the ‘entanglement of human and non-human’ (Knox 2014, 75) that goes into making learning analytics ‘speak’ – ‘the software, algorithms and codes’ (Knox 2014, 73). It was a small but vibrant part of the rethinking of online education that came along with a large doctoral project, and as such there is still more of its story to come.

5. Conclusion

I want to subvert method by helping to remake methods … that imagine and participate in politics and other forms of the good in novel and creative ways; and that start to do this by escaping the postulate of singularity, and responding creatively to a world that is taken to be composed of an excess of generative forces and relations. (Law 2004, 9)

In this article I proposed speculative method as a research approach which could be of significant value in the field of digital education, supporting researchers to engage with not-yetness in ways that approaches such as design-based research and randomised controlled trials cannot. I focused on and explored epistemological, temporal and performative dimensions of speculative method, and demonstrated its potential applications through the examples of the teacherbot, artcasting and tweeting book projects.

The extent to which these kinds of projects can be oriented to ‘making a difference’, as Lury and Wakeford (2012, 11) say inventive methods must, needs to be considered from two perspectives. First, for such approaches to be credible in research terms, we need to be able to discuss and defend their value in the context of paradigms which approach credibility in quite different ways. To do so could involve drawing from and working with concepts of quality being explored in other social science contexts – for example, Tracy’s (2010) ‘big tent’ criteria for qualitative research. Tracy identifies eight criteria: a worthy topic, rich rigor, sincerity, credibility, resonance, significant contribution, ethics and meaningful coherence, while avoiding ‘tying these markers to specific paradigmatic practices or crafts’ (839). A number of these criteria would need to be engaged with critically in the context of speculative methods, but by working through these and similar frameworks, educational researchers keen to explore such methods might find helpful building blocks for conceptualising and justifying the value even of methods which do not, as yet, exist.

Second, speculative approaches need to produce findings and conclusions which are generative enough to serve as invitations for further work, and indeed to offer insights for educational practice. Educational researchers who engage with these kinds of methods need papers, talks and social media contributions to be bold in suggesting what new approaches and concepts might flow from their work, and what further questions it might generate. We need to work to make a bigger space in digital education for the impact that comes from engaging the world and its messiness more creatively and critically, more imaginatively and inventively.

Notes on contributor

Dr Jen Ross is co-director of the Centre for Research in Digital Education, and deputy director of Research and Knowledge Exchange in the School of Education, at the University of Edinburgh. Her research interests include online distance education, digital cultural heritage learning, open education including Massive Open Online Courses (MOOCs), digital cultures and futures, and online reflective practices.

Acknowledgements

Many thanks to my colleagues for their comments and suggestions on this paper: Siân Bayne, Michael Sean Gallagher, Jeremy Knox and Hamish Macleod.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

The ‘Artcasting’ research described in this paper was supported by the Art and Humanities Research Council [grant number AH/M008177/1].

Notes

1. This is not an issue just for educational researchers – disciplines such as sociology also find themselves seeking methodological innovation to ‘resist conceding relevance, competence and expertise to actors with principally commercial interests’ (Wilkie, Michael, and Plummer-Fernandez 2015, 81).

2. Law’s definition and use of ‘assemblage’ has been suggested to be less radical than that of Deleuze and Guattari: ‘where he is only prepared to trouble the distinction between single and multiple, they reject it altogether’ (Buchanan 2015, 386).

3. Many thanks to one of the anonymous reviewers of this article for suggesting this term, which I have interpreted in a way that may diverge from what they had intended.

4. http://www.artcastingproject.net.

5. https://www.coursera.org/course/edc.