Reimagining and reexamining assessment in online learning

We are pleased to introduce this special issue of Distance Education with its focus on assessment in online and distance learning. As has been pointed out by several of the authors in this collection, assessment is an often-overlooked topic and an often-neglected aspect of successful online teaching and learning.

We have observed that researchers’ concerns with online assessment are global; they exist everywhere. The responses to our call poured in from all corners of the world. However, for all the normal reasons, not all corners of the world are represented in the limited number of articles in this issue. That said, there is representation from the Pacific Region, from Europe, from Asia, and from North America.

In this difficult era of the worldwide COVID-19 pandemic, online learning has experienced—endured?—a rapid and startling, though not surprising, rise in implementation. Has the dramatic increase in online learning by traditional institutions been fully embraced and appreciated by academics and teaching support staff? Perhaps the jury is still out on that important question, although Dianne’s response would be “no,” based on her experience of one-on-one mentoring of university faculty making the transition from classroom to online teacher as well from abundant reading on the topic, both academic and casual.

Gabi agrees and asks what the established distance education community has learned from the pivot to online teaching during the pandemic. In her experience, which involves supporting academics with learning design in a higher education context in the United Kingdom, some of the tried and tested “old” methods (particularly asynchronous approaches, such as the use of discussion forums) have been overlooked in favor of newer, synchronous methods (symbolized by the ubiquity of Zoom memes) in an attempt to replicate traditional classroom practices. Those of us with experience in online education have had perhaps as much to learn from this shake-up as those for whom the very notion of online teaching was previously unthinkable.

The required transition from classroom or traditional teaching to online teaching is so multifaceted, so expansive, and very time- and labor-intensive, not just on the part of those instructors and teachers on the frontlines, but also by institutions and their systems, administrators, course designers, and whoever else is touched by these dramatic changes. And students, of course, are often thrust without adequate preparation into completely new learning lives.

The import and impact of such a transition have consequences far beyond the actual issue of learning. It is our contention that learning will go on, although altered in modality, at very much the same pace. But the emotional and social effects will linger, widespread, and will continue to manifest in many ways.

This special issue, however, deals only with one critical aspect of online learning: assessment. Assessment sits, in many cases, as the judge at the end of the race. Who won? Who did well? Who succeeded and who didn’t? And, in the case of education, did the methods, instruments, and tools used (whether old or new) produce the appropriate results—the quality and quantity of the measurements that we have come to accept as viable outcomes?

The recent shift to online learning has necessitated a consequence of “pulling back the curtain” in education. That is, whatever mystique or secrecy that was allowed to exist behind closed classroom doors has become less so. Curricula and syllabi now need to be shared or examined; teams work together to construct new and better courses. Experts in web design, course design, and technology combine forces to fulfill the need for online material. These are welcome developments in the evolution of higher education, but they have also created new uncertainties that need to be resolved.

As a part of the curtain-raising, educators now ask themselves—or should ask themselves: Was the assessment structure that I have always used the right structure? Were my assessment instruments as effective as they could have been? Did my assessment contribute meaningfully to the learning cycle?

Our experience as long-time distance educators and consultants tells us that the answers to all these questions are often “no.” No, assessments have not been effective. No, they did not adequately reflect or ask for reflection on the course content. No, they were not integrated meaningfully into the learning cycle but rather tacked on to the end of a course as a final hoop through which to jump.

But there’s more to the assessment landscape—another wrinkle that has become more prominent in recent times, and this one falls into the much larger realm of social justice and equity, of individual rights, of learners’ rights. We refer, firstly, to the issue of privacy and surveillance, and secondly, to issues of inclusivity and accessibility. As technology has changed the application of assessment, so too has it changed the morality of assessment. The field is considering assessment through new eyes; and several of the articles in this special issue present those novel views.

What we face now, therefore, is a time of reckoning; not just in meeting demanding targets for transitioning to effective and sound online learning at all levels of education, but also in critically (re)examining our assessment practices, where approaches have often been grounded in mechanistic, positivist philosophies that have not encouraged critical or higher-level thinking or deep learning.

This special issue, therefore, is both timely and important. The articles presented here range across a number of assessment-related topics and demonstrate a variety of research approaches. There are both conceptual pieces and empirical pieces; they present challenges to our current understanding of online assessment as well as capturing the diversity of approaches to online assessment. As editors and as practitioners, we found strategies and practices that were new to us; we found concepts and connections that we thought were novel and exciting.

While working on the production of this issue over the past months, we have made many observations. In the background is the fact that producing academic work in the time of COVID-19 has been taxing; on the other hand, the pandemic as a topic has infiltrated much of the work in our field. The existence of COVID-19–driven transition has produced new threads of inquiry and research, some of which are already evident in this issue—in both the empirical studies undertaken and in the supporting references from the literature. The paper by Byrne, Hogan, Dhingra, Anthony, and Gannon focuses most directly on this topic, in their exploration of the experience of graduate student instructors in the pivot to emergency remote teaching in an American context.

One theme in the collection of papers that stood out for Dianne as surprising was the interest in the potential for academic dishonesty—or, to put it bluntly—cheating on online examinations. I (Dianne) can think of a couple of reasons for my surprised reaction to receiving two articles that deal with this unhappy phenomenon. The first is disciplinary. In my practice in the social sciences at graduate level, a long-time adherence to authentic assessment has appeared to suppress the opportunity of academic dishonesty in both summative and formative assessment occasions. We don’t assume its total absence; but vigilance and close attention to the learners’ writing style and cognitive abilities, in addition to the creation of authentic instruments, have diminished the pervasiveness of academic dishonesty. Secondly, my (Dianne again) online teaching has lessened in the last year; so, whereas many online teachers are coping with increased teaching loads with learners who are new to online learning and therefore, perhaps extra-nervous about exams—and consequently, perhaps more prone to academic dishonesty—this has not been my experience. Gabi was less surprised to see cheating emerging as a hot topic, as she has been drawn into many conversations with academics, since the start of the pandemic, about issues arising from the attempt to apply the conventions of closely supervised in-person examination conditions to an online setting. Moreover, her research into the recognition of open, online learning has shown her that a huge cultural shift is required for many people from traditional higher education contexts to accept any form of online education as valid unless it is accompanied by strictly proctored or invigilated examinations. That said, this issue features two articles on that very topic: double attention, as it were, in articles authored by Gudiño Paredes, Jasso Peña, and de La Fuente Alcazar from Mexico; and Stadler, Kolb, and Sailer from Germany. Both articles feature discussions on the use of artificial intelligence in the struggle against academic dishonesty as educators seek out methods, strategies, and equipment to maintain academic integrity.

The argument against technological surveillance tools and platforms such as Proctorio, Respondus, and ExamSoft, while raising the issue of ethics, eventually settles on authentic assessment and design. Aware of the accusations of focusing overly on surveillance and control, which is the purpose of proctoring software, some institutions such as California’s Stanford University and Canada’s McGill University have banned such software amid calls for “a kind of model where there’s space for agency and trust” between students and faculty” (Feathers, 2020). Meanwhile, other authors in this issue—notably DeWaard and Roberts, Karunanayaka and Naidu, and Marinho, Fernandes, and Pimentel—argue for authentic assessments as a way to circumvent the need to monitor cheating, emphasizing the benefits of prevention over penalization.

Authentic assessments—those which mirror as much as possible a real problem or challenge—have been shown to offer deeper and more meaningful grounds for measuring learning. Karunanayaka and Naidu provide a case study in which they investigated the embedding of graduate attributes into the learning outcomes of a program for educators and found that this seemingly simple step had profound effects in terms of providing the foundations for the design of authentic assessments. Marinho, Fernandes, and Pimentel consider authentic assessment in their case study on the use of digital portfolios, also in an education program. In both these papers, the researchers aimed to increase the authenticity of the assessment tasks, not only to decrease opportunities for cheating but also to increase student agency and ownership of the learning process and to enhance the validity of the assessment.

Returning to the topic of artificial intelligence, Gamage, Staubitz, and Whiting’s literature review of peer assessments in MOOCs includes data about the implementation of a tool that is being applied to support peer assessment, although it is only one aspect of the broad turf they cover in their systematic review. Among the numerous genres of academic writing, the literature review stands out not necessarily as a superior vessel but as a very useful vessel, a sturdy vessel—one that can launch a thousand ships or at least provide a foundation as a starting point for colleagues. Also included in their paper are discussions on the multifaceted and intricate aspects of peer review, including providing scores or marks, providing feedback to peer colleagues, and interacting with peer colleagues. On that note, we extend congratulations to the authors of this article for their communication across three countries—Sri Lanka, Germany, and the United States of America. Similar congratulations for “walking the talk” of distance education go to Marinho, Fernandes, and Pimentel for their Brazil-Portugal collaboration, and to Karunanayaka and Naidu for their Sri Lanka-Fiji partnership. As editors working across two continents with journal administration also bridging continents, we are well aware of the challenges that distance can bring.

The remaining articles range across the broad field of online assessment. Some of the articles are challenging; we are referring specifically to the piece by Hickey and Harris, who took the call to reimagine assessment to heart—and reimagine assessment they did. These authors take exception to what have become conventional approaches to assessment and present, at length, a new schema for achieving valuable formative and summative assessments. A part of their reasoning is based on the notion that teachers may experience burnout as a result of time-consuming exchanges with individual students in the grading process. While this may indeed have been experienced by many practitioners, systems of avoiding such burnout are not often discussed. Hickey and Harris cover a great deal of ground in their paper, from both theoretical and practical perspectives, and the authors continue to work toward formal and widespread implementation of their reimagined assessment processes.

It was a pleasant surprise to receive an article that draws on the seminal work of Paulo Freire and creates a context for his work on our topic of online assessment. While wearing our editorial hats, we often notice that our exciting and dynamic field surges ahead with research and discussions of current situations or developments and does not hark back to the important foundations that underpin many of today’s theories and/or pedagogy. It’s especially uplifting to see the wisdom of the adult educator-guru Freire transported into the online world by, in this case, DeWaard and Roberts, who draw links between Freire’s social justice approach and authentic assessment.

This special issue on assessment has only scratched the surface of an area both broad and deep within the study of online teaching and learning. We were heartened by the response that we received to our initial call for papers and we suggested to those whose work we could not accept that there could be another such special issue in the future. Certainly, the interest in assessment—and the challenges it presents—will only, should only, increase in the future.

Our stint as guest editors of this prestigious journal’s special issue has been a rollicking adventure—a journey of learning, of expansion, of humility, of insight. Editing a journal comprises a lot of work and endless emails. We salute our colleagues around the world who take on this task in the spirit of collegiality and scholarship. Here, particularly, we are grateful to Executive Editor Som Naidu and Distance Education personnel for their guidance and patient assistance. Thank you for the opportunity.