Learning with virtual humans: Introduction to the special issue

Abstract

Virtual humans are embodied agents with a human-like appearance. In educational contexts, virtual humans are often designed to help people learn. In this special issue, we see six representations of this field from different perspectives, including system analysis and evaluation, value-added studies, and reviews of the literature. This diversity of research exemplifies the exciting field of virtual humans. In this introduction to the special issue, we discuss virtual humans in educational contexts, highlight the articles in this special issue, and outline productive future research directions.

Keywords:

• Virtual human
• pedagogical agent
• embodied agent
• learning

Innovations in educational technology continue to help instructors find ways to improve the online learning experience. Virtual humans are one innovation with over a quarter century of research that is still ongoing with new and exciting areas. Over the years, virtual humans have gone by several names within the research literature depending on how they are used, such as avatars for human-controlled virtual humans, and agents (including conversational, embodied, or pedagogical) when controlled by a computer system (Craig & Schroeder, 2018). Two studies (Bondie et al., 2021; Howell & Mikeska, 2021) in this special issue discuss the use of avatars within teacher training. It has been argued that avatars can be very important for supporting learning by supporting collaboration within virtual environments (Bailenson, 2008) and adding social presence (Hodge et al., 2008; Lowenthal, 2010). The other four articles in this special issue focus on the use of agents. The articles in this special issue focus on virtual humans used within educational environments, which have traditionally been broadly referred to as pedagogical agents.

Pedagogical agents are embodied representations of software agents that appear within the user interface, and they are designed to help people learn (Moreno, 2005). Since their inception, pedagogical agents have been researched in a variety of learning environments to help learners through a variety of instructional roles. Early studies provide examples of the variance in how pedagogical agents have been used. For example, Johnson and Rickel (1997) presented a virtual human named Steve that could teach procedural tasks, and Lester et al. (1997) presented Herman the Bug, who delivered different types of feedback and guidance. Through the years, one of the essential questions around the use of pedagogical agents is how they should appear to the learner, or what type of physical embodiment they should have. While pedagogical agent researchers conducted many studies with non-human-like characters (e.g. Atkinson, 2002; Moreno et al., 2001), recently this trend has subsided and virtual humans have been featured in recent work across many disciplines.

Virtual humans can be more than simple human-like representations on a computer screen, and there are many ways in which virtual humans and other pedagogical agents can aid learning. For example, Clarebout et al. (2002) proposed that they could play six different roles in the learning environment, including demonstrating, scaffolding, modeling, testing, supplanting, or coaching. Virtual humans have been implemented in a variety of learning environments, spanning narrated videos (Davis et al., 2019; Schroeder et al., 2021), intelligent tutoring systems (Graesser, 2016, 2017; Johnson et al., 2017), and educational games (Rowe et al., 2009), to name a few. As creating virtual humans and the environments they appear in becomes more approachable to those without programming and animation knowledge, it seems plausible that the number of potential use cases will continue to increase. This is demonstrated through the increased variance in virtual human use outside of strictly education and learning domains, such as medical and mental health care fields. Research surrounding virtual humans has included topics such as older adult self-care (Morrow et al., 2020), health behavior change (Olafsson et al., 2020), genetic risk communication (Zhou & Bickmore, 2019), and virtual patient interactions (Hirumi et al., 2016).

Given the broad range of virtual human research in the literature, it is natural to question their efficacy for facilitating learning. While specific, synthesized insights about virtual human efficacy are challenging to find in the literature, within the last decade work in the broader field of pedagogical agents has been systematically synthesized. Heidig and Clarebout's (2011) systematic review found that no significant differences were generally found for either learning or motivational outcomes between pedagogical agent and non-agent conditions. This is notable, as pedagogical agents can be time consuming and potentially expensive to create. Meanwhile, Schroeder et al.'s (2013) meta-analysis found that pedagogical agents produced a small, positive effect on learning outcomes. However, this effect was significantly moderated by features of the research, such as the age of the participants. We view the conclusions of Heidig and Clarebout and Schroeder et al. as largely complementary, in that for pedagogical agents to be effective, we must thoughtfully design them and implement them when their strengths can be highlighted and their weaknesses minimized (Craig & Schroeder, 2018). Yet, research is still needed to understand what these strengths are and when other interventions may be more effective.

This special issue

The pace at which virtual humans are being implemented in learning technologies is rapidly increasing, and virtual humans may be more commonly implemented than ever before. While this infusion of technology is exciting, there is a lack of research to support the wide implementation of virtual humans. This special issue focuses on the implementation and design of virtual humans specifically, rather than other types of pedagogical agents, and identifies the limitations of current work. We are pleased to include the following six articles in this special issue, of which two are examples of systems analysis and evaluations, two are examples of value-added research studies, and two present reviews of existing literature.

System analysis and evaluations

A critical step in developing any educational technology is the analysis and evaluation of the system. Two studies in this special issue examine different approaches to analyzing and evaluating a virtual human-based system.

Virtual humans have successfully functioned as conversational partners for many years (Graesser et al., 2017; Johnson & Lester, 2016), and recent work has started to look at the impact of dialogs with multiple virtual humans (see Millis et al., 2018). Howell and Mikeska (2021) provide an example of how this would work with the Go Discuss Project. Go Discuss provided teachers with a virtual classroom in which they had to teach and monitor virtual students. They use this example to explore the role of environment authenticity when teaching with virtual humans. They investigate the questions of the implications of the participants’ understanding of the goals and mechanism of the simulation and when does the authenticity of the virtual human’s matter. Their article specifically highlights the need for greater attention to representation, decomposition, and approximations of practice. This paper poses interesting new directions for virtual human research highlighting the need to better understand the learner’s individual differences to fully understand learning with virtual humans.

In their evaluation of MentorPal, Nye et al. (2021) provide an example of virtual mentors, which are one type of conversational virtual human. This paper provides an example of usability evaluation during the rapid development process. This type of evaluation is unfortunately underutilized within the intelligent tutoring system and conversational agent world (Chughtai et al., 2015). However, it has great potential for establishing the system’s feasibly and for finding implementation problems that will impact adoption (Roscoe et al., 2018). Their evaluation indicated an overall usable system that was effective but found a drawback that could impact later adoption of the system and can be addressed during the rapid development process.

Value-added research

While analyzing and evaluating virtual human-based educational technologies is a critical step, in many cases these systems are used to answer research questions that can further inform the design of the technology. Mayer (2019) refers to these types of studies that compare the same system with different features as value-added studies. Two studies in this special issue present value-added research studies.

Much of the research literature around interactive virtual humans falls within the domain of intelligent tutoring systems. AutoTutor is a long-standing intelligent tutoring system that can provide communication with the learners through natural dialogue (Li & Graesser, 2021). The design of such interactive systems is of importance for both research and practice, and one aspect of virtual human design is how it communicates with the learner. Li and Graesser (2021) explored the impact of AutoTutor’s communication style during a three hour interaction with learners using either a formal language style, an informal language style, or a mixed language style. Their article highlights the complexity of language and various ways to measure learners’ writing quality.

From Betty’s Brain (Biswas et al., 2005; Roscoe et al., 2013) to SimStudent (Matsuda et al., 2013, 2018), teachable agents have a long history within the virtual human and learning literature. Silvervarg et al. (2021) revisited this area to investigate how these agents can mitigate critical constructive feedback. Their study indicated that when feedback is mitigated through teachable agents there is a positive effect on students' responses. According to their findings, teachable agents decrease the direct negative impact of feedback, which leads learners to be more responsive to the feedback, increasing posttest performance.

Reviews of the literature

Finally, it is critical to conduct reviews of the literature in order to synthesize what the results of individual studies collectively mean for the field. Two studies in this special issue present reviews that can inform research and practice.

Historically, how a learner perceives a virtual human has been viewed as an important question and the field has long investigated the so-called persona effect (Lester et al., 1997). One way of measuring the persona of a virtual human is the Agent Persona Instrument (API; Ryu & Baylor, 2005). Researchers have continued to use the API as a measure of agent persona and recently began investigating if the API is related to learning outcomes (Schroeder et al., 2017, 2018). Davis et al. (2021) systematically reviewed the literature to examine if the ratings of agent persona, as measured by the API, were associated with measures of learning. They did not find compelling evidence that persona effects are related to significant differences in learning outcomes, however they specifically noted the importance of considering context in virtual human research. Specifically, they denote the differences between claims from interactive systems compared to one-way information delivery (e.g. virtual human communicates with the learner, but the learner does not communicate with the virtual human).

Virtual humans have been used within teacher training simulations to provide guided practice and skill-building (Bradley & Kendall, 2014). Bondie et al. (2021) conducted a literature review to identify the best practices for designing teacher training simulations. While the reviewed literature was modest, the authors were able to identify five important design considerations which should be reported for teacher training simulations with virtual humans. First, the learning design should explicitly describe all elements within the simulation. Second, avatar and interactor learning should be investigated by analyzing the learning that avatars demonstrated through their responses. Third, the simulation will only be as effective as the interactor training, so detailed descriptions of training used must be provided. Fourth, time and distance can be suspended, compressed, and extended within the learning session and these features should be specifically addressed. Finally, complex practices in a sociocultural context are very important, so learner characteristics such as language, race, culture, and power must be explicitly considered.

The future is bright for virtual human research

Each year that passes tends to bring new technologies that make creating virtual humans more approachable or improves the programmer’s ability to create a lifelike virtual human. As the software engines (e.g. Unity (Unity Technologies, 2020), Unreal Engine 4 (Epic Games Inc., 2020)) and animation databases (e.g. Mixamo (Adobe Systems Incorporated, 2020)) continue to improve and expand, we anticipate that the use of virtual humans will continue to increase, as will research focusing on and incorporating virtual humans.

We note that while the field is moving forward at an exciting pace, the field has long-standing limitations that still need to be addressed, such as a lack of strong measurement techniques, underreporting on cost-effectiveness, and the lack of longitudinal studies (Clark & Choi, 2005; Schroeder & Gotch, 2015). Longitudinal studies are particularly warranted to better understand how learners interact with virtual humans over time (Veletsianos & Russell, 2014), especially as they are adopted into learning systems designed for more than a single use. Additionally, we encourage researchers to leverage previous work in the area and conduct systematic lines of research rather than ‘one-shot’ studies that hold little relation to existing research in the area, as well as well-document their research and development methodologies. In regards to the design and development of virtual humans, researchers may consider designing their virtual humans with respect to established design frameworks, such as those presented by Domagk (2010) and Heidig and Clarebout (2011). Only through well-documented, systematic lines of research will the field move forward to help answer the question of in what situations virtual humans may be the most effective intervention.

The diversity of the articles in this special issue highlight various research streams in the field of virtual humans. The empirical work shows much of the lifecycle of developing an educational technology, from usability testing (Nye et al., 2021) through longer-term interventions (Li & Graesser, 2021) and classroom-based implementations (Silvervarg et al., 2021). Similarly, the review articles in this special issue help highlight the maturation of a field, from novel ways of thinking about existing systems (Howell & Mikeska, 2021) through analyses of extant empirical findings to guide future work (Bondie et al., 2021; Davis et al., 2021). Together, these studies provide insights into the design of virtual human-based systems.

The studies in this special issue also stand in contrast to much of the work in the field of pedagogical agents. While some early studies in the field revolved around interactive agents that would respond depending on student input (e.g. Johnson et al., 2000; Moreno et al., 2001), there are many examples of non-interactive systems where the agent primarily lectures to the learner (e.g. Baylor, 2005; Craig et al., 2002; Moreno & Flowerday, 2006; Veletsianos, 2010). The articles in this special issue highlight how the field of virtual humans is transitioning away from non-interactive agents that simply deliver content and toward virtual humans that can respond to learner input and interact appropriately. It seems plausible that these lines of work around interactive virtual humans may increase with time given the effectiveness of intelligent tutoring systems (see Kulik & Fletcher, 2016; Ma et al., 2014; Steenbergen-Hu & Cooper, 2014) and other conversational agent systems (Graesser & McNamara, 2010; Graesser et al., 2017). The types of research highlighted in this special issue, system analysis and evaluation, value-added research, and reviews of the literature, will be essential in helping to continue moving the field of interactive virtual humans forward into the future.