Close Encounters of the Virtual Kind: Agents Simulating Copresence

ABSTRACT

Results of a field study of an open-access collaborative virtual environment in actual use suggested that awareness of others significantly increases the level of presence experienced by participants. Given the importance of copresence, this paper argues that, in the absence of other human collaborators in a collaborative virtual environment, copresence can potentially be simulated using agent technology. A controlled experiment deploying a prototype embodied conversational agent was conducted to investigate the potential of such agents to simulate copresence. This paper briefly introduces the concepts of presence and copresence, summarizes experiences drawn from the field study, reports on the controlled experiment, and discusses its results. Results suggest that even limited copresence as provided by the current prototype agent is sufficient to help users feel presence in the environment.

This paper outlines a theoretical and experimental study of the relevance of copresence and the feasibility of agents simulating copresence in collaborative virtual environments. A brief overview of the concept of presence and an overview of the CyberAxis field study is presented first, and is used to build an argument for the importance of copresence in collaborative virtual environments. This argument is then developed into a case for software agents simulating copresence: Essentially, that the use of such agents potentially facilitates “presence in absence” of users, even though the users themselves may not be directly active in the environment. An overview of the agent developed for the current research is then presented. Finally, the paper outlines experimental work, conducted within a collaborative virtual art gallery, which illuminates issues concerning agents simulating copresence.

PRESENCE

Blascovich (2002 ) argues that much discussion has taken place in the literature regarding the concept of presence without producing a universally accepted definition of “presence.” Considering presence to be the prima facie key-added value in collaborative virtual environments (CVEs), researchers have just begun to analyze what presence is, what cognitive variables are connected to presence, how it is generated in CVEs, and what are its benefits for education and training. In the absence of a universally agreed definition, the basic definition provided by Lombard et al. (2000), that characterizes presence as the subjective sense of being in an environment, is deemed sufficient within the context of the current research. The environment referred to in this definition need not be restricted to computer-generated environments, as people can also experience presence when reading a book or watching a movie. Indeed, the phenomenon of presence has been researched in other domains: In film theory it is known as the diegetic effect (Burch 1979), and in literature theory it is known as transportation (Gerrig 1993).

Tan (1996) defines the diegetic effect as the experience of the fictional world as the environment, in that the film creates the illusion of being present in the fictional world. Burch (1990) argues that the diegetic effect in film is based on the general visual stimuli that paintings and photographs provide, which result in drawing the viewer into a position that is defined in relation to an imaginary space behind the window and formed by the picture plane and the frame. Bordwell et al. (1985) claim that the viewer of a film becomes an onlooker on an environment, and that viewers experience the fictional events as if they were happening around them.

“Transportation” relates to research on the persuasiveness of narrative media, such as books, and is defined by Green and Brock (2000) as a convergent process, where all of a person's mental systems and capacities become focused on the events occurring in the narrative. The physical environment (for example, a reading chair) is suppressed in favor of an alternative cognitively imagined environment, with the difference between the two to be understood as metaphorical travel, i.e., the transportation to another environment.

Thus, presence can emerge from both the perception of sensory stimuli, such as pictures and sound, and the understanding of symbols, such as words; either way, Schubert et al. (2001) argue, a spatial model of the cognitively imagined environment is constructed. Schubert et al. further argue that users report presence in a film as well as in a fictional narrative, and in each case, the experience is mediated by the construction of cognitive representations as another mental layer. Further, narrative is also important, especially in the absence of sensory stimuli, since literature needs narration to produce transportation, and the diegetic effect in film strongly profits from narration (Burch 1990). In CVEs, however, narrative elements have been largely ignored so far, with the exception of the games genre (Schubert et al. 2001).

Presence is closely related to immersion (Johns et al. 2000), in that immersion is seen as the experience of an intense sense of presence, with one part of reality receiving total attention to the exclusion of other parts of reality (Tromp 1995b). According to the Oxford English Dictionary, immersion is synonymous with absorption. Thus, Psotka et al. (1993) characterize immersion as the cognitive state of being mentally absorbed into some kind of environment. Similarly, Tromp (1995a) argues that immersion shares many aspects with absorption, defining absorption as “a state of total attention, which involves a full commitment of available perceptual, motoric, imaginative, and ideational resources to a unified representation of the attentional object, be it a landscape, a human being, a sound, a remembered incident, or an aspect of oneself.” The individual, who is absorbed in a novel, film, or CVE, uses information from an external source to generate and elaborate imaginings that temporarily become the main focus of attention.

Absorption is a phenomenon addressed by a wide range of academic literature in areas such as hypnosis, meditation, expanded awareness, peak experiences, mysticism, aesthetic experiences, altered state of consciousness, and drug effects. Studies in this area have revealed significant correlations between hypnotic responsiveness and the ability to become absorbed in activities involving imagining, such as watching a movie, reading a novel, daydreaming and, dramatic acting (Barber and Wilson 1979). Shor (1970) explains how responding to hypnotic suggestions is similar to becoming involved in imagining while reading a novel. Hilgard (1970) concluded from empirical research that the responsive hypnotic subject is especially capable of creating and becoming immersed in a process of imaginative mental production that is initiated by an external source such as music, a film, a novel, or by the suggestions of the hypnotist. Similarly, Loomis (1992) argues that individuals' immersive tendencies are related to their experience of presence, the sense of being there within a simulated environment (Sheridan 1992).

Social Presence and Copresence

Social presence is the most common theoretical model used to investigate mediated communication systems such as CVEs (Fulk et al. 1987; Walther 1996; Rice 1993; Caldwell et al. 1995; Palmer 1995). The concept of social presence has been used as a theoretical basis for comparing face-to-face interactions to mediated interactions and comparing different mediated interactions to one another (Short et al. 1976, Walther 1996). Short et al. (1976) define social presence as the degree of salience of the other person in the interaction and the consequent salience of the interpersonal relationships. Bull (1983) extended this concept to include the sense that occurs when one person feels another person is “there.”

Based on a review of existing theories and measures of social presence, Biocca et al. (2001) identify three dimensions of social presence: copresence, psychological involvement, and behavioral engagement. Copresence is the degree to which users believe they are not alone and secluded, i.e., their level of awareness of other users and their sense of the degree to which other users are aware of them. Psychological involvement is the degree to which a user allocates focal attention to the other users, empathically senses or responds to the emotional states of the other users, and believes that they have insight into the intentions, motivation, and thoughts of other users. Behavioral engagement is the degree to which users believe their actions are interdependent, connected to, or responsive to other users and the perceived responsiveness of the other users to their own actions.

The theoretical construct of copresence focuses on evaluating the sense of connection with another mind or other minds (Nowak 2000). The term copresence originated in the work of Goffman (1963), who explained that copresence exists when people sense that they are able to perceive others and that others were able to actively perceive them. Goffman argued that the full conditions of copresence have been achieved when users sense that they are close enough to be perceived in whatever they are doing, including their experiencing of others, and close enough to be perceived in this sensing of being perceived. Further, Ciolek (1982) emphasized the importance of attention or responsiveness to others in this sense of copresence. Nowak (2001) argues that the concept of copresence has a dual nature in that it involves users' perceptions of their partners' involvements in the interactions, i.e., perceived copresence, and the users' own involvements in the interactions, i.e., self-reported copresence.

CYBERAXIS FIELD STUDY

Some evidence for the importance of copresence in collaborative virtual environments was found in a field study conducted by Gerhard (2003) within an existing virtual art gallery called CyberAxis. CyberAxis was built as an interactive three-dimensional virtual representation of an art gallery, providing a means to view and discuss artworks over the Internet. Visitors to CyberAxis could walk around the virtual space by manipulating their avatars. Images of artworks exhibited in CyberAxis were selected from the Axis database. Online artist residencies complemented a series of professionally curated exhibitions. During these residencies, artists were available in the virtual gallery represented by their avatars, and the public was invited to join live debates.

The artists in residence were required to be online for 20 hours during their exhibition, that is 10 sessions of two hours, with the times and duration for each session being published in advance. At these times, the artist represented by their avatar was online in the CyberAxis gallery to discuss their works. The artist residency debates were seen as core to the success of CyberAxis. The residencies allowed the artist to gain experience in, and benefit from, articulating their experience to the Internet community. In addition, the Internet community could listen to artists explaining their visual art practice and possibly engage in debate about it.

The blaxxun Virtual World Platform (VWP) software was used to implement the virtual gallery. A VRML multiuser server and a community administration interface are integrated into VWP, enabling the development and operation of scalable CVE applications.

Findings

Analysis of the CyberAxis field study suggests a direct relationship between the feeling of being present in a collaborative virtual environment as experienced by participants and the copresence of other participants (Gerhard 2003). It emerged that people who were alone in the collaborative virtual environment, experiencing a lack of copresence, consequently, experienced significantly reduced levels of presence. The evidence suggests, that is, that copresence significantly increased the feeling of presence. This intuitively suggests that it might be beneficial to extend the level of copresence by deploying agents simulating the copresence of other users, and thus increase the feeling of presence as experienced by the participants.

PROTOTYPE AGENT

The blaxxun VWP contains an application programming interface (API) that allows the developer a comprehensive system for adapting and expanding its capabilities. The agent API offers access to agent scripting from blaxxun. This allows application developers to link predefined agent actions with various system events. Via the agent API, the blaxxun agent can be integrated with complementary AI technologies. The agent server, performing event handling and response selection processes, is responsible for appearance and animation of the agent's avatar. Given the results of a series of experiments on avatars (Gerhard et al. 2002) and the observations made on avatars during the field study, a humanoid embodiment of the agent (rather than a cartoon or abstract representation) was chosen (see Figure 1). Using the blaxxun agent script it is possible not only to display an avatar and assign scripted behavior to be interpreted by the blaxxun agent server, but also to interface with external applications to extend the functionality of the agent. Thus, it is possible to incorporate advanced chat skills to create an embodied conversational agent.

Figure 1 Appearance and animation of the agent.

Crucial for the effectiveness of the agent is not only its appearance, but also its conversational behavior, whether it can communicate with others on behalf of the absent user. A (disputed) yardstick for judging machine intelligence is whether it can play what Alan Turing called the imitation game, now known as the Turing test. No computer has actually passed the test, but since 1991 the Loebner competition has put conversational agents to the Turing test and annually awards the best agent with the so-called Loebner prize. The winner of this competition in 2000 and 2001 is an open-source project called Alice. Alice is an abbreviation for Artificial Linguistic Internet Computer Entity. It is an open-source environment dedicated to promoting networked artificial intelligence and natural language processing.

Alice uses AIML (Artificial Intelligence Mark-up Language), a nonstandard XML-based specification for bots, and includes a large AIML general knowledge base containing 40,000 categories, i.e., units of knowledge, on which a bot can speak. Further, special domain knowledge modules are available for downloading (concerning, for example, artificial intelligence), and additional knowledge modules can be built (in the current context a module containing domain knowledge about art was created). The prototype agent has been named Art-Fairy. The embodification of the agent is humanoid and its alleged capabilities cover general conversational skills and some (admittedly, rather limited) knowledge of the art gallery environment. Although a knowledge module was developed for the prototype agent providing some background information about the virtual gallery space and the current exhibition of artworks, the knowledge base of the prototype was largely based on extensive pre-existing AIML modules available at the time of development.

However, as Bates et al. (1992) have argued, it is not always necessary for an agent to have deep knowledge of a domain in order for it to generate behavior that is believable. Given this, and given Alice's humanoid appearance, implemented via the blaxxun agent API, and given Alice's conversational skills, underlined by winning the Loebner prize, it could be argued, prima facie, that the prototype agent may be capable of simulating copresence in the virtual gallery. A key aspect of the current research was to experimentally test this prima facie argument.

CONTROLLED EXPERIMENT

Considering recent progress in agent technology, the development of systems that simulate the copresence of other users is now feasible and a suitable prototype agent for experimental use was developed. Given that no agent-populated CVE is currently available for descriptive empirical study, it is argued that controlled experimentation is the appropriate approach to investigate the effects of agents in CVEs. Further, the controlled experimentation approach has the advantage of contributing to the internal validity of results obtained (Tuckman 1999).

Variables

It was argued earlier that presence is the key indicator for the usability of the CVE. Since presence cannot be directly measured or manipulated in a concrete way, it was the intervening variable of this experimental setting. The CyberAxis field study provided evidence that copresence increases the level of presence as experienced by participants. In the current experiment, the effects of copresence simulated by agents were investigated; copresence was therefore the independent variable to be manipulated to determine how it affected the degree of presence the participants experienced during the trials. In the current experimental setting, there were two different levels of copresence: copresence simulated by agents and no copresence.

Within this experimental framework, two variables functioned as secondary independent variables: the composition of the CVE and the interaction model used within that CVE. For this series of experiments, these variables were constant. CVE and interaction were therefore the moderator variables. The dependent variable is the response variable. To observe the impact of copresence simulated by the agent, its effects on presence had to be investigated. This was attempted by measuring participants' experience of immersion, involvement, and awareness, which were therefore the dependent variables in this setting. The previous experience of users within the specific environment or within a similar environment and the individual immersive tendencies of users determines the individual characteristics of participants (Tromp 1995b). For the purpose of this experiment, these variables needed to be controlled and their effects on presence neutralized. User experience and immersive tendencies were therefore the control variables. The hypothesized relationship and the combination of all variables involved are illustrated in Figure 2.

Figure 2 Combined variables.

Control and Manipulation

To reduce the effects of maturation (rehearsal effect) being a potential source of internal invalidity, a between-groups design was applied to guarantee subjects participate only once in the experiment. The two variables that needed to be controlled were the subject's previous experience and individual immersive tendencies. It could be assumed that the experience of participants within this specific environment was not an important factor as all participants were in the same situation, in that they all entered the test environment for their first time. However, previous experiences in other, similar virtual environments, as well as different levels of expertise in the area of visual arts, could have been influential and needed to be controlled. Further, the individual absorption abilities or immersive tendencies might have varied greatly among participants and potentially affected the outcome of this experiment. Therefore, prior to the experimental sessions, each subject completed a pre-experiment questionnaire in an attempt to determine, quantify, and equally distribute experience and individual immersive tendencies between subject groups.

The deployment of copresence was the independent variable to be manipulated to investigate changes in participants' responses. Participants were divided into two groups of 10 participants each dependent upon their responses to a pre-questionnaire measuring the control variables, in an attempt to provide equivalence of groups. Participants assigned to one group (group B) did not experience any copresence: Each subject was alone in the gallery during their experimental session. Participants assigned to the other group (group A) found a virtual agent in the gallery, simulating the copresence of another gallery visitor (see Figure 3).

Figure 3 Subject interacting with agent in the gallery.

Experimental Procedure

In total, 20 participants took part. The sample was selected randomly from a population of university students, schoolteachers, and visual artists registered on the Axis database. A task was needed to stimulate participants to explore the virtual gallery. To achieve this, participants were given the task of selecting one favorite artwork in each of the three exhibition rooms. Further, each participant was asked to fill in a pre-experiment questionnaire and a post-experiment questionnaire, which consisted of attitude statements with (seven-point) Likert scales and open-ended questions. The pre-experiment questionnaire attempted to identify and measure possible individual differences of subjects' previous experience and their abilities or tendencies to immerse themselves in different environmental situations. In this experimental setting, therefore, it measured the control variables and was used to assemble subject groups where these variables are evenly distributed. The post-experiment questionnaire addressed the subjective experience of participants in the simulated environment. It was designed to identify and measure the degree to which aspects of the virtual environment could engender a sense of presence.

The images of artworks were taken from the Axis database (the British National Artists Register). Inside the virtual gallery, 45 images of contemporary artworks were exhibited, 15 in each of the three exhibition rooms. The implementation of this experiment was fully Web-based. Questionnaires used were implemented as CGI/Perl online forms to be submitted by participants electronically. The CyberAxis virtual gallery model was used. The gallery consisted of one reception room and three exhibition rooms. The blaxxun Virtual World Platform VRML multiuser server was used to make the virtual gallery accessible on a Web server and enable avatar and chat interaction. The blaxxun Contact VRML browser was used on the client side. Terminals to access the system were provided at locations within Leeds Metropolitan University and Axis premises.

Results

Results of the post-experiment questionnaire are summarized in Table 1, which shows the scores for the dependent variables (subscales 1–3). Table 1 also shows the scores for the moderator variables (subscales 4 and 5).

Table 1 Results Post-Questionnaire

	X1 (agent)		X2 (no-agent)
	Mean	Stdev	Mean	Stdev
1. Immersion	5.34	0.99	3.80	0.65
2. Involvement	5.05	0.76	3.85	0.63
3. Awareness	4.80	0.82	3.35	1.03
4. CVE	5.60	1.08	5.20	0.63
5. Interface	4.90	0.92	4.37	1.38

Presence is the combined results from subscales 1–3. Table 2 shows the data for presence: mean (for X1: 5.12 and for X2: 3.75), as illustrated in Figure 4, standard deviation (for X1: 0.71 and for X2: 0.52) and standard error (for X1: 0.23 and for X2: 0.16). Table 2 also specifies a 95% confidence interval for the presence means, and gives the minimum and maximum score for each condition X1 and X2.

Table 2 Descriptive Statistics for Presence Means

Variable: Presence
					95% Confidence interval for mean
	N	Mean	Std. deviation	Std. error	Lower bound	Upper bound	Minimum	Maximum
X1	10	5.1240	.7137	.2257	4.6135	5.6345	3.62	5.77
X2	10	3.7530	.5197	.1644	3.3812	4.1248	2.92	4.46
Total	20	4.4385	.9294	.2078	4.0035	4.8735	2.92	5.77

Figure 4 Mean presence scores.

Since, in this experiment, each subject performed under only one of the conditions, this experimental setting is said to have one treatment factor with no repeated measures. This setting is also known as the completely randomized experiment (Tuckman 1999). Two samples of scores, one for each group, were obtained from the experiment. A group mean is taken to be an estimate of people's typical level of performance under a particular condition. But individual performance can vary widely and at times did indeed deviate markedly from the group mean.

Levene's test for equality of variances was carried out (Table 3) to determine whether the variations of scores for the two groups were the same, whether or not equal variances could be assumed, and which t-value was appropriate. As the significance value is larger than .05 (here it is .476), equal variances were assumed.

Table 3 Independent Samples Test of Presence Mean

Levene's test for equality of variances			t-test for equality of means
	F	Sig.	t	df	Sig. (2-tailed)	Mean diff.	Std. error diff.	95% C.I.
								Lower	Upper
Equal  variances  assumed	.529	.476	4.911	18	<0.0005	1.3710	.2792	.7845	.529
Equal  variances  not assumed			4.911	16.451	<0.0005	1.3710	.2792	.7805

The value for significance (Sig. 2 tailed) is less than .0005; therefore, there is a significant difference in the mean scores on presence for each of the two groups. This in itself gives evidence that the difference could not have occurred by chance. However, to provide an indication of the magnitude of the difference between the two groups, the effect size was also calculated. There are a number of different effect size statistics, the most commonly used is eta squared (Kirk 1982). Eta squared can range from 0 to 1 and represents the proportion of variance in the dependent variable that is explained by the independent variable. The formula for eta squared is as follows: Eta Squared = t²/t + (N1 + N2 − 2). Where t is the t-value provided in Table 2. Replacing with the appropriate values from above (t = 4.911, N1 = 10, N2 = 10):

Expressed as a percentage, this means that 57% of the variation in the presence scores can be explained by the existence (or nonexistence) of the prototype agent.

Qualitative Data

The analysis of the qualitative data collected was intended to capture participants' individual perspectives and experiences. Within the post-experiment questionnaire three open-ended questions were asked. The question Do you have any other comment on this experiment? was intended to cover any general problems that might have occurred during the experiment. One subject reported that the agent had not responded to their chat attempts; no other problems or views were reported. The question Beside you, how many persons were in the virtual gallery? was put only to participants receiving the agent treatment. It was intended to reveal the attitude of participants towards the notion of Art-Fairy being another person, as opposed to being another object within the gallery. When asked how many other persons were in the gallery during the experiment, six out of the 10 participants in the agent group answered “1” to the question, seemingly regarding Art-Fairy as another person,or at least as some kind of entity that can be referred to as a person within the virtual gallery context.

The questions How do you feel about Art-Fairy? Did you talk to her? Did her presence enhance the gallery environment? Do you think the existence or nonexistence of other gallery visitors was significant for the experience? (Please explain your answer.) was also put only to participants receiving the agent treatment. It was aimed at capturing all possible issues concerning the agent itself, which participants might have experienced during the experiment.

Statements significant with respect to revealing the shortcomings and strengths of the prototype were of special interest. These were extracted and separated into critical and positive statements as follows.

Critical Statements:

	sometimes responded like a robot, did not give appropriate replies
	didn't directly answer my questions, asked unrelated questions
	didn't understand misspelled words, voice support was missing
	didn't know much about the artworks, didn't help with the task
	I had no awareness of the environment, had no sense of where I was or looking at
	seemed very static, wouldn't follow me to other rooms, didn't respond to gestures

Positive Statements:

	good for company, created a friendly atmosphere
	good to have someone to talk to
	helped me understand what was going on, told me the names of the artists
	looked very realistic, at times like talking to a real person on the net
	was fun and exciting, silly replies made me laugh, enjoyed talking to her
	was trying to be helpful

Further qualitative data, contained in the form of conversational transcripts extracted from the chat log files, was downloaded from the blaxxun VWP server; full transcripts are reported in Gerhard (2003). The chat log files containing the agent-subject conversations in textual form were analyzed in detail, by applying to each utterance linguistic evaluation criteria as argued for by Gerhard (2003). Further analysis of the files revealed that three out of the 10 participants receiving the agent treatment did not make any attempts to communicate with the agent at all. Further, three participants terminated the conversation with the agent almost immediately when the system's natural language recognition/understanding failed.

The remaining four participants engaged in lengthy conversations with the agent, with some participants showing great patience when continuing to interact with Art-Fairy, despite not receiving the task-related information they were trying to retrieve. On a few occasions the system worked well and did trigger a number of relevant and task-related responses. Overall, the analysis of these conversations revealed that the agent did follow some of the dialogue conventions of natural human dialogues.

Further, the system's reaction to users' turns on a basic level was satisfactory in that a response, making fairly concise use of the English language, was triggered by the agent. However, at times it proved to be less successful on a semantic level. Consider, for example, the agent's response to the last question in Figure 5 ([Subject]: What are the names of the artists? [Art-Fairy]: “That information is confidential.”). The response seems to be a valid agent turn on a syntactic level. However, the agent's response did not contain the information wanted by user, thereby suggesting that the agent did not understand the user's turn on a semantic level. Indeed, it led to the subject terminating the conversation.

Figure 5 Chat log extract 1.

On a different occasion a similar question was put to the agent by a different subject ([Subject]: Who are the artists? [Art-Fairy]: “George Hainsworth, Maggie Angus Berkowitz, Jo Holland, Emily Bates … to name just a few. They are all on the Axis database.”); this time the agent's response did contain the desired information (see Figure 6). This example shows that at times the agent was not able to give a meaningful response to one question (What are the names of the artists?), although the related domain knowledge was held within the system and was used to respond to a similar question (Who are the artists?). This inability to cater to different linguistic expressions of the same semantic content exposes one of the shortcomings of the AIML module developed for this prototype agent.

Figure 6 Chat log extract 2.

When user input was inconsistent or ambiguous, the system at times did seize the initiative to repair the failure or misunderstanding (e.g., [Subject]: This painting is by Heather Easters, is it an abstract painting? [Art-Fairy]: “What do you refer to?”). However, in most cases it failed to clarify misunderstandings, and instead pretended that the user's input was understood by giving a bogus reply (example continued from earlier … [Subject]: The style of painting [Art-Fairy]: “Oh now I see.”).

DISCUSSION

It was predicted that the deployment of an embodied conversational agent (ECA) and the experience of presence would be positively related. Results show that the mean difference was indeed significant between the subject group provided with the agent treatment and the control group, who did not experience the agent in the gallery. The implication of this is that, by simulating the copresence of another entity within the experimental environment, the prototype agent did succeed in increasing participants' experience of presence. Despite the small sample sizes of 10 participants in each of the experimental group and the control group, the statistical results of this study strongly suggest therefore that participants experience a higher degree of presence when ECAs simulating other participants are copresent, compared to being alone in the virtual world. This is an interesting outcome considering that three out 10 participants in the agent group, who did not communicate with the agent at all, still experienced more presence than participants in the control group. This can perhaps be seen as an indication for the importance of the agent's embodiment, since, although some participants within the agent group might not have communicated with the agent, they were still aware of the agent's existence through its avatar.

Within this experimental setting, the task was kept rather simple to encourage exploring of the environment without the need of any arts knowledge on the part of the participants. However, it is believed that the task was sufficient to engage participants in the environment to an extent that enabled the subsequent measurement of presence. Participants did not report any difficulties filling in the questionnaires or any other general problems that might have occurred during the experiment.

On the other hand, building an agent that can carry out a proper face-to-face conversation with a human being was not the aim of the prototype development and arguably is beyond the current state-of-the-art. The availability and use of such a system could be considered as the ideal scenario for this kind of experiment: an ECA capable of passing a face-to-face Turing test (Cassell et al. 1999) where a person has to determine whether a screen was part of a video-conferencing set-up, displaying another human being filmed in another room, or whether the screen was displaying an ECA running on a computer. With such systems not yet available nor indeed for the foreseeable future, the prototype used here was merely a system capable of some elements of small talk; it would not pass the Turing test. Although being embodied by an animated humanoid avatar and having the most advanced conversational capabilities available at its disposal, the prototype agent would certainly not be a challenger in a face-to-face Turing test. Nevertheless, the agent proved to be sufficient to simulate some form of copresence, and consequently, to significantly increase participants' experience of presence.

Analysis of the qualitative data extracted from answers to the open-ended questions as well as chat log files was aimed at further investigating agent design issues and their effects on participants. Interestingly, it turned out that all participants in group A without exception became aware of the existence and physical appearance of the agent, with a number of participants regarding the agent as another virtual person within the virtual gallery context. User expectations towards the agent varied greatly. It has to be noted that three out of the 10 participants receiving the agent treatment did not make any attempt to communicate with the agent. Analysis of the qualitative data revealed that two of these participants did not feel the need to consult the agent because the ease of the task did not require any help or assistance. One subject attempted but failed to communicate with the agent due to interface usability problems (specifically, the subject didn't know that it was necessary to invite Art-Fairy for a chat, by right clicking the agent's avatar and selecting invite chat, or by mentioning the agent's name).

A further three participants terminated the conversation with the agent almost immediately. It can be speculated that once these participants had realized that it was an agent and not another human being in the gallery they lost all interest in the conversation. Despite this, however, these participants still seemed to experience greater presence than their counterparts in the control group. This further suggests, perhaps, that even limited copresence is sufficient to help users feel presence in the environment. Furthermore, the results seem to be consistent with similar findings by researchers in the related field of intelligent tutoring systems, where the effect of a speech-driven embodied interface agent was investigated (Moundridou and Virvou 2002).

Moundridou and Virvou (2002) claim that the results from their study confirm that such a pedagogical agent can enhance students' learning experience as perceived by the students, though they could not find clear evidence showing that the presence of their agent did improve short-term learning effects of the system.

Not unexpectedly, the perceived shortcomings concerning the linguistic behavior of the Art-fairy agent were numerous. The nonexistence of automatic spell check or voice support decreased the usability of the system considerably and proved to be a nuisance for users. Further, it became obvious that the agent gave a number of bogus responses, where some basic language recognition succeeded and a seemingly reasonable response was triggered, but where the system nevertheless did not understand the user (e.g., [Subject]: Which painting did George Hainsworth create? Art-Fairy: “The big one, I think.”). The agent contained only very limited subject knowledge and was rarely able to help participants with their task.

The perceived strengths of the prototype agent were far fewer; positive points noted were the general reliability of the speech/language system (namely, all user input did trigger a response), and the agent's ability to generate clear and concise language output. Further, the analysis of the subject-agent conversations revealed that the agent was able to perform some elements of small talk, followed largely the conventions of natural human dialogues in terms of turn taking, and that the vast majority of agent utterances contained information somehow related to the environment or to participants' previous turns.

The physical appearance of the agent was humanoid, similar to those of the participants, and this appeared to be very successful, with 60% of participants regarding the agent as another person within the virtual gallery. However, considering the context of the question, it can be assumed that participants are likely to have taken the term person figuratively, not considering the agent as an actual human being, but as some form of entity representing a virtual actor within the CVE. Furthermore, concerning its bodily behavior, the agent's capabilities were very limited; the agent was not able to change its position or walk within the environment, did not generate meaningful gestures or facial expressions, was not aware of the location of objects and other avatars in the world, and did not perceive gestures or facial expressions of avatars triggered by participants.

Overall, results are, perhaps, important and encouraging for the future use of ECAs in CVEs, for it seems to suggest that even relatively limited agent capabilities suffice to increase perceived levels of presence. Although the statistical results suggest that the prototype agent did succeed in simulating some degree of copresence, it is believed that within this specific setting the agent did not necessarily succeed in simulating the copresence of a human being but that of some kind of entity, possibly that of a digital companion, as the agent took the role of a nonintrusive, at times informative, companion.

CONCLUSIONS

In sum, despite all the shortcomings, the prototype agent does seem to have increased participants' experience of presence. Evidence was found that even limited copresence as provided by the current prototype is sufficient to help users feel presence in the environment. The results reported in this paper, it is argued, are important and encouraging for the future use of embodied conversational agents in collaborative virtual environments, for it seems to suggest that even relatively limited agent capabilities suffice to increase perceived levels of presence.