AN EMERGENT EMOTION MODEL FOR AN AFFECTIVE MOBILE GUIDE WITH ATTITUDE

Abstract

This article describes a novel emergent emotion model for a context-aware “Affective Mobile Guide with Attitude,” a guide with emotions and personality that accompanies visitors touring an outdoor attraction. The interest lies in modelling the conditions to the emergence of emotions instead of prescripting emotions. The emotion model regulates the guide's emotions, behavior and beliefs, allowing the guide to adapt to its interaction environment flexibly. The guide adopts improvisational storytelling techniques taking into consideration the user's interests, its own interests, and its current memory activation in narration. It has emotional memory that stores its previous experiences allowing it to express its perspectives while narrating the stories. The guide's internal state is expressed through a 2-dimensional face where different facial features vary from one state to another along the arousal-pleasure emotional space. The guide exhibits potential in improving overall tour experience verified by a user evaluation.

The idea of improving tour assistance by agents capable of responding to the needs and emotional states of the users is not new, yet not much has been achieved so far. Users often lose their interest while interacting with current virtual guides due to unmet expectations about the character intelligence and a lack of “life.” One way to tackle this problem is to provide the agent with an emotional system inspired by the human's so that it can generate behavior that produces an “illusion of life.”

This article reports on a novel-emergent emotion model aiming at the creation of a biologically plausible “Affective Mobile Guide with Attitude,” hereafter termed as the Affective Guide. This work is part of the result of a doctoral thesis (Lim, 2007). What we mean by “attitude” is that the guide is capable of presenting stories from its own perspective. The resulting system provides guidance, interesting and engaging interaction around outdoor attractions on a mobile platform. The Affective Guide commences the tour by introducing itself. This is followed by an ice-breaking session where it extracts information about the user's name and interests. Then, it chooses attractions that match the user's interests, and plans the shortest possible route to these destinations. After a brief introduction to the tour, it navigates the user to the chosen locations by giving directional instructions. Upon arrival at a planned destination, it notifies the user and starts the storytelling process. We move away from a guide that recites facts to a guide that tells stories by improvisation. The guide tells facts as well as its autobiography by linking its memories—semantic and emotional—its own interests, and the user's interests to the physical location, thereby augmenting the real world with an additional layer of virtual information. It continuously receives input from the user and shapes the stories according to his/her interests. The user inputs feedback through a graphical user interface (GUI) and receives output by means of text, speech, and visual animation.

We start with a review of work that has inspired our research. Following this, we present a brief description of the Affective Guide. The core element of this research, the biologically inspired emotion model for an Affective Guide is then exposed, featuring the integration of an emotion model, a storytelling system, emotional memories, and a facial animation system. Since the storytelling system and emotional memories (Lim and Aylett, 2007b) and the facial animation system (Lim and Aylett, 2007c) have been published elsewhere, we will describe these components only briefly and focus on detailing how the emotion model drives these components. The algorithms involved are documented. Reports on evaluations with real users to establish the impact of the inclusion of emotions and attitude in the guide can be found in Lim and Aylett (2007a) and Lim (2007).

INSPIRATION FOR OUR WORK

Much work has been done to create context-aware applications that personalize information presentation. Some of these systems are mobile while others are virtual. Some examples of mobile context-aware tour guide systems are C-MAP (Sumi et al., 1998), HIPS (O'Grady, O'Rafferty, and O'Hare, 1999), GUIDE (Cheverst et al., 2000), HyperAudio (Petrelli and Not, 2005), and PEACH (Stock and Zancanaro, 2007). All these systems utilize multimodality to overcome the static constraints of the environment by dynamically changing the user's perception of the environment through the use of multimedia techniques and changing the user's physical location through suggestions of where to go next. While HIPS and HyperAudio do not include life-like interface character, C-MAP, GUIDE, and PEACH do. Other examples of systems that employ life-like interface character are virtual tour guide systems such as Kyoto Tour Guide (Isbister and Doyle, 1999), the system by Almeida and Yokoi (2003), and the SAGRES Virtual Museum (Bertolleti, Moraes, and da Rocha Costa, 2001). Evaluations of these systems reveal that the characters' human-like behavior offers users a more friendly interaction interface and increases the suspension of disbelief in users, at the same time helping them to learn and encourage them to explore for more information.

According to Nass, Steuer, and Tauber (1994), the individual's interaction with computers is inherently natural and social. Because affective communication occurs naturally between people, it is expected by people when they interact with computers. Picard (1997) argued that for computers to be truly effective at decision-making, they will have to have emotion-like mechanisms working in concert with their rule-based systems. Hence, a guide that shows variable emotions, attitude, and adaptive behavior is likely to be perceived as more intelligent and believable. We view the human emotional system to be consisting of a successful body-mind link (Damasio, 1994) where physiological responses influence cognitive processing and vice versa, resulting in a continuous feedback loop between the two levels. Thus, we based our emotion architecture on Dörner's model (Dörner, 2003; Dörner and Hille, 1995), “Psi,” that integrates motivation (physiological drives), emotion, and cognition for human action regulation. Additionally, we aim to develop a biologically plausible agent and the “Psi” model has successfully replicated human behavior in complex tasks (Bartl and Dörner, 1998; Dörner, 2003; Dörner, Gerdes, Mayer, and Misra, 2006).

Motivation in “Psi” represents needs that influence survival and can be existential (e.g., need for water and affiliation) or intellectual (e.g., need for competence and certainty). Emotions are not explicitly defined but emerge from modulations of cognitive processes while cognition refers to those processes that control adaptive behaviors, including perception, action-selection, planning, and memory access. Modulating parameters of cognitive processes that define emotions are:

	Arousal: Speed of information processing
	Resolution level: Carefulness or attentiveness of behavior
	Selection threshold: Tendency to concentrate on a particular intention.

Complex behaviors become apparent when the modulating parameters' values are modified by needs resulting in what can be termed as emotions. For example, fear is experienced under conditions that produce high needs, and as a result, is characterized by a high arousal (quick reaction), low-resolution level (inaccurate perception and planning), and low selection threshold (easily distracted by other cues within the environment in an attempt to detect dangers). The interaction between motivation, emotion, and cognition is a continuous loop within the organism, regulated by both internal and external circumstances. Whenever a need deviates from the set point, it activates the corresponding motives. Several motives may be active at a particular time and one of these motives is selected according to an expectancy-value principle. The selected motive is the actual intention that is executed. There are three different modes of execution: automatism, knowledge-based, or trial-and-error. If the “Psi” agent is highly experienced in performing the current intention, a complete course of action to achieve the goal is carried out automatically; otherwise it generates a plan to achieve the goal. If both these fail, it explores the environment to collect information that will lead to the goal. Additionally, it learns by experience and possesses a memory system in which all perceptions and activities are continuously recorded.

Many other interesting emotion models exist, but these models concentrate either on the physiological aspects (Cañamero, 1997; Pezzulo and Calvi, 2007; Velásquez, 1998) or the cognitive aspects (Gratch and Marsella, 2004; Ortony, Clore, and Collins, 1988; Turrini, Meyer, and Castelfranchi, 2007) of emotions, lacking a communication bridge between the two aspects. The physiological models operate at a nonsymbolic level which is too low for the Affective Guide framework. The guide requires cognitive capabilities for planning and storytelling. In contrast, the cognitive models based on appraisal theories perceive emotions as arising from a certain kind of cognition, with little or no attention to the other important aspects of emotion such as the physiological, behavioral, and expressive components. Additionally, we adopt the “affect as interaction” view (Boehner, DePaula, Dourish, and Sengers, 2005), thus ruling out models that define emotions explicitly. Operating in a real-world environment means having to deal with many uncertain circumstances; hence, an architecture that is flexible and robust such as “Psi” is necessary to allow the agent to cope with uncertainty, react to unanticipated events, and recover dynamically in the case of poor decisions.

For storytelling, we must acknowledge the work of Ibanez (2004), in particular its emphasis on improvisational storytelling. The work focused on improvisational virtual storytellers in a virtual environment, hence, lacks the user's interaction. While Ibanez's system omits user interests during narration, we consider this together with the user's feedback throughout the tour session as important factors that may affect the overall tour experience. The Affective Guide construct stories based on historical facts, its past experiences, and the user's interests.

Facial behavior has been commonly categorized using two judgment procedures: categorical and dimensional. A particular emotion can usually be represented by several facial expressions, depending on intensity, and the discrete approach suffers from the flaw of rigidity, with its one-to-one mapping. The dimensional approach eliminates this restriction, conveying a wider range of affective messages. Using the dimensional approach, the only consistent finding across experiments for classifying facial behavior is the arousal-pleasure dimensions (Ekman and Friesen, 1982) leading to its application in mapping facial expressions to emotions (e.g., Russell (1997) and Grammer and Oberzaucher (2006)). The dimensional approach supports the Affective Guide's underlying architecture for emerging emotional states. However, the mapping adopted in the Affective Guide differs from previous works in that it mapped different facial features (eyes, mouth, and eyebrows) rather than facial action units (Ekman and Friesen, 1982) onto the arousal-pleasure dimensions of emotion.

THE AFFECTIVE GUIDE

The Affective Guide integrates a PDA with a global positioning system and an embedded text-to-speech system. Due to limited resources on the PDA, a server is utilized to hold the data, perform processing, and transmit information to the hand-held unit on demand. Communication between the PDA and the server is through wireless LAN as depicted in Figure 1.

FIGURE 1 The overall system architecture.

Figure 2 presents the main interaction interface during the tour session. The text is displayed on the screen allowing the user to read any information that may have been missed when listening to the speech. The user has the freedom to stop, pause, resume, or repeat the presentation whenever they wish. The guide has abilities to detect the user location, replan the route when the user strays from the planned path, generate and present stories automatically. Notification of current location using both speech and message boxes are provided periodically to ensure continuity in system behavior.

FIGURE 2 The main graphical user interface.

On the way to the destination, if the user is attracted to a site which is not in the preplanned route, he/she can activate the storytelling process manually using the “Tell” button. This facility gives the user freedom to wander around the attraction site at will during and after the planned tour. At any time, the user can always find information about the function of buttons from the “Help” menu. After each storytelling cycle, the user can choose to listen to more stories about the current location or move to the next location by touching the “More” or the “Continue” button, respectively. At this stage, the user has to provide feedback on his/her degree of interest in the stories and degree of agreement with the guide's argument(s). Each rating ranges from 0 to 10 on a sliding bar and reflects the user's current feelings and opinions about the story content, useful for personalizing subsequent stories.

THE EMERGENT EMOTION MODEL

The Emergent Emotion Model is the novel element of this research. Taking the psychological model—“Psi” (Dörner, 2003; Dörner and Hille, 1995) as a basis, we have added a storytelling system, emotional memories, and a facial expression mechanism. We put more emphasis on social interaction with humans and maintenance of cognitive needs, whereas Dörner's work is more concerned with “survival” related issues. In our architecture, motivation is represented by the needs (built-in motivators) of the guide to maintain its performance and establish an acceptable condition for interaction with the user. Emotions are emergently determined by the modulators while cognition is represented by the information processes in PERCEIVE, GENERATE INTENTION, SELECT INTENTION, RUN INTENTION, RECOVER, as well as in the MEMORY OF INTENTION represented by the dotted boxes in Figure 3.

FIGURE 3 The emergent emotion model.

Functionally, the guide perceives the environment continuously. It reads the user's inputs which include the user's degree of interest in the stories (DoI) and the user's degree of agreement to the guide's arguments (DoA); the system feedback, either success or failure in performing a selected intention; and the GPS information. Using this information, it updates its built-in motivator values and generates intention(s). More than one intention can be active at any point in time and the decision on which and how to perform the intention is based on the guide's current emotional state, influenced by the built-in motivators and the modulators. Finally, the execution of intention will produce a success or failure feedback into the system and recovery will be performed as necessary.

When the interaction between the user and the guide starts, the guide's long-term memories, both semantic and emotional, contain a complete set of location-related information and the guide's past experiences, respectively. In contrast, the user's interests model is empty. The guide makes assumptions about the user's interests based on the initial information from the ice-breaking session. As interaction proceeds, the guide constantly updates its user's interest model so that narration is always personalized. The following discussion provides detailed descriptions for each of these processes.

Built-in Motivators

In our context, the guide has two built-in motivators to maintain—competence and certainty. The level of competence refers to the guide's ability to narrate stories and cope with the user's differing perspective about an issue or event, whereas the level of certainty is the degree of predictability of the environment (GPS reliability) and of the user interests. For example, if the user disagrees with the guide's opinion, its level of competence decreases due to lack of confidence in coping with the situation. If the user likes the stories, its level of certainty increases because its prediction about the user's interest is confirmed. Figure 4 illustrates the interaction between the environment variables and the built-in motivators in the PERCEIVE process.

FIGURE 4 Interaction between stimuli and the built-in motivators.

From Figure 4, it can be seen that two signals—positive and negative—exist for each built-in motivator: efficiency versus inefficiency for competence and certainty versus uncertainty for certainty. Additionally, each built-in motivator has intensity and need values. Every time a new input is received, all the attributes of each built-in motivator will be updated. The value of the positive and negative signals are calculated as shown in Equations (1) and (2), respectively.

In the equations, pSignal represents the positive signal (efficiency or certainty) while nSignal represents the negative signal (inefficiency or uncertainty). pSWeight and nSWeight are the weights of the positive and negative signals, respectively. f _i denotes the environment variable values, which can be the GPS reliability, DoI, DoA and the system's own feedback. k refers to the total number of variables that affect each built-in motivator's signals. iWeight refers to the degree of impact a particular variable has on the motivator. In the current version, both GPS reliability and DoI influence certainty but in the ratio of 1:3, which means the DoI has a higher impact on the motivator than the GPS accuracy. This is because the feelings of the user are viewed as more important. The same ratio applies for system success or failure and DoA in the case of competence. Thus, in the current system, the value of k is 2 because certainty and uncertainty are weighted sums over GPS reliability and DoI while efficiency and inefficiency are weighted sums over system feedback and DoA. Using the values of the opposing signals, the intensity of the respective motivators is generated using Equation (3). This intensity value is maintained across each cycle to ensure a gradual change in the guide's internal state:

Subsequently, calculations of needs are performed using Equation (4). A constant value of 1.7 (an approximation of e-1) is applied to Equation (3) to ensure that need values in Equation (4) range from 0 to 1. The need measures deviation from the set point indicator, referring to the need for competence and the need for certainty. A need will tend to increase in value unless it is satisfied. The guide's aim is to keep these values as low as possible. These need values will affect the modulator values, discussed in the next section. The need values also determine the state of the motivators, used for intention selection. Each built-in motivator can be in one of the following five states at any time instant: LOW, MEDLOW (medium low), NEUTRAL, MEDHIGH (medium high), or HIGH depending on its urgency as shown in Figure 4. These states form a fuzzy set that allows the guide to reactively select different coping strategies under different need ranges to respond to its current interaction circumstances.

Modulators

The guide has three modulating parameters of its emotional state: arousal level, resolution level, and selection threshold. Like the motivators, these modulators have an intensity that falls in the range of 0 and 1 inclusive and a weight, a constant value that will shape the guide's personality, discussed in Lim, Aylett, and Jones (2005). The influence of the built-in motivators on the arousal level that further influences other modulators is presented in Figure 5.

FIGURE 5 Interaction between the built-in motivators and the modulators.

Arousal level refers to the speed of processing, that is, the guide's readiness to act. When arousal level is high, fast decisions and actions are sought. Thus, a high arousal tends to narrow attention and inhibit in-depth planning. Its value (aIntensity) is calculated by summing the needs as shown in Equation (5). After the story generation process, if emotional story element(s) have been included, a final arousalIntensity that combines both current and memory arousal is calculated (Equation (6)). It is assumed that the arousal strength of current experience is stronger compared to the retrieved emotional event; thus a weight of 0.4 is applied to the value from memory while a weight of 0.6 is applied to the current arousal value. This assumption is made based on the fact that the guide is directly involved in the currently happening event, hence experiencing a more intense feedback from this event than the event it recalls from memory. If no emotional element has been included in the story, the final arousalIntensity takes the value of the current aIntensity:

Arousal affects both the resolution level and the selection threshold values. Resolution level (Equation (7)) covaries inversely while selection threshold (Equation (8)) covaries directly to the arousal level. Both resolution level and selection threshold are distinguished from arousal level so that the influence of weight parameters (in Equations (7) and (8)), which determine the guide's personality, can be brought to bear. Resolution level determines the carefulness and attentiveness of the guide's behavior. When the resolution level is low, usually when the user dislikes the stories or disconfirms the guide's opinion, the guide provides only brief facts about a particular location. A high resolution level on the other hand, will lead the guide to perform more extensive memory retrieval and story generation, thus, presenting longer stories including its perspectives on the topic (refer to Lim and Aylett (2007b) for detail). Lastly, selection threshold is the limit competing motives have to cross in order to become active. The higher the selection threshold, the more difficult it is for another motive to take over and get executed. This modulator prevents oscillations between intentions.

Generate and Select Intention

The guide has three possible intentions (goals): UPDATEBELIEF (update its belief about the user's interests), NEWTOPIC (adjust the stories presentation), and STORYTELLING (perform storytelling). Each intention becomes active in a different interaction situation. In every interaction cycle, two intentions are generated based on needs defined by the built-in motivators state and strength of the intention, which takes account of the intention tendencies (likelihood to perform the intention), the guide's previous experience and expectation of success. The strength of each intention is a measure of its relevance to the current situation. The intention based on needs is called currentIntention while the intention based on strength is called leadingIntention, and they can be identical or differ from each other. The pseudocode for generation of these intentions is presented in Figure 6.

FIGURE 6 Intentions generation and selection.

The leadingIntention is determined based on the principle of expectancy-value where the intention with the highest strength is selected. It is obtained by applying Equations (9)–(16) in sequential order. The calculation in Equations (12)–(16) are applied once to each of the possible intentions. It has to be noted that the choice of parameters in the equations was for purely empirical reasons. The equations are the result of mapping the original equations in Dörner (2003) to the context of the Affective Guide operations. The presence of a large number of parameters makes the choice and adjustment of initial parameter values to achieve a plausible starting behavior tedious. However, once this is achieved, subsequent behaviors emerge from interaction and learning through experience. In the equations, storytellingTendency, newTopicTendency, and updateBeliefTendency are the guide's likelihood to perform storytelling, story adjustment, and update its belief about the user's interests, respectively. Experience represents the ability of the guide in performing an intention that is affected by successCount, the number of successes so far in performing the specific intention divided by totalPerformance, the total number of times the intention was performed. SuccessExpectation is the probability of success for an intention and tendency represents storyTendency, newTopicTendency, or updateBeliefTendency. Lastly, finalStrength is the strength of the intention after application of decay through multiplication by an intentionFactor:

The guide selects one of its active intentions for execution depending on the importance of the needs and the selection threshold value. The leadingIntention will overtake the currentIntention if and only if its strength is greater than the strength of the currentIntention plus selection threshold. When either UPDATEBELIEF or NEWTOPIC becomes active, a decay is applied to the selected intention to stabilize the guide. This is achieved by multiplying a low intentionFactor of 0.25 to the strength. This decreases the probability of the same intention being chosen again for a short while, avoiding continuous change of beliefs or adjustment of topics. This factor is increased after the finalStrength is calculated and in subsequent iterations until it reaches a maximum value 1.0.

Run Intention

In order to execute an intention, the guide decides whether to explore for more information, to design a plan using the available information, or to run an existing plan depending on which intention is selected and its emotional state. Prompt response occurs when there is no story to tell or the story for the current location has finished, at which point the guide informs the user of the unavailability of any further story. Planning is performed for STORYTELLING. In the case of UPDATEBELIEF and NEWTOPIC, the guide will explore the database for information so that appropriate changes to its belief and story topic may take place.

EMOTIONAL MEMORY

Since much of the information we encounter daily holds emotional significance, we view emotional memory as important and necessary for the Affective Guide. Emotional recollection of past experiences should allow the guide to tell more believable and interesting stories. Holding to this view, the Affective Guide possesses emotional memory including “arousal” and “valence” tags (refer to Lim and Aylett (2007b) for detail), generated through simulation of past experiences. When interacting with the user, the guide is engaged in meaningful reconstruction of its own past (Dautenhahn, 1998) retrieved from its emotional memory, at the same time presenting facts about the site of attraction. The user will be “Walking Through Time” as the guide takes them through the site presenting its life experiences and reflecting the emotional impact of each experience. Through perspective information and past experiences, the guide is able to show its attitude towards certain historical events, thus creating awareness in the user about the existence of such a viewpoint. The emotional tag values combine with the guide's current built-in motivator values to determine the arousal level and valence level (Equations (17) and (18)), allowing the guide to show appropriate expressions while narrating the stories. For valence calculation, it is again assumed that the guide's current experience has a greater emotional impact than the retrieved experience with a weight ratio of 3:2. The final value falls in the range −0.5 to 0.5.

THE STORYTELLING SYSTEM

In the prototype version, the “Los Alamos” site ¹ of the Manhattan Project has been chosen as the narrative domain because it contained many characters with different personalities and ideologies that can be used as Affective Guides. The buildings in the “Los Alamos” technical area are mapped onto the Heriot-Watt Edinburgh campus buildings. Hence, all stories are related to the “Making of the atomic bomb.” It is noteworthy that the Affective Guide includes attitude information only if it is currently competent and highly certain of the user's interests, in other words, when its resolution level is high enough. Following are some sample cases of how the storytelling system works in concert with the Emergent Emotion Model to perform adaptation of the Affective Guide's emotions, behavior, and beliefs on the user interests.

Case 1: If the guide's prediction about the user's interests is correct (high certainty) and the user perspective is consistent with that of the guide (high competence), the guide may experience low-to-medium arousal level and selection threshold with a medium resolution level. In this case, the guide may be diagnosed to experience pride because it can master the situation. It is not so easy for another goal to become active. The guide performs an elaboration to the stories and includes its attitude. The guide's belief about the user's interests is strengthened, consistent with Fiedler and Bless's (2000) argument that an agent experiencing positive affective states fosters assimilation which supports reliance on and the elaboration of its existing belief system. It continues to present stories that match the user's interest attributes in its memory.

Case 2: If the guide's prediction about the user's interests is right (high certainty), but the user's perspective is in conflict with the guide's perspective (lower competence), then the arousal level of the guide will be higher than in the previous case. The resolution level decreases while the selection threshold increases. The guide has some difficulties in coping with the differing perspective, but since it has anticipated the situation, it is motivated to concentrate on the specific goal. The increased arousal due to the user's disagreement may lead the guide to experience a negative affect (e.g., anxiety) and is taken as a failure feedback, leading to requirements for new information and inhibiting the application of current accessible beliefs (Clore and Gasper, 2000). Thus, the guide performs an adjustment to the story topic by modifying its own interest on the related topic. At the same time, it generates shorter stories that contain no or less perspective-based information to prevent further disagreement from the user. Furthermore, since it has allocated its resources to gather information and perform story adjustment, fewer resources are available for generation of comprehensive stories.

Case 3: In the case that the guide's prediction about the user's interests is wrong (low certainty), but the user's perspective is consistent with the guide's viewpoint (high competence), the arousal level of the guide may be equal to, higher, or lower than the second case depending on the user's rating. The selection threshold and the resolution level remains the same, decreases, or increases accordingly. The guide is still in control of the situation making the uncertain environment look less threatening. Nevertheless, the guide may be disappointed about its wrong prediction. Since the guide is experiencing a negative emotional state, it performs more detailed and substantive processing that will lead to mood repair. Its action involves a high user's input fidelity where the information is utilized to change its beliefs about the user's interests. Once more, it generates shorter stories that contain only facts because most of its resources have been allocated for beliefs update. This is again supported by Fiedler and Bless's (2000) discussion that negative states trigger accommodation processes, allowing beliefs to be updated.

From these examples, it can be observed that UPDATEBELIEF usually occurs when the user dislikes the stories, implying that the guide's prediction about the user's interest is incorrect. When this occurs, the guide's need for certainty increases. Since the guide's aim is to keep its needs as low as possible, a self-regulatory process takes place. This regulative process enables effective action regulation, allowing the guide to perform adaptation by modifying its mental model about the user's interests and opinions continuously. NEWTOPIC on the other hand, occurs when the user disagrees with the guide's argument, that is, when the guide's need for competence is high. In such situations, the guide will present less perspective-based information about the current topic or choose a new topic for presentation.

In all cases, the execution will end with story presentation by the guide. The extensiveness of story generation depends on the guide's current emotional state, particularly the resolution level. Normally after UPDATEBELIEF or NEWTOPIC, a quick response is required as the guide has a high need for certainty or a high need for competence, in other words, it is highly aroused, causing it to perform shallow planning and thus generate short stories that contain only facts or general information. By doing so, the guide not only performs adaptation to user's interests but also in terms of its processing strategy based on interaction circumstances. These complex and wide variations of behaviors due to the interaction between built-in motivators and modulators are what allow the guide to be perceived as emotional. Additionally, when stories contain only facts or general information, the possibility of further disagreement from the user is reduced. Meanwhile, the guide uses its updated belief about the user's interests for subsequent story generation until another discrepancy occurs and this process is continuous.

2D GUIDE CHARACTER

Having internal states, the guide needs a more obvious mechanism for expression in addition to its behavior modulation. The most common means of expressing emotions is through facial expressions. Adopting the “affect as interaction” view (Boehner et al., 2005), we build the Affective Guide based on patterns that are familiar to the user but without explicit labelling of the generated emotional expressions. Ekman and Oster (1979) suggested that the individual's ability to express and judge facial expressions varies, which further confirms the appropriateness of our approach—putting the freedom of interpretation with the observer.

The emergent internal states of the Affective Guide are reflected through a simple 2-dimensional animated talking head. We propose a simple approach of facial expression mapping onto the arousal-valence emotional space. This approach is flexible and capable of producing a wide range of expressions, at the same time making effective use of the scarce resources on the PDA. Research has shown that arousal has the greatest impact on the eyes (Morris, deBonis, and Dolan, 2002; Partala, Jokiniemi, and Surakka, 2000). On the other hand, valence affects the mouth curvature (Breazeal, 2003; Darwin, 1948). Additionally, we take into consideration the eyebrows, which have been found to be as influential in facial expression recognition (Sadr, Jarudi, and Sinha, 2003). We mapped these three facial features: the eyes, the mouth, and the eyebrows onto the arousal and valence dimensions, where each dimension influences a facial feature more strongly than the others. The valence value moving from negative to positive will move the lip curvature from a downturn U to an upturn U. In the case of extreme pleasure, the cheek raiser is visible below the eyes. In contrast, for extreme displeasure, wrinkles are formed beside the wing of each nostril due to the action of the naso-labial fold. Along the arousal dimension, the size of the eye opening increases with increasing arousal and reduces with decreasing arousal. As for the eyebrows, they are influenced by both the arousal and valence values. Under a positive valence, when arousal is low to medium (<0.5), the eyebrows will have a slight V curve. The eyebrows become more and more relaxed and straightened with increasing arousal. When arousal is very high (> 0.8), the eyebrows will be raised slightly and the raised inner eyebrows cause delicate wrinkles to be formed across the forehead. On the other hand, the opposite takes place under negative valence. The resulting facial expressions along the arousal and valence dimensions are shown in Figure 7.

FIGURE 7 Different facial expressions on the arousal-valence space.

CONCLUSION

We have proposed an emergent body-mind architecture for emotions, behavior, and belief regulation. Our architecture allows the design of a flexible virtual guide that mediates between internal and external stimuli to elicit an adaptive behavioral response that serves self-maintenance functions. The emotions of the guide are triggered by conditions that are evaluated as being of significance to its “well-being,” establishing the desired relation between the guide and its interaction environment. Emotions have not been handcrafted or pregenerated, but emerge from modulation of cognitive processing, hence, producing a rich set of expressions.

The guide presents personalized stories by improvising taking into consideration the user's interests, its own interests, and the previously told stories in priority order. The guide expresses its attitude through perspective information stored in its emotional memories, hence telling the user facts as well as its autobiography. Throughout the tour session, the guide performs a continual update of its beliefs about the user's interests and adjusts the stories based on the user's feedback to ensure that its presentation is always relevant to the user's expectation. By adapting its behavior, the guide's emotional responses mirror those of biological systems, consistent with what a human might expect, hence they should seem plausible to a human. The resulting values of cognitive modulation have a corresponding affective display. The user evaluations (Lim and Aylett, 2007a; Lim, 2007) provide some evidence that the Affective Guide can indeed make the stories more interesting, engaging, and improve overall tour experience.

The authors would like to thank the EU FP6 Network of Excellence HUMAINE (Human-Machine Interaction Network on Emotion, http://emotion-research.net) for funding this work.

Notes

http://www.lanl.gov/