TOWARD REUSABLE ROLEPLAYERS USING AN APPRAISAL-BASED ARCHITECTURE

ABSTRACT

Pedagogical drama uses engaging role-play to train the learner in dealing with the simulated situations. An important asset for such dramatic interactions could be embodied software agents that can take on different roles in the simulated domain. This paper describes ActAffAct, a bottom-up approach to constructing emotionally and dramatically believable and reusable characters for interactive drama. Emotions are central in engaging drama. The conflicts between the characters in a play and the emotions involved in resolving them are the constituents of a dramatic structure, a plot. This idea leads to the assumption that a cast of characters driven by a simulation of the process described by the appraisal theory of emotion might realize dramatic structures by simply interacting in an environment that is prone to conflict. By abstracting domain-independent mechanisms, these characters should lend themselves to reuse in different scenarios.

PEDAGOGICAL DRAMA

The capabilities of animated agents have been improved dramatically in recent years and their use seems promising for the pedagogical area where they can be used to provide responsive learning interactions that are tailored to the individual learner.

When embodied software agents are employed in learning environments, they often take on the role of a pedagogical expert or at least use knowledge from pedagogical expertise. In some cases, however, the technology is used in a simulation of the domain about which the user may learn by interacting with the simulation and the learning companions in it. Chih-Yueh et al. (2003) define a learning companion as “a computer-simulated character, which has human-like characteristics and plays a non-authoritative role in a social learning environment.” Several possible roles for such an agent have been tackled in research projects (e.g., the teachable agent [Biswas et al. 2004], the troublemaker [Aimeur et al. 1997], the collaborator [Hietala and Niemirepo 1998], or the motivator [Baylor 2003]).

An interesting possibility for learning companions is to use them in a simple dramatic role-play; interacting with each other and the learner, the agents enact a dramatic presentation of the learning content, while responding to the actions of the learner. A well-known example of such a pedagogical drama with predefined discrete choice-points is Carmen's Bright IDEAS described in Marsella et al. (2000).

An architecture for characters that can be used in different worlds—always reacting believably and in accordance with their role—could greatly alleviate the current need for laborious handcrafting of the dramatic structure for such applications. It is a special challenge to reduce the domain-dependent aspects of such an architecture, thereby allowing for the reuse of characters in a different scenario with as little effort as possible. It seems obvious that the actions of agents, and other entities that are present, differ from domain to domain, but (parts of) drama theory and emotion research seem to suggest that the dynamics that form the structure of an engaging plot can be harnessed independent of any specific scenario. This should leave less work for the author of dramatic interactions, i.e., the declaration of domain-dependent actions and behaviors as well as abstract parameters for the agents, both of which they can use and (possibly adapt) during the enactment of a “scene.”

This paper presents the ActAffAct project, an abbreviation for Acting Affectively affecting Acting, a bottom-up approach towards providing emotional characters that interact in a simple story-world with the goal of achieving a plot-like structure while limiting the required top-down directing control. The ideal level of top-down control would be no prescriptive control, just (descriptive) specification of desired performance qualities, resulting in the emergence of a coherent plot from the characters' interaction, although the issue of ensuring sensible outcomes has to be addressed.

As a premise for this endeavour and inspired by drama theory (Martinez and Scheffel 2002; Vogler 1996), we assume that a key to engaging drama is to build characters that use a (simple) simulation of emotions. Emotional reactions connect the elements in a dramatic structure so that a plot is formed. The other necessary precondition for drama is an environment that is ripe for conflict. ActAffAct is a system for experimenting with multiple agents in an environment that is conducive to dramatic conflict. The environment is visualized using 2D graphics and basic animations. The agents themselves use plans for task-directed behavior, but they are mainly driven by emotion-related plans and actions resulting from subjective appraisal of their perceptions. Emotions of the agents are not reified (Petta and Trappl 1997); an appraisal component evaluates the relevance of perceived events as an ongoing process that is inspired by cognitive appraisal theories. ActAffAct is based on the foundation laid in earlier work on the TABASCO framework (Petta 1999; 2003) and is described in more detail in Rank (2004).

RELATED PROJECTS

The computational treatment of emotion by choosing features of a psychological theory as a model has a long history (Colby and Gilbert 1964; Elliott 1992; Moffat and Frijda 1995), as does the quest for believable agents and interactive drama (Bates 1992; Hayes-Roth et al. 1995; Petta 1999; Isla et al. 2001; Prendinger and Ishizuka 2002; Laird et al. 2002; Mateas and Stern 2002; Cavazza et al. 2002).

An effort of the Virtual Theater project at Stanford University included the implementation of principles and techniques of the Commedia dell'Arte.

Another example of pedagogical applications is the support and guidance architecture (SAGA) that was applied in Teatrix (Machado et al. 2001), a virtual story creation environment for children. The dramatic theory used to formalize the process of story creation is mainly based on the ideas from Propp's analysis of the Russian folktale (Propp 1968). In Teatrix, children control the actions of several characters; a director component keeps track of the flow of the story and, if unexpected actions are taken, will question the child for its motives.

One can easily imagine software agents taking on the roles of missing story characters, as in FearNot!, a system that enables children to witness and handle bullying situations in a simulated world (Paiva et al. 2004). FearNot! is an advanced system that used ideas about empathy from different areas of psychology to create an engaging environment for children with autonomous characters, that were often hard to control. Therefore, an important topic for their research is now also the question of the “narrative paradox,” i.e., the balance between autonomy of the characters and the control of the storyline (Sobral et al. 2003a; 2003b).

Carmen's Bright IDEAS (Marsella et al. 2000) is an interactive pedagogical drama designed to help mothers of pediatric cancer patients. The characters in the story are autonomous agents, and two other agents—a director and a cinematographer—manage the flow of the story. The agent models employ emotional appraisal as a key component to influence action selection using a two-step appraisal process. The interaction with the user is accomplished through predefined dialog choices and the dramatic structure is similarly pre-authored.

Another related learning environment populated by software agents is the Mission Rehearsal Exercise, intended to teach decision-making skills to soldiers (Hill et al. 2003). The virtual humans in this project at the USC Institute for Creative Technologies also model emotion by using appraisal and coping mechanisms implemented in SOAR, a reasoning system that has also been used in the virtual tutor Steve (Rickel and Johnson 1997).

Gratch and Marsella (2004) describe EMA, the most recent product of the activities at the USC. EMA is a framework for modeling emotion that tries to be domain-independent by harnessing concepts from appraisal theories of emotion.

FEATURES OF DRAMA THEORIES

Drama is the art that deals with a comprehensible depiction of emotional interaction between individuals. Drama theories describe plays, film scripts, and stories in general as the changing constellations of emotional characters. These ideas provide a starting point and success criteria for the creation of dramatic story-worlds. The goal is to simulate characters whose behavior is coherent in the context of drama. Explicit directions, however, given by a “director” component, should not be necessary. The driving vision for the present proof of concept is that the dramatic structure should emerge from the very interaction of the characters, because their behavior is generated by emotional systems.

There are different types of theories about drama, depending on what purpose they serve. The main function of most drama theories is to analyze already existing drama in retrospect; only a few of them try to provide guidance in writing successful or good drama (which is not necessarily the same).

We might also learn from looking at drama theory how to choose a sensible cast of characters or roles: What types of characters will be needed, and what kinds of actions must their environment support in order to make dramatic interaction possible at all? What is a minimal version of dramatic interaction? The result will probably not be interesting by itself, but it should be sufficient for testing whether the behavior of the participating characters can be called dramatically appropriate or, more daringly still, believable.

Analysis and Formalization

An example of a rigid structural analysis of stories is the “Morphology of the Folktale” (Propp 1968), which focuses on Russian folklore. It is an attempt to derive a suitable classification scheme for tales based on their structural properties. According to Propp, the majority of folktales in his study are constructed from so-called plot moves taken from a sequential list: Not all of them are present in every story, but their relative order is always the same.

In Russian folktales, there are only about thirty of these plot particles which are termed “functions of the dramatis personae.” A function is seen as the action of a character in terms of its significance for the rest of the tale's plot (Propp 1968, p. 21).

A tale is analyzed as starting from an initial situation α, connecting a selection of functions acted out by one or more of the cast until it ends (usually in a wedding). All of these functions represent categories of possible actions. Examples of these functions are the “violation” of an “interdiction” shortly after the beginning of a tale, “trickery”—usually executed by the villain—and “punishment” at the end of the story (usually of the villain). Functions range from rather specific, like “provision or receipt of a magical agent,” to very general categories, like “difficult task.” The latter function, for example, could be fulfilled by milking a herd of wild mares, riddle guessing, or winning a game of hide and seek. The former bears a significant resemblance to the plot moves of Campbell (see the following), as several others of the functions do. Some of these functions can be grouped according to by whom they are normally executed. Specific members of the dramatis personae tend to enact specific functions. The classes of characters in Propp's analysis include a villain and a hero, the princess and her father, a donor, a helper, a false hero, and a dispatcher.

Martinez and Scheffel (2002) wrote another theoretical book that looks at the general process of telling stories (albeit in the form of literature). The general questions posed are how something is narrated and what is told. The “how” part is tailored to descriptive analysis and categorization and was not applicable to our project, as the goal of ActAffAct is to generate a plot without varying the mode of telling it. The “what” part tries to define important terms for describing a story, it also stresses that the plot of a story is defined not only by a sequence of events but mainly by the causal relationship between these events. They cite an illustrative example, originating from the English novelist and essayist E.M. Foster.

“The king died, and then the queen died,” is a story.“

The king died, and then the queen died of grief,” is a plot.

Please note that the causal connection between the two events is an emotional reaction.

The Monomyth

Campbell (1968) wrote a very influential book that describes every existing story as the derivation of an archetypal monomyth. Though it can be doubted that truly every successful story must adhere to the principles he lists as defining the steps of a hero's journey, many a book and screenplay has been shown to work according to this monomyth. Vogler (1996) wrote a newer take of this basic story specifically targeted at screenwriters, and the software “Dramatica Pro” ¹ also owes a lot of its concepts to Campbell.

The basic premise of the monomyth is that every story can be seen as the journey of a hero, from her ordinary surroundings into an unknown and challenging world. It can either be a physical one, searching for a special item for example, or an inward journey of the mind and heart. Any good story needs the hero to grow and change, quoting Vogler:

[…] a journey from one way of being to the next: from despair to hope, weakness to strength, folly to wisdom, love to hate, and back again…

Note again the abundance of emotional terms.

This journey is the story of the archetypal hero, during which she meets several other significant types of characters, which need not be embodied as actually different characters, but there are ones that mainly fulfill one of these functions. To give an impression how this monomyth can be described, here is a short summary of the prototypical hero's journey, adapted from Vogler (1996).

The hero starts in her common-day world (her status-quo), from where she is called to adventure as a problem or lack is spotted. After first refusing the call, she meets her helper and eventually crosses the threshold into the unknown world, by overcoming the threshold guardian. She is then subjected to trials and tests ending in a supreme ordeal (the central crisis of the story). Be it passing or failing the ordeal, afterwards the hero is changed and she can take the road back home. This is complicated by a last climactic difficulty, and finally she returns into the ordinary world, changed by the events and possibly bringing along the remedy for the initial problem.

While this is a far cry from a formalized structure, it can nevertheless help at a qualitative evaluation of generated stories. Campbell (1968) is analytical in his nature, while Vogler (1996) tries to harness the ideas in the form of a guide for writers.

Descriptions of the elements of the monomyth bear a striking resemblance to the core relational themes postulated by one of the emotion theories (Lazarus 1991; Smith and Lazarus 1993) as the characterization of appraisals on the “molar” level that arise from the evaluations of simpler components on the “molecular” level. Elliott et al. (1998) is another example that sees a direct connection between the structures that are used to describe drama and those used for emotional episodes.

Characters with a rather rigid predefined profile, so called stock characters, are the principal building block used in 16th century Commedia dell'Arte (also used in Hayes-Roth et al. [1997]). Each participating actor chose one of the well-known roles and they all agreed on a basic theme and a suitable ending for the play. This kind of theater relied heavily on the improvisational talent of the participants, but nevertheless there were rather rigid rules of combining the possible bits of comedy into a coherent whole (Duchartre 1966).

According to the system used in “Dramatica Pro,” there are eight archetypal characters; see Tables 1 and 2 that include possibly corresponding terms from Campbell/Vogler (although the mapping is not as simple as shown here), and some examples from well-known films. They can be seen as pairs of characters: Some of them drive the action, others are more passive. Any two, but for those that form an opposing pair, can be combined as one actual member of the cast. These characters are seen as separating the different motivations in a human, which are then explicitly dealt with in a narrative.

Table 1 Dramatica's Eight Archetypal Characters as Opposing Pairs (terms from Campbell or Vogler in parentheses)

Protagonist (Hero)	↔	Antagonist
Guardian (Mentor)	↔	Contagonist (Threshold Guardian)
Reason (Herald)	↔	Emotion (Shadow)
Sidekick	↔	Skeptic (Shapeshifter, Trickster)

Table 2 Example of Clear-cut Archetypes in Film

Dramatica	Star Wars	The Matrix
Protagonist	Luke	Neo
Guardian	Obi-Wan Kenobi	Morpheus
Reason	Princess Leia	Trinity
Skeptic	Han Solo	Cypher
Antagonist	The Empire	Agent Smith

Minimal Requirements for a Story

In all of the theories reviewed, emotional terms feature prominently when describing the structure of a narrative. Emotions are used for the causal links between events in a story.

What kind of a character constellation should be used as a starting point? We chose to use four players: a hero (protagonist), a villain (antagonist), a mentor, and a victim. The goal is for them to act believably in their environment and to choose courses of action that qualify as dramatic. To make conflict possible, the characters must be guided by their intentions and at least two of them need to have opposing goals (with respect to the shared environment).

Their actions should be understandable in the light of their history in this environment and what we already know about them. Hints about the motives, that originate in a character's back-story, are very varied and often subtle in stories, and as the goal here is only to generate suitable courses of action, these problems will be ignored. The aim is to have several (i.e., four) autonomous characters interact with each other so that their reactions to each other can be interpreted as part of a storyline. The environment and the actions feasible therein should be as simple as possible, examples might be stealing objects from somebody, using a rope to bind somebody, or giving a key as a reward for solving a simple puzzle.

So, imagine a simple world with four characters, two of which have opposing goals, like getting a third one to love them (and only them, otherwise, there would be no conflict). The world is filled with simple objects to be used for good or bad purposes, like flowers, ropes, or swords. What could be a minimal storyline for such a world?

Hero falls in love with Victim and wants to give her a flower. This fails, because Villain has bound her with a rope, so Hero has to solve Mentor's puzzle of stacking blocks, in order to get the keys to the treasure from him. He then bribes Villain with the treasure and saves Victim by cutting her free with his sword. She falls in love with Hero instantly, of course.

The results of the project, i.e., the “storyness” of the generated sequences, were only evaluated qualitatively. A final test of generated stories would be to use the system in an interactive way and to question the audience, but this project concentrates only on implementation issues and the refinement of a reusable domain-independent mechanism.

APPRAISING AGENTS

What can be implemented in computational characters to create the impression of emotional behavior and believable reactions? This section will give a short overview of selected psychological models of emotion and the emotion process with an emphasis on the eclectic features used in the realization of the ActAffAct system.

Emotion is a way to assess and cope with the (likely) subjective significance and the causes of events and actions we observe. To this end, it is possible to make sense of others by using simplified models of them to explain their actions. This is also the way in which emotions are used as explanatory parts of drama theory. But what process in an individual is causing emotion and the actions that are attributed to emotions?

Processes involving emotion are now considered to be an important, if not downright essential, part of the control structures of boundedly rational situated beings. They allow them to react adaptively in a dynamic, changing, and highly complex environment. They are highly flexible (compared, e.g., to reflexes and drives [Smith and Lazarus 1990]) and still do not necessarily require conscious or voluntary reasoning.

Several psychological theories give an account of emotion; cognitive theories of appraisal describe models in which emotions are elicited by an evaluation of situations and events according to certain subjective dimensions or criteria (Frijda 1986; Roseman et al. 1996; Roseman and Smith 2001; Ellsworth and Scherer 2003). An entity's situation in the environment is checked with every perception for changes of relevance to the individual. When a perception is found to be of importance (negative or positive, approach or withdraw, etc.), this does not, however, lead to an overt response directly. Appraisal triggers first a motivation for possible further action. While there is a broader consensus about what criteria are needed for the appraisal process, there is less on how such checks are exactly executed.

The Ortony2003 Model

A very influential work on appraisal, concerning implementation and application, has been published in Ortony et al. (1988). An evolved version of this so-called OCC-model appeared in Ortony (2003).

Ortony classifies the types of emotional reactions on the one hand, and the types of emotional responses on the other. The first scheme includes what can be subject to appraisal and according to what it might be appraised. A basic distinction is made between positive and negative reactions to a situation. Apart from that, the object of the appraisal can either be an event, an action (by somebody), or an object itself. Events might be relevant to an individual's goals, to the standards it tries to uphold, or to its tastes. Table 3 shows the types of reactions an appraisal can result in.

Table 3 Valenced Reactions (from Ortony [2003, p. 194])

to…	Positive reactions	Negative reactions	against…
Event	…because something good happened (joy, happy-for, gloating)	…because something bad happened (distress, sorry-for, envy)	Goal
	…about the possibility of something good happening (hope)	…about the possibility of something bad happening (fear)
	…because a feared bad thing did not happen (relief)	…because a hoped-for good thing did not happen (disappointment, sadness)
Action	…about a self-initiated praiseworthy act (pride, gratification = pride + joy)	…about a self-initiated blameworthy act (shame, remorse = shame + distress)	Standard
	…about an other-initiated praiseworthy act (admiration, gratitude = admiration + joy)	…about an other-initiated blameworthy act (reproach, anger = reproach + distress)
Object	…because one finds something appealing or attractive (love, liking)	…because one finds someone/–thing unappealing (hate, dislike)	Taste

To be able to process its situation in such a manner, the individual needs to have goals, standards, and tastes. To make its reactions believable, i.e., consistent, these values have to be relatively stable and coherent. Goals of an individual directly affect its actions; humans are said to have a hierarchy of goals dependent on each other, with long-term higher-level goals on top, and transient subgoals beneath. Standards and norms are responsible for moral value in the individual's world-view, and tastes or preferences give certain values to single important objects of perception. All these perceptions can also be directed inward, perceiving bodily changes or cognitive processes.

The emotion instances given in parentheses in Table 3 are only examples that fall under the respective category, they are by no means the definitive members of their group: They are just emotion words used in everyday language to explain reactions that might belong to this category. Each of the emotion types is associated with a varied group of reactions or behaviors. The appraisal of an event, action, or object as belonging to one of these types triggers an inclination to use one of these behaviors, a response tendency, as Ortony calls it.

There are mostly involuntary expressive reactions, on the one side of the spectrum, changes in the style of internal information processing and attention, and coping activities that include complex planned actions, on the other end. Figure 1 shows the three categories of response tendencies and related subgroups (examples in parentheses).

Figure 1 Emotion response tendencies (from Ortony [2003, p.198]).

Frijda defines an emotion as a change of action tendency (e.g., “approach” or “avoidance”), a change in the readiness to alter one's relation to external or internal objects. Ortony's response tendencies can therefore be seen as realizing a change of action tendency, on the one hand, when they trigger plans and alter internal states, and on the other hand, as the result of a change of action tendency, when they deploy expressive reactions. One theorized purpose of expressive response tendencies is to reveal the elicitation of an emotion directly to others that are watching, at a subconscious level (e.g., Reisenzein [2001]).

Effects of information processing can either result in redirecting attention or changing general evaluations about other agents or values that the individual has. Coping is a very wide category (and a prominent term in emotion research [Gross 1999]). Ortony identifies two kinds of importance for believability: Problem-oriented coping is the most direct approach to solving the cause of an emotional reaction, trying to use a plan that brings the situation under control, thus improving a bad situation or realizing or extending a good one. Emotion-oriented coping is more involved; it is directed either at oneself or another participating individual. By trying to change oneself or someone else, the individual tries to change or stabilize the situation indirectly.

Ortony also elaborates on models of personality as an important part of modeling a consistent, and therefore, believable character. Several models of personality exist, most of them rely on the characterization of people along a small number of dimensions, e.g., the big five traits (Pervin 1994).

A proposal for an approach to implement a mechanism in that way is to start with only few parameters as, for example, the distinction of promotion and prevention focus. This could influence the choice of coping strategies directly. Promotion focus would tend to choose plans that strive for pleasure, whereas prevention would first try to prevent harm.

The Process of Appraisal

From an engineering standpoint, appraisal is under-specified, many of the steps from a perception to an action are unclear. There also must be an underlying model of the perceiving entity; it must be capable of remembering its goals, standards, and preferences; it must be able to act and perceive, to expect and anticipate.

In our model, the basic action-reaction loop starts with the perception of the environment. Every perception passes through appraisal. This might lead to an entry in an appraisal registry, which in turn causes several response tendencies to be considered by the action part of the model. This in turn effects the environment in a hopefully beneficial way.

This is a vast simplification for the purpose of modeling; the processes that supposedly happen in a human agent do not justify a clear-cut distinction between perception and response. It could be described more aptly as a sensorimotor circuit where sensations and actions happen in a close feedback loop (Dewey 1896; Pfeifer and Scheier 1994). We assume that the simplification is valid in this case, because in the proposed model, actions do have direct access to all current perceptions to actively modify their parameters, and the actions themselves are processed as perceptions.

Among appraisal theorists, there seems to be a general consensus on at least the stimulus evaluation criteria in the following list:

	relevance, goal significance, focus
	standards compliance, blameworthiness
	intrinsic pleasantness, valence, appealingness
	novelty, unexpectedness, suddenness, familiarity
	responsibility
	coping potential

Following the Ortony2003 model, the first three out of this list of evaluation criteria can be seen as corresponding directly to the three necessary value types in an agent—goals, standards, and preferences—whereas the latter three are applicable to every appraisal.

Novelty modifies the intensity of an appraisal: If the perception it deals with has not been expected, the intensity of the appraisal is higher, as opposed to, for example, the expected outcomes of one's own actions.

Responsibility for an action attributes causality to an agent; this is directly usable in directing coping plans or expressive actions, and for changing evaluative processing about this agent.

Coping potential represents the possible or likely degree of control about the situation. It could be higher for perceptions that can easily be answered with several possible coping plans or for those where there is already a suitable plan in execution; when there is no plan available, the coping potential would be low. The value representing the degree of control could also modulate the intensity of the appraisal, as with novelty.

But first we need a basic intensity value for an appraisal. The first three criteria can be seen as subjective assessments of the constituents of a change as differentiated by Ortony (2003) (event, action, or object). They may be implemented as quantitative evaluations that provide a basis for an intensity value, but this only provides an initial account of an appraisal intensity neglecting the dynamics of intensity. The laws of hedonic asymmetry in Frijda (1986) point to a fundamental difference in the changes of positive and negative intensities, respectively; furthermore, repeated sensation of the same type of perception should lead to a decreasing intensity. In the proposed model, however, the intensity values are easily calculated, compared against a threshold, and simply decrease over time when there is no further perception reinforcing the same appraisal.

Perceptions about the success of an action can be matched directly against an agent's standards to yield standards conformance. The standards should attribute a moral value to a specific action or more realistically a group of actions. The value of the responsibility check is used to differentiate between one's own actions and those initiated by others. The preference for agents or specific objects in the environment can also lead to an appraisal about one's feeling about them. This of course must be preceded by a change of this value.

It can be argued that determining what counts as failure of an action in the real world can be rather complicated, even more so for the prospect of something happening. In a simulated world, it is easy to work around this problem. It would, of course, be much more challenging to solve this more realistically. Another point of improvement would be to incorporate pre- and post-conditions of goals into the matching process. A perception that corresponds to a precondition for a goal would also be conducive to this goal; preventing a precondition would be obstructive. But this would mean a more complicated reasoning process involved in appraisal, although it should still be possible without relinquishing the idea of appraisal as a basic low-level function.

Response Tendencies

An appraisal can have a threefold effect, corresponding to the distinction of response tendencies in Figure 1. If the appraisal is intense enough, there should be an immediate expressive reaction, revealing it to others. Information processing effects would be changing the values about the relationship to the responsible person or involved objects in proportion to the intensity, this would be the main way of changing preferences for others.

The most important effect is the installment of goals for coping activities, which could in turn lead to the execution of a suitable plan and the spawning of new goals. The underlying model is responsible for the pursuit of goals, scheduling of concurrent plans, and sorting out the conflicts between action tendencies.

ActAffAct's Agents

For the actual implementation, an existing implementation of an agent model was chosen that already included many of the necessary features: JAM, the Java Agent Model (Huber 1999). JAM belongs to the family of belief-desire-intention (BDI) agent models, based on ideas about resource-bounded practical reasoning (Bratman et al. 1988). It provides its own plan representation language, meta-level and utility-based reasoning, and goal-driven and event-driven behavior. An agent is defined as a collection of its beliefs (facts it believes to be true), its desires (goals), and its capabilities (specified as plans in a JAM specific language with definable primitive actions at the lowest level). Capabilities scheduled to achieve a goal form the third name-giving component, intentions.

Figure 2 gives an impression of the overall architecture including the additions for this project, most notably the appraisal component (Staller and Petta 2001). The intended scenario includes four agents that run concurrently, therefore, some changes had to be done to achieve thread-safety and a way for pausing and resuming the agents synchronously. One reactive agent is used for the simulation of the environment; it can be polled by the synthetic actors for perceptions, and action requests are directed towards it. The Batik toolkit ² is used to display simple animated SVG graphics (Scalable Vector Graphics ³ ) that illustrate the actions of the actors. The sequence diagram in Figure 3 gives a high-level view of the important threads during an episode and Figure 4 is a snapshot of the stage as displayed by the viewing application. The bottom part contains list and tree views of the agent's internal state, i.e., their beliefs and intentions, respectively.

Figure 2 The ActAffAct agent architecture (based on [Staller and Petta 2001]).

Figure 3 Sequence diagram: Interaction of the stage threads.

Figure 4 The AAAViewer application.

Further changes to the agent model include “auto-boredom,” i.e., the reduction of the utility of a goal if it cannot be processed successfully, an idea taken from Moffat (1997). Specific interpretation plans that match perceived facts and transform them to beliefs relative to the agent's current situation are used. During this step, beliefs about the success or failure of an action or the possibility of a future action are formed; this can be seen as a first step of appraisal. This interpretation process should be an evolving subsystem, but in this simple setting, a static set of rules is used. The concept of a focus of attention has for now been neglected, but a primitive form is realized by these interpretations. In order to allow matching generic patterns of facts, JAM's beliefs were extended with a simple placeholder that matches any value.

The domain, the agent's initial roles, i.e., their beliefs and desires, and the actual calculation of appraisal values are described in the next section.

SETTING THE STAGE

In this section, we describe the cast and its environment in the implemented scenario, as well as the components and dynamic aspects of the cognitive architecture.

The Cast and Its Environment

The cast consists of four graphically distinct agents that use the same underlying architecture. The choice of names for the four different agents is influenced by the notion of character archetypes and cliché plots that should be easier to generate. There are no architectural differences, but while one actor is in principle not limited to a particular role in a story, their initial beliefs and goals differ, thus suggesting a particular role.

In order to achieve minimal storylines that are at least intelligible, the characters have to be discernible not only by their behavior, but also by their appearance. The choice of very abstract names that nevertheless imply a bit about their role in a dramatic interaction shall further the inclination of the viewer to interpret actions as dramatically significant.

• Hero: the lead character, the one who should substantially change.

• Villain: the antagonist, trying to disrupt the hero's action.

• Victim: the possibly suffering one, in need of help.

• Mentor: leads the hero's way and lends a helping hand.

There is a thin line between encouraging the viewer to look for drama and avoiding the exploitation of our innate urge to “make sense” of what we see. Humans are very motivated to interpret even unsensible things as coherent, which needs to be considered when the results are evaluated. This project is, however, not intended for a broad audience and thus it can safely be left to the individual viewer to decide whether action sequences are sensible or not.

Apart from the actors, several objects can be manipulated in the environment. Table 4 enumerates the possible activities that the objects in the ActAffAct world might be used for.

Table 4 Objects in ActAffAct's Environment

Object	Possible use
Bomb	activate or deactivate
Rope	bind someone
Flower	give it as a present
Key	unlock the treasure
Sword	loosen the rope or threaten someone
Block[1–3]	stack them
Treasure	give it as a present

These objects and activities were chosen with conflict in mind and to provide the possibility of solving a puzzle. For most of the actions, it should also be obvious whether they are beneficial of detrimental for the target. Thus, the objects can be used to influence how others think of an actor. The villain of the story would be recognizable as the one who threatens and hinders others. The “good guy” on the other hand might help others solve the puzzle and use items to gain acceptance by others. The main puzzle of this small world is the stacking of blocks in the correct order. This quest is, however, not explicitly posed by any of the actors; it is implicit in the circumstances. Such a problem that needs solving would be part of the beginning of a minimal storyline.

Apart from encapsulating the dynamic aspects (the “physics”), the environment is a container for facts that describe the current state of the world. It carries out actions requested by the actors, which are also tracked by facts in the world model, including facts about the success, or, if something prevented it, the failure of actions.

The facts that are visible to onlooking actors can be divided into static and dynamic information about the environment; the former group comprising facts that name the agents and objects that are present and facts about the (discrete) positions and paths in the environment. The dynamic aspects include the current whereabouts of objects and agents, state-related information about objects, and action-related facts that might be seen as a separate category.

The actions that are possible in the environment are: moving around, using objects and moving them around, and, last but not least, expressive actions. The latter ones substitute all the delicate ways in which people express their emotional state, voluntarily or not, so that others can react to them. Expressive actions are represented graphically mainly (and inadequately) as speech bubbles (see Figure 5).

Figure 5 Villain steals flower; Hero awaits a block.

Any combinatorially possible “status quo” can be set up for the four actors in this environment. When the stage is loaded, the actors and objects are randomly positioned in the environment, but for the purpose of testing the variability of the agent's responses in the same situation, specific initial setups are also used.

Elements of Situated Cognition

To get a plot started, at least one of the characters needs a task-oriented motivation to begin. To foster drama, two characters with opposing ambitions should be present. The goal in setting up the initial situation is seeding it with potential conflicts, which means conflicting interests and goals.

The initial concerns (top-level goals of the intention structure) of the four agents along with their initial preferences and their standards are chosen so that conflicts are likely, mainly because the Hero character wants someone to love him, and the Villain is determined to prevent anyone from being happy or loving someone. All of the actors have a permanent goal, their main “concern,” that they keep on their agenda throughout their lifetime. This “task-oriented” goal generates behaviors, and its subgoals are used to evaluate the perceptions of the actor as conducive to or obstructive of that goal. The standards are also used as set points for this evaluation and include the implicit goal to uphold the standard.

The actors are not limited to one concern and a more complex environment would ask for several conflicting concerns in every agent from the beginning. Even so, in this toy environment, single concerns lead to a much more understandable initial cause of action. Further top-level goals are introduced when the actors express emotions and try to cope with important appraisals, so the possibly fruitful complexity of inner-agent conflicts is not avoided at this level.

Apart from the facts that describe the current state of the environment, an actor's knowledge includes the interpretations about past, present, and future actions of others and himself, and its standards and preferences, which set numeric positive or negative values for groups of actions and other agents or objects, respectively.

The plans that an actor uses to achieve its goals are organized in a hierarchy (Petta 2003). The lowest layer contains all those plans that try to act on the environment, i.e., that post action requests. These are used to build action packages, which are employed by the behavior level. This latter level is special as it is also used during appraisal, and its plans use timed control structures. Mediating between the top-level goals or concerns of an agent and the behaviors is the level of activities. In the top layers, i.e., concerns and activities, variability of behavior and the selection of suitable targets are the main issues, while at the bottom, the focus lies in the robust execution of the behavior that was decided upon.

The behaviors that actors know about can also be seen as the main parts of a story, which would also include their justifications: either a top-level concern or a coping activity initiated by an appraisal. Plans at this level are often subdivided into different phases that influence how easily they can be interrupted or replaced by a different plan. Initially, all the necessary preconditions are evaluated, and if the behavior is possible as far as the actor knows, then it is repeatedly tried over a fixed time-slot. If the plan reaches a certain stage of completeness and only final steps are needed, it can then only be stopped by the repeated failure of the subgoals still pending.

The repeated failure during an earlier stage triggers the reconsideration of the behavior by the calling activity. Nevertheless, behaviors can be interrupted by more important coping goals that get posted to the intention structure.

Appraisal and Coping

The agents use a sequential execution cycle: observing, appraising changed facts, selecting—and possibly intending—applicable plans, and then executing one step of the highest-utility plan. During appraisal, new facts in the world model are evaluated to determine their significance for the actor. To check whether a new fact is relevant, it is compared to the possible matching goals and standards; the numerical values “relevance” and “conformance” are calculated and compared against thresholds. If the thresholds are exceeded, an object is created for this appraisal that immediately imposes its effects on the world model and the intention structure of the agent. It is kept in an appraisal register and will be visited each cycle to update its effects, until it is found too old to be of any significance, at which point it is finally removed. If the preference for an object or agent exceeds the corresponding threshold, a similar appraisal object is registered and treated likewise.

If the appraisal of a situation calls for a response, a coping goal is posted. The utility of the goal that decides whether it is executed or not depends on the intensity of the appraisal. The appraisal type can be one of the categories taken from the Ortony2003 model. This coping goal also explicitly includes the reason for the appraisal, i.e., the behavior and the responsible actor as well as the actual arguments and whether it was successfully completed or not (yet).

The coping strategies actually used in ActAffAct include, e.g., the retribution for a behavior that made someone angry, trying to hinder an actor from achieving what it supposedly wants to do, or cleaning up attempts of hindering if the behavior the actor wanted to prevent has already happened.

As mentioned above, the facts that are considered for appraisal are those that deal with behaviors: They either state that an actor completed a behavior, that it failed to do so, or that it seems to contemplate that behavior. The facts can therefore be categorized by the success value the fact indicates—attempt, failure, or success—and the responsible actor, which can be anyone, including the appraising actor.

All the behavior-related facts are matched with the actor's behavior goals; matching with standards is only used for successful actions. For each standard, however, there is a virtual goal to uphold this standard, and it is used for the calculation of the relevance of prospects about actions that are relevant to this standard.

The values for relevance, conformance, and preference always range from − 1 to 1. To calculate the relevance value of a fact, the utility of the matched goal is used, but as JAM only considers the utilities of top-level goals, the intention structure has to be walked to retrieve the significant value. Furthermore, the result needs to be normalized to the range from 0 to 1. The sign of the relevance value is determined by the success value of the fact and the type of behavior goal: Prospects and successful behaviors are represented as positive values as opposed to failures, the same holds true for goals of helping and doing something as opposed to hindering. For standards, the moral value of the standard is used as conformance measure.

If the value for an appraisal exceeds the corresponding threshold, an appraisal object is created and registered. The intensity of the appraisal is calculated by combining its value so far with the recency of the appraised fact and a crude approximation of the concept of coping potential, i.e., if there is already a plan in effect that tries to cope with an identical appraised fact, the current one will receive a lower intensity. The recency of any appraised fact will initially be 1, indicating a new fact, but as cycles pass, each update of the appraisal will lower the intensity value according to the age of the fact. This intensity value is used for the effects of the appraisal.

First, another threshold determines whether the intensity is high enough for an expressive reaction. If so, a goal is posted to the intention structure to show an impulse, with all the details of the appraisal to modulate the expression. In ActAffAct, however, this always results in a speech bubble with a font-size proportional to the intensity. This impulse goal is only posted if the agent is not currently showing an impulse with a higher intensity, as the actors cannot mix the display of concurrent emotions.

Second, the preference for the responsible actor is changed in proportion to the intensity and according to the type of the appraisal.

Third, a coping goal is posted with a utility proportional to the intensity, which will possibly lead to a behavior that is better suited for the current circumstances of the actor.

The goals that are used for coping and showing impulses are subjected to the normal planning procedure of JAM agents, and therefore, are accessible to each other, enabling an actor to suppress the display of emotions or to fake it, but only if there is a “conscious” effort to do so.

RESULTS

This chapter discusses episodes that were observed in the working ActAffAct system. The question is, whether the resulting sequences engendered by the interplay of different instances of the same agent control architecture can be seen as sensible mini-dramas without having to resort to an overly impoverished idea of dramatic interaction.

All of the following descriptions are only qualitative, there was no quantitative analysis of the action sequences produced in different runs of the system, as the intention is only to prove the possibility of “emerging” drama. As will be pointed out at the end of this chapter, more practical work is still needed, as well as the development of a principled approach towards formalizing actions in an environment.

Approach: Varying Initial Setup

The premise of this project is that a cast of autonomously acting characters whose behavior is driven by emotional considerations will lead to dramatically interesting interactions if there is enough potential for conflict in the initial setup. An obvious question is what happens to these characters if there is no conflict in their world. As there are no conflicting initial concerns in any one of the agents, this can be accomplished easily in the ActAffAct setup by simply leaving the Villain character out.

When the Villain is excluded, the action that takes place loses a lot of variety. The most prominent expressions of emotion in an episode with this small cast are hope, admiration, pride, and joy. The only negatively valued appraisal is sadness resulting from an unfulfilled, i.e., false, hope.

This spectrum is enriched substantially when we introduce the antagonist to the cast. When the full cast of Mentor, Hero, Victim, and Villain is used, the interaction gets considerably richer and the appropriateness of the sequences for drama can be discussed. With an opposing force, expressions of fear and relief or anger are more common and influence the plans that the actors use. The results depend largely on the sophistication of the plans used for coping. These plans can be tweaked to get better-suited responses from actors.

The aim of ActAffAct is to provide a template for organizing plans in intelligent actors that lead to suitable responses combined with the appraisal method of evaluating the actor's surroundings. Using behavior goals and plans that explicitly state the intent of favoring or hindering the actions of other actors directly provides goals that can be used sensibly during coping. This structure is beneficial for the evaluation during appraisal and also for the implementation of coping activities.

When the initial situation is set up randomly, the task-directed behavior of the actors is also influenced by objects that they can reach at the beginning of planning. This is the first source of variation that does not rely on appraisal.

A Successful Episode

The following is a free-form description of the beginning of a typical generated episode using a setup of all four agents.

1. Hero wants to give the flower to Victim, he does that and her preference for him rises.

2. Meanwhile, Mentor gets Block1.

3. Villain wanted to use the bomb to threaten Victim, but as he sees that Hero gave her the flower, he feels anger and therefore decides to steal the present.

4. Villain puts the bomb down again and goes towards Victim.

5. While Villain succeeds in stealing the flower, Hero hopes for the currently needed block from Mentor.

6. Villain now expresses joy about the success of his devious plan of retribution; all others can observe this and are overwhelmed by anger.

7. Victim still feels the need to cope with her fear ⁴ and flees from Villain.

8. Villain on the other hand, after his joy and pride are gone, intends to steal the block that was exchanged between Mentor and Hero.

9. Hero now feels fear at the prospect of Villain approaching while he himself holds a valuable object in his hands. He therefore copes with this fear by removing himself from the source of the emotion, i.e., he flees from Villain.

10. Villain fails to steal the block, feels sad about it, and decides to threaten the Hero with the bomb.

11. After his escape, Hero uses the block he got to start solving the blocks puzzle.

12. Shortly after Hero puts the block on the table, thus completing the first step of the blocks puzzle, Villain threatens him with the bomb.

13. Hero is frightened and runs away, but Mentor and Victim were nearby and saw what happened. Both try to defuse the bomb and Mentor succeeds.

14. The result is a stronger “bond of preference” between Hero, Victim, and Mentor.

In the following, Villain is forced to use other methods like deterring Hero with the sword, but most of the time his obstacles are overcome. Finally Hero solves the blocks puzzle and can give the treasure to Victim. In the end, Victim expresses love for Hero (and often for the helping Mentor too).

Variations

Assuming that Villain succeeded in stealing the block from Hero, he would then decide to focus on Victim and to bind her with the rope. This would lead Hero and Mentor to try to free Victim again, which is a very positive deed, that would increase the preference that Victim would have for them even faster, leading to an earlier feeling of love. In this case, the solving of the blocks puzzle would not be needed, as the main problem would be to free the bound Victim.

In both cases, the motivations of the actors are visible in the succession of their planning activities, and although they currently are expressed inadequately in ActAffAct, the knowledge could be used to provide the audience with the necessary elements for emotional participation. The end of this account of an episode is, of course, a little bit idealized. Because of the limited repertoire of actions that the actors can use, a lot of repetition is encountered until the episode comes to an “ending.” This can be solved by simply providing a larger set of actions to chose from, along with the tendency to use new methods if available.

More important problems arise through the current inability to combine simultaneous emotions. Several emotions are manifested at once and they are also expressed, but only the most important one will be coped with, without the knowledge about the other appraisals that happened at almost the same time.

This problem is one candidate for meta-level planning, but several other types of meta-information could also be included in the ActAffAct planner to enhance the believability of action sequences. Timing of plans and metering their success rate could be used to implement a simple learning scheme. This and a controllable bias of actors toward either exploration of new behaviors or exploitation of already known successful ones should yield substantially better plan selection.

Meta-level plans can also be used to replace the crude scheme of utilities, but more explicit meta-data for plans would be of great use. That would include relations between plans like “plan A facilitates plan B” or “plan A hinders plan B,” the same kind of information that can be used for appraisal. If the pre-appraisal step of interpretation leads to the assumption that another actor is executing plan A, one's own knowledge about the relations between plans can be used to determine whether that would be conducive or obstructive for one's own plans. This way to make the simple scheme of plans for helping or hindering others more explicit would also provide valuable information for normal task-directed planning.

Although the methods used in ActAffAct leave much to be desired, it is still possible to see that the use of the appraisal process is beneficial for the implementation of intelligent actors. In short action sequences, the causes for actions and reactions are understandable (i.e., in the terms of the simulated world), and though there were no explicit precautions to ensure a coherent long-term planning of the actors, the effects of appraisal and coping lead the actors to a sensible succession of goals and plans.

“Storyness”?

Let us recapitulate which aspects of the requirements for drama are dealt with in ActAffAct and what is needed on the road towards engaging interactive drama.

ActAffAct addresses the initial need for conflict to start the sequences toward resolving it. Only two actors with conflicting goals suffice to produce a wide range of emotional reactions and diverse reactions. Appraisal and coping provide the actors with causes for their actions, a substantial feature for a plot, and the long-term effects needed to achieve a sensible order in the sequence of actions.

Patterns of dramatic functions (Propp 1968) can be discerned as arising naturally, like, for example, the “violation” of a standard that is followed by the “punishment” of the wrong-doer. As another case to accomplish his tasks, Hero is assisted by Mentor who gives him the necessary objects, not only a correspondence with Propp's functions, but also with the role of the mentor in the monomyth (Vogler 1996).

Such interpretations of the simple actions that are possible in ActAffAct are probably overly ambitious, but nevertheless the bottom line of this project is a positive one. The first signs of the necessary components for believable drama are visible and the methods used are conducive to the efficient planning of emotionally justified actions.

CONCLUSION

This project was started to get closer to the goal of dramatically and emotionally believable software agents that can easily be reused in a role-playing scenario. Starting from notions in the theory of drama, it was assumed that characters and their emotions drive the plot of a story.

In the setting of a very limited simulated world, the actors choose actions in a sequence that can be seen as dramatically appropriate. They are driven mainly by activities that are targeted at coping with the appraisal of a recent perception of their surroundings, thus providing the connected causes or motivations that turn sequences of separate actions into a plot.

To simplify the task at hand, a lot of relevant issues were excluded from the project or dealt with only rudimentarily. One of them—although it is certainly one of the biggest issues on the road towards believable interactive drama—is the interaction itself, i.e., the interaction between one or more human users and the system. The area of different styles of interaction, from modifying the environment to modifying the agent's personality parameters, is left for further projects.

To our mind, using a simplified version of the appraisal process takes us closer to the goal of domain-independent reusable actors, but a lot of work remains to be done in order to achieve a principled way of specifying behavior plans for actors and actions in an environment, so that the necessary matching during appraisal can always be accomplished. Further work will be focused on common domain ontologies for the actors, including meta-data about the actions available to them and a format for relations between plans, and between plans and actions, that can be used for task-related planning as well as the process of appraisal.

Considering pedagogical applications, it is especially important to tackle the question of “directing” the characters in a simple, possibly domain-independent, but effective way, in order to strike a balance between autonomy and flexibility, on the one hand, and control of the desired outcome, on the other. By using the described psychologically inspired architecture for our characters, we hope to limit the need for directing to subtle modifications.

The Austrian Research Institute for Artificial Intelligence is supported by the Austrian Federal Ministry for Education, Science and Culture and by the Austrian Federal Ministry for Transport, Innovation and Technology. This research is carried out within the European Network of Excellence Humaine (IST-2002-2.3.1.6 507422) with support from the Austrian industrial research promotion fund (FFF 8088818/2970KA/SA). This publication reflects only the author's views. The European Union is not liable for any use that may be made of the information contained herein.

Notes

http://www.dramatica.com (last visited 20th Aug. 2004).

http://xml.apache.org/batik (last visited 20th Aug. 2004).

http://www.w3.org/TR/SVG/ (last visited 20th Aug. 2004).

Since the intensity of the appraisal is still higher than the utility of her task-directed behavior.