AUTHENTIC INTERACTIVE REENACTMENT OF CULTURAL HERITAGE WITH 3D VIRTUAL WORLDS AND ARTIFICIAL INTELLIGENCE

Abstract

We investigate an innovative design of information and communications technology for preservation and reenactment of cultures. Through an extensive literature review we identified the key components associated with a culture, among which are the environment, objects, knowledge, and institutions. Based on the acquired knowledge we developed a formal model of culture that builds a computational preservation and simulation of cultures along their multiple dimensions. We present the design of an immersive, intelligent, and interactive computational preservation—3I technology—based on tight fusion of 3D virtual worlds and artificial intelligence techniques. Markedly, virtual agents are central to the proposed model. They are both the knowledge carriers and the links between the environment, objects, and knowledge. For testing the model and supporting technological components we developed a research prototype that simulates the culture of the ancient City of Uruk 3000 B.C. (an essential representative of Sumerian culture in the cradle of civilization) within the virtual world environment of Second Life.

INTRODUCTION

The way general public learns about ancient cultures is shaped by the methods used by the subject matter experts to preserve the cultural knowledge. These methods rely on studying the results of archeological excavations and available written sources to produce text descriptions, drawings, and various mock-ups of artefacts and scenes associated with a given culture. Figure 1 illustrates the traditional culture preservation approach. It relies on the expertise of archeologists, historians, anthropologists, and art experts, to name a few. The primary focus of traditional methods of culture preservation is on understanding the significant events, significant technological advances, significant buildings, and customs. At this macro-level the cultural elements related to the lives and behaviors of ordinary culture carriers, including their rituals, movement styles, social norms, and etiquette, often remain out of sight. A recent attempt to address this drawback in nonfiction films that consider past cultures is based on reenactments that use re-creation of the environment and actors to portray culture carriers. The influence of producer's subjective bias, however, is inevitable.

FIGURE 1 Traditional approach to culture preservation.

The advance of technology offers an opportunity to advance our methods of preserving and learning ancient cultures. The use of video and/or 3D visualization for simulating cultural heritage is on the rise, but to understand how the existing practices can be improved we must first understand the nature of the concept with which we are working. Further, in the article we identify the elements that constitute a culture and analyze the techniques and methods that can be used for a more accurate and complete preservation of cultural heritage.

Understanding the Concept of Culture

There is a disturbing lack of agreement among researchers as to what constitutes a culture. Most existing definitions of culture come from anthropologists, but even between them there is no consistency. Stone axes and pottery bowls are culture to some (Kroeber and Kluckhohn, 1952), but no material object can be culture to others (White, 1959). Some researchers claim that the environment itself constitutes an important part of the culture and strictly defines how the culture evolves (Herskovits and Jean, 1955), whereas other researchers consider cultures being the knowledge transmitted by nongenetic means (Nishida, 1987) and neither objects nor the environment are related to the culture in their view. We do not take any side in this debate, but consider culture being tightly connected with both the environment and the material products created as the result of utilizing the cultural knowledge. Our assumption is based on the following observations:

	The majority of the existing culture preservation methods and learning about extinct cultures are structured around discovering, preserving, and learning from the objects produced by this culture.
	The environment also provides important clues that help to fill the gaps in the existing knowledge.
	In some cultures, (e.g., the culture of indigenous Australians), the environment is so tightly integrated with all human actions, beliefs, and traditions that ruling it out, as being irrelevant to the culture, makes it impossible to understand most of the respective cultural knowledge.

The aim of our work is to discover the elements associated with learning and preserving cultures and to produce an integrated framework, enabling culture preservation and learning, that incorporates all those elements. While studying the existing techniques for cultural preservation we identified printed materials as the most popular way of preserving cultural knowledge and museums as the way of preserving a culture in terms of objects. The most popular techniques that link a culture to a particular environment are movies and 3D virtual environments. Through further analysis of these techniques we selected 3D virtual environments (and their subclass, 3D virtual worlds) as the most affordable, dynamic, and interactive option for integrating the environment, objects, and knowledge associated with a culture.

Virtual Heritage and 3D Modeling

Using 3D virtual environments to reconstruct lost sites of high historical and cultural significance has become very popular during the last decade (Addison, 2000). The primary focus of the majority of such approaches is on reconstructing ancient buildings, objects, and even entire cities that are partially or completely destroyed at present. “Rome Reborn” (Guidi et al., 2006) recreates a historically accurate reconstruction of a large part of ancient Rome. The reconstruction includes virtual models of nearly 7,000 buildings and the terrain of the city in the time of Constantine the Great in A.D. 320 (Institute for Advanced Technology in the Humanities, 2007).

Visitors to virtual heritage sites, similar to “Rome Reborn,” are normally able to browse through digital models of the heritage objects and inspect them from different angles and proximity. They can also select a particular heritage object (i.e., Roman Colosseum) and closely explore its architectural details. Although such an approach allows general audiences to examine the architectural details of the heritage site and the corresponding objects, it still does not help an observer to understand how this site has been enacted in the past, limiting the exploration of the knowledge aspect of the given culture.

Virtual Heritage and Avatars

The virtual heritage concept has the potential to incorporate some of the knowledge aspects through the use of avatars. Populating a virtual heritage site with virtual agents that behave similar to the ancient citizens that used to occupy given heritage site could bring the heritage preservation to a new level. Through the use of artificial intelligence (A.I.) the virtual agents can absorb the relevant knowledge and become the knowledge carriers.

This potential, however, has not been properly developed. The majority of the work in this direction that has focused on simulating ancient people in virtual heritage uses the so-called virtual crowds (Gutierrez et al., 2007). Such crowds normally consist of a large number of avatars dressed as local citizens of the reconstructed site. The state of the art in combining crowd simulation and 3D heritage can be observed on the example outlined in Mam et al. (2007) where a virtual City of Pompeii is populated with a large number of avatars that walk around the city avoiding collisions. In this work the avatars are simply moving around and are not involved into historically authentic interactions. So their presence is not much different from the presence of moving objects, and they only extend the atmosphere of the culture simulation.

Few attempts that use rich individual models of virtual agents for capturing some of the cultural attributes are mostly focused on building culturally adaptive agents. Such agents try to adjust their personality to better fit with the culture of the user. Some recent works are concerned with developing agents that strongly adhere to the given culture. An approach to encode some of the global cultural dimensions into the agent behavior is presented in Aylett et al. (2009). Illustrating some attributes of a given culture through the use of rituals is outlined in Mascarenhas et al. (2009). Although being successful in highlighting a range of some cultural dimensions, the aforementioned works do not rely on a coherent formal model of culture and do not provide a general formal mechanism for culture preservation.

Virtual guides are another popular direction in deploying avatars in virtual heritage. These agents help users to navigate the virtual world that either preserves a given culture in a traditional form of a museum (Oberlander et al., 2008) or re-creates the actual site as a virtual environment (Palace Museum and IBM, 2009). The later case, the “Forbidden City” project, better uses virtual worlds technology in regards to the heritage preservation. The project aims at simulating the culture of ancient China and replicating the one- square-kilometer palace grounds called The Virtual Forbidden City. Similar to other virtual heritage solutions, a significant effort has been put into a realistic re-creation of the architecture of the city, whereas a much smaller effort has been spared on the development of virtual agents. The agents in the Forbidden City are supplied with very limited “intelligence.” Their actions are highly scripted, and their ability to interact with the users is limited to scripted monologues. The number of available agents is also quite low, and the majority of those act purely as guides rather than as virtual inhabitants of the city. As one of the most advanced virtual heritage applications available on the market, the forbidden city shows the current limits in using the virtual human factor. The most significant drawback, however, is that the agents are not behaving in a way that the relevant cultural knowledge is authentically presented to the human observer.

Our Contribution

Introducing virtual humans into cultural heritage applications, in our view, is an important step toward integrating all three key dimensions of a culture: knowledge, environment, and objects. Existing crowd simulation oriented approaches are not normally concerned with having virtual agents as immersed knowledge carriers but rather use them as moving decorations. Those works that do not use crowds normally explore a very limited set of cultural attributes, often rely on scripted behaviors, and are very limited in preserving the cultural knowledge that modern A.I. techniques can provide.

In this article we analyze the mechanisms required for capturing and preserving cultural knowledge through virtual agents. To do so we suggest focusing on individual agents rather than crowds and adapt a number of A.I. techniques that allow the preservation and simulation of a wide range of cultural attributes.

To avoid the debate about the role of environment and objects in a culture, we introduce the notion of virtual culture, which is a combination of cultural knowledge, environment, and objects preserved in a 3D virtual world. Within this framework the key contributions of our work are as follows:

	Specifying the role of virtual agents as an important element in the preservation of virtual cultures.
	Producing a formal model of virtual culture that facilitates culture preservation and learning of that culture by the visitors and provides the foundations for the computational enactment of a culture.
	Developing a computational framework that helps to preserve and simulate virtual cultures.
	Testing and demonstrating the developed model and the framework through a case study.

Structure

The remainder of the article is structured as follows. In Section 2 we identify the key elements that constitute a culture by using existing definitions and mathematical models. As the result of it, Section 3 proposes a formal model of virtual culture. Section 4 presents a technological solution for implementing a virtual culture based on the developed formal model. In Section 5 the resulting model is applied to the development of a prototype aiming at preserving the culture of the ancient city of Uruk, 3000 B.C. Finally, Section 6 presents concluding remarks and directions of future work.

BACKGROUND: DEFINITIONS AND EXISTING CULTURE MODELS

Replicating a culture inside a computer-simulated environment requires to have a formal model of a culture. The majority of research efforts focused on creating such artificial cultures originates in the field of artificial life (Upal, 2006). To our knowledge, none of the existing works provides a comprehensive formal model of a culture that can be used for preserving a culture along its multiple dimensions. Therefore, in this section we analyze the existing models and the available informal definitions of culture to understand which existing models to rely on and how they should be extended.

Definitions

Most existing conceptualizations consider culture being some sort of knowledge. One of the first and most popular definitions of culture still accepted by the majority of modern researchers was produced by Edward Burnett Tylor. He defines culture as “that complex whole which includes knowledge, belief, art, morals, law, custom, and any other capabilities and habits acquired by man as a member of society” (Tylor, 1871).

The definition proposed in Tylor (1871) treats culture as a list of elements, which motivates us to look through other available definitions and identify those elements that can be included into the resulting formal model. Through the analysis of the definitions linking culture to knowledge we identified the following elements that constitute such knowledge: beliefs (Tylor, 1871), morals (Tylor, 1871), law (Tylor, 1871), customs (Tylor, 1871), habits (Kroeber and Kluckhohn, 1952), techniques (Kroeber and Kluckhohn, 1952), values (Kroeber and Kluckhohn, 1952), ideas (Kroeber and Kluckhohn, 1952), standards (Herskovits and Jean, 1955), behavior patterns (Kroeber and Kluckhohn, 1952), and rules of behavior (Herskovits and Jean, 1955).

Culture is also believed to have a functional dimension, as suggested by White (1959). It is not only considered as knowledge, but also as an evolving mechanism of using this knowledge to better adapt to the environment and control it.

Although not making a direct connection to physical objects in his definition, in his works Tylor also connects culture to human possessions. Specifically, he enumerates beliefs, customs, objects—“hatchet, adze, chisel,” and so on—and techniques—“wood-chopping, fishing, fire-making,” and so on (Tylor, 1871). The view that the culture concept is associated with certain objects created by its carriers is also shared by Kroeber and Kluckhohn (1952). Herskovits and Jean (1955, p. 305) take an extreme materialistic view and consider culture being “the man-made part of the environment.”

In Tylor (1871) many of the attributes constituting a culture correspond to humans. In our approach we consider two clusters of knowledge elements: (1) those that are closely related to humans as culture carriers – (we emulate humans as the carriers of this knowledge) and (2) those that though distributed among the culture carriers have some kind of a unifying nature and can be preserved independently from their carriers.

The later elements are mentioned in the above definitions as rules of behavior, techniques, standards, patterns, and customs. We introduce the notion of institutions as the concept uniting all the above terms and find this notion to be central to culture preservation. Although strongly associated with knowledge, the institutions are rather a global type of knowledge with individual agents having little impact on changing this knowledge. In summary, we can generalize that a culture is associated with institutions accepted by a virtual society, significant objects and culture carriers that possess culture specific knowledge and behaviors.

Existing Models

Culture has been studied by social and computer scientists in the realm of the emergence of social consensus (e.g., Axelrod, 1997; Carley, 1991), of which culture is a particular case. In Axelrod (1997) it is observed that individuals can be characterized by their cultural features, such as language, religion, technology, style of dress, and so forth. Hence, the cultural traits of such features characterize each individual. Therefore, given a population of agents Ag, Axelrod characterizes each agent ag _i  ∈ Ag by a vector of cultural features , each one taking on a value to define an agent's cultural traits. Some of these traits change over time with the dissemination of culture, whereas other traits remain unchanged because an agent might be closed-minded or simply a given trait is not under its control (e.g., ethnicity). Similarly, Carley (1991) considers that culture is a distribution of facts among people, namely who knows what facts (e.g., a belief in God). Therefore, both Axelrod and Carley propose basic models to characterize cultural knowledge.

Despite the many definitions in the literature about culture, everyone agrees that people learn from each other. Hence, the dissemination of culture among people is based on the notion of social influence. Axelrod (1997) and Carley (1991) incorporate a well-known regularity in the social world: “homophily” (Cohen, 1977), or the tendency to interact with similar people. Thus, similarity among people's cultural features drives interactions. As a result of an interaction, two agents start sharing cultural features or knowledge that were different before their interaction. Therefore, existing models of dissemination of culture agree on the need for a local dissemination function that each agent uses for changing her cultural knowledge or features based on her social influences (interactions).

Recent studies (mostly in complex systems) on the emergence of social conventions agree on the importance of the structure (topology) of social relationships. Such network of interactions among agents accounts for the local geography in Axelrod's model (Axelrod, 1997). Indeed, Miguel et al. (2005) empirically show that the network of interactions is important to reach consensus on either a single, global culture or on multiple cultures. Hence, a model of culture must also take into account a model of social relationships.

Some new models propose to differentiate cultures through a set of global measurable attributes (i.e., individualism vs. collectivism, masculinity vs. femininity, degree of uncertainty avoidance, short-term vs. long-term orienation, etc.). (Aylett et al., 2009). An attempt to structure cultural knowledge along role-specific collective patterns of behavior (rituals) was proposed and modeled with virtual agents by Mascarenhas et al. (2009).

At this point we can compile the fundamental components identified in the literature to computationally model a culture. Thus, a culture can be characterized in terms of models of (1) cultural knowledge or features, (2) dissemination of culture, and (3) social relationships. We also identified the missing bit in these models. In Section 2.1 we highlighted the role of institutions. Hence, here we advocate that institutions shape social relationships with varying degrees of social influence. This is particularly true in ancient societies where family customs, social norms, dogms of religion, or tribal code represent a significant part of the culture.

FORMAL MODEL OF VIRTUAL CULTURE

Based on the analysis of the definitions and existing formal models of culture presented in Section 2, we produced a formalization of virtual culture. Our aim was to develop a model allowing for preserving a culture along as many of its attributes as possible. This aim is very different from the aims behind the models presented in Section 2.2, where the key motivation was to investigate a particular aspect of cultures (i.e., the dissemination of culture) and the simplicity of the model was rather a positive than a negative factor. In our work we rely on those existing models but extend them with additional elements, identified in Section 2.1.

In producing the formalization of culture we first rely on the fact that a culture is an aggregation of objects, environment, and knowledge. The cultural knowledge can be grouped into facts, culture-specific individualistic and group behaviors, and institutions. Researchers from the field of distributed A.I. have been working on formalizing behaviors and institutions for over a decade. Electronic institutions (Esteva, 2003) is a popular formalism that merges both and, in terms of formalizations of behaviours, closely matches the one proposed by Mascarenhas et al. (2009).

A virtual culture develops in a virtual environment, reflecting the actual physical environment where a culture is situated and enacted by virtual agents whose interactions occur in the framework of and are constrained by electronic institutions. To accurately capture the context, each electronic institution is associated with a certain space within the virtual environment (e.g., temples, markets). To accurately capture interactions, virtual agents use virtual objects in their interactions, namely virtual replicas of artefacts that mimic the actual physical objects (e.g., spears, pottery) being used by the members of modeled culture. Therefore, we can regard virtual places along with artefacts as the objects produced by a culture. Wrapping all the above elements, we can characterize a virtual culture as a tuple:

where E is the virtual environment, P stands for the set of virtual places occupied by a virtual culture, O stands for the objects produced by a virtual culture (buildings and artefacts), Ag stands for a set of virtual agents, I stands for a set of electronic institutions constraining the interactions of virtual agents, and l: I → P is a function mapping each electronic institution to its context (location), namely to some virtual place.

An electronic institution is treated as a composition of roles, their properties and relationships, norms of behavior in respect to these roles, a common language (ontology) used by virtual agents for communications with each other (e.g., this ontology must allow virtual agents to refer to places and artefacts), acceptable interaction protocols representing the activities in an institution along with their relationships, and a role flow policy establishing how virtual agents can change their roles. As mentioned in Section 2.2, we take the stance that social relationships in the realm of an institution establish social influences of varying degrees that must be considered for culture dissemination. So, an Institution ∈I in our model is characterized as

where R is a set of roles, ssd ⊆ R × R and sub ⊆ R × R stand for relationships among roles (incompatibility of roles and subsumption of roles, respectively), N is a set of norms of behavior, Ont is a common language (ontology), PS stands for a graph defining the relationships among interaction protocols and role flow of the agents, and ℐ ⊆ R × R stands for a set of directed arcs between roles, where w: ℐ → ℛ⁺ labels each arc with a degree of social influence. For a detailed explanation of electronic institutions, see Esteva (2003).

From the agent perspective, we take the stance that virtual agents are culturally characterized by their appearance (e.g., dress, facial features, etc.) and their cultural knowledge, namely their beliefs. We also assume that virtual agents are endowed with patterns of behavior, namely plans of actions, that allow them to act in different institutions. Based on its beliefs, a virtual agent selects a pattern of behavior to perform in an institution. Moreover, the definition of culture presented in Nishida (1987) suggests that a culture is not transmitted by genetic means but mostly via social learning mechanisms. Hence, a virtual agent requires a social learning function modeling the dissemination of culture, namely the way a virtual agent's knowledge changes after interacting with some other agent in the framework of an institution depending on the role each agent plays (because a role determines the degree of social influence). Following the above considerations, we can characterize the components of a Virtual Agent ∈Ag as a tuple:

where Ap is the appearance of a virtual agent, K is the agent's knowledge, B is a set of patterns of behavior the agent can perform, π: K × I → B is a behavior selection function that allows a virtual agent to choose a behavior, a plan of actions, and δ: K × R × I × Ag × R → K is a social learning function.

The social learning function transforms the knowledge and patterns of behavior of an agent. This occurs after interacting with another agent in the context of an electronic institution. By taking this perspective, the social learning function can be regarded as the local dissemination function (similar to models of dissemination of culture we referred to in Section 2.2) that each agent uses for changing its cultural knowledge or features based on its social influences (interactions). However, we go one step beyond by considering that the social learning function must also take into account the institution where agents' interactions take place as well as the roles played by the agents during their interactions.

3I TECHNOLOGY

The formal model of a virtual culture forms the basis for preservation of a given culture along all the identified dimensions. Next, we show how this formal model can be practically converted into a technology, based on two families of techniques: virtual worlds and A.I.

Approach

We use the virtual worlds technology for re-creating significant heritage objects from the results of archeological excavations and available written sources. In a virtual world all participants are embodied as avatars and can freely move within the virtual world, interact with other participants, and change the virtual world itself (very often in a dynamic manner using in-world building facilities). Therefore, we also extensively rely on the direct participant involvement in the process of re-creating the heritage site as well as in populating it with virtual humans. Our approach to preserving and simulating cultures is shown in Figure 2.

FIGURE 2 Our approach to preserving and simulating cultures.

The 3D virtual world is used for both preserving the culture and for teaching the resulting culture to the visitors. It is accessible by two types of participants: “visitors” and “experts.” Visitors participate in the environment with the aim to learn about the given culture through immersing in the virtual world and interacting with its virtual inhabitants. Experts are a key element in culture preservation. Through embodied interactions with other experts in the virtual world, they share their knowledge and refine the appearance of the heritage environment, validate the correctness of the reconstructed buildings and artefacts, and help to refine the behavior of virtual agents. As the result of the joint work of historians, archeologists, designers, and programmers, the resulting heritage site is re-created in the virtual world and populated with virtual agents that look and behave similar to the actual people that used to live in the given area. Through the virtual institutions technology (Bogdanovych, 2007) the agents can engage into complex interactions with other agents and humans while adhering to the social norms of the reconstructed culture.

Virtual Worlds Component

Virtual worlds technology is used to model the environment, places, and objects in the formal model of virtual culture. In our particular case we use the virtual world of Second Life. ¹ This virtual world offers on-site collaboration using the same environment for both object building and navigating the environment. The process of modeling the virtual heritage site according to our approach consists of the following phases:

1.	The site is mapped by archaeologists using the existing site plans.
2.	A virtual three-dimensional model of the buildings is constructed using the given measurements.
3.	Each building model is positioned in the virtual world according to the plan of the site.

Figure 3 outlines these phases on the example of constructing the Ziggurat in the Virtual City of Uruk. The comprehensive model of the resulting city is presented in Figure 7.

FIGURE 3 Three phases of building the virtual environment.

Artificial Intelligence Components

The use of A.I. caters for the Agents and Institutions in our model. The agents act as the mechanism of preserving the cultural knowledge and then simulating the required elements of the culture to the visitors on demand. The Institution maps to the concept of virtual institutions (Bogdanovych, 2007) and helps to formalize the virtual environment and take into account social norms, role hierarchy, interaction protocols, and role flow policy for both human-controlled avatars and virtual agents.

Virtual Institutions

Virtual institutions are a new class of normative virtual worlds that combine the strengths of 3D distributed environments and normative multiagent systems, in particular, electronic institutions (Esteva, 2003). In this “symbiosis” the 3D virtual world component spans the space for visual and audio presence, and the electronic institution component takes care for establishing formal rules of interactions among participants.

The virtual institutions technology (Bogdanovych, 2007) provides tools for formal specification of institutional rules, verification of their correctness, mapping those to a given virtual world, and enforcing the institutional rules on all participants (both humans and autonomous agents) at deployment.

The specification is expressed through three types of conventions and their corresponding dimensions: Conventions on language–Dialogical Framework determines language ontology and illocutionary particles that agents should use, roles they can play, and the relationships among the roles; Conventions on activities–Performative Structure establishes the different interaction protocols (scenes) the agents can engage in, and the role flow policy among them; Conventions on behaviour–Norms captures the consequences of agents' actions within the institution, such consequences are modeled as commitments (obligations) that agents acquire as consequence of some performed actions and that they must fulfill later on.

At deployment, the specification is connected to the virtual world and the state of the virtual world is mapped onto the state of the institution (normative platform). In our implementation we are assuming that every avatar is controlled by either a human participant or an autonomous agent, where human-controlled avatars represent visitors and experts and the agent-controlled avatars are virtual humans enacting a particular role in a given culture. Having the institutional formalization aligned with the virtual world enables the decision making of virtual agents controlling the avatars. Each agent has access to the institutional formalization and can sense the change of the institutional state and reason about the actions (both own actions and actions performed by other participants) that resulted in the state change. In our implementation the agents use this information in their decision making. Hence, for an agent it makes no difference if it interacts with an agent or a human-controlled avatar as the result of the other party's actions can be sensed through the normative platform.

The technological solution that enables this is based on the fact that every avatar is assigned with a certain institutional role. Each role imposes a number of restrictions on the enacting agent on entering certain parts of the virtual environment and performing certain actions there. The virtual institution establishes the mapping between different areas within a virtual world and the scenes in the normative platform as well as the mapping of actions of the virtual vorld to the messages requesting the state change in the normative platform. Every agent request for performing a certain action is first verified against the institutional rules and only then passed for execution if the institutional permission is granted; otherwise, the action is blocked and the action performing agent is notified about the grounds for blocking it.

Figure 4 illustrates the deployment architecture and presents an example of how the institution controls the admission of participants to certain activities (rooms). An event is generated as the result of a human participant positioning the mouse pointer over the door and clicking the left mouse button (requesting the avatar in the virtual world to open a door) or an agent initiating the “Touch” request. Each event that requires institutional verification has an associated script and the name of this script is stored in the Action/Message table. The Communication Layer of the deployment architecture consults with this table to find the institutional message that represents this event in the Normative Control Layer (built on top of AMELI; Esteva et al., 2004). In case such a message is accepted by AMELI, the response message is sent back and the Communication Layer again consults the Action/Message table to transform this response into the name of the action that has to be executed in the virtual world. Next, the action is performed by executing the corresponding script. In the given example this action results in opening the door and moving the avatar through it. Further details of the institutional part are presented in an example in Section 5.6; an extensive coverage of the virtual institutions technology can be found in Bogdanovych (2007).

FIGURE 4 Runtime architecture and norm enforcement in virtual institutions.

Agents in the Virtual World of Second Life

The availability of agents in Second Life is established through the libopenmetaverse library (openmetaverse.org). This library represents a reverse-engineering attempt to implement the client–server protocol of Second Life. One of the problems our agents face is that libopenmetaverse is based on the UDP protocol, which is fast but is subject to packet loss. Another problem facing the application of many standard A.I. techniques, like path planning, is asynchronous communication with the server.

The use of agents in classical A.I. solutions is quite different to the use of agents in virtual worlds like Second Life. The main difference lies in the way the agents are integrated into the virtual environment. In the case of Second Life the agents reside in a separate platform from the Second Life server. The actions of other participants can only be sensed on the server and the own actions of the agent can only be performed through the server. Thus, the agent must continuously exchange network packets with the server. This creates a range of problems caused by the network latency.

To address all the aforementioned problems we developed synchronization and message delivery verification modules into our agents, so that no packets are lost and synchronous communication is possible.

Interacting with the Environment

There are two parts of the environment the agents are involved in: the visual part rendered by Second Life and the normative part supported by virtual institutions technology. The agents interact with the virtual world by sending commands to the Second Life server through libopenmetaverse. Through these commands they are capable of moving around the virtual world, perform certain animated behaviors, perform commands on the objects in the environment, and interact with other agents or humans. The normative part enforces the social norms on all the Second Life participants. Agents interact with this part by sending and receiving illocutions (text messages) to the AMELI system.

Agents can interact directly with both parts of the environment, whereas the humans can interact only with the visual part. The normative part mediates their actions there and blocks the actions that are inconsistent with the social norms expressed in the normative part.

In the visual part the agents are capable of sensing the changes of the environment state, including the movement of the objects or avatars, new objects or avatars, actions performed by all participants, and so on. In the normative part the agents can send and receive the institutional illocutions and sense the state changes resulted by these illocutions. The state of the visual part of the environment represents parameters like the time of the day, positions and transformations of the objects, and agents in the virtual world. The institutional state corresponds to the set of currently active scenes and the state of each scene. Figure 5 outlines the elements of the virtual world and the normative platform that can be sensed by the agents.

FIGURE 5 Agent perception of the environment.

Agent Architecture

Our model presented in Section 3 identifies some functionalities a virtual agent must implement to successfully operate within a virtual culture. This agent model is tightly integrated within the virtual institutions technology. The agent must be capable of sensing the environment, reasoning about the acquired data, and acting on the environment. Our agent architecture is outlined in Figure 6. It is based on the BDI (belief desire intention) model and functions similarly to the jack intelligent agents platform (Howden et al., 2001).

FIGURE 6 Agent architecture.

The use of libopenmetaverse imposes a requirement to use C# as the programming language for the agents. Existing BDI agent solutions in C# do not satisfy our requirements: hence, we developed our own agent library (VIAgents) for developing agents that follow our formal model. It provides them with communication facilities, planning, and goal oriented behavior. Each agent has a number of beliefs that map to a state of the virtual world and the institutional state, a number of goals the agent wants to achieve, and the number of plans the agent can perform to achieve these goals. As the result of sensing the changes in the environment or events received from the institutional infrastructure or other agents, the agent may establish new goals or drop the existing goals and will start or stop corresponding plans accordingly. The BDI architecture is also extended with a learning mechanism that enables the agent to learn by example from a human expert. This part of the agent architecture is outlined in Bogdanovych et al. (2008).

Communicating with Humans

The agents can chat with human visitors on topics, related to the heritage environment, through the chat mechanism provided by Second Life. To participate in a conversation an agent uses the ALICE chat engine based on the AIML language (Richards, 2003). Each agent uses a number of AIML files that represent what can be seen as a common sense database. Additionally, to this database every agent is supplied by personalized AIML files that reflect on its personality and the data relevant for its role within the virtual society. Each agent can extend its knowledge (update the corresponding AIML files) through interactions with subject matter experts.

Actions

The actions of the agents include (1) chat messages they can exchange with human users, (2) institutional illocutions that they send to the institutional infrastructure or to other agents with the goal to change the institutional state, and (3) actions performed directly on the virtual environment that are not related to the change of the institutional state, including locomotion, playing an animation, or performing operations with objects. Both illocutions and virtual world actions are essentially represented in a text form, where every action performed upon the virtual world is a request packet being sent to the Second Life server.

Gestures and animations are the most frequent actions performed by agents in the virtual world. There is a standard set of animations provided by Second Life, but custom gestures are often required too (i.e., pulling a jar with water from the well, lighting a fire, or spear-fishing). In such cases we use a full-body MVN XSENS motion capture suit that enables capturing a particular movement and importing it into Second Life.

Locomotion

The locomotion of the agents is based on artificial potential fields (Ge and Cui, 2002). Each agent applies artificial emitters of repelling forces to all obstacles in the environment and uses these forces to avoid collisions while moving between different positions in the environment. Although this is a fairly standard technique in gaming environments, the implementation of potential fields based locomotion in Second Life faces the problems caused by network latency, packet loss, and asynchronous nature of locomotion. We developed a Second Life specific locomotion library that introduces synchronized locomotion, deals with packet loss, and tries to minimize the effect of network latency through movement interpolation.

Object Use

To convincingly simulate the daily life of the ancient people, it was important to enable agents to use objects in the environment (i.e., take a spear, jump on a boat, and go fishing). Object use is one of the aspects of virtual heritage that hasn't received appropriate attention. In contrast to the majority of other virtual heritage systems, we implemented a fully fledged solution to object use for the heritage context. We developed a designated library that allows agents to identify an object in the virtual world, attach it to the default attachment point, play a certain animation (i.e., rowing) associated with a given object, wear an object that is a piece of clothing, detach the piece of clothing, drop an object to the ground, and detach the object and hide it in the avatar's inventory.

Gaze

For the sake of believability our agents are supplied with a programming solution dealing with idle gaze behavior. When the agents are moving around their gaze is not fixed but is changing using the attention based model similar to the model presented in Cafaro et al. (2009). The agent would shift its gaze between objects and avatars depending on the level of its interest in those. It predominantly follows the movements of avatars approaching the agent at close proximity.

CASE STUDY: THE CITY OF URUK, 3000 B.C

To test the validity of our formal model and the feasibility of the presented approach, we implemented a prototype of a virtual culture and conducted a case study. The case study aims at re-creating the ancient city of Uruk from the period around 3000 B.C. in the virtual world of Second Life and letting history students experience how it looked like and how its citizens behaved in the past (more about the Uruk Project as well as the prototype video can be found in Uruk Project, 2009). The resulting virtual world provides a unique collaborative environment for history experts, archeologists, anthropologists, designers, and programmers to meet, share their knowledge, and work together on making the city and the behavior of its virtual population historically authentic.

Uruk was an ancient city located in present day Iraq. Many historians and archeologists believe that Uruk was one of the first human built cities on Earth. Uruk played a major role in the invention of writing, emergence of urban life, and development of many scientific disciplines including mathematics and astronomy.

Prototype

The prototype aims at enhancing the educational process of history students by immersing them into daily life of the ancient city of Uruk, so that they gain a quick understanding of the advance of technological and cultural development of ancient Sumerians. The prototype was built following the formal model presented in Section 3 and based on the technological solution from Section 4. Next we outline the details of each major component in the formalization of the virtual culture and how it was supported by our prototype.

Uruk Environment

The knowledge about the Uruk environment mostly comes from various written sources. Such sources are normally based on deciphered clay tablets produced in ancient Uruk. It is known that Uruk was located on the bank of a large river (Euphrates) in a flat desert-like area with very little vegetation. The rainfall was extremely low at around 150 mm per annum. Rain was confined to the winter months, and summer heat was very intense. Outside of the Uruk city the area is known to be semidesert, unirrigated, where sheep and goats could be grazed and scrub and thorn bushes were collected for fuel. There were not many natural resources of stone, timber, or metal around the area. The citizens of Uruk had many domestic animals, mostly donkeys and sheep. Eagles are known to be one of the most popular birds. Fish were plentiful in the Euphrates river and were one of the key sources of food in Uruk. Due to the lack of other resources, reed and clay were the most popular building materials (Crawford, 1991).

Based on the above data we re-created the environment of the city of Uruk in the virtual world of Second Life. The virtual environment features a large flat area textured as a desert. It contains very few trees and features the aforementioned animals. The river with a large variety of fish in it was also created. The appearance and behavior of the citizens of virtual Uruk also reflect on the climate of the environment and harsh weather conditions.

Places in Uruk

It is known that Uruk was a very large city surrounded by walls. It was separated into a number of districts and housed about 50,000 people. We didn't have the capacity to re-create the entire city, so we decided to select the key buildings and key areas only. So, our virtual Uruk is an approximation of the actual city and includes the following scenes: temple, ziggurat, school, market, fishing place, and a number of residential houses. At the current stage of the prototype only the following areas are used by the virtual agents: fishing, well, two residential houses, and fireplace.

Uruk Objects

Based on the results of archeological excavations and the available written sources, we re-created the buildings and artefacts that were available in Uruk. Both modeling of the city and programming of the virtual humans populating it were conducted under the supervision of respective domain experts. The object designers used input from our domain experts, who provided them with sketches, measurements, and positions of the objects. Many of the artefacts were replicated from the artefacts available in museums.

The key buildings include the ziggurat of Uruk, Eanna temple, a number of private residences, school, local market, and the city well. Similar to the actual city, virtual Uruk lies on the bank of a river and is surrounded by a large wall. Figure 7 shows virtual Uruk re-created in the Virtual World of Second Life.

FIGURE 7 City of Uruk, populated with buildings and objects in the virtual world of Second Life.

Uruk Agents

The agents reenacting the life of the ancient Sumerians in our prototype are developed using the architecture outlined in Figure 6. For the purpose of this study we selected fishermen daily life of ancient Uruk to illustrate our model of virtual culture. We created four agents that represent members of two fishermen families. Each family consists of a husband and a wife. Every agent has a unique historically authentic appearance and is dressed appropriately for the period around 3000 B.C.

The agents literally “live” in the virtual world of Second Life. Their day is approximately 15 minutes long and starts with waking up on the roof of the house (where they slept to avoid high temperatures). The wives would wake up first to collect water from the well and prepare breakfast for their husbands. The husbands normally start their day by having a morning chat while waiting for the breakfast to be prepared (eating and cooking are not currently implemented). After breakfast the fishermen would collect their fishing gear and walk toward the city gates, as shown in Figure 8a. Outside the gates on the river bank they would find their boat, which they both boarded and started fishing. One of the agents would be standing in the boat with a spear trying to catch the fish and the other agent would be rowing. Figure 8b illustrates the fishing process. After fishing the men exit the boat, collect the fishing basket and spear, and bring them back to their homes. This daily cycle is then continuously repeated with slight variations in agent behavior.

FIGURE 8 Virtual agents in the City of Uruk.

Each agent enacts one of four social roles. Agent Fisherman1 plays the “SpearOwner” role. He is the young male fisherman possessing the fishing spear and is capable of catching the fish with it. He and his brother jointly own a fishing boat. His daily routine consists of waking up on the roof, having a morning chat with the agent Fisherman2, fishing, bringing the fishing gear back home, and climbing back on the roof to sleep.

Wife1 Andel enacts the “WaterSupplier” role. She is the young wife of Fisherman1, who is responsible for collecting water from the well. Her daily routine consists of waking up on the roof, collecting the water from the well, doing housework, and climbing back on the roof to sleep there. As any other typical fisherman wife in Uruk she does not have any recreation time and is constantly working. The agents Wife1 Andel and Fisherman1 Andel are outlined in Figure 9.

FIGURE 9 Fisherman Family 1: Fisherman1 Andel and Wife1 Andel.

Agent Fisherman2 is the older brother of Fisherman1 playing the social role “BoatOwner.” He lives with his wife in a separate house next to his brother. Both families are very close and spend most of their day together. Fisherman2 possesses a fishing basket and paddles for rowing the fishing boat. His daily routine consists of waking up on the roof, having a morning chat with Fisherman1, fishing, bringing the fishing gear back home, and climbing on the roof to sleep.

Wife2 Jigsaw, the wife of Fisherman2, plays the “FireKeeper” role. Starting the fire and preparing food are her direct responsibilities. Her daily routine consists of waking up on the roof, starting the fire, doing housework, and climbing back on the roof to sleep. The second fisherman family is shown in Figure 10.

FIGURE 10 Fisherman Family 2: Wife2 Jigsaw and Fisherman2 Jigsaw.

Uruk Institution

In Section 4.3 we described how an institution regulates the interactions of all participants in a virtual world and helps the virtual agents to simplify their environment and interpret the consequences of the actions performed in it by human-controlled avatars. The extended description of the process and the methodology used for formalizing the Uruk institution are presented in Bogdanovych et al. (2009). For the purpose of this article we only focus on the key components present in the resulting Uruk institution.

Figure 11 outlines the Performative Structure, the roles of participants, and gives an example of a Norm and an interaction protocol (Scene). The Performative Structure shown in Figure 11a is a graph defining the role flow of participants among various activities. The nodes of this graph feature the identified scenes and the arcs define the permission of participants playing a given role to access certain scenes. Arcs labeled with “new” define which participants are initializing the scene, so that no other participants can enter it before the initialization occurs.

FIGURE 11 Some components of the Uruk institution.

The “root” and “exit” scenes are not associated with any patterns of behavior and simply define the state of entrance and exit of participants into the institution. Apart from them each of the scenes in the Performative Structure is associated with a Finite State Machine defining the interaction protocol for the participants that are accepted into the scene. To change the scene state a participant has to perform an action accepted by the institutional infrastructure.

Figure 11c defines the role hierarchy and indicates that the institution can be accessed by the agents playing the following four roles: SpearOwner, BoatOwner, WaterSupplier, and FireKeeper. Here SpearOwner and BoatOwner are two subroles of the role Fisherman and WaterSupplier and FireKeeper are the subroles of role wife. The Performative Structure also includes the following roles: Fire, Boat, House1, House2, Well. These roles correspond to dynamic objects that change the state of the environment by performing some actions in it (such objects are treated as agents). The interaction of the agents with such objects must be formalized appropriately in the specification of the institution to ensure correct behavior.

Figure 11b illustrates how a scene protocol is formalized. The scene protocol here defines in which sequence agents must perform the actions, at which point they can join and leave the scene and what they should do to change the scene state. In virtual institutions we consider every action that changes the state of the institution being a speech act (text message). Every action (i.e., grabbing an object or clicking on it) a participant performs in a virtual world is captured by the institutional infrastructure.

As an example, Figure 11b outlines the institutional formalization of the Fishing Scene (associated with the area around the boat). Once a scene is initialized its initial state becomes “W0.” While the scene is in this state Fisherman2 and Boat can join the scene and both Fisherman1 and Fisherman2 can leave the scene (the “Boat” is part of the scene when it is activated). Fisherman1 can only enter the scene after Fisherman2 successfully enters the boat. This occurs when the avatar of Fisherman2 boards the boat by performing the action “f2:enterboat” (labelling the transition from “W0” to “W1”), making the scene evolve from state “W0” to state “W1” (BoatOwner on board). After Fisherman1 enters the boat, by performing the “f1:enterBoat” action, the institutional infrastructure makes the scene evolve to state “W2” (SpearOnwer on board) and notifies all participants about the state change. Then Fisherman1 may request to start the fishing, by performing action “f1:startFishing,” which would bring the scene into “W3” (Fishing). The result of this is the change of the boat state from “standing” to “afloat,” Fisherman2 will start rowing and the boat object will move. In state “W3” the only action that can be performed is informing all the participants by Fisherman1 that fishing is finished. When the fishing is finished Fisherman2 must return the boat to the initial position, park it there, drop the paddles, take the fishing basket, and exit the boat. Fisherman1 will also have to exit the boat. No participants can leave the scene in this state and must wait until the scene evolves to “W0.” While the scene is in “W0” again, the Boat object will change its state to “docks,” this being captured as a “finish” action by the institutional infrastructure that makes the scene evolve to its final state “W4.” This deactivates the scene and makes it impossible for the participants to join it and act on it (no participant will be able to sit inside the boat).

Similar to the Fishing scene the interaction protocols have to be specified for other scenes present in the Performative Structure. We would like to point out that the scene protocol does not define how the actual fishing should take place but simply provides the key states within the scene so that the agents can have a formal understanding of the performed actions.

The final significant component of every institutional specification is the formalization of “norms” and “obligations.” In the context of virtual institutions the term “norms” refers to the interscene obligations acquired by the agents as the result of performing some actions in the institution. Figure 11d shows an example of a norm that instructs a fisherman agent to bring all his catch home and pass it to his wife once the “stopFishing” illocution is sensed within the Fishing scene.

Validation

Our approach to modeling cultures is based on the 3D virtual worlds technology. The importance of this technology for transmitting knowledge was highlighted by the outcomes of the research summit on the role of computer games in the future of education (Summit on Educational Games, 2006). The outcomes suggest that 3D virtual worlds is an important technology for teaching higher-order thinking skills such as strategic thinking, interpretative analysis, problem solving, plan formulation and execution, and adaptation to rapid change. Based on the opinions of over 100 experts in education the summit concludes that virtual experience is beneficial for learning as it helps the students to maintain a high level of motivation and goal orientation (even after failure), enables personalized learning, and, under certain conditions, is associated with unlimited patience. The key identified benefits of using virtual worlds for learning are personalisation, active learning, experiential learning, learner-centered learning, and immediate feedback (Summit on Educational Games, 2006).

To test the validity of our particular approach to modeling the selected culture in a virtual world, we conducted additional validation from two different perspectives: expert validation and learners feedback. To verify our simulation of the culture of the city of Uruk, 3000 B.C. we collaborated with two domain experts. The experts helped us in revising the scenarios and 3D models of the objects. Once the prototype was completed, the experts confirmed that the created prototype indeed reflects the way of life of ancient Sumerians from the city of Uruk and confirmed its historical authenticity.

To conduct the learners feedback validation, we selected 10 people (students and staff members) from two Australian universities. The key selection criteria for our sample was that those people possess minimum knowledge about ancient Mesopotamia and the city of Uruk. To ensure this, before conducting the study all candidate participants were tested about their previous knowledge in this respect. We aimed at analyzing the impact of our simulation on people from different genders and different age groups. To ensure this, another level of candidate screening was associated with their age and gender. As a result, we selected 10 people from the set of people with lowest history text results (5 men and 5 women) with their age evenly distributed between 23 and 63 years old.

During the study each test subject was asked to sit in front of the computer screen and was given a very brief introduction. The introduction mentioned that what is shown on the screen is the 3D reconstruction of the city of Uruk in 3000 B.C. After this the participant was given instructions on how to navigate in the virtual world and was asked to interact with each of the four virtual agents present in our prototype. The interviewee was giving commands as to which direction to go and which avatar to approach. Once the participant successfully observed the key activities in the life cycle of the selected agent he or she was asked to switch to another avatar. For the purpose of this study we kept the duration of this experience under 20 minutes for each participant.

At the end of this experiment the virtual world browser was closed and the participant was interviewed about the experience in Uruk. The aim of these interviews was to verify whether the users of our simulation are able to learn about the culture of ancient Mesopotamia along all the dimensions we identified. Each interview consisted of 18 questions. The first 16 questions aimed to test what was learned by the participant about the Uruk culture along each of the four dimensions covered by our model. For example, some of the questions focused on the environment, asking the test subject to describe the climate, vegetation, and weather. Other questions targeted the institutional structure, that is, social relationship between the observed virtual agents as well as information about their social roles and interaction protocols. Another two groups of questions focused on agent behavior and on objects in Uruk city. Finally, the last two questions aimed at evaluating the overall experience, asking the test subjects to summarize what they learned about the culture, identify any of their concerns, and list the key highlights of the virtual experience.

The results of the study confirm that participants were able to acquire new knowledge about the Uruk culture along every dimension we identified. None of the participants gave 100% correct answers, but all of them provided at least 70% of correct information. The incorrect responses were not biased along any of the dimensions and seemed to be highly individual. Some of the wrong answers had clear correlation with the lack of skills in controlling the interface.

CONCLUSION AND FUTURE WORK

Through extensive analysis of the literature and existing computational models of culture we produced a formal model that is suitable for preserving a variety of cultural attributes and for simulating a given culture to the public. Our model is based on the virtual institutions technology (Bogdanovych, 2007) with virtual agents being the carriers of the cultural knowledge. The 3D virtual world provides a necessary environment for visualizing cultures. The resulting model was used for creating the research prototype of the city of Uruk 3000 B.C. The prototype simulates the culture of two fishermen families in ancient Mesopotamia. The validation of the prototype ensures the feasibility of the selected approach and suggests that it is possible to preserve cultural knowledge along all identified dimensions.

The scalability of the implementation of the culture reenactment model depends on (1) the bandwidth of agent communication and (2) the efficiency of the rendering algorithms. The maintenance of the model include (1) updating the institutional structure and norms, which requires recompilation of the institutional part, and updating of the virtual representations, including spaces, avatars, and artifacts. The validity of these updates is checked following the expert validation procedure described in Section 5.7.

Future work includes extending the scenarios with more agents, improving the agent architecture, introducing more variety in agent behavior, and further formalization of the Uruk institution. In the future we will support the dissemination of culture, namely how culture spreads and evolves. We will also work on gathering more scientific evidence in favor of our approach to modeling cultures by comparing it with conventional approaches. In particular, we will investigate whether the users of our simulation are able to learn more about a particular culture than those accessing the same information through printed materials.

Notes

http://www.secondlife.com