Potential role of monkey inferior parietal neurons coding action semantic equivalences as precursors of parts of speech

Abstract

The anterior portion of the inferior parietal cortex possesses comprehensive representations of actions embedded in behavioural contexts. Mirror neurons, which respond to both self-executed and observed actions, exist in this brain region in addition to those originally found in the premotor cortex. We found that parietal mirror neurons responded differentially to identical actions embedded in different contexts. Another type of parietal mirror neuron represents an inverse and complementary property of responding equally to dissimilar actions made by itself and others for an identical purpose. Here, we propose a hypothesis that these sets of inferior parietal neurons constitute a neural basis for encoding the semantic equivalence of various actions across different agents and contexts. The neurons have mirror neuron properties, and they encoded generalization of agents, differentiation of outcomes, and categorization of actions that led to common functions. By integrating the activities of these mirror neurons with various codings, we further suggest that in the ancestral primates’ brains, these various representations of meaningful action led to the gradual establishment of equivalence relations among the different types of actions, by sharing common action semantics. Such differential codings of the components of actions might represent precursors to the parts of protolanguage, such as gestural communication, which are shared among various members of a society. Finally, we suggest that the inferior parietal cortex serves as an interface between this action semantics system and other higher semantic systems, through common structures of action representation that mimic language syntax.

Keywords:

• Area 7b
• Mirror neurons
• Action recognition
• Action semantics
• Functional equivalence
• Equivalence relations

INTRODUCTION

In human society, knowledge is usually expanded and shared through linguistic communications among society members, and mutual observation of various acts also contributes to such knowledge distribution. Non-human animals, without linguistic faculty, understand the goal-directed actions, to share common understandings among conspecifics about the situations that the actions evoked.

The mirror neuron system has been thought to subserve this faculty (Iacoboni, 2005; Iacoboni et al., 2005; Oztop, Kawato, & Arbib, 2006; Rizzolatti & Craighero, 2004). Mirror neurons are neural substrates found originally in the monkey premotor cortex (Rizzolatti & Craighero, 2004) and later in the intraparietal lobule (IPL) (Fogassi et al., 2005; Tanaka, Yokochi, & Iriki, 2004), which activated equally during executing and observing corresponding goal-directed actions. Thus, monkeys could sense the actions of others, at least at the cellular level.

Understanding of other's action through mere observation would be quite beneficial for animals in terms of learning novel but effective behavior emitted by that individual. However, non-human animals in wild habitats rarely imitate the actions of others (Tomasello & Call, 1997), although they may notice that they are being imitated (Paukner, Anderson, Borelli, Visalberghi, & Ferrari, 2005). Although many reports have claimed imitation in non-human animals, many of the observations could be interpreted as “local enhancement”, “stimulus enhancement”, or some other mechanism (Zentall, 2006). That is, there is an obvious discrepancy in behaviour: The animals can recognize similarity of actions performed by themselves and others, both behaviorally and neurally, but they do not copy the actions of others with their own body.

This discrepancy may be partly understood by assuming that their actions are embedded, as a whole, into the environment that is strongly connected to biologically-important events such as food and conspecific animals. Indeed, through the mirror neuron activities, the goal of action (e.g., food) that is directly linked to its cause (view of that food) in the environment could be inferred regardless of the agents that execute the action, but precise structures and the form of the action per se are not able to be extracted through neuronal discharge patterns. This lack of analysis would prevent them from analysing actions into elements, and from further structuralizing the components into a series of actions leading to a goal. Lack of precise structure in the mirrored representation of actions in monkeys would prevent them from imitation (Iriki, 2006). As we will discuss, most of the mirror neuron activity found in monkey F5 were strongly connected to the goal of action.

In contrast, human linguistic expressions of actions possess explicit components of actions—namely, agent, form, and consequence of actions, which in turn are represented as parts of a sentence, such as subject (S), verb (V), and object (O), integrated by syntactical structures. Thanks to these structures, the listener or observer can understand the actions and their embedded context without actually participating in the situation where the actions occurred. Because the imitation requires not only attendance to the goal of the target action (i.e., stimulus enhancement) but also reproduction of the behavioral sequences leading to that goal, it is likely that humans are exceptional in analysing action streams into smaller components and flexibly manipulating them through a specific rule.

In this article, we propose a possible framework for investigating multiple levels of symbolic representation to produce a human-specific action semantic system based on two types of evidence: one coming from the literature on mirror neurons, and the other based on observations performed by the authors that, although quite anecdotal in nature, are nonetheless interesting for speculating about semantic equivalence. By showing fragments of evidence found in mirror-neuron activities in the parietal area of Japanese monkeys interacting with an experimenter, we hypothesized a possible developmental course of abstract representation through actions, underlining an aspect of linguistic communication. Actually, the observed neuronal data were unstable and rarely replicable, possibly because of unique characteristics of activities in the recorded area and our experimental condition, which was intended not to firmly tune up the monkeys for a specific task, but to abstract “natural” neural activities of untrained, communicative behavior. Still, we try to find signs of abstract action analysis from the monkey data, rather than abandon them as unreliable, to bridge between direct and symbolic encoding of actions by observation.

BRAIN AREAS FOR INTEGRATING MEANINGFUL ACTIONS

Recognition and integration of meaningful actions by inferior parietal cortex

The coordination of higher-order motor functions—including the sequencing and orchestration of elementary motions and the use of objects or tools to achieve an intended purpose—relies on internally represented plans for, and predictions about, meaningful actions as well as sensorimotor integration for guiding those actions. When we plan an action—say, to pick up a glass of water and take a drink—we usually only hold the general goal or desired outcome in mind; the detailed forms, fine motion and postural adjustments, kinematics, degrees of force, and sequential structures of the intended action are selected automatically, outside of awareness, as the action unfolds. Monkey experiments (Graziano & Gross, 1998; Rizzolatti, Luppino, & Matelli, 1998), brain imaging studies (Blakemore, Wolpert, & Firth, 2002), and clinical evidence (Buxbaum, 2001; Leiguarda & Marsden, 2000) suggest that planning actions and guiding body movements are closely integrated in both the parietal and frontal lobes working in parallel.

Thus, an action's intention and ultimate goal should be represented in one brain area independently of its kinematic or structural details but must nonetheless be functionally linked to another circuit that is responsible for orchestrating those details. This independence would predict two kinds of related neural responses to physically observed or performed actions. There must be (1) neurons that respond differentially to identical acts when they are components of larger action sequences performed to achieve different purposes (e.g., sticking out a finger to push a button versus using the finger to point to a distant object), and (2) other neurons that respond equally to differently structured actions when they are made to fulfil an identical goal (e.g., picking up a mug by either its handle or its sides in order to take a drink). Also, in the same cortical areas in which such information processing is performed, there should exist neurons with transitional properties between the two cognitive components described above, reflecting their integration.

Where in the cerebral cortex should we look for these neural characteristics? The most likely candidate is the inferior parietal cortex, where both multisensory and sensorimotor integration are known to take place (Mountcastle, Lynch, Georgopoulos, Sakata, & Acuna, 1975). For a long time, the inferior parietal cortex has been regarded as a sensory associative area that integrates somatosensory and visual information. In classical single-cell recording experiments in monkeys, the inferior parietal cortex also appears to subserve higher-level cognitive functions related to action (Blakemore et al., 2002; Buxbaum, 2001; Graziano & Gross, 1998; Leinonen & Nyman, 1979; Leinonen, Hyvärinen, Nyman, & Lirmankoski, 1979; Rizzolatti et al., 1998). In addition, this area is known as a site for controlling the neural responses of various top-down mental processes such as attention, motivation, and context-dependent behavior (Hyvärinen, 1981, 1982; Mountcastle et al., 1975; Robinson & Burton, 1980; Rolls et al., 1979). In monkeys, its anterior portion (area 7b) is known to be interconnected with area F5 of the premotor cortex (Matelli, Camarda, Glickstein, & Rizzolatti, 1986; Rozzi et al., 2006), where neurons with similar properties have been detected (Rizzolatti et al., 1988).

Action understanding in premotor and parietal neurons

In animal communities, including human societies, actions exhibited by community members tend to share common semantic and kinematic characteristics by dint of shared anatomies, bodily dynamics, and cognitive abilities. These commonalities should provide grounding for perceptual generalizations and categorizations about movements and, by extension, about the motivations or goals behind those movements. Neural response properties that fulfil these assumptions do indeed exist in the cerebral cortex. Mirror neurons, originally reported in the premotor cortex (e.g., Rizzolatti, Fadiga, Gallese, & Fogassi, 1996), and later in IPL (Fogassi et al., 2005; Tanaka et al., 2004), respond equally well during observation of others executing goal-directed actions as they do during execution of the same actions by the subjects themselves.

This unique property could derive from generalization of actions’ appearances across different agents. In Figure 1, we categorize the response properties of mirror neurons from the observed data, with different levels of abstraction in a behavioral context involving agents (subjects, S), action forms (verb, V), goals (objects, O), and other variables (others, Ot), together with generalized expressions with symbol (“Coding” column) and its cognitive function (“Neuro-cognitive processes” column). The first rows of the figure show the “Classical mirror neuron” response property exhibited by F5 (Rizzolatti et al., 1996) and 7b neurons, where equivalent action forms (V) and goals (O) by different subjects (X, Y) are coded equally. This response property can be expressed as “* – (V − O)”, where * denotes compatibility of the subjects (i.e., indifferent of specific agent), and the parentheses indicate the specificity (i.e., uniqueness) of the element. For these premotor mirror neurons to encode actions as being equivalent among different agents based on similar action kinetics, actions executed by either hand or tool (Ferrari, Rozzi, & Fogassi, 2005) should be firmly connected to some biologically relevant stimuli or events.

Figure 1.  Summary of observed and predicted properties of mirror neurons in premotor (F5, from Rizzolatti et al., 1996 and Figure 2) and inferior parietal cortex (7b, from Fogassi et al., 2005, Figure 3, and Figure 4). Each action sequence can be analysed and described by agents (X, Y, general form in “S”, like subject), action forms (A, B, general form in “V”, like verb), goal of action (objects, general form in “O”), and other variables (others, general form in “Ot”). These elements are expressed in symbols as shown in the “Coding” column. In this column, “*” denotes compatibility with any substitutes. Elements in parentheses indicate that these must be specific, not compatible with any substitutes. Possible “Neuro-cognitive processes” are proposed in the rightmost column. The bottom row depicts our predictions based on these results, suggesting that when mirror neurons with five different properties are integrated into a system, the “Equivalence relations” will emerge in the brain to produce a human-like flexible linguistic system.

Establishment of semantic equivalence through the parietal mirror neuron system

However, as the relationship between behavior and object increases, involvement of the parietal lobe becomes critical. Indeed, Buccino et al. (2001) reported that, in human subjects, besides activations found in premotor cortex, the parietal lobe was involved more when an observed action involved a specific object goal than when it did not involve it, as in mimicking the action. Thus, the parietal lobe is responsible for “pragmatic” description of object-related actions, in contrast to “semantic” object description by the infero-temporal lobe (Jeannerod, Arbib, Rizzolatti, & Sakata, 1995). This interpretation is consistent with activations of mirror neurons found in the IPL by Fogassi et al. (2005). Two different types of action were executed, by both subject monkeys and a human experimenter: reaching for a food pellet and putting it in a container, and reaching for a food pellet and eating it (“Contextual dependency” as shown in Figure 1). Some neurons encoded identical actions (V) differently, because in each case, the action was being performed in the service of a different final goal (Ot). Additionally, the neurons showed the same selective firing pattern during the observation of each action performed by the experimenter (S). Thus, these parietal mirror neurons conceptually extend their responses on the basis of context, intention, and purpose behind an action, regardless of its kinematic similarities of actions and temporal proximity between action and intended goal (* − V − O−(Ot)). Hence it is likely that these neurons are crucial for the higher-order conceptual understanding of events. This faculty should be particularly beneficial in social contexts, where exchanges of information about external and internal events are critically important.

EXPERIMENTAL OBSERVATION OF MIRROR-NEURON ACTIVITY IN AREA 7B

Available lines of evidence suggest that a neural system that crucially includes multiple types of mirror neuron exists and serves as an interface between physical action sequences and action semantics. However, one key piece of evidence still lacking is the second class of neuron described above (2)—that is, neurons that respond equally to different and dissimilar actions (V) that have common or identical behavioral meanings (O) in a given context (* − * − (O)).

We have found that such neurons do in fact exist in the monkey inferior parietal cortex. We recorded activities of such neurons from area 7b of four hemispheres of three Japanese monkeys (Macaca fuscata) using a single unit recording method.1 The details of the methods for preparation of animals for electrophysiological experiment, and histological identification of the brain areas were identical to those described in previous publications (e.g., Iwamura, Tanaka, Sakamoto, & Hikosaka, 1993; Yokochi, Tanaka, Kumashiro, & Iriki, 2003). Most of the time during the experiments, no particular training was required for the monkeys, and they were interacting with the experimenter using various kinds of food rewards, instruments, and container.2

The neurons in parietal area 7b discharge when the monkey observes actions related to objects performed by the experimenter, such as placing, grasping, and reaching, and there are some neurons having somatosensory receptive fields, which tended to be distributed around the face or neck region and often extended from the hand to the upper arm. Among these neurons, we observed neuronal activities exhibiting mirror neuron properties: responding both when the monkey itself executed an action and when it watched the experimenter executing a similar or related action.

Neural response with mirror neuron properties

The recording sites of the neurons with mirror-like properties are shown in Figure 2 through 4, together with the precise locations of the recorded neurons. No particular pattern of localization was observed with respect to these neurons’ functional properties, nor was any laminar bias in their distribution detected by the histological examination. Figure 2A shows the typical response pattern of 80 neurons, which was similar to those of premotor mirror neurons, discharging both while executing a particular goal-directed action and while observing another performing the same action. These neurons exhibited an inhibitory form of mirror neuron-like response (Figure 2B), with sustained, spontaneous discharges punctuated by selective periods of inhibition while the monkey or the experimenter performed the same action, such as picking up a piece of food with a precision grip.

Figure 2.  Generalization process of action understandings. A: A representative example of inferior parietal neurons whose response patterns resemble those of classical mirror neurons in the F5 premotor cortex. Neural discharge in raster plots (top of the graph) and histograms showing numbers of spikes occurring per 100 ms bin (ordinate) are shown along the time axis (abscissa). Behavioural events were examined frame by frame (30 frames per second). This format is identical for all neural activity graphs in Figures 2–4. In this condition, the experimenter reached into the container, which sat in front of the monkey on the table, picked up the reward with his or her fingers and handed it to the monkey, which then grabbed the food, brought it to its mouth and ate it. This neuron discharged both when the monkey observed the experimenter picking up a piece of food with a precision grip (red bars under the abscissa, and red circle in the inset) and when the monkey picked up food in the same manner (blue underline and circle). B: Neural recording sites. Data from four hemispheres of three monkeys are projected onto the right parietal area (area shown by the square in the inset) of one monkey, normalized in relation to species-general configurations of intraparietal sulcus. cs, central sulcus; ips, intraparietal sulcus; ls, lateral sulcus. The red dot indicates the electrode tract from which the neurons depicted in A were recorded.

Activity modulated by reward value

Some of the neurons having mirror neuron properties responded differentially to picking up different types of food, even though the forms of the grip or the actions’ kinematics were nearly identical. Figure 3A represents a typical example of such neurons. The neuronal activity was recorded as the experimenter picked up the food with forceps, brought it to the monkey, and let the monkey pinch the food away and eat it. The neurons fired differentially when the final reward was peanut, cookies, or raisins. The strength of the activity was dependent on the monkey's preference: activity was the strongest with the favourite food (i.e., cookies).

Figure 3.  Early processes for establishment of functional equivalence of action semantics. A: A representative example of neurons that exhibited activity similar to mirror neurons, firing equally during grip actions performed by the experimenter (using forceps, indicated by red bars under the abscissa) and by the monkeys themselves (using thumb and index finger, indicated by blue bars). They differed from classical mirror neuron properties in that they fired differently depending on the monkeys’ preference of objects to grip (indicated in the graph). In this example, from the left, the experimenter used forceps to pick up and hand over the rewards to the monkey, and then he picked up three different rewards, cookies, peanuts, and raisins, one by one. This sequence was repeated twice to see replicability of the differential activities. Insets illustrate actions by the experimenter (red circle) or the monkey (blue circle). B: Neural recording sites. The red dot indicates the electrode tract from which the neurons depicted in A were recorded.

Activity representing categorization of different actions by context

If the neurons in the inferior parietal cortex generalize their responses to actions regardless of agent and operand, including external objects (i.e., tools) as well as original body parts (see Figures 2 and 3), it is possible that they code the meanings or semantics per se of actions. Figure 4 depicts two instances in which the monkey was handed a lidded container with food sealed inside. In Figure 4A, the recoded neuron discharged (1) when the experimenter opened the container, (2) when the experimenter closed the container, and (3) when the monkey opened the container. It is noteworthy that the neuron discharged equally on either opening or closing the container, because the kinematics of these two actions is completely opposite. The response property of such neurons was not a generalization of agents and action kinematics as found in ordinary mirror neurons but more an arbitrary categorization of actions based on the context in which the actions were embedded. Fogassi et el. (2005) found the context-dependent activity by area 7b neurons, which would be in contrast to that shown in Figure 4A: differential responses depending on the contexts in the former, and categorized responses depending on a common context in the latter.

Figure 4.  Late functional equivalence process or preliminary process for establishment of stimulus equivalence among action semantics. A: A representative example of inferior parietal neurons that discharged equally to different actions embedded in the same behavioural context. In this condition, the container was sealed with a lid after the experimenter put a piece of food in it, and then handed to the monkey, which then removed the lid, grabbed the reward from inside, brought it to its mouth and ate it. This neuron discharged when the monkey observed the experimenter either opening (red bars under the abscissa, and red circle in the inset) or closing (red dashed bars under the abscissa, and red dashed circle in the inset) the container and also when the monkey opened the container (blue underline and circle). B: In this condition, the container could be opened using either of two distinct actions—by pushing a button at its base or by pulling up the lid. To teach the monkeys to push the button, the experimenter guided the monkeys’ hand to push the button for several times. Then they gradually learned to push the button to get food items inside within a day. A representative example of neurons in which spontaneous discharges were inhibited equally when the monkey observed the experimenter opening the container by lifting its lid (red bars under the abscissa, and red circle in the inset) and also when the monkey pushed the lid-opening button on the device on which the container was placed (blue underline and circle). C: Neural recording sites. The red dots indicate the electrode tracts from which the neurons depicted in A and B, respectively, were recorded.

Activity representing categorization of different actions by shared function

Different types of categorization were also found in area 7b neurons. Figure 4B depicts an activity pattern of such a neuron. This neuron exhibited spontaneous firing that was suppressed during two different epochs: when the experimenter opened the lid of the container by pulling it up with her fingers, and when the monkey pushed the button on the device to open the lid mechanically. Thus this neuron responded when either the experimenter or the monkey performed an action that resulted in the lid opening—even though the actions were completely different in form.

INTEGRATION OF ACTIVITIES TO ESTABLISH HUMAN-ACTION SEMANTICS

Neural response with classical mirror neuron properties in area 7b

In area 7b of Japanese monkeys, we found neurons with response properties similar to those of the “classical” mirror neurons that were originally found in the F5 premotor cortex (e.g., Rizzolatti et al., 1996; Figure 1). In addition, we found other mirror-like neurons whose response patterns differed notably in two ways.

The first class closely resembles classical premotor mirror neurons (Figure 2). These cells generalize the kinematics or appearances of actions, together with their immediate goals, across agents. This allows concrete phenomenal components to be shared among community members and may resemble the phonetic structures of language contingent on specific behavior. Indeed, these classical mirror neurons are most frequently observed in the premotor cortex or area F5 in monkeys (Rizzolatti et al., 1996), which are proposed to be homologous with Broca's area in humans, where phonetic motor–linguistic functions are encoded (Rizzolatti & Arbib, 1998).

Activity with reward dependency

In the second class of neuron (Figure 3), activity corresponding to a given action element is significantly modulated by goal and context—by the overarching action structure in which the element is embedded (Fogassi et al., 2005) and by the action's intent or purpose. Thus, these neurons encode at least a preliminary form of action semantics. In daily life, no action (except perhaps pantomime) stands alone; all elementary actions are embedded in meaningful, goal-directed sequences of action. Based on an ideomotivational state to achieve an intended goal, the agent estimates the consequences of its movements while making continuous fine adjustments as its behavior unfolds. Through this process, fragments of movement are combined into meaningful actions whose components are closely related semantically, through a common goal.

Activity representing different actions with common functions

The third class of neuron encodes the semantics of actions without respect to their kinematics or appearances. These neurons fire equally to different actions executed by different agents if the meanings of their actions are nonetheless the same. That is, pinching using forceps by the experimenter or using a precision grip by either monkeys or the experimenter were coded as equivalent (Figure 3). These neurons generalize the meaning of action further than those that respond to the use of tools but less than those that respond to the use of hands, as reported earlier for neurons in the premotor cortex (Ferrari et al., 2005; Umiltà et al., 2008). When the actions with different kinematics appear in a common context, they are categorized into the same class of behavior, which induces the same neural activation pattern (Figure 4A). The meaning was further abstracted by the neurons that responded to completely different actions for opening a lid: by pulling it up or by pushing a button (Figure 4B). Thus, the abstract semantic equivalence of actions, either self-executed or observed, is established regardless of their concrete appearances. These neural properties are the exact inverse of the other known class of parietal mirror neuron mentioned earlier (Fogassi et al., 2005), which responds differentially to identical kinematics when they are embedded in different contexts and thus have distinct meanings. That is, this class of neuron nicely forms reciprocal and complementary combinations with the former class to comprise a parity of elemental equality and inequality involved in the forms and semantics of actions. Response properties from both of these classes of neurons could be derived from those of the first class, which perform a top-down process of generalization that interacts with the bottom-up additive integration processes. Thus, in concert, these inferior parietal mirror neurons, perhaps in cooperation with premotor neurons, are able to extract functional or semantic equivalence among actions.

As seen in the “Coding” column in Figure 1, the properties of the inferior parietal neurons that show mirror-neuron-like activity can be described as combinations of three categories of elements: agents (S), action forms (V), and their goals (O). Some of these elements have compatibility with any alternative, denoted as “*”, whereas those in parentheses indicate that the elements are specific. In the case of the “Classical mirror neurons” shown in Figure 2, the neurons are indifferent to the individual agency (X, Y) but do respect combinations of action forms and goals (*−(V − O)). The neurons with “Reward dependency” (Figure 3), in contrast, respond selectively to an action's ultimate or intended goals, although agents (S) and action forms (V) are treated as equivalent (* − V − (O)). This property is consistent with the study reporting area 7b neurons influenced by motivational and behavioral variables (Hyvärinen, 1981, 1982). In this respect, they are akin to premotor neurons (Kohler et al., 2002) that are known to have reciprocal projections with area 7b (Matelli et al., 1986). The neurons shown in Figure 4 deal only with the objects of the action (O), not with agents or action (* − * − (O)). Note that when these neurons work together, parietal mirror neurons can encode both each behavioral segment and groups of segments. With these properties, it seems likely that they contribute to higher-order action recognition such as serial integration and segmentation of actions (as opposed to lower-order functions such as coding individual action elements or sequences).

In humans, fMRI studies of normal subjects show right inferior parietal activation during action observation (Buccino et al., 2001) and while viewing actions in context (Iacoboni et al., 2005). In Buccino et al. (2001), activity of inferior parietal region was modulated whether the performed actions had goals (e.g., grasping + apple) or just mimicking. In the case of Iacoboni et al. (2005), different activity in this region was observed depending on whether same action (grasping a cup) was embedded in the different contexts (drinking from it, cleaning it up). Thus, the neurons in this region have not only the classical mirror-neuron property but also the property of coding “functionally related to the observed act”. This highly abstract mirror neuron system is then free from physical similarity, not only between the agents, but also the action kinematics: It can classify highly dissimilar action sequences from multiple modalities as being equivalent just by dint of their having the same final goal. This system could be a very important contributor to several higher cognitive functions, such as categorization and concept formation in one direction, and imitation of another's action in the other direction.

The human supramarginal area—which corresponds to the monkey brain area that we studied (Passingham, 1998)—is regarded as the centre of symbolic function, representing the meanings of abstract forms (Farah, 1989; Goldenberg & Hagmann, 1997), and as one of the most greatly expanded areas along the course of hominine evolution (cf. Holloway, 1996). On these bases, we propose that the monkey inferior parietal area possesses a mechanism for integrating and abstracting the semantics of actions having common purposes, which are shared by community members and which in turn may have served as a precursor to the human linguistic faculty to support the semantic aspects of the coding.

Linguistic faculty viewed beyond equivalence of action semantics

The advanced mirror properties demonstrated in our experiment seem to comprise a correlate of the behavioral phenomenon called “functional equivalence”—an equivalent classification formed by dint of sharing the same consequence (Schusterman & Kastak, 1998; Vaughan, 1988). That is, if the responses to stimuli A, B, C, and D all lead to the same behavioral consequence (e.g., reward) whereas E, F, G, and H lead to another (e.g., no reward), then animals classify the former four stimuli and the latter four stimuli into two separate classes. The stimuli within each class are equivalent in terms of their functions. With this ability, animals are able to form concepts of objects which do not necessarily share physical similarities. Semantics thus represented, and shared among society members, could conceivably serve as fundamental components of a communication system.

Multiple coding and activities of parietal mirror neurons

Assuming that functional equivalence is built upon the neural properties depicted in Figures 2and 3, the activity of the neurons in Figure 4 would be related to a primitive symbolic system that has later developed into the abstract linguistic system in humans. In this system, multiple elements often code for a single event or action using stimuli from multiple modalities such as auditory, visual, and motor stimuli. Such duplicate coding is superior to single coding because it remains useful if any of the modalities are not available. That is, a multiple-coding system makes communication stable across diverse situations and between individuals belonging to the same community. As multiple codes signify a common event, this coding system should be directly dependent on the faculty for functional equivalence. The parietal region is the most plausible candidate for a multiple-coding site based on multimodal integration, which would be comparable to the evidence of multimodal integration in premotor cortex (Kohler et al., 2002).

From functional equivalence to stimulus equivalence

Evidence of establishment of functional equivalence alone, however, would not be a sufficient condition for stimuli to be equivalent. In other words, even if some nonhuman animal were shown to have acquired multiple coding, presumably supported by inferior parietal activity, it is doubtful whether they could manipulate these codings as substitutable (or exchangeable) for each other beyond spatiotemporal constraints, as humans often do.

In order to establish a human-level linguistic faculty, representations of action equivalence and semantics would need to be associated with an arbitrary code shared among society members. At this stage, the mirror neurons in the monkey 7b area with generalizing functions (Figure 4B) would afford such shared coding. This code could become a set of general signs for conveying a set of meanings among members, thus becoming a form of language. This phenomenon could come about through the establishment of a kind of inference called “stimulus equivalence” (e.g., Sidman, 1994). Stimulus equivalence refers to establishment of emergent exchangeable relations among arbitrary, sometimes multimodal, stimuli through a minimum matching-to-sample training set. Because this type of relational learning seems to be particularly beneficial in vocabulary learning in human children, and is exhibited almost exclusively by humans (see Yamazaki, 2004 for a review), it surely plays a vital role in the evolution of language and other uniquely human kinds or degrees of cognition, and could have only evolved after the group size of the human ancestors became large enough for communication among members outside immediate relatives to become demanding. This faculty might be extrapolated from the semantic action-equivalence properties of the inferior parietal neurons described in the following analysis.

The establishment of equivalence relations is best understood by assuming acquisition of the meanings of words. When we learn that object A is called “A”, then, without further instructions, we can naturally recognize that the name “A” stands for object A, not object B. Such bidirectional (but not necessarily logically correct) association between object and name must greatly facilitate the acquisition of language in younger children. The bidirectional association can be expanded to more complex relationships—that is, when taught “if A, then B” (A→B) and B→C, both children and adults readily draw not only the bidirectional inference (B→A, C→B, “symmetry”) but also the transitive inference (A→C, “transitivity”) and its bidirectional inference (C→A, “equivalence”) (Sidman, 1994). How, then, could the elementary components of such equivalence inferences be represented in the monkey inferior parietal cortex, and how could evolution have adapted these mirror-like functions to use stimulus equivalence as a language precursor?

To fill in the missing links in the chain leading from functional equivalence to equivalence relations, we offer some predictions about the possible integrated neural properties derived from the interaction of the three types of neuronal activity shown in Figures 2–4 and obtained from Fogassi et al. (2005). In this type of coding, subject, action, and goal are grouped into one unit, and this unit is exchangeable for other units composed of different subject, action, and object (* − * − *). In this cording, not only physical but also functional similarities become irrelevant to the meaning of a set of action. Whether or not one can recognize the meaning of that set depends on rules of a community or a group to which individuals belong. That is, the meaning of a given set of subject, action, and object is determined arbitrarily by community, so it requires highly developed cognitive ability for abstraction. Because of this abstraction of any types of sensory stimuli, we could expect that similar activities by the inferior parietal lobule for supporting functional equivalence should play a critical role in development of this type of coding, in concert with premotor mirror neurons which would support abstraction of actions.

The above type of coding is presumably found only in humans, exemplified in gestural language systems, symbolic ritual customs, and various types of arts, such as painting, dancing, and drama. However, the most effective media to recognize highly abstract sets of action would be words, with which humans could convey the arbitrary meanings of them independent of spatiotemporal factors. If these units are substituted by specific words, then the words can be even translated into other languages, contributing to broaden linguistic systems of action semantics. For these relations to be established, neural and behavioral faculties of equivalence relations should be prepared in the agents.

The expansion of human social interaction, not only within but also between communities, would have been a critical factor contributing to the changes in our coding system that established uniquely human activities. Several cognitive processes would be operative during such translation, such as multisensory integration and visuomotor coordination—both of which have strong ties to the activities of the inferior parietal region. Thus we expect that this part of the brain should contain a still-undiscovered group of neurons that show similar responses to each other and to different combinations of the elements (again the elements are from three categories: subjects, actions, and goals) once one of them is conditioned to be related to another element from another combination (lower part of Figure 1). For example, in the case of Figure 4B, when [experimenter-pulling up-food from the container] (call set A, for example) shares functional equivalence with another set having the different subject and action, but same goal [monkey-push button-food from the container] (set B), sets A and B would become functionally equivalent sets, supported by neural activities. If, then, there is another set, having different subject, action, and goal, but the goal shares a context [experimenter-close the container-food in the container] (set C), then sets A, B, and C may become equivalent via a mechanism supported by neural activities found in Figure 4A. At this point, the elements eliciting similar activity become equivalent and interchangeable through different levels of coding mechanism. Various multimodal stimuli related to actions and their consequences are equivalent to each other, regardless of physical similarity or spatial or temporal proximity. By developing this type of coding, humans would have acquired flexible communication, initially using actions and contingent stimuli, and then gradually establishing equivalence between the stimuli or events.

CONCLUSIONS

The neural representations used by members of a primate society for drawing common semantic inferences from actions embedded in shared situations may comprise the foundational neural mechanisms of language. In this paper, we have described three classes of neuron (Figures 2–4) in the monkey inferior parietal cortex with mirror-like or mirror-related properties that encode semantic equivalence between the actions of self and others despite their imperfect equality and sometimes completely different appearance. We propose that these three neural populations interact to produce a flexible, powerful semantic system, called equivalence relations, which has become uniquely superdeveloped in humans (Figure 1). We argue that this line of neural properties should bring about a semantic-equivalence system in which any element may become connected to any other semantic unit within the system.

The preliminary stage in the development of our semantic-equivalence system is represented by neurons with classical F5 premotor mirror properties (Figure 2), in which the kinematics or appearances of actions together with their immediate goals are coded without regard to which agent (whether self or other) performs them. As such, representations of concrete phenomenal components could be shared as discrete elements among community members, perhaps prefiguring the phonetic aspects of language.

In the next set of neurons, the representations of the above elements are modified depending on the context in which they are embedded. Activity corresponding to kinematically identical actions differs if the actions have different purposes or goals (Figure 3) (Fogassi et al., 2005). Thus, these neurons are more deeply attuned to the semantics of actions, rather than to their immediate appearances.

The third set of neurons identified by the present study encodes an even deeper level of action semantics (Figure 4). These neurons fire equally to different actions performed by different agents regardless of kinematic appearance so long as the actions share the same purpose in the global scheme of the whole act—thus establishing the semantic equivalence of the actions, either from executed (intrinsic somato-sensorimotor) or observed (extrinsic visual) information.

We assume that the instability of the neuronal activity in area 7b would be a key characteristic in representing abstract aspects of the action events. We also assume that there is not much difference in area 7b activity between monkeys and humans. Rather, there may be differences in connection in which the action representations are categorized into independent components and further encoded as linguistic faculties. By being supported by this encoding system, the representation of the actions in humans could become much more abstract and flexibly interconnected with the other action representations.

Based on these several mirror-like neural response properties observed in the inferior parietal cortex, we expect additional sets of neurons that encode a yet further abstract level of semantic equivalence could be established in advanced primates (lower part of Figure 1). We surmise that, using these various representations of meaningful actions, together with contingent stimuli from several modalities, common understandings of semantic contents emerged in the human brain through the gradual establishment of equivalence relations. The inferior parietal cortex is suggested to interface the action semantics system with a language system that results in far more flexible and reliable communication among community members. This function would represent the precursor to uniquely human cognitive functions that allow us to regard disparate phenomena as equivalent. The establishment of such equivalence relations is thought to be one of the critical factors leading to the human linguistic faculty.

Acknowledgements

We thank Professor Leonardo Fogassi for valuable comments on an earlier version of the manuscript. This study was supported by JSPS and MEXT Japan.

Additional information

Notes on contributors

Hiroko Yokochi

The first and the second authors contributed equally

Notes

¹The experiments were approved by the Animal Experiment Committee of the RIKEN Brain Science Institute and the Animal Care and Use Committee of the Tokyo Medical and Dental University. All husbandry and experimental procedures were in accordance with the Guide for the Care and Use of Laboratory Animals of the National Research Council (1996) and the Guidelines for Animal Experimentation of the RIKEN Brain Science Institute and the Tokyo Medical and Dental University.

²The experimenter always sat at the same table position to the right of the monkey and gave him food rewards—sweet potatoes, peanuts, raisins, or cookies cut into 1 cm cubes. Brief descriptions of the experimental conditions can be found in the captions of Figures 2–4.