The orchestration of the sensory-motor systems: Clues from Neuropsychology

Abstract

Research over the last several decades has led to clear and empirically tractable proposals about the representation of conceptual knowledge in the brain. Here we argue that there are already sufficient data from neuropsychology to strongly constrain extant hypotheses about the representation of conceptual knowledge. One constraint imposed by these neuropsychological data is that recognition of actions and understanding of objects do not necessarily depend on the ability to produce object-associated actions. This conclusion compels a reconsideration of the role played by motor planning and/or execution processes in action and object recognition and understanding.

Acknowledgments

Preparation of this manuscript was supported in part by NIH grant DC04542 to AC, and by an NSF Graduate Research Fellowship to BZM. We are grateful to Jorge Almeida, Leslie Gonzalez-Rothi, Raffaella Rumiati, Kyle Simmons, and Naomi Zack for comments on an earlier version of this manuscript.

Notes

“To claim, as we do, that an action like grasping is multimodal is to say that (1) it is neurally enacted using neural substrates used for both action and perception, and (2) that the modalities of action and perception are integrated at the level of the sensory-motor system itself and not via higher association areas” (p. 459, emphasis original). Gallese and Lakoff (2005) situate these proposals in a broader theory of human cognition that treats a wide range of issues, from the principles that guide praxis, to the architecture of the sensory-motor system, to whether human cognition is mediated by symbols. We will not discuss the issues of imagination, parameters, metaphor, structured neural computation, nor the implications of their basic assumptions for a theory of language (for recent discussion of the available hypothesis space on the latter issue, see Fitch, Hauser, & Chomsky, in press; Pinker & Jackendoff, 2005).

It is sometimes not clear if Gallese and Lakoff (2005) are arguing for multimodality or against modularity. These authors distinguish their use of the term “multimodality” from “supramodality”: “This is important theoretically because supramodality is consistent with the idea of strict modularity, while multimodality is not. Supramodality accords with a picture of the brain containing separate modules for action and for perception that need to be somehow ‘associated.’ Multimodality denies the existence of such separate modules” (p. 459, emphasis original). The claim that “[m]ulti-modality denies the existence of… modules” (p. 459) does not accord with all construals of either multimodality or modularity. In the way modularity was originally proposed (Fodor, 1983), it was not a property of cognitive processes that distinguished (either empirically, or a priori) between modality-specific and multimodal systems. One could, by hypothesis, have modular multimodal processes; perhaps the process(es) underlying the McGurk effect illustrate(s) one such example (Fodor, 1983; Footnote 13 therein).

For instance, it might be argued that selection of the “correct” interpretation of perceptual input is guided by motor planning and/or execution programs (e.g., Viviani & Stucchi, 1992).

Where clarity of exposition requires, we will distinguish “transitive” from “intransitive” pantomimes. Following conventions in the literature “transitive pantomimes” are mimed actions that are associated with objects (e.g., hammering), whereas “intransitive pantomimes” are mimed actions that do not involve an object (e.g., waving goodbye).

Pantomime agnosia cannot be attributed to a “general” agnosia, for two reasons. First, patients with visual agnosia for objects do not necessarily present with pantomime agnosia (Schwartz, Barrett, Crucian, & Heilman, 1998); and second, the patient reported by Rothi and colleagues (1986) who was impaired at discriminating and identifying correct pantomimes, was not impaired at imitating pantomimes. The fact that the patient was not impaired at pantomime imitation indicates that the inability to identify actions cannot be due to a “general” visual impairment.

For example: “Multimodality does everything that supramodality has been hypothesised to do, and more. Multimodal integration has been found in many different locations in the brain, and we believe that it is the norm…”(Gallese & Lakoff, 2005, p. 459).

The fact that BS was unimpaired at imitating transitive and intransitive actions indicates that a general input deficit cannot explain the patient's inability to imitate meaningless gestures. Similarly, this deficit cannot be attributed to a production deficit, since the patient was not markedly impaired at producing transitive and intransitive gestures across a number of modalities of input. Following the model of praxis proposed by Rothi and colleagues (1991), the impairment in BS would be localised to the “sub-lexical” route for extracting motor information from visual input. This mechanism is similar to the orthography-phonology conversion mechanism postulated by many models of language processing, in that it is motivated by patients (e.g., BS) who are impaired at imitating meaningless gestures (cf. reading nonwords) but not meaningful or learned gestures (cf. reading real words).

Nontrivial predictions can be extrapolated from these generalisations. For instance, recall the study on mirror neurons by Kohler and colleagues (2002). In that study, it was observed that a population of neurons responded both to the auditory and to the visual presentation of actions. The position which holds that recognition of actions is independent of production processes at the level required for successful recognition predicts that it should be possible to observe apraxic patients who are not impaired at recognising the correct sounds associated with those actions that they are not able to correctly produce.

This portrayal of the hypothesis space goes back to antiquity: “As to the nature and number of the first principles opinions differ. The difference is greatest between those who regard them as corporeal and those who regard them as incorporeal, and from both dissent those who make a blend and draw their principles from both sources. The number of principles is also in dispute; some admit one only, others assert several” (Aristotle, De Anima, 405a, 30–35).

In fact, Gallese and Lakoff (2005) advance the even stronger claim that the “inferential structure” of abstract concepts can also be reductively traced back to sensory-motor information/processes. Here we restrict our discussion to the representation of concrete concepts, on the assumption that if [sensory-motor content + simulation] is not sufficient to represent concrete object concepts, then [sensory-motor content + simulation + metaphor] will not be sufficient to represent abstract concepts.

The arguments developed by Gallese and Lakoff (2005) assume that evidence for the existence of simulative processes is ipso facto evidence for the claim that such simulation grounds conceptual content. For instance: “According to our hypothesis, understanding requires simulation. The understanding of concrete concepts—physical actions, physical objects, and so on—requires sensory-motor simulation. But sensory-motor simulation, as suggested by contemporary neuroscience, is carried out by the sensory-motor system of the brain. It follows that the sensory motor system is required for understanding at least concrete concepts. We see this as an insurmountable difficulty for any traditional theory that claims that concrete concepts are modality-neutral and disembodied” (p. 468, emphasis original). We will take up in the final section ‘PULLING IT ALL TOGETHER’ the issue of whether demonstrations of simulative processes present an insurmountable difficulty for disembodied theories.

Two types of simulative processes that Gallese and Lakoff (2005) take to bear equally on their proposal must be distinguished. Some types of simulative processes may be understood as being voluntary and/or explicit (e.g., mental imagery, mental rotation, etc.), while some types of simulative processes may be understood as being automatic and/or implicit (e.g., the behaviour of mirror neurons and canonical neurons). It is not at all clear that the evidence for voluntary simulation is relevant to a theory of the representation of conceptual knowledge. The reason is that such evidence does not bear on either of two claims, one of which must be intrinsic to the content-reductive hypothesis: (1) simulation over sensory-motor content exhausts conceptual content; (2) simulation over sensory-motor content is necessary for understanding concrete concepts.

This pattern constitutes one side of a double dissociation: The other side concerns patients who are impaired for conceptual knowledge of objects but relatively unimpaired at using objects correctly (e.g., Negri, Lunardelli, Gigli, & Rumiati, in press; Sirigu, Duhamel, & Poncet, 1991).

On the terminology of some authors (e.g., Gallese & Lakoff, 2005) any systems/processes that “orchestrate” sensory-motor systems are, by definition, “disembodied.” Furthermore, there seems to be some antipathy toward postulating cognitive processes that are in this sense disembodied, presumably because it is believed that disembodied cognitive processes are not tractable constructs from a biologically/neurologically oriented account of behaviour. It is not obvious what the weight of this charge might be; this use of the term “disembodied” seems to suggest cognitive processes that could not (in principle) be described as neural processes, or could not (in principle) exist “in the brain.” Another way to view these relations is that no cognitive processes are “in the brain” or “in the body,” in the way that, for instance, coffee is “in the cup” or the car is “in the garage.” Rather, mental processes are the functioning of the brain, in the sense that digestive processes are the functioning of the digestive system. We have adopted the embodied–disembodied terminology in order to facilitate connection of the issues discussed herein with those discussed by other authors.