From intermediate shape-centered representations to the perception of oriented shapes: response to commentaries

ABSTRACT

In this response paper, we start by addressing the main points made by the commentators on the target article’s main theoretical conclusions: the existence and characteristics of the intermediate shape-centered representations (ISCRs) in the visual system, their emergence from edge detection mechanisms operating on different types of visual properties, and how they are eventually reunited in higher order frames of reference underlying conscious visual perception. We also address the much-commented issue of the possible neural mechanisms of the ISCRs. In the final section, we address more specific and general comments, questions, and suggestions which, albeit very interesting, were less directly focused on the main conclusions of the target paper.

KEYWORDS:

• Object-centered representation
• mid-level vision
• visual consciousness

1. Introduction

The light captured at the level of our retinas is progressively transformed into conscious percepts through a cascade of processing stages. How this transformation takes place remains largely unknown. We recently reported the detailed study of Davida, a young woman presenting with an extraordinarily clear and informative visual disorder that invites new hypotheses about the organization and nature of intermediate processing stages (Vannuscorps et al., 2022). Her results in more than 100 experiments indicated that Davida perceives 2D regions of space bounded by some types of edges (sharp luminance and chromatic edges) alternating between their correct orientation and all the other orientations that would result from their mirroring across one or both axes of their own “shape-centered” coordinate system, their rotation by 90, 180 or 270 degrees around their center, or both (see Figure 1).

Figure 1. Example of stimuli (in black ink) and corresponding potential types of errors (in red ink). Davida’s errors consist in seeing the stimulus as if it were rotated by 90 degrees around its center (A), reflected across one (B–C) or both (D) of its own shape-centered axes, or resulted from a combination of these types of errors (E–G). However, she never reported perceiving tilted stimuli as if it were flipped vertically (H), horizontally (I) or both (J).

Some aspects of Davida’s perceptual disorder remained difficult to explain. In particular, as noted by many commentators (Armendariz et al., 2022; Cavanagh, 2022; Coltheart, 2022; Harris, 2022; Vaziri-Pashkam & Conway, 2022), her almost systematic, below-chance, failure to report the accurate orientation of the stimuli remained particularly puzzling. Nevertheless, we argued that her highly selective visual disorder motivates three main conclusions about the nature of the mechanisms involved in transforming retinotopically represented visual primitives into the conscious perception of objects in environmental coordinates. First, we argued that an intermediate stage in the process of constructing the conscious representation of a visual scene involves segmenting the initial retinotopic space into mutually exclusive bounded regions of space and representing them in a shape-centered coordinate system centered and aligned on the shape itself (e.g., on the elongation axis of elongated shapes). This conclusion was supported by three features of her disorder: (1) it affects similarly tasks assumed to rely on spatiotopic and body-centered representations of visual information, (2) it affects bounded regions of the visual field (but not compound shapes) independently from each other and from their background, and (3) she sees these shapes rotated around their own center or reflected across their own axes (see also Vannuscorps et al., 2021). We proposed to refer to this level of representation as “intermediate shape-centered representation” (ISCR).

Second, in discussing the implications of the fact that her disorder is specific to shapes bounded by only some types of edges (sharp medium-to-high luminance and chromatic edges), we suggested that ISCRs computed by edge detection mechanisms operating on different visual information, such as color, luminance, texture, motion, and binocular disparity are computed in parallel. According to this hypothesis, a single region of the retina (e.g., a triangular region filled with red color on white background) bounded by edges that can be captured by different types of detectors (e.g., both color and luminance) will be represented multiple times (e.g., once in a map resulting from the detection of the changes in luminance and another from the detection of the chromatic change).

Third, we noted that this proposal is committed to the additional claim that a subsequent step in vision must specify the relation of the ISCRs to “higher” representational frames, where conscious representation of objects characterized by several features and of their spatial relationship emerges. And we argued that the limited and systematic types of errors made by Davida supported this idea given that all the types of errors she made corresponded to a specific failure to specify the correct axis correspondence and axis polarity correspondence necessary to map ISCRs onto higher frames (McCloskey et al., 2006).

Besides these conclusions, we speculated on the possible neural basis of ISCRs in the visual system and we also considered possible implications of the fact that the types of visual properties leading to Davida’s impaired vs. intact perception followed a line that parallels that distinguishing the information originally derived from the Parvocellular and Magnocellular subcortical channels, respectively.

The paper attracted 19 commentaries from 33 researchers from complementary fields of research (computational, psychophysical, behavioral, physiological, and neuropsychological). These commentaries focused on different aspects of our study and conclusions (the nature of intermediate representations, their representational frame, their locus in the brain), contributed new hypotheses about different aspects of her disorder (e.g., her below chance accuracy), and brought additional empirical evidence from electrophysiology to experimental psychology and neuropsychology to our attention. We are grateful to all the commentators.

In this response paper, we first address the reactions to our article’s main theoretical conclusions and related speculations: the existence and characteristics of intermediate shape-centered representations in the visual system (2.1), how they are eventually reunited in higher order frames of reference underlying conscious visual perception (2.2), what this means for theories about how orientation is computed (2.3), the possible neural substrate of the ISCRs (2.4), and the implications of Davida’s disorder for theories about the contribution of the subcortical parvocellular and magnocellular pathways to visual perception (2.5).

Then, we address more specific and more general issues that, albeit very interesting, were less directly focused on the main conclusions of the target paper. In section 3.1., we address disagreements and alternative characterizations of Davida’s disorder. We argue that Davida does not suffer from Meares-Irlen syndrome and explain why her disorder cannot be explained by “an extreme oblique effect”, a disconnection between V2/3 and V4 or a disconnection between V4 and IT. In section 3.2., we address specific questions and requests for clarification. Among other things, we introduce new findings indicating that Davida is able to generate a correctly orientated visual image of the types of shapes that she misperceives and that her perception was similar when tested with 2D shapes presented in white on black background. We also clarify that motion parallax may underlie Davida’s advantage for 3D objects since her head was not fixed when she was tested with 3D stimuli and discuss the issue of the integrity of her visuomotor grasping system. Finally, in section 3.3 we discuss possible implications of Davida’s disorder for understanding mirror letter confusions in individuals diagnosed with dyslexia and for computational theories of vision.

Two important additional issues were addressed by the commentators. Several commentators addressed Davida’s significantly below chance accuracy, stressing that Davida’s “avoidance” of the correct orientation implies that she does have some information about the correct mapping parameters but avoids or “inhibits” it (Armendariz et al., 2022; Cavanagh, 2022; Coltheart, 2022; Harris, 2022; Vaziri-Pashkam & Conway, 2022). In the words of Armendariz and colleagues, she “did not so much fail to get the correct answer as she managed to avoid it”, which indicates that “some information about the actual stimulus orientation was available at some point in the visual processing hierarchy”. Other commentators noted that Davida’s perception is mostly modulated by the presence/absence of parvocellular input (Almeida, 2022; Casco, 2022). As Casco notes, the blurring of stimuli that help Davida “hardly affects low spatial frequencies but attenuates high spatial frequencies” (see also Almeida’s comment). This seems to imply that her disorder is mediated by the presence/absence of parvocellular input, irrespective of the presence/absence of magnocellular input. According to Almeida, this suggests that “the presence of parvocellular-biased computations (putatively impaired) overrides information derived from the magnocellular pathway (that, in this case, would allow for the perception of the correct orientation)”. These commentaries led us to explore the hypothesis that Davida’s disorder may result from abnormally strong parvo-on-magno inhibition. We take up this issue in a separate article, where we present new experimental results in line with this hypothesis (Vannuscorps, Houbben, Galaburda & Caramazza, submitted).

2. Implications of Davida’s disorder

2.1. Intermediate shape-centered representations (ISCRs)

In our target article (Vannuscorps et al., 2022), we reported findings about Davida’s disorder that we summarized as follows: Davida sees shapes bounded by some types of edges (sharp color and luminance edges) but not others (severely blurred edges, very small luminance contrast, 3D, stimuli defined by motion) as if they were rotated across their own center or reflected across their own axes. From this, we argued that there must be an intermediate, unconscious, stage of processing where the visual system represents bounded region of space computed by edge detection mechanisms operating on different visual properties (e.g., luminance, color, motion, disparity) in a perceptual frame composed of orthogonal axes, aligned, and centered on geometrical properties of the shape itself (e.g., on the elongation axis of elongated shapes; see also Vannuscorps et al., 2021). We proposed to refer to this new type of visual representation as “intermediate shape-centered representation” (ISCR).

It has been proposed that some type of object-centered representations could underlie object recognition (Caramazza & Hillis, 1990; Marr, 1982; Marr & Nishihara, 1978), the programming of eye movements (Olson, 2003), bodily interactions with objects (Hayward, 2012) or the allocation of attention (Halligan, 1991; Tipper & Behrmann, 1996). ISCRs, in contrast, are assumed to constitute a necessary building block for any visual processing. Once the ISCRs are computed, they are made available to the rest of the visual system for all purposes, such as object recognition and bodily interactions with objects.

The idea that the visual system computes some type of shape-centered representations is far from new. However, this idea has lost some traction in the last decades (but see, Almeida et al., 2020 for instance). In their comment, Armendariz et al. note that shape-centered representations are absent from modern computational models of vision, for instance. Likewise, most theories on human object recognition favor egocentric (viewer-centered) representations instead of shape-centered representations (Hayward, 2012, for discussion). Part of this skepticism originates from the demonstration that the types of behavioral, neuropsychological, and neurophysiological data typically argued to support the existence of object-centered representations are open to alternative interpretations (Driver & Pouget, 2000; Mozer, 2002; but see Subbiah & Caramazza, 2000). Vaziri-Pashkam and Conway (2022), Cavanagh (2022), and Popovkina and Pasupathy (2022) discussed the data reported in the target article in the same spirit. They all suggested that Davida’s disorder could be explained without appealing to shape-centered representations. However, it appears that the argument in those commentaries may have underappreciated aspects of the data that we reported.

According to Vaziri-Pashkam and Conway (2022), Davida’s deficit would rather be of “seeing images flipped vertically”. According to Cavanagh (2022), Davida’s errors would be better characterized as reflections of 2D shapes “across their vertical or horizontal axes”. If so, then, Davida’s disorder would have to be reinterpreted as a disorder occurring in a frame of reference composed of a horizontal and a vertical axis, such as a body-centered frame of reference aligned with Davida’s (upright) retina, head or body or a spatiotopic frame of reference aligned on the gravitational vertical or with the computer screen. In this framework, as Vaziri-Pashkam & Conway put it, Davida’s disorder could result from a difficulty of “ … using cues that allow to set the correct up-down and left-right of the retinal image”, for instance. However, these interpretations are not compatible with Davida’s percept of tilted stimuli. As discussed in the target paper, focusing on Davida’s perception of tilted stimuli is critical because it allows discriminating whether errors of reflection occur across an axis aligned with the shape itself (as we argue) or with her (upright) body, her (upright) retina or some environmental vertical axis (e.g., gravitational vertical, the vertical meridian of the computer screen). Davida never reported perceiving a tilted stimulus “flipped vertically” as suggested by Vaziri-Pashkam & Conway or reflected across a vertical or horizontal axis as suggested by Cavanagh (see Figure 1 for an illustration). Rather, as reported in the Set of results §3 and §4 of the target article, Davida consistently perceives tilted stimuli as if they were reflected across axes aligned on geometrical properties of the shape itself, irrespectively of its orientation on the screen or with respect to Davida herself. We have since additionally reported that Davida’s errors consist precisely of reflections of shapes across orthogonal axes aligned and centered onto the most elongated segment of elongated shapes and, for symmetrical shapes deprived of a straight segment, aligned to their axis of symmetry, and centered on their centroid (Vannuscorps et al., 2021). It is unclear how this type of error could be accounted for by a difficulty of “using cues that allow to set the correct up-down and left-right of the retinal image”. These characteristics of Davida’s disorder are also incompatible with Popovkina & Pasupathy’s suggestion that “the deficit may stem from transforming representation from an eye-centered to a head, body or world centered coordinate system” or that “Davida’s deficit could stem from faulty egocentric encoding in the dorsal pathway”.

Popovkina and Pasupathy (2022) raised an additional point and queried whether Davida’s disorder in copying shapes could rather have to do with a difficulty of perceptually aligning the orientation of the paper and the screen. Three properties of Davida’s behavior speak against this possibility. First, Davida copies perfectly shapes that are blurred, presented with low luminance contrast or constituted of a collection of unconnected elements. A disorder in orienting a sheet of paper with respect to a screen would not be expected to affect only some types of visual stimuli. Second, such a disorder would be expected to lead to “mirror” errors affecting both orientation and location. Davida locates stimuli perfectly. Third, Davida makes the same number and type of errors in copy tasks as in tasks that do not require switching from a screen to a sheet frame of reference, such as when she is asked to read orientation sensitive letters (Experiment 1.6), to name the orientation of an arrow (Experiment 1.1., 3.1), or to click on the tip of an arrow (Experiment 3.2), for instance. Perhaps even more convincingly, Davida’s perceptual disorder affected her performance in tasks in which she was not asked to judge or process the orientation of any stimulus, such as when describing her perception of a color after-effect (Experiment 1.8), to name a picture presented below an arrow (Experiment 3.6), or to judge which of two lines is the longest (Experiments 1.12–1.13). Thus, her disorder is not of judging the orientation of a shape, or in copying it: it occurs at a stage in the visual system at which orientation computation is unconscious and automatic.

Other commentators responded favorably to our proposal that ISCRs constitute a crucial step of perceptual processing (Almeida, 2022; Armendariz et al., 2022; Breitmeyer, 2022; Freud & Ahsan, 2022; Harris, 2022; Leek & Reppa, 2022; Ling & Klimova, 2022; Medina, 2022; Silson & Morland, 2022; Taylor & Xu, 2022). Several commentators noted that in many respects our proposal is in line with prior empirical research. Breitmeyer noted that our proposal that ISCRs may constitute a building block of object recognition is supported by the finding that, in a shape recognition task, the strongest masked priming effects occur when participants view bounded regions of space rather than a mere collection of separate unconnected elements (Breitmeyer & Jacob, 2012). This finding suggests that unconscious visual processing can occur in the visual system at least up to a level at which shape representations are computed.

Although our proposal was empirically motivated by an attempt to account for the nature of Davida’s disorder, it is interesting to note that it ended up being similar to other proposals based mainly on theoretical grounds. As noted by Ling and Klimova (2022) and by Leek and Reppa (2022), our proposal is in line with several surface-based models of shape representation suggesting that an intermediate level of representation consists in representing mutually exclusive 2D bounded regions extracted from the retinal image by edge detection mechanisms operating on different sources, such as color, luminance, motion and binocular disparity (Leek et al., 2005; Marr, 1982; Palmer & Rock, 1994). That Davida perceives a bicolor shape as two separate misoriented bounded regions when the two colors are separated by a sharp edge, but as one single misoriented perceptual unit when the two colors are progressively blended into each other (Experiment 5.6), for instance, is in line with Palmer and Rock’s (1994) hypothesis that 2D bounded regions emerge from edge detectors that are selectively sensitive to sharp spatial changes in a given visual property (e.g., luminance). Breitmeyer additionally notes similarities between our proposal and Grossberg’s (2003) hypothesis that vision relies on the integration of the operations of a “boundary-contour system” that identifies edges determined by different types of visual features and computes bounded regions (e.g., a circle), and of a “feature-contour system” that fills its surface properties (e.g., “light red”). Breitmeyer further notes that our hypothesis that ISCRs are computed at an unconscious level of processing is in line with Grossberg’s proposal that the boundary contour system may operate completely unconsciously. In addition, our proposal that conscious visual perception of objects is the result of the integration of independent mappings of ISCRs computed in parallel extrastriate pathways into “higher” representational frames is in line with Grossberg’s proposal that conscious visual perception occurs only after the surface filling-in process is completed.

In the target article, we did not address the possible functional relevance of the ISCRs. While we do not have a definite answer to this question, an interesting possibility discussed by Breitmeyer is that ISCRs may play a critical role in maintaining coherent binding of different features of the same object (possibly of its surface and contour information) despite their distributed and asynchronous processing. According to Armendariz et al. (2022), ISCRs may (also) be a critical step for robust object recognition. Armendariz and colleagues note that although computer vision has now developed algorithmic solutions for object recognition that reach levels of performance comparable to humans, current artificial networks still fail to recognize some images when small image changes are distributed across the image of an object or when texture is misleading. A teapot with the texture of a golf ball is typically recognized as a golf ball, for instance (Baker et al., 2018). Armendariz and colleagues suggest that ISCRs may be a critical step that allows human object recognition to weigh shape information more appropriately and, thereby, protect object recognition against this type of adversarial attacks. Still another possibility, notably discussed by Palmer and Rock (1994), is that ISCRs are computed because one main function of vision is to parse the visual field into distinct objects and that very often regions of the visual field of homogeneous property correspond to single objects or surfaces.

2.2. A deficit in setting the axis correspondence and axis polarity correspondence during the mapping of correctly computed ISCRs onto higher frames

The existence of ISCRs implies that a subsequent step in vision must be to specify the relation of the ISCRs to “higher” representational frames, where conscious representation of scenes composed of multiple shapes and their spatial relationship emerges. In the target article, we argued that the selectivity of Davida’s disorder to some specific aspects of shapes’ orientation supports this conclusion and provides insight into the mechanisms involved in this mapping. Indeed, all the errors made by Davida corresponded precisely to the errors that would be expected under the assumption that her disorder is of setting the accurate axis correspondence and axis polarity correspondence during the mapping of a correctly computed ISCR onto higher representational frames (McCloskey et al., 2006). Several commentators disagreed with that conclusion. We briefly respond to these disagreements below.

The most fundamental disagreement came from Casco (2022), who suggested that Davida’s disorder is not of perceiving the orientation of shapes. Rather, her disorder would be of binding local features that compose 2D shapes with sharp edges. Binding errors (for instance seeing├ instead of ┤, or p instead of q) would lead to percepts that we misinterpreted as orientation errors. We read this provocative proposal with much interest but realized that it faces several problems. Casco motivates her proposal by pointing out Davida’s ability to tell which of two lines is parallel to a third one (Experiment 2.5). A binding disorder would explain why Davida perceives accurately simple lines but not shapes composed of several line segments. This line of reasoning is based on a wrong interpretation of Davida’s performance in this experiment, however. Although Davida performed this experiment flawlessly, she reported seeing these three lines both in their displayed orientation and as if they were rotated by 90 degrees. In other words, her percept of the three lines used in Experiment 2.5 was identical to the percept of the lines used in all the other tasks (Experiments 1.9, 1.12, 1.13) and we found no evidence that Davida’s perception of simple lines differed from that of more complex stimuli. In addition, the unadorned binding hypothesis fails to explain fundamental features of Davida’s performance: (1) why Davida would make only a particular subset of all possible binding errors, which precisely correspond to what the shapes would look like if they were reflected across their axes, (2) how a binding error would result in a percept in which a whole stimulus is perceived as if it were rotated by 90 or 270 degrees, and (3) why binding errors never affect the integrity of the shape. In contrast, all these features are easily explained under the hypothesis of a specific disorder affecting the axis correspondence and axis polarity correspondence during the mapping of correctly computed ISCRs onto higher frames. In sum, the binding deficit hypothesis is not a good fit for the data reported in the target article.

While agreeing with the idea that Davida suffers from a deficit affecting the perception of the orientation of shapes, Orban (2022) contested the conclusion that Davida has an intact ability to encode the tilt of the shapes that she misperceives. In the target article, we reported that Davida is flawless when asked to decide which of two lines is perfectly aligned with another one (Experiment 2.5). Orban focused his criticism on that observation, arguing that Davida may solve this task by relying on the location of the endpoint of the lines rather than on the tilt of the line per se. In other words, Davida may not be able to code tilt, but she does not make tilt errors in this task because tilt can be inferred from the location of the endpoints of lines. This is an interesting proposal but, if adopted, it would fail to account for other aspects of Davida’s performance. Although the proposal that the location of endpoints of a line provides information about its orientation seems reasonable, if this information were available and used by Davida, it is unclear why it would allow her to derive tilt information, but not to avoid all the other types of errors that she makes, such as perceiving lines as if they were rotated by 90 degrees (Experiment 1.9) and asymmetrical shapes in 7 different orientations (see Figure 7 in the target article). Besides, and more importantly, our conclusion that tilt is preserved in Davida is also supported by the type of errors that she makes in all the other tasks. As illustrated in Figure 7 of the target article, virtually all the errors made by Davida can be interpreted as a consequence of a specific failure to compute the axis correspondence and axis polarity correspondence between the ISCR and higher frames, leading to axis correspondence errors, axis polarity correspondence errors and their combination. In contrast, we found no error that could indicate a failure to correctly compute shapes’ tilt (magnitude or direction). In other words, this alternate hypothesis cannot account for, and is in fact incompatible with, the empirical facts that we reported in the target article.

Taylor and Xu (2022) note that although it is possible that Davida’s disorder emerges during the mapping of ISCRs onto a higher frame, “it remains possible that orientation processing deficits could have occurred much earlier for these shapes during cortical processing” or “much later, when objects are represented in their transformed reference frames”. We appreciate this comment (and the opportunity to respond to it) but it seems difficult to reconcile with the empirical data reported in the target article. The idea that Davida’s disorder may occur earlier than the stage at which ISCRs are mapped onto higher frames seems to face two main difficulties. The first is that Davida perceives ISCRs reflected across their own axes or rotated across their own center. This severely constrains hypotheses about the functional locus of her disorder to the stage at which or after which bounded regions have been extracted from the image and have been ascribed their own shape-centered frame of reference (e.g., ISCRs have been computed). Under the assumption that visual processing proceeds in several successive stages which progressively aggregate the retinotopically-organized primitives represented in the primary visual cortex to form increasingly complex representations of the visual scene, a deficit occurring before that stage would not be expected to affect the perception of the ISCRs with respect to their own axes and center, but instead either some local features that compose the ISCRs, such as the perception of the size, orientation, or location of their edges, or the retinal orientation of the ISCRs. However, as discussed above, these hypotheses are not compatible with Davida’s disorder. As for the idea that her deficit could occur later than the stage in which ISCRs are mapped onto higher frames, it is logically possible but committed to two additional unnecessary and unmotivated assumptions. First, this would require assuming that Davida suffers from multiple (yet quantitatively and qualitatively virtually identical) deficits affecting the representation of ISCRs in the different types of higher reference frame that support perception and action (e.g., spatiotopic, body-centered). Second, this would require additional assumptions to explain why she perceives accurately compound shapes. On our account of the locus of Davida’s deficit, the perception of the orientation of compound shapes emerges after individual ISCRs have each been mapped onto a higher frame and, therefore, is independent from Davida’s deficit in perceiving the orientation of each ISCR.

In sum, none of these criticisms posed a real challenge to our interpretation of Davida’s performance and its implications for the role of ISCRs in visual perception. Medina (2022) additionally (and accurately) stressed that Davida’s disorder therefore provides additional support for the coordinate-system hypothesis of orientation representation (COR) hypothesis (McCloskey et al., 2006).

2.3. Orientation computation: a multiplicity of processes

A third important implication of Davida’s profile of performance concerns the issue of the multiplicity of processes involved in deriving orientation information at different levels of processing in the visual system. There seems to be general consensus that orientation computation is not monolithic, that different stages and processes exist. However, what these stages and processes are remains to be clarified. Davida’s profile forces a distinction between at least three levels. The first is that of the initial retinotopic orientation of the edges that, once detected, located, oriented, and integrated, form the contour of an ISCR. This level is intact in Davida. If it were not, she would have difficulties forming ISCRs. The second is that of computing the orientation of the ISCRs in non-retinotopic frames, such as body-centered and spatiotopic frames. This level is impaired in Davida, as discussed above. The third level is that of the orientation of compound shapes in non-retinotopic frames, which is intact in Davida.

As proposed by Medina (2022), as well as by Castaldi et al. (2022), several other studies have contributed to this issue by documenting other neuropsychological cases who suffered from a disorder affecting their ability to perceive, name, judge, memorize, reproduce and/or compare some aspect of objects’ orientation. Among these cases, Medina notes that some presented with an orientation perception deficit that affected multiple sensory modalities, such as B.C. (Valtonen et al., 2008) who showed a combination of left-right inversion errors and difficulty to reproduce the tilt of a stimulus in both the visual and tactile modalities. Other cases like P.R. (Pflugshaupt et al., 2007) and L.M (Riddoch & Humphreys, 1988) seemed to present with a general confusion between left and right that affected visual judgment tasks and was accompanied by mirror writing from memory. A noted by Medina, such profiles suggest some supra-modal representation of orientation.

We propose that supra-modal deficits do not emerge from deficits in representing the orientation of objects or shapes, but from deficits in representing the environment itself. That is, we suggest that such multimodal disorders are more easily explained as deficits that affect the computation of body-centered and/or spatiotopic frames of reference. If so, we would expect such multimodal deficits to (1) affect not only the orientation but also the location of shapes in the environment, (2) always similarly affect lines, bounded regions and compound shapes and (3) to occur in a spatiotopic or body-centered frame of reference and, therefore, affects either perception or action tasks. One study discussed by Castaldi and colleagues (2022) seems to support this view. In that study (Castaldi et al., 2018), the authors report that some children suffering from lesions to the periventricular white matter fail to discriminate tilted lines in both the visual and haptic modalities. Importantly, they also observe that the deficit (1) affects also compound lines, (2) takes place in a spatiotopic frame of reference and (3) does not affect the type of posting task that has been shown to depend on the vision-for-action visual subsystem (Milner & Goodale, 2006). Hence, the reported deficit is compatible with a disorder affecting the representation of the polarity of the axes of a spatiotopic frame of reference.

Besides retinotopic, spatiotopic and body-centered frames of reference, there are also case reports suggesting that daily life activities may require computing the location and orientation of shapes with respect to other frames of reference. A very interesting case, A.H., often perceives stimuli as if they had been left-right reflected and often mislocates stimuli where they would have been if they had been reflected across a vertical axis crossing the point where she focusses her attention, suggesting that the location and orientation of shapes may also be represented in an attention-centered frame of reference (see McCloskey, 2009 for an extensive discussion of A.H.).

Still other patients seem to suffer from difficulties that are specific to only some visual tasks, suggesting that such tasks may require additional orientation-related abilities (Cooper & Humphreys, 2000; Davidoff & Warrington, 2001; Harris et al., 2001; Karnath et al., 2000; Martinaud et al., 2014; Priftis et al., 2003; Robinson et al., 2011; Turnbull et al., 1995, 1996; Turnbull & McCarthy, 1996). In some of these studies the patients erred mostly or only when asked to compare the orientation of several shapes or objects. For instance, Patient G.R., reported by Priftis and colleagues (2003), erred when presented with three line drawings and asked to identify an “odd-one” that is reflected across the shape’s secondary axis, but the patient had no difficulty when asked to copy shapes or to name orientation-sensitive letters. Similarly, patient R.J., was unable to discriminate lateral mirror images of stimuli from each other and to match the orientation of a shape to the orientation of an arrow but had no difficulty to orient a line drawing in its upright orientation or to point to a part of the object (Turnbull & McCarthy, 1996). In other studies, the patients struggled especially when asked to maintain information about shapes’ orientation in memory, or to retrieve from memory information about the typical orientation of letters and shapes. For instance, W.B. could not tell if a letter is upright or inverted and made numerous confusions in naming orientation-sensitive letters but was flawless in copying them on a separate sheet of paper (Solms et al., 1988). L. M could not discriminate lines oriented at-45 and 45 degrees presented sequentially but had no such difficulty when they were presented simultaneously (Riddoch & Humphreys, 1988).

Together with the fact that in all these cases the deficit was specific to only some aspects of shapes’ orientation (e.g., some types of tilt or some type of reflection errors), and sometimes specific to stimuli characterized by some specific visual features (McCloskey et al., 2006; Pflugshaupt et al., 2007), an obvious goal of future research will be to identify more precisely the type of unit that is under investigation (edges, bounded regions, or compound shapes), the type of frame of reference with respect to which orientation is tested (retinotopic, spatiotopic, body-centered, attention-centered), the sensory modality under investigation (magnocellular or parvocellular information, tactile, etc.) and the precise parameter under investigation (i.e, with respect to McCloskey’s COR framework).

2.4. ISCRS are housed in LO1-LO2

A major goal of vision science is to characterize the form that mental representations take along the visual hierarchy. This implies specifying two main aspects of these representations. The first is the “representational content”, the nature of “what” is represented at any given stage. The second is the reference frame with respect to which this content is represented. In the target article, we argued that current evidence about both the content and frame of reference of visual information in LO1-LO2 (the equivalent of V4d in the Monkey) make it a likely candidate to house ISCRs. This proposal seemed reasonable to most commentators but it was questioned by others (Armendariz et al., 2022; Breitmeyer, 2022; Freud & Ahsan, 2022; Popovkina & Pasupathy, 2022; Silson & Morland, 2022).

Silson & Morland’s commentary focused on the reference frame issue. They query whether the frame of reference that characterizes LO1-LO2 is compatible with our proposal. They argue that “ … the predominance of retinotopy within LO1-LO2 themselves and their neighboring territory (i.e., LO) suggests that perhaps ISCRs are more likely computed in adjacent, but anterior regions of cortex”. We agree with Silson & Morland’s characterization of LO1-LO2 and welcome their note of caution. However, we also need to clarify that our hypothesis is not incompatible with the fact that the majority of neurons in LO1–LO2 are retinotopically tuned. What we suggest is that both shape-centered and retinotopic information is represented in LO1–LO2. Indeed, we assume that ISCRs must retain at all time information about their relation (size, location and orientation) to another frame of reference (extrinsic to the shape itself), be it the retinotopic frame in which it originally emerges, or subsequently a body-centered or spatiotopic frame of reference (see Figure 1 of the target article). In other words, we propose that neurons in LO1-LO2 encode “shape-centered” information about shapes’ contour, but also information about their retinal size, position and orientation, allowing distinct but complementary visual computations to be performed in LO1-LO2. Retinotopic coding may provide an effective frame of reference (organizing principle) for the computation of ISCRs from different types of visual primitives and constitute a prerequisite for subsequently relating ISCRs to other frames of reference. In short, the coexistence of retinotopic and shape-centered frames of reference is a cornerstone of our proposal. And indeed, there is evidence that LO/V4d contains both populations of neurons that are sensitive (Dilks et al., 2011; Hatfield et al., 2016) and insensitive (Eger et al., 2008; El-Shamayleh & Pasupathy, 2016) to the retinotopic size, location and orientation of a given shape.

How exactly this dual coding is embodied in LO1-LO2 is unclear. One possibility is that there are heterogeneous populations of neurons in LO1-LO2. Ling and Klimova (2022) draw an analogy between this possibility and the properties of simple and complex shapes in V1: the information about phase coded in the simple cells is abstracted away in complex cells, which provides V1 sensitivity to both phase (in simple cells) and to motion direction and stereoscopic depth (in complex cells). In this framework, an interesting possibility suggested by Silson and Morland (2022) is that of a gradient of representation within LO1-LO2, from more retinotopically bound posteriorly to more shape-centered anteriorly, similar to findings reported within scene-selective regions (Çukur et al., 2016; Lescroart & Gallant, 2019). An alternative, however, is that one single population carries the joint encoding of these two types of properties. Evidence compatible with such dual coding comes from the finding that neurons in V4d that exhibit position-invariant and size-invariant shape contour tuning are nevertheless also modulated by the retinal size and position of these shapes (El-Shamayleh & Pasupathy, 2016; Pasupathy et al., 2018). This suggests that neurons in V4d that code shape contour information in a way that is independent from the retinal location or size of the shape, nevertheless code information relative to the retinal size and location of these stimuli — exactly what we would expect from neurons participating to representing ISCRs.

Other commentators queried whether current evidence about the representational content of LO1-LO2 is compatible with our proposal (Armendariz et al., 2022). They note that V4 might not exclusively respond to bounded regions alone, but also to configurations of unconnected segments that compose a shape. This is potentially an important point, but we failed to find convincing empirical evidence in support of this claim. In support of this claim Armendariz et al. (2022) cite a study by Gallant et al. (1996) in which some cells in Monkey’s V4 respond selectively or disproportionately to some specific types of gratings, composed of unconnected elements. However, in that study unique configurations of unconnected segments were composed of uniquely oriented single segments (Gallant et al., 1996). It is thus possible that cells’ sensitivity identified by the authors is to oriented single ISCRs rather than to unique configurations of unconnected segments. Whether V4 codes ISCRs or also compound shapes is an important topic of future research. It would be particularly interesting to study potentially shared and distinct neural representations of oriented compound and single shapes.

Other commentators stressed that there are equally (or more) likely alternatives (Armendariz et al., 2022; Freud & Ahsan, 2022). According to Freud & Ahsan, ISCRs might not be a unique property of the ventral visual pathway and could also reflect computations carried out by regions in the posterior parietal cortex (PPC). As pointed out by Freud & Ahsan (and to a certain extent by Taylor & Xu, 2022), the PPC indeed shares several of the properties that makes LO1-LO2 a likely candidate to house the ISCRs, like its intermediate location in visual processing, its connectivity to upstream regions in both the ventral and dorsal cortex and, more importantly, its coding for shapes independently from their size, orientation and location (Konen & Kastner, 2008; Lehky & Sereno, 2007; Sereno & Maunsell, 1998; Vaziri-Pashkam & Xu, 2019). Although these arguments are convincing, a problem with the PPC hypothesis is that, to our knowledge, damage to the PPC does not lead to the type of visual perceptual disorder that would be expected in case of an incapacity to compute ISCRs (e.g., a perceptual disorder affecting the ability to compute shapes). This contrasts with ample evidence that LO1-LO2 damage has severe perceptual consequences, often leading to a complete breakdown of shape processing (Goodale et al., 1991; James et al., 2003; Silson et al., 2013).

Another alternative proposed by Freud & Ahsan is that the PPC represents only ISCRs computed from magnocellular input. In this case, it could be that perceptual disorders do follow PPC lesions but that they often go undetected due to the focus of neuropsychological tests on stimuli biased towards the parvocellular subcortical pathway (Medina, 2022). This proposal is also compatible (1) with the PPC’s sensitivity to 3D structure and depth cues, from it being the main recipient of magnocellular input, (2) with the findings that the PPC is less sensitive than LO1-LO2 to 2D objects (Konen & Kastner, 2008), (3) that lesions of the PPC indeed seems to affect the perception of 3D shapes (Van Dromme et al., 2016), and (4) that sensitivity to 3D structure in PPC resists bilateral lesions of the ventral stream (Freud et al., 2017).

In this framework, Davida’s disorder would be one of mapping ventral ISCRs. Yet, we see two remaining difficulties for this hypothesis. First, it is unclear to us whether, and if so how, this hypothesis accounts for previous neuropsychological reports of patients with dorsal stream damage who failed to discriminate mirror images of stimuli composed of visual cues that were clearly originally tuned towards the parvocellular pathway such as line drawings and geometric shapes (Davidoff & Warrington, 2001; Harris et al., 2001; Martinaud et al., 2014; Priftis et al., 2003; Turnbull & McCarthy, 1996). A second difficulty is that there seems to be evidence that V4d/LO1-LO2 contains cell assemblies specialized in the processing of shape defined by either sharp or blurred edges (Oleskiw et al., 2018), and by either high or low luminance contrast with the background (Bushnell et al., 2011). This suggests that V4d/LO1-LO2 is likely to represent ISCRs computed from information originally biased toward both the parvocellular and the magnocellular pathway. Of course, it is possible that ISCRs derived from the magnocellular pathway are computed in both PPC and V4d/LO1-LO2; however, we see no reason to favor this more complex hypothesis given the current state of evidence.

According to Armendariz et al. (2022), another candidate for the neural substrate of ISCRs may be inferotemporal cortex (IT). Indeed, neurons in the monkey inferotemporal cortex have been reported to code for oriented medial axes in shapes (Hung, Carlson & Connor, 2012; Lescroart & Biederman, 2013). Still, we are not sure that IT is a likely candidate. The type of medial axis representation that has been decoded in IT is different from the properties of the ISCRs: the medial axis representation of most shapes does not include any representation of the shapes’ “center” or centroid, and it does not include any axis perpendicular to the shapes’ main axis. In fact, medial axis representation in IT provides a coding that is more resistant to transformations that change the objective shape of objects (such as rotations in depth, stretching, bending or shearing) than ISCRs, in line with the finding that IT neuronal responses show some invariance to rotations in depth (Booth & Rolls, 1998; Freiwald & Tsao, 2010; Ratan Murty & Arun, 2017). Such invariance to depth rotation is not expected at the stage of ISCRs. A more likely hypothesis is that IT and its medial axis representations could play a role at a stage of visual processing, subsequent to that of the V4/ISCRs, specialized in the representation of shapes for their recognition. In line with this proposal, the medial axis has been shown to constitute a basis of shape categorization and perceptual similarity judgments (Ayzenberg & Lourenco, 2019; Lowet et al., 2018).

2.5. A segregation between ISCRs derived from the magnocellular and parvocellular subcortical pathways

As we discussed in the target article, a particularly striking aspect of Davida’s disorder is that the types of cues leading to intact vs. impaired perception follow a dividing line between the type of information that is originally carried by parvocellular and magnocellular subcortical neurons. This raised the speculative but exciting possibility that her disorder could reflect a distinction between a normal ability to map onto a higher frame the ISCRs computed from information derived from the magnocellular pathway in the context of a disability to map ISCRs computed from information derived from the parvocellular pathway. Most commentators seemed to agree that this proposal is reasonable. In fact, we read no argument problematic for this proposal.

At first glance, Vaziri-Pashkam and Conway’s (2022) commentary may seem problematic for this conclusion, but it is not. Vaziri-Pashkam & Conway’s comment focuses on two main points. First, they stress that subcortical channels do not remain segregated through V1 toward the extrastriate streams. In support of their argument, Vaziri-Pashkam and Conway’s (2022) refer to the results of a lesion study (Ferrera et al., 1994), which they present as evidence that “extrastriate areas receive equally strong inputs from magnocellular and parvocellular layers”. This is a surprising interpretation of the results of that study, where 3/14 V4 sites (21%) at which neuronal activity was recorded after blockade of both the parvocellular or magnocellular layers of the LGN were shown to be completely selective for either magnocellular or parvocellular inputs: their response did not change when one pathway was blocked while they became completely unresponsive when the other pathway was blocked. In any case, this point is only tangentially related to our hypothesis. We were careful in the target article to formulate our hypothesis in terms of a distinction between the ability to process visual cues “originally derived” or “originally biased” towards the subcortical parvocellular or magnocellular pathway. All our hypothesis requires is (1) that different types of visual features be derived disproportionately from the subcortical parvocellular or magnocellular pathway and (2) that some part of the extrastriate cortex be specialized in computing shape information from these different types of visual features, and not that subcortical channels remain completely separate in the extrastriate cortex. It seems that both (1) and (2) are largely reasonable assumptions. In line with (1) there is evidence that magnocellular neurons are more sensitive than parvocellular neurons to visual stimuli of low contrast, low spatial frequency and high in temporal frequency, while parvocellular neurons are more sensitive to stimuli defined by chromatic information, high luminance contrast, higher spatial frequency and lower temporal frequency (Edwards et al., 2021). In line with (2) there is evidence that shapes defined by different types of cues (e.g., color vs. luminance, very high vs. very low spatial frequency) evoke segregated cortical responses in several retinotopically organized areas of the extrastriate cortex such as V2, V3, V4, MT (Bushnell et al., 2011; Conway et al., 2007; Conway & Tsao, 2009; Ferrera et al., 1994; Nassi & Callaway, 2009; Oleskiw et al., 2018; Tanigawa et al., 2010; Tootell & Nasr, 2017; Yabuta et al., 2001), as well as in the parietal lobe, where an inferior-high spatial frequency to superior-low spatial frequency organization has been reported (Mahon et al., 2013). Even more convincingly, there is direct evidence that brain lesion may affect the ability to process shape from only some specific cues despite intact low-level vision (Cowey & Vaina, 2000; Fine et al., 2003; Humphreys & Chainay, 2001; Rizzo et al., 1995).

As a second point, Vaziri-Pashkam and Conway note that “despite dogma and contrary to the arguments put forth by Vannuscorps et al., spatial frequency by itself is insufficient to isolate magnocellular and parvocellular channels”. Our target article did not make such a claim. There is evidence that isolation of the magnocellular and parvocellular pathways requires specific combinations of spatial frequencies, contrast, and temporal frequencies. Sensitivity to stimuli of medium-to-high contrast and high spatial frequencies is impaired by lesions to the P cells, but not by lesions of the M-cells. In contrast, perception of stimuli of very low contrast and low spatial frequency is selectively impacted by M-cell lesions (see Edwards et al., 2021 for review). More importantly, we found no evidence that spatial frequency by itself influences Davida’s perception. Davida’s performance improved when a shape was blurred, but this transformation decreased both spatial frequency and the edge/background luminance contrast. Accordingly, we interpreted her profile as possible evidence that “ … her perception would be intact when ISCRs are computed from information derived only from the magnocellular channel (i.e., for stimuli of very low spatial frequency and contrast)”. This combined influence is well in line with the hypothesis of a segregation between ISCRs derived from the magnocellular and parvocellular subcortical pathways.

Is the parvo-/magno delineation the right overarching framework to unite the different cues that are associated with worsening or improving Davida’s perception? Of course, we cannot be sure. However, we read no plausible alternatives in the commentaries. Vaziri-Pashkam & Conway propose that Davida’s disorder may spare stimuli that are blurred and of low contrast because “ … these image manipulations provide cues used by the brain to compute scene structure”—a proposal suggested by the fact that artists often report that squinting allows them to perceive the spatial organization of scenes more clearly. However, beyond the combined impact of shapes’ luminance contrast and spatial frequency, one also needs to explain that Davida’s performance is modulated by temporal frequency, the presence of 3D cues, and motion in a sense that is exactly predicted by our hypothesis of a delineation along the M-P distinction: she has difficulty for equiluminant stimuli but improves when the temporal frequency increases, and her performance is flawless when stimuli are in 3D or when shape is conveyed by motion signal.

Castaldi and colleagues argued that instead of a disorder affecting ISCRs computed from information originally carried in the parvocellular pathway, Davida’s disorder may be better characterized as a contrast-dependent deficit in which “contrast is correctly computed in V1, but then the read-out of this first stage by higher order areas (such as V4 in the case of Davida) is impaired”. According to the authors, this simpler hypothesis may account for the reported effect of motion, blur and for Davida’s ability to perceive accurately compound shapes because these stimuli have also less contrast overall than still 2D shapes. A limitation of this proposal, however, is that it accounts neither for Davida’s difficulty when presented with isoluminant stimuli (experiments 6.1 and 6.2) nor for the fact that she is not in difficulty with real 3D stimuli characterized by high luminance contrast and sharp edges (Experiment 6.17).

Popovkina and Pasupathy (2022) suggested that “Davida’s deficits could relate to weak encoding of impoverished stimuli—2D shapes defined solely by luminance contrasts or line drawings—in the midlevel processing stages of the ventral visual pathway”. Davida’s disorder would be for impoverished stimuli, not for stimuli originally biased toward the parvocellular pathway. We agree that some aspects of Davida’s profile are consistent with this characterization, for instance 3D shapes and compound shapes (which she perceives flawlessly) are arguably more complex than 2D bounded shapes (which she perceives incorrectly). However, this proposal is at odds with other aspects of Davida’s profile, such as the fact that she is perfectly able to see 2D shapes and line drawings defined by very weak luminance contrast (Experiments 6.3–6.9), or when the stimulus is strongly blurred (Experiments 6.11-6.14).

3. Specific comments on the nature and functional locus of Davida’s disorder

Besides the commentaries directly focused on the main theoretical conclusions of the target article, there were also other more specific and more general comments, questions, and suggestions that, albeit very interesting, were less directly focused on the main conclusions of the target paper. We address these below.

3.1. Some disagreements and alternative characterizations

3.1.1. Meares-Irlen syndrome?

A puzzling aspect of Davida’s disorder is that she sees these stimuli alternating piecemeal through a gradual transition “ … as if the letter was fading in, fading out in different orientations”. Although Popovkina and Pasupathy (2022) agree that one possible explanation is that “Davida experiences a multi-stable percept, where different sub-groups of neurons encoding different directions become successively dominant over time”, they suggest that an alternative view is that Davida may suffer from Meares-Irlen syndrome, a condition characterized by a lack of visual stability or clarity (blurry, dancing letters on a page) in (very) bright environments. However, this alternative is incompatible with three findings from Davida: (1) unstable percepts never occurred for shapes such as crosses (with arms of equal length), squares, diamonds, or circles that would not be affected by axis polarity and axis correspondence errors; (2) Davida has very high acuity for subtle differences in tilt or shape, and (3) her disorder is not related to general brightness, for instance she is in difficulty when presented with shapes displayed on a low luminance (20cd/m2) isoluminant background. Thus, Davida does not suffer from Meares-Irlen syndrome.

3.1.2. An “oblique effect”?

Orban (2022) proposes a radically different interpretation of Davida’s disorder. According to him, Davida’s disorder may be better qualified as “an extreme oblique effect”, whereby some neurons in her visual system would be “tuned only to cardinal orientations”. The oblique effect refers to a larger difficulty to code precisely the orientation of stimuli that are obliquely oriented than the orientation of stimuli that are oriented horizontally or vertically. Some assume that during brain development a decrease of the oblique effect (i.e., the coding of oblique orientations improves) requires binocular coordination (Schoups et al., 1995). Hence, the interest of Orban’s hypothesis is that it may relate Davida’s two known perceptual disorders: her difficulty to perceive the orientation of 2D shapes would be a consequence of her rapidly alternating ocular suppression. However, we have found no evidence suggesting that Davida may indeed have more difficulty for oblique stimuli (e.g., Experiments 4.1 to 4.4) than stimuli presented with their “intrinsic” axes (e.g., axis of elongation) oriented vertically or horizontally (Experiments 1.1. to 1.14 and 3.1 to 3.6.). In Experiment 1.7, for instance, Davida’s number and distribution of errors was virtually identical when she had to use the computer mouse to click as precisely as possible on the tip of the arrow displayed in a cardinal (0—upright, 90, 180 or 270 degrees) or oblique (20, 45, 70, 110, 135, 160, 200, 225, 250, 290, 315 or 340 degrees) orientation. According to Orban, his hypothesis would be supported by Davida’s “ … peculiar pattern of orientation errors, with all orientations reported clustering along the cardinal axes”. The trouble here is that Davida’s errors did not cluster along cardinal axes (see Figure 7 for detail on Davida’s errors).

3.1.3. A disconnection disorder?

In previous attempts to draw conclusions from neuropsychological cases of individuals who suffered from a deficit that was selective to the perception of some features (orientation and location) of visual shapes biased toward the parvocellular pathway, authors assumed that these visual features are computed independently in separate transient-magnocellular and sustained-parvocellular visual subsystems (McCloskey, 2004; Pflugshaupt et al., 2007). A disorder affecting one of these systems would not affect the perception of shapes biased toward the other system. In the target article, we proposed a different hypothesis according to which ISCRs computed from information derived from the parvocellular or the magnocellular channels are collectively differentially connected to shared dorsal mapping mechanisms. We argued that an advantage of this hypothesis is that it can account more easily for the fact that although deficits affecting orientation can be specific to P-or M- biased information, they generally affect indiscriminately all types of stimuli. Selective deficits are interpreted to be a consequence of a selective disconnection from the ISCRs to the (putatively dorsal) shared mapping mechanisms. Across-the-board deficits are interpreted to be a consequence of an impairment of the shared mapping mechanism. In contrast, the hypothesis that the visual system is divided into M-based transient and P-based sustained visual subsystems which both compute the location and orientation of shapes is forced to account for across-the-board deficits as a result of damage affecting selectively the same functional components of two independent systems (McCloskey, 2004; Pflugshaupt et al., 2007).

In his commentary, Medina (2022) invites caution about our premise that most previous cases may have an across-the-board deficit. According to him it is possible that the prevalence of across-the-board disorders is overestimated, notably because in both research and clinical settings, attempts to study the visual system overwhelmingly focuses on types of stimuli that are more strongly biased toward the P-/sustained than the M-/transient visual system. It could be then, that patients with selective deficits to the P-/sustained system are probably often diagnosed with an across-the-board visual disorder. On the other hand, as discussed by Medina, patients with selective deficits to the M-/transient system may be underdiagnosed. We agree that stimuli used in clinical settings are typically biased towards the P/sustained system (e.g., steady, high contrast line drawings made of fine lines). Yet, it also seems to us unlikely that selective disorders are often mistaken for across-the-board disorders. Many situations in daily life naturally favor one pathway or the other, such as reading in natural light or in the dark. Therefore, it seems unlikely that a selective deficit would go undetected. In the three documented cases with a disorder that was selective to the shapes originally derived from P-information (Davida, P.R. and A.H.), for instance, the selectivity of the deficit was rather obvious. Davida spontaneously reported having difficulties only with certain types of 2D stimuli and that it was easier for her to read in the dark. A.H. led an apparently normal life (McCloskey, 2009) and P.R. reported that she could read normally under flickering lighting conditions (Pflugshaupt et al., 2007). But this is speculative, of course, and stresses the importance of including tasks and stimuli specifically targeted as evaluating the perception of stimuli biased towards the magnocellular and parvocellular pathways in clinical settings.

3.1.4. Neural locus of Davida’s disorder

In the target article, we speculated that Davida’s disorder arises at the level of connecting ISCRs computed from parvocellular information and putatively housed in LO1-LO2 to mapping parameters housed in the dorsal stream. Two commentators proposed very different hypotheses.

Orban suggested that Davida’s deficit would be better characterized as a deficit that “concerns the white matter (WM) including the connections between areas V2/3 and V4”, possibly specific to “the disruption of the parvocellular inputs to V4”. He argues that the basis for our hypothesis would seem to suffer from a “classical problem of confusing absence of evidence for evidence of absence”. He notes that although remapping has mostly been investigated in the dorsal stream for both theoretical and practical reasons, remapping also occurs in the ventral stream. We agree with Orban that remapping occurs also in the ventral stream. A core proposal of the target paper is precisely that ISCRs emerge in the ventral stream (i.e., there is a first remapping of retinotopically based primitives onto shape-centered coordinates) and are then remapped on a spatiotopic frame of reference for subsequent processing stages in the ventral stream. This hypothesis is in line with evidence for shape-centered and spatiotopic coding of visual information in the ventral stream (see Melcher & Morrone, 2015 for review). Relatedly, it may be useful to repeat here that our hypothesis is based on previous neuropsychological evidence that brain lesions affecting selectively or disproportionately the ability to discriminate mirror images have been so far exclusively reported in the context of dorsal stream damage. Our hypothesis is speculative, of course. However, its force stems from the absence of equally parsimonious alternatives.

For instance, given the current state of knowledge about visual processes occurring in V2/3, Orban’s alternative seems unlikely. To our knowledge, V2 is widely assumed to be a retinotopic area that computes simple properties of local elements, such as color, line conjunctions, line endings, line orientation, angles, and curves—not yet shape per se (Anzai et al., 2007; Pasupathy et al., 2018). While the function of V3 remains unclear, it also seems to be a purely retinotopic area (Kaas et al., 2015). Hence, it is difficult to imagine how a disconnection between V2/3 and V4 could lead to a deficit in reflecting a correctly computed shape rotated around its own center or reflected across its own axes. In the target article, we argue that this aspect of Davida’s disorder implies that it occurs after ISCRs have been correctly computed. We stand by this reasoning. In terms of functional brain organization, the earliest stage at which shape representation in a shape-centered frame start to emerge seems to be V4 (Pasupathy et al., 2018). Accordingly, we tentatively propose to locate her disorder at or after V4.

Another alternative, proposed by Breitmeyer (2022) is that Davida’s disorder could be caused by a disconnection between V4 and IT. Breitmeyer suggests that this disconnection could affect mostly the stimuli that are characterized by high spatial frequency (P based) because M-based stimuli would be able to use another route to IT, mediated by the orbitofrontal cortex (OFC; Bar, 2003). One difficulty with this hypothesis is that it seems to require several assumptions for which we could not find clear motivation: (1) the hypothesized disconnection of V4 to IT must affect specifically the ability to compute axis correspondence and axis polarity correspondence between ISCRs and a higher frame; (2) this disconnection should also affect V4’s connectivity to the regions of the dorsal stream that are involved in mapping ISCRs onto the type of body-centered representation involved in pointing and finger positioning onto a 2-D line. This, plus the fact that there is independent evidence that lesions to the dorsal stream leads to the type of errors that we observed in Davida suggests that our explanation is currently more parsimonious.

3.2. Questions and clarifications

The commentaries also included requests for clarification or additional information about the nature of Davida’s disorder.

3.2.1. Multiple objects

Popovkina and Pasupathy (2022) wondered whether Davida would be able to compare accurately the orientation of two objects better than she is able to report their individual orientations. This is not the case (Experiments 3.4 and 3.5).

3.2.2. Objects sharing a boundary

Popovkina and Pasupathy (2022) also wondered what would happen if two objects were to share a boundary. In particular, they wondered whether Davida would be able to judge their orientations accurately, or at least their orientations relative to each other. It depends on the type of boundary (Experiment 5.6): shapes that share a sharp boundary are processed independently from each other but shapes that share a blurred boundary are processed as one single shape.

3.2.3. Contrast or luminance of the background

Popovkina and Pasupathy (2022) queried whether Davida’s difficulty could emerge when stimuli are presented on a highly luminant background, rather than when there is a medium to large luminance contrast between the edge of the shape and the background, as we suggest. In our opinion, two sets of results reported in the target article argue against this alternative explanation: (1) Davida presents with the same difficulty when the background has a low luminance (20 cd/m2) (Experiment 6.1) and (2) her perception improves when the luminance of the stimuli increases and the background luminance is kept constant (e.g., Experiment 6.2–6.7). Of particular relevance here, we should note that Davida also performed several tasks in which 2D shapes were presented in white on black background. We had decided not to report the results of these tasks in the target article because they were virtually identical to those obtained with the opposite luminance contrast (black figure on white background).

3.2.4. Vision in periphery

Based on Davida’s accurate perception of blurred shapes, Armendariz et al. (2022) predict that she should also perceive better the orientation of shapes presented in the periphery—where visual acuity is lower. We agree with this rationale but must acknowledge that we have not been able to document this effect of peripheral vision. In Experiment 3.1 of the target article we reported that Davida’s performance was not different when small stimuli (arrows subtending 1.5 × 3 degrees of visual angle) were displayed where Davida fixated her gaze (in central vision) or 10 degrees in the visual periphery. Of course, it could be that she would improve more if the stimuli were presented even more peripherally. Indeed, her performance improves only when stimuli are considerably blurred (see Figure S35 of the target article, for instance).

3.2.5. Cardinal vs. oblique orientations.

Castaldi and colleagues (2022) wonder if Davida would also succeed in perceiving accurately the orientation of stimuli presented in conditions that yield high performance, such as at a low level of contrast, if they were presented obliquely. This is the case, as shown in Experiments 6.3 to 6.16. In all these experiments, stimuli were presented not only in cardinal orientations, but also obliquely and when Davida performed flawlessly for stimuli presented in cardinal orientation, she also performed flawlessly for stimuli presented in oblique orientation.

3.2.6. Mental imagery and mental rotation

Breitmeyer (2022) notes that visual perception and mental imagery share some resources in the visual cortex and asks whether Davida’s perceptual errors when viewing 2D images could also affect mental imagery, for instance her ability to mentally rotate 2D images (see also Popovkina & Pasupathy’s commentary). This does not seem to be the case.

In the target article, we reported that Davida is flawless when asked to close her eyes and either name or copy orientation-sensitive letters upon tactile exploration of 3D wooden letters or the integration of a path traced on the back of her hand by the experimenter (Experiments 2.8.). There is some evidence that tactile identification of shapes’ orientation and (Braille) letter recognition may rely on visual imagery (Bola et al., 2019; Sathian et al., 1997; Sathian & Zangaladze, 2002; Zangaladze et al., 1999). If true, then, this finding suggests that Davida does not suffer from any visual imagery difficulty.

Nevertheless, the role of visual imagery in the tactile identification of letters is debated (at least in the sighted; see Cohen et al., 1997) and, therefore, the evidence reported in the target article is not totally satisfactory. For that reason, it is relevant to report here two findings that we did not report in the target article. First, we also tested Davida in an experiment in which she was asked to name orientation-sensitive letters traced on her hand, but only after having tried to visualize that letter for 5 s. This “imagery” condition of the tactile recognition task has been shown to lead to severe difficulties in a previous case, AH, with an orientation perception deficit that also affected mental imagery (McCloskey, 2009): although AH had no difficulty to name the letters immediately (100% correct), she was severely impaired when asked to first visualize them (53% correct). By contrast, Davida performed the task just as flawlessly and easily in the imagery condition. Of course, given visual imagery's inherently private nature, it is possible that Davida simply did not try or succeed in generating a visual image of the letter. In our opinion, this is unlikely. When interviewed after the experiment, she reported that she was indeed able to “see an image of the letter” and that this image “did not switch or change” in a way similar to what happens when she sees the letter. Second, it may also be relevant to add that Davida was not only perfectly able to draw orientation sensitive letters on oral command as reported in the target article (Experiment 2.9), but she was also able to draw them rotated by 90 degrees or 180 degrees. This suggests that Davida was able to generate a visual image of the letter, to rotate it in her mind, and then draw it.

3.2.7. Cues responsible for Davida’s intact perception of real 3D objects

In Experiment 6.17 of the target article, we reported that Davida was able to determine flawlessly the orientation of black 3D arrows and letters on a white surface, both binocularly and monocularly. Breitmeyer (2022) points out that we failed to report whether Davida’s head was fixed or not in this Experiment. This is indeed an important missing piece of information. Fixing the head would have deprived Davida of monocular motion parallax. Davida’s head was not fixed for the task. Although she did not appear to execute visible voluntary head movements during the experiment, we cannot exclude that she made small movements that allowed using monocular parallax. Monocular motion parallax is thus a possible candidate cue for Davida’s advantage for 3D objects. Other cues proposed by Breitmeyer (binocular parallax, shape from shading, pictorial depth cues) and Orban (stereopsis) can be discarded on the basis of the fact that Davida is also flawless when looking at 3D shapes monocularly (excluding stereo processing and binocular parallax as potential explanations; Experiment 6.17) and when looking at 2D objects with 3D pictorial depth cues like linear perspective (e.g., Experiment 1.3).

3.2.8. Interacting with 2D shapes with haptic feedback

Davida does not have any difficulty perceiving and interacting with 3D objects. In contrast, when asked to place her index finger to a particular part of a 2D shape (e.g., to the tip of an arrow) or her index and thumb onto the extremities of a 2D line “as if she were grasping it” (experiments 1.9 and 1.10), the placement of Davida’s finger(s) was driven by her distorted representation of the shape’s orientation. She typically places her index where the tip of the arrow would have been if the arrow were rotated by 90, 180 or 270 degrees, for instance. In their commentary, Whitwell & Goodale (2022) wonder whether Davida could nevertheless accurately post a card through a slot that would appear 2D-enough to induce a misperception of its orientation. Unlike the tasks that we used, the posting task would provide haptic feedback when she inserts the card correctly, which she could then use to improve her posting. This is an interesting question and an important topic of future research. In our target article, we proposed that ISCRs represent a mandatory building block for any subsequent visual processing. Once an ISCR is computed, it is made available to the rest of the visual system for all purposes, such as object recognition and bodily interactions with objects. In this framework, we interpreted Davida’s disorder as the result of a selective deficit during the mapping of ISCRs computed from information originally derived from parvocellular information into subsequent frames of reference. In other words, we assumed that Davida’s selective difficulty with 2D shapes results from a deficit occurring at a stage that is preliminary to the functional specialization of the visual system in subsystems devoted to representing visual information into body-centered and spatiotopic frames of reference. The finding that Davida’s perception of 2D shapes improves when haptic feedback is available (or expected) would force us to revise this proposal.

3.3. Other implications of Davida’s performance profile

3.3.1. Identity precedes orientation

We reported that Davida is able to name line drawings of objects (e.g., a crocodile, an apple, a bed, a bike) that she perceives as misoriented (Experiment 1.14). According to Harris (2022), this suggests that “ … the shape and identity of an object is extracted before an orientation vector that describes how the object is positioned in space is assigned to the object”. In this, Davida’s performance would be in line with previous neuropsychological and experimental data suggesting that object identity is determined faster than object orientation, and Corballis’ hypothesis (1988) that one must know an object’s identity before one can determine its orientation. We thank the commentator for introducing this topic and for the attempt to relate Davida’s profile to this interesting hypothesis. However, in our opinion Davida’s disorder actually provides evidence that is incompatible with this hypothesis. If identity precedes orientation perception, one should expect Davida to correctly recognize orientation-sensitive letters accurately. In contrast with this hypothesis, Davida often confuses orientation-sensitive letters. She names them not as they appear on the screen, but as she perceives them. This suggests that the identity of the letter is determined after the letter has been assigned an inaccurate orientation.

3.3.2. Mirror letter confusions in developmental dyslexia

Like Davida, a subset of individuals who suffer from a persistent reading disability despite normal intellectual abilities, socio-economic and educational opportunities —often referred to as “developmental dyslexia”— struggle to discriminate mirror letters (Fernandes et al., 2014; Fernandes & Leite, 2017; Peter et al., 2021). So far, the origin of this difficulty remains unclear. As efficient discrimination between mirror images requires literacy in a script composed of mirror letters (Fernandes & Kolinsky, 2013; Kolinsky et al., 2011), one may be tempted to attribute this difficulty simply to the dyslexics’ weaker reading level. However, this explanation is undermined by evidence from first-graders and ex-illiterates suggesting that only rudimentary reading abilities are sufficient to develop efficient mirror discrimination abilities (Ahr et al., 2016; Fernandes & Kolinsky, 2013; Kolinsky et al., 2011). Other researchers have proposed that mirror errors may result from a phonological deficit (Peterson & Pennington, 2015). However, this hypothesis is undermined by the fact that this difficulty is also found for geometrical stimuli (Fernandes & Leite, 2017). By relating some aspect of Davida’s profile to recent evidence obtained with individuals with dyslexia, Casco’s commentary (2022) suggests a new hypothesis.

As discussed in the target article, neurophysiological and neuropsychological studies have shown that damage to the dorsal visual stream affects monkeys’ and human patients’ ability to discriminate mirror images of objects. Accordingly, we interpreted Davida’s deficit as a difficulty to access these dorsal mechanisms when information is originally derived mostly from the parvocellular subcortical pathway (see section “A disconnection disorder”). In her commentary, Casco raises two additional points. First, she suggests that an additional assumption is required to account for Davida’s deficit. Given that blurred and sharp stimuli mainly differ in the amount of high spatial frequency information (not so much the amount of low spatial frequency information), one also needs to assume that the correct orientation contributed by the processing of low spatial frequency information (derived from the magnocellular pathway) is somewhat inhibited by information originally derived by the the parvocellular pathway—a “parvo-on-magno masking effect” (see also Almeida’s comment for a similar point). We agree with this point and discuss it further in a separate article (Vannuscorps, Houbben, Galaburda & Caramazza, submitted). Second, and more importantly for the point adressed in this section, Casco relates this hypothesis to recent evidence of magno-on-parvo masking in developmental dyslexics (Ciavarelli et al., 2021). The results of that study complement those of previous behavioral (Ciavarelli et al., 2021; Lovegrove et al., 1980; Peters et al., 2019) and anatomical (Livingstone et al., 1991) studies suggesting that some people with developmental dyslexia may indeed have a magnocellular-dorsal deficiency. By pointing to this parallel, Casco implicitly raises the exciting possibility that mirror letter confusions in some individuals with developmental dyslexia may be caused by insufficient magnocellular-dorsal activation or parvo-on-magno masking, as in Davida. In other words, the individuals with developmental dyslexia who struggle to discriminate mirror letters (and other 2D sharp high contrast shapes) could be precisely those who show signs of magnocellular-dorsal deficiency or parvo-on-magno masking.

To our knowledge, this hypothesis has not been tested yet. A fascinating prediction of this hypothesis is that it should be possible to improve dyslexics’ mirror letter discrimination by decreasing the contribution of the parvocellular pathway to perception, for instance by reducing the luminance contrast of the letters or blurring their edges. In line with this possibility, some studies have reported that visual-search times and reading speed of individuals with developmental dyslexia improve more than those of control participants when the text is blurred or presented with weak contrast (Williams et al., 1995). In fact, Williams et al. (1995) already proposed to account for this improvement by suggesting that blurred or low-contrast text maximizes the developmental dyslexic’s insufficiently activated magnocellular pathway. Interestingly, this hypothesis would also explain why dyslexics who struggle to discriminate letters (high-contrast 2D stimuli) do not appear to have any difficulty to perceive the orientation of daily life 3D objects, which are naturally more biased toward the magnocellular pathway (like Davida).

3.3.3. Interactivity in the visual system

State-of-the-art artificial networks underlying machine object recognition process information sequentially — they are feedforward architectures. However, behavioral, neurophysiological, and computational evidence suggests that many aspects of visual perception additionally require feedback mechanisms (Kreiman & Serre, 2020). In line with this evidence, Vaziri-Pashkam and Conway (2022) suggest that Davida’s perceptual experience of 2D shapes alternating between different orientations is difficult to reconcile with a strict feedforward view of the visual system (see also Breitmeyer’s commentary on this issue). Vaziri-Pashkam & Conway ask: “How could damage to any stage lead to a multi-stable perception? Instead, the data from Davida require a dynamic interplay across stages in the putative visual-processing hierarchy that is engaged as her visual system attempts to sort through the plausible interpretations of the retinal image”. We agree that this aspect of Davida’s perceptual experience remains puzzling. However, we are hesitant to follow Vaziri-Pashkam & Conway’s suggestion that it implies “a dynamic interplay across stages in the putative visual-processing hierarchy”. We have two reasons to remain cautious about this explanation. First, how this hypothesis can account for her disorder remains unclear to us. One hypothesis we can think of would be that Davida’s unstable perception results from an attempt of her visual system to resolve a conflict between the feedforward spatiotopic representation of visual information when it feeds back onto earlier stages of processing and is incompatible with the content of her intact retinotopic representation of visual information, leading to repeated cycles of feedforward error generation and feedback error detection. However, as discussed in the target article, this hypothesis seems committed to the prediction that her perception should settle when an accurate spatiotopic representation of the orientation of the shape is computed and we know that this is not the case (e.g., Experiment 4.3). A second reason is that this aspect of her perceptual experience may be accounted by the hypothesis that visual perception is underlined by rapid cycles of feedforward processes (Singer, 1999; VanRullen, 2016, see the target article for a fuller discussion).

3.4. Concluding comment

In responding to the commentaries to our paper “Shape-centered representations of bounded regions of space mediate the perception of objects”, we have addressed three broad sets of issues. First, we tried to clarify the results themselves. The original paper is very long, with dozens and dozens of data points, making for difficult reading and the possibility of misinterpreting some results. We hope we have succeeded in clarifying occasional misunderstanding of the results. Second, we addressed alternative interpretations of the results offered by the commentators. Some of the alternative interpretations were motivated in part by a misunderstanding of individual results or failure to consider the full set of results. In these cases, we pointed out how the proposed alternative failed to fully account for the reported results. Here, too, we hope to have succeeded in clarifying the core theoretical inferences that are and are not supported by the data. Third, and most important, we have emphasized the distinction between the evidence that informs our claims about the nature and form of representations computed during visual object recognition and the evidence that speaks to their neural instantiation — a distinction that is an inherent feature of neuropsychological research. Regarding the claims about the types of representations computed in the process of constructing the conscious representation of a visual scene, we have argued that Davida’s performance provides clear evidence for a stage of processing where bounded regions of space are represented in a shape-centered coordinate system centered and aligned on the shape itself (e.g., on the elongation axis of elongated shapes) — ISCRs. The evidence we have provided for this conclusion is overwhelming. Less clear is the conclusion we have been able to reach regarding the neural mechanisms underlying the mapping of ISCRs onto higher-level frames of reference. One aspect is relatively clear: Since Davida’s impaired vs. intact perception is aligned with whether the information is originally derived from the Parvocellular and Magnocellular subcortical channels, respectively, it is plausible to conclude that the construction of the representations that underlie conscious perception involves the interaction of magno—and parvo-derived representations. However, the precise way in which this operation takes place remains to be fully articulated.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This research was supported by the Mind, Brain and Behavior Interfaculty Initiative provostial funds to A.C.