Use of 2D Similarity Metrics for 3D Object Recognition

Abstract

How the brain recognizes three-dimensional objects is one of the fundamental open questions in visual neuroscience. The challenge is to determine the information the visual system uses for making recognition decisions. It is generally accepted that shape attributes play an important role in the recognition of many object classes. However, the term ‘shape attributes’ is loosely defined and encompasses not only the projected two-dimensional shapes of the objects, but also their three-dimensional forms. It is unclear whether, for the purpose of recognition, the visual system favors the use of projected shape over depth structure or viceversa. We have developed an experimental paradigm that allows us to directly compare the relative efficacies of the two kinds of information in a simplified object domain, and thereby provides a rigorous method for addressing this important question. Our results indicate that while it is possible for humans to use depth information when explicitly instructed to do so, their default recognition strategy is overwhelmingly biased towards the use of two-dimensional projected shape. We discuss possible reasons for this bias and also consider the implications of these results with regard to the issue of the nature of internal representations for three-dimensional objects.

Indexing terms:

• 3D object recognition
• Representations
• Similarity metrics

Additional information

Notes on contributors

Pawan Sinha

Pawan Sinha is an assistant professor of computational neuroscience in the Department of Brain and Cognitive Sciences at MIT. He got his undergraduate degree in computer science from the Indian Institute of Technology, New Delhi and his Masters and doctoral degrees from the Department of Compuer Science at MIT. Using a combination of experimentation and modelling. Sinha's research attempts to uncover brain mechanisms of high level visual processing in normal and compromised populations. Sinha is a recipient of the Alfred P Sloan Foundation Fellowship in Neuroscience. His work has appeared in several important journals including Nature, Nature Neuroscience and the Proceedings of the National Academy of Sciences. He serves on the editorial board of ACM's Journal of Applied Perception. He is a founder of Imagen, Inc, a company that applies insights regarding human image processing to challenging machine vision problems. Imagen was the winner of the prestigious MIT $50K competition.