First-order and second-order statistical analysis of 3d and 2d image structure

Abstract

In the first part of this article, we analyze the relation between local image structures (i.e., homogeneous, edge-like, corner-like or texture-like structures) and the underlying local 3D structure (represented in terms of continuous surfaces and different kinds of 3D discontinuities) using range data with real-world color images. We find that homogeneous image structures correspond to continuous surfaces, and discontinuities are mainly formed by edge-like or corner-like structures, which we discuss regarding potential computer vision applications and existing assumptions about the 3D world. In the second part, we utilize the measurements developed in the first part to investigate how the depth at homogeneous image structures is related to the depth of neighbor edges. For this, we first extract the local 3D structure of regularly sampled points, and then, analyze the coplanarity relation between these local 3D structures. We show that the likelihood to find a certain depth at a homogeneous image patch depends on the distance between the image patch and a neighbor edge. We find that this dependence is higher when there is a second neighbor edge which is coplanar with the first neighbor edge. These results allow deriving statistically based prediction models for depth interpolation on homogeneous image structures.

• Natural scenes
• visual system

Notes

Notes

[1] In this article, a relation is first-order if it involves two entities and an event between them. Analogously, a is second-order if there are three entities and (at least) two events between them.

[2] In this article, chromatic 3D range data means range data which has associated real-world color information. The color information is acquired using a digital camera which is calibrated with the range scanner.

[3] In this article, for the sake of simplicity, junctions are called corners, too.

[4] Note that XYZ and RGB coordinate systems are not the same. However, detection of gap discontinuity in XYZ coordinates can be assumed to be a special case of edge detection in RGB coordinates.

[5] Note that using bigger planes have the disadvantage of losing accuracy in positioning which is very crucial for the current analysis.

[6] Singular value decomposition is a standard technique for fitting planes to a set of points. It finds the perfectly fitting plane if it exists; otherwise, it returns the least-squares solution.

[7] By unit-planes, we mean planes that are fitted to the 3D points that are 1-pixel apart in the 2D image.

[8] In other words, the Euclidean image distance between the structures should be less than N.

[9] In the following plots, the distance means the Euclidean distance in the image domain.