Improved seeded region growing for detection of water bodies in aerial images

Abstract

In aerial images, near-specular and specular reflection often appear in water bodies. They often lead to irregular brightness or color changes in water bodies and even produce hot spots, harmful to radiometric normalization. Therefore, water bodies must be eliminated when calculating radiometric differences during radiometric normalization of aerial images. In this paper, a simple method to detect water bodies in aerial images based on texture features is presented, an improved seeded region growing (SRG) method. A texture feature is calculated using the relative standard deviation index (RSDI) and a coarse-to-fine procedure is employed. The proposed method includes a multiple partition strategy and a refinement in gradient image that improves the reliability and accuracy of water body detection. By fusing water bodies detected in multiple images, hot spots in these water bodies are also detected. Experiments validate the feasibility and effectiveness of the proposed method.

Keywords:

• aerial images
• water
• specular reflection
• hot spots
• radiometric normalization

1. Introduction

Digital aerial orthoimages are widely used in surveying, mapping, geographic information systems (GIS), visualization, and other applications. When generating digital aerial orthoimages, radiometric normalization, i.e. radiometric equalization or color balancing is an essential processing stage to eliminate radiometric differences. Water bodies, however, have an important influence on the effect of radiometric normalization because the reflection property of water bodies is much different from land areas. The amount of light reflected by a given surface depends on the orientation of the surface, the position of the light source, and the type and roughness of the surface (1). Natural scenes are often not Lambertian, i.e. they do not have a constant reflectance in all directions. Instead, the surface displays a different brightness or color from various sensor-viewing directions. The surfaces can even have some near-specular or specular reflection even producing hot spots in the image (2). Since surfaces of water bodies are much smoother than that of land areas, near-specular and specular reflection occur frequently in aerial images and often lead to irregular brightness or color changes in water bodies and even produce hot spots. As a consequence, radiometric differences caused by different sensor-viewing directions over water bodies are large. In contrast, for land areas, near-Lambertian or Lambertian reflection is the dominant reflection type. Therefore, radiometric differences caused by different sensor-viewing directions over land areas are much smaller. It is necessary to eliminate water bodies, including hot spots to calculate radiometric differences between images in order to improve the radiometric normalization effect for digital aerial images (3).

In remotely sensed images, in contrast to land areas, water bodies strongly absorb sunlight and present a low reflectivity across the whole spectrum. Water absorbs long-wavelength bands and near-infrared bands, rather than short-wavelength bands. The reflectivity of water is close to zero in the near-infrared and mid-infrared bands, so nearly all the radiation energy is absorbed by water bodies. Therefore, water typically looks blue or blue-green as a result of stronger reflectance at shorter wavelengths, and looks darker if viewed at red or near-infrared wavelengths. Its digital number (DN) values are relatively lower than other features in remotely sensed images (4–6). Thus, infrared bands (the near-infrared and mid-infrared bands) are often used to detect water bodies in remotely sensed images. Zhang adopted the unsupervised iterative self-organizing data analysis technique (ISODATA) algorithm to extract urban rivers based on band 3 (the red band), band 4 (the near-infrared band), and band 7 (the mid-infrared band) of TM images (7). Du et al. rejected water pixels by certain thresholds in band 4 (the near-infrared band) and band 5 (the mid-infrared band) of TM images during the radiometric normalization (8). Sivanpillai and Miller used ASTER images (the near-infrared, red and green bands) for detecting smaller water bodies in the Powder River Basin using the ISODATA algorithm (9). Many kinds of water indexes have been proposed to extract water bodies. The normalized difference water index (NDWI) was developed to enhance water information based on the spectral characteristics of water and is used in many applications (5, 10, 11). Building on the original NDWI, Xu presented the modified normalized difference water index (MNDWI) (12). Lu et al. combined the difference between NDVI and NDWI (NDVI–NDWI) with SLOPE and near-infrared (NIR) band to map water bodies with HJ-1A/B satellite imagery (13). In addition, other data sources were also used to extract water bodies. Wu et al. fused spatial information from LIDAR point cloud data with the color information from aerial images (6). Yuan and Sarma used airborne laser swath mapping (ALSM) data and imagery in a spatially constrained model-driven clustering method (14).

In many cases, infrared bands are not available for aerial images only visible bands. Moreover, aerial images are often acquired at a flying height about 1000−3000 m; so the possibility of specular reflection increases greatly in water bodies. When specular reflection occurs on water bodies, hot spots will appear. Of course, if solar zenith angles, a digital terrain model, and orientations of images are available, the position of the specular center can be computed geometrically (3). The accuracy of the computation depends on the accuracy of data and furthermore, the scopes of hot spots still cannot be computed. In addition, waves in water bodies also add the uncertainty. Therefore, some well-known commercial softwares, e.g. ERDAS and OrthoVista, provide functions such as manual drawing or vector data import that determine water bodies and their hot spots, and eliminate these areas for calculating the radiometric differences to improve the quality of radiometric normalization. In this paper, we propose a simple automatic detection method to extract water bodies in aerial images based on texture features.

2. Methods

Water bodies are much different from land areas in aerial images. Generally, they have a smooth texture and appear as homogeneous regions; texture features, therefore, are used to extract water bodies. The major steps of our method include rough detection, optimization by multiple partitions, and refinement in gradient image.

2.1. Rough detection based on texture features

A rough detection method is used to determine the rough water bodies based on the texture features of water bodies. One of the most simple and effective ways to describe a texture feature is using the statistical moments of histogram. Assuming z is a grayscale random variable in the interval [0, L−1], and p(z_i) (i = 0, 1, 2, …, L−1) is its corresponding histogram, then the nth order moment of z about the mean is:(1)

where m is the mean value of z, given asfollows:(2)

Moreover, the second-order moment, i.e. the variance, is:(3)

The second-order moment is a measure of the grayscale contrast that can represent the texture intensity and the dispersion of pixel values. The standard deviation, σ(z), is used frequently as a measure of texture as standard deviation values are intuitive (15). A small standard deviation value means pixel values are concentrated more closely around the mean value. A large standard deviation value means pixel values are more dispersed. Usually, water bodies have much smaller local standard deviation values than land areas. Based on this characteristic, the relative standard deviation index (RSDI) is used as a measure of texture during rough detection. The steps for the rough detection of water bodies are as follows.

(1)	Calculate the luminance image;
(2)	Divide the luminance image into small blocks without overlap based on a given scale;
(3)	Calculate the value of RSDI for each image block;
(4)	Determine candidate water bodies with a RSDI threshold. The threshold can be obtained from sample areas or from experience. Image blocks with a larger RSDI value than the threshold are detected as water bodies.

The RSDI of image block i is defined as:(4)

where i is the image block number, σ_i is the standard deviation of image block i, m_σ is the mean value of standard deviations for all image blocks of the image, and n is the power (n ≥ 1.0). m_σ is used to normalize the standard deviation of each image block. n is used to exaggerate or depress the value of . If m_σ > σ_i, the value of will be exaggerated, otherwise it will be depressed. Generally, the standard deviation of an image block for water bodies is much smaller than m_σ and the standard deviation of an image block for land areas is close to or larger than m_σ. Therefore, the gap in the value of between an image block for water bodies and an image block for land areas can be enlarged.

Other simpler attributes have also been considered, e.g. mean and range of pixel values. In aerial images, water bodies usually show up as different colors corresponding to different mean values. The color of a water body is related to many factors, e.g. the view angle, sediment content, waves in water bodies caused by the wind or flowing water. Hence, it is hard to distinguish water from land using mean values. It is also hard to distinguish water from land using the range of pixel values which is determined by the minimum and the maximum pixel values. There are ships and waves caused by wind or flowing water as well as objects in water bodies. If even only a few pixels have very small or large values, the range of pixel values will be large. However, a few pixels with very small values or large values will have little influence on the value of the standard deviation. Therefore, the standard deviation is a more suitable measure for distinguishing water from land and RSDI, i.e. other form of standard deviation, is a measure for texture that exaggerates the difference between water and land.

In practice, water bodies may be very large in some images with only a small part belonging to land areas. Thus, for some images, the value of m_σ may be close to the standard deviation values of image blocks belonging to water bodies. Then, the RSDI threshold for determining water body candidates will be hard to obtain. In such situations, this problem can be resolved using values of m_σ calculated for other images in the same survey area. For each image, a value of m_σ is obtained. Thus, for the same survey area, the maximum value of m_σ will be taken as the unified value of m_σ for each image to calculate values of RSDI.

2.2. Optimization by multiple partitions

In rough detection, if the given scale, i.e. the size of the image block is too large, water bodies smaller than this size of the image block may not be detected. A smaller size, of course, can be used to improve the accuracy of the detected water bodies. However, the size of image block is too small, rough detection reliability cannot be ensured because some objects may also have a smooth texture in small areas. In order to solve the contradiction between the size of image block and rough detection reliability, a multiple partition strategy is adopted to further improve reliability. The same image is partitioned multiple times to obtain water bodies in each partition based on the method described in Section 2.1. The water bodies detected through multiple partitions make up the final candidate water bodies.

The idea of the multiple partition strategy is derived from the super-resolution image reconstruction technique. The premise is that the use of more than one low-resolution image for the same scene improves image resolution. These low-resolution images are the repeated observations or conversely, they can be regarded as more than one sampling with a sub-pixel offset in the same scene. If the offset is known or accurately estimated and if these low-resolution images are combined, a super-resolution image can be reconstructed (16).

Figure 1 shows a demonstration of two partitions with the given scale for the same image. As compared to the first partition, there is a half-scale offset in the x and y directions in the second partition, respectively. The image also can be partitioned more times to further improve the reliability. In each partition, rough detection will be carried out and RSDI values for image blocks will be calculated. Candidate water bodies will be determined by a RSDI threshold with candidate water body results for each partition. By combining more than one result of candidate water bodies, the reliability of rough detection can be improved. As shown in Figure 1, the shadow area represents a water area. In the first partition of Figure 1(a), the shadow area is divided into four image blocks, each image block only contains a small part of the shadow area. The water area may not be detected because the standard deviation values of the four image blocks may be close to those of other image blocks in land areas. However, the shadow areas are mostly contained in one image block in the second partition of Figure 1(b); therefore, the detection is easy. Fusing the detection results in two partitions, the reliability of rough detection can be improved significantly. More partitions can further improve the reliability but also add the computation burden.

Figure 1. An illustration of the two partitions.

2.3. Refinement in gradient image

The seeded region growing (SRG) algorithm is extended to obtain more accurate water body results based on the rough detection. In our method, the SRG is carried out in gradient image and the candidate water bodies detected by the rough detection act as the seed region instead of a seed point. Based on Refs. (17, 18), the region growing around the seed region can be calculated by:(5)

where g(x) is the gradient value at image pixel x. mean(g(y)) and std(g(y)) is the mean and standard deviation calculated from pixels that already belong to the desired water bodies, respectively. t is the threshold for the separation of water bodies and land areas. From experience, a value around 2.0 is chosen for t.

In the refinement, the gradient image is calculated first using the luminance image and the Roberts gradient operator (15). Then the SRG algorithm is used to obtain accurate water body regions. The initial values of mean(g(y)) and std(g(y)) are computed by the seed region and updated during processing. The reasons for using gradient values instead of pixel values directly in refinement are as follows. On the one hand, changes for pixel values in water bodies are usually slow and pixels in water bodies have small gradient values. The boundary of water and land is relatively clear and corresponding to relative large gradient values. On the other hand, pixel values in water bodies still remain a gradual change in trend. Therefore, if a water area is large, the range of its pixel values may also be large, i.e. the water area may also have large standard deviation value for pixel values. While in gradient image, the gradient value is only related to the changes of adjacent pixel values. The gradual change trend in pixel values will not appear in gradient image. Therefore, the gradient image is more suitable for the separation of water bodies and land areas to obtain the accurate water bodies.

Hot spots are caused by specular or near-specular reflection in water bodies and can be detected by fusing water bodies detected in multiple images. They are not only related to the roughness of the surface, but also related to the viewing angle, the camera field angle, solar altitude, exposure position, the regions of water bodies, and so on. Thus, the locations of hot spots are different for multiple digital aerial images. It is possible to detect hot spots in water bodies by fusing the detected water bodies from multiple images with overlaps. The geometric misalignment of multiple images has limited impact on the accuracy of water body detection because of the strict geometric accuracy requirement for orthoimage products, e.g. the geometric misalignment of adjacent orthoimages is not larger than 2 pixels for 1:500, 1:1000, and 1:2000 orthoimages produced in China. An illustration of hot spot formation is shown in Figure 2. Assume that i is the view angle, 2θ is the camera field angle, and h is the solar altitude. Then, if h ≥ i−θ, there may be hot spots in water bodies. We can hover mitigate hot spots by designing the flight plan to avoid hot spots in images since the solar altitude can be calculated according to the planned flight time and exposure positions.

Figure 2. Hot spot formation.

3. Experimental results and discussion

3.1. Experimental results

Our algorithm was implemented in C++ language based on the principles described in the previous sections. The geospatial data abstraction library (GDAL) was used to read and write image files. A number of digital aerial orthoimages were used to test the proposed algorithm. The data-set for the experiment presented in this paper are digital color orthoimages covering a study area in the Three Gorges, China. These images have an approximate overlap of 60%. The resolution of the orthoimages is 0.5 m and the image size is approximately 2000 × 2000 pixels.

Figure 3 shows one of the auto-detection experiments for water bodies and their hot spots. Figure 3(a) is an orthoimage with water bodies and hot spots. These water bodies are known as the Yangtze River. The given scale for partition, i.e. the size of image block, was 250 × 250 pixels. Figure 3(b) shows the RSDI histogram for the first partition. The RSDI threshold for rough detection was 50 and the power of RSDI was 3.0. In our experience, the value of 3.0 for the power of RSDI is enough to achieve satisfactory results. Figure 3(c) shows the detected rough water bodies based on three partitions, i.e. the highlighted areas in Figure 3(c). There is a one-third scale offset in the x and y directions, respectively, for each partition. Figure 3(d) shows the refined water bodies using gradient image, i.e. highlighted areas in Figure 3(d). The hot spots in water bodies were not detected in Figure 3(d). Figures 3(e) and (f) show two adjacent orthoimages of Figure 3(a). Hot spots appearing in Figure 3(a) disappear in Figures 3(e) and (f). Fusing the detected water bodies seen in Figures 3(e) and (f), the final detected water bodies for Figure 3(a) are shown in Figure 3(g), i.e. highlighted areas. Water bodies and their hot spots are all detected and some ships in water bodies are also eliminated. Figure 3(h) shows the detected water bodies with the SRG algorithm, i.e. highlighted areas. The result is detected by single orthoimage, i.e. Figure 3(a). Figure 3(i) is the final detected water bodies by SRG. This result was obtained by fusing detected water bodies in these three orthoimages (Figures 3(a), (e), and (f)) with the SRG algorithm. Figure 3(j) shows the final detected water bodies by SRG in gradient image. The marked points in Figures 3(a), (e), and (f) are hand-selected seed points of SRG algorithm. From the comparison, it can be seen that the gradient is more suitable for detection of water bodies.

Figure 3. Detection results of water bodies.

Figure 4. The relationship between overlaps and hot spots in water bodies.

To further evaluate the performance of our method, we compared the water bodies detected by our method to the reference water bodies detected by manual tracing, SRG, and SRG in gradient image, as shown in Table 1. A total of 1,817,065 pixels were counted in the reference water bodies. In the experiment, 758,858, 1,793,883, and 1,809,869 water body pixels were detected by SRG algorithm, SRG in gradient image, and our method, respectively. Because of the waves in water bodies, most of water bodies were not detected by SRG algorithm. When the gradient image was used, the result of SRG algorithm was improved greatly. Our method, however, achieved the highest efficiency because the rough detection is very fast and the following refinements are restricted to relatively narrow areas. This strategy reduces the computation of the original SRG algorithm.

Table 1. Comparison of SRG, SRG in gradient image, and our method.

Item	Reference	SRG	SRG in gradient image	Our method
Water bodies (pixel)	1,817,065	758,858	1,793,883	1,809,869
Time (ms)	N/A	4511	7720	4391

3.2. Discussion

The applicability of the proposed method for the detection of hot spots must be discussed. The relationship between overlaps and hot spots in water bodies using frame aerial images (digital or film based) is demonstrated in Figure 4. Assuming images are captured ideally by vertical photography, in the direction of the baseline, let L be the width of the ground coverage of a frame, a (while a ≤ 100%) be the overlapping degree between adjacent images, and G1, G2 as centers of hot spots in two adjacent images. Then, the width of overlap between two adjacent images is aL, the width of tri-overlap is (1−2×(1−a))L=(2a−1)L, the distance between two adjacent exposure positions is (1−a)L, and the distance between G1 and G2 is also(1−a)L. In water bodies, adjacent images usually have a similar roughness of surface, viewing angles, camera field angles, and solar altitudes. Thus, the sizes of hot spots in adjacent images are also similar. If two adjacent images are used to detect hot spots in water bodies, there are two sufficient conditions as follows.

(1)	In order to ensure that any area is at least covered by two images, there must be certain tri-overlaps. Namely, there is (2a−1)L ≥ 0. It is equivalent to a ≥ 50%, i.e. the overlapping degree between adjacent images should not be less than 50%.
(2)	In order to ensure that hot spots appearing in one image do not appear in the same region of another image, the region of hot spots should not be greater than (1−a)L.

If these two conditions are met, hot spots in water bodies will undoubtedly be detected by fusing water bodies detected by adjacent images. This discussion only concerns the situation when hot spots are detected using two adjacent images. In practice, other overlapping images also can be used to detect hot spots. They can be determined by fusing water bodies detected from several images with overlaps.

4. Conclusions

This paper presented an improved seeded region growing method to detect water bodies in aerial images based on texture features. Considering that infrared bands are not usually available for aerial images, the proposed method uses the texture features of water bodies for detection. A coarse-to-fine procedure is carried out including rough detection based on texture features and refinement in gradient image. The RSDI is used to describe the texture features during rough detection. Then, the rough detected water bodies are refined in gradient image using the SRG algorithm to obtain more accurate regions of water bodies. To improve the reliability of the proposed method, a multiple partition strategy is employed in rough detection. The strategy is derived from the super-resolution image reconstruction technique. Considering that hot spots appearing in one image must disappear in the same region of another image, we fuse the water bodies detected from multiple images to detect hot spots in water bodies. The applicability of the proposed method is discussed for detection of hot spots. Experimental results indicate that water bodies and their hot spots can be detected effectively. We also compared the detected water bodies by our method with the reference water bodies by manual tracing, the SRG, and the SRG in gradient image. This comparison indicates that the proposed method achieves very competitive accuracy with good efficiency. Our method is in fact an extension of SRG algorithm, but without the hand selection of seed points and with higher efficiency.

Nevertheless, there are still drawbacks in the proposed method. The primary drawback is that our method is only suitable for the detection of water bodies not smaller than the given scale for partition. A water body smaller than the given scale might not be correctly detected because it may have a relative large RSDI value and appear similar to image blocks belonging to land areas. In addition, if there are objects with a texture similar to water bodies, the proposed method may fail and other auxiliary information should be used to distinguish water bodies. These issues will be addressed in future work.

Notes on contributors

Jun Pan, PhD, is currently an associate professor at the State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing (LIESMARS), Wuhan University, China. His research focuses on high-resolution remote sensed image processing, including image restoration, image dodging and quality evaluation, and seamless image mosaicking, and radiometric normalization.

Mi Wang, PhD, professor of LIESMARS, Wuhan University, is mainly engaged in image processing for digital photogrammetry and remote sensing of new-type airborne and spaceborne imaging sensors.

Funding

This work was supported by the National Basic Research Program of China (973 Program) [grant number 2012CB719901]; the National Natural Science Foundation of China (grant number 91438112), and the Foundation for the Author of National Excellent Doctoral Dissertation of China (FANEDD) [grant number 201249].