Detecting shadows in multi-temporal aerial imagery to support near-real-time change detection

Figures & data

Figure 1. Bi-temporal aerial frame image sets used in this study. (a and b) Hospital facility at morning and afternoon; (c and d) office park at morning and afternoon; (e and f) campus of San Diego State University during the afternoon with ~15 min between image acquisitions; and (g and h) a bridge overpass in Albuquerque, NM, imaged in the morning and afternoon. The top of each frame is oriented northward, except for (g) and (h), in which rightward is northward.

Figure 2. Examples of images associated with the transient shadow classification procedure. Transient shadows are visible in the intensity images at (a) time-1 and (b) time-2. This figure shows the time-2/time-1 ratios of (c) intensity (I_ratio) and (d) intensity-normalized blue (B/I_ratio), wherein higher values correspond to brighter areas in grayscale. Threshold classification of I_ratio and B/I_ratio images was used to derive (e) the bi-temporal transient shadow map. The clockwise arrangement of frames (a)–(e) reflects the workflow order.

Figure 3. Illustration of the approach for deriving a hue/intensity ratio (H/I) classification threshold value based on the transient shadow region of an image: (a) histogram distribution of H/I pixels extracted from the transient shadow region, showing mean and standard deviation measures and (b) histogram of original H/I image and position of the threshold from (a) used to classify shadow pixels in the time-1 and time-2 images. Histograms (a) and (b) are from the hospital facility scene.

Table 1. Hue-intensity classification coefficients based on the mean (µ) and standard deviation (σ) of the transient shadow regions in each image.

Scene	Image	Threshold formula
Hospital facility	t-1	µ – 1.3σ
	t-2	µ – 1.3σ
Office park	t-1	µ – 0.8σ
	t-2	µ – 0.8σ
University campus	t-1	µ – 0.1σ
	t-2	µ – 0.1σ
Bridge overpass	t-1	µ – 0.5σ
	t-2	µ – 0.5σ

Figure 4. Workflow diagram outlining the procedure for shadow classification used in this study. Items with dashed perimeters signify image data and classification products, linked to procedural step items with gray backgrounds and solid perimeters. Arrows indicate the flow of data.

Table 2. Hue-to-intensity (H/I) threshold values used to classify shadows based on visual identification of the inter-modal inflection points, and those based upon statistics derived from the transient shadow regions.

Scene	Time	Inter-modal inflection point	Threshold based on transient shadows
Hospital facility	1	0.63	0.62
	2	0.62	0.62
Office park	1	0.47	0.49
	2	0.55	0.49
University campus	1	0.51	0.55
	2	0.53	0.55
Bridge overpass	1	0.57	0.56
	2	0.50	0.56

Table 3. Summary statistics of shadowing effects on intensity and intensity-normalized blue for the paired pixel block samples, and spectral-radiometric differences observed in directly-illuminated areas of the image pairs. Bold, italicized values are compared with the I_ratio and B/I_ratio thresholds used in shadow classification: 0.9 and 1.1, respectively.

Image		Shadowed (S) and directly-illuminated (DI) areas of interest								Differences between t-1 and t-2
		Ishadowed /Idirectly illuminated (Iratio)				B/Iratio				Iratio	B/Iratio
		min	max	μ	σ	min	max	μ	σ	μ	μ
Hospital facility	t-1	0.42	0.89	0.59	0.15	1.06	1.35	1.21	0.08	0.06	0.08
	t-2	0.26	0.89	0.46	0.22	1.06	1.38	1.23	0.10
Office park	t-1	0.35	0.63	0.45	0.08	1.17	1.32	1.23	0.04	0.03	0.05
	t-2	0.35	0.74	0.52	0.09	1.09	1.29	1.18	0.05
University campus	t-1	0.36	0.75	0.54	0.10	1.11	1.31	1.21	0.05	0.09	−0.11
	t-2	0.54	0.83	0.67	0.07	1.05	1.21	1.13	0.05
Bridge overpass	t-1	0.32	0.67	0.48	0.11	1.05	1.38	1.18	0.08	0.01	0.04
	t-2	0.38	0.67	0.49	0.08	1.11	1.42	1.28	0.08

Figure 5. Bi-temporal shadow classification maps of the four study scenes, corresponding to (a) frames (a and b) (hospital facility), (b) frames (c and d) (office park), (c) frames (d and e) (university campus), and (d) frames (f and g) (bridge overpass) shown in Figure 1. The shadow maps shown were filtered using clump and sieve operations.

Table 4. Accuracy assessment results for (a) maps based on the Otsu thresholding approach and (b) maps based on the novel approach evaluated in this study.

	Based on verification				Accuracies
	No shadow	Shadow t-1	Shadow t-2	Persistent shadow	Producer’s accuracy	User’s accuracy
(a) Based on classification
No- shadow	358	50	45	6	88.2	78.0
Shadow t-1-1	4	303	2	29	79.1	89.6
Shadow t-2-2	43	5	309	28	85.4	80.3
Persistent shadow	1	25	6	384	85.9	92.3
				Average:	84.6	85.1
(b) Based on classification				Overall accuracy:		84.7
No- shadow	390	8	15	4	96.1	93.5
Shadow t-1-1	7	372	3	38	96.6	88.6
Shadow t-2-2	8	0	341	15	94.2	93.7
Persistent shadow	1	5	3	390	87.2	97.7
				Average:	93.5	93.4
				Overall accuracy:		93.3

Figure 6. Comparison of intensity image difference products based on the normalized versus original intensity images. The original intensity images at (a) time-1 and (b) time-2 were subtracted to generate (c) a difference image, wherein neutral (no or little change) is gray while black and white shades represent high negative and positive intensity changes, respectively. The frames labeled (d) and (e) represent the time-1 and time-2 intensity images, which were normalized in transient shadow areas, and frame (f) shows their arithmetical difference.

Figure 7. Magnified subsets of the intensity and arithmetical difference products, based on normalized versus original intensity images. The subsets are located within the (a) hospital facility, (b) office park, (c) bridge overpass, and (d) university campus scenes. Subset images are co-organized as: [1] original intensity image at time-1 and [2] time-2 used to generate [3] an arithmetical difference image; subsets [4] and [5] represent the time-1 and time-2 intensity images which were normalized in transient shadows, and [6] their arithmetical difference. Neutral (no or little change) is gray while black and white shades represent negative and positive changes, respectively.