Validating the geometric accuracy of high spatial resolution multispectral satellite data

Abstract

Uses of high spatial resolution data obtained from satellite-based sensors include creating land cover maps, deriving large-scale quantitative assessments such as vegetation indices, and visually assessing an area for qualitative information only assessable from large-scale digital data. One of the more popular uses of high spatial resolution data is to use the image as a base map for on-screen digitizing spatially dependent vector products. Since most geographic information system (GIS) databases store a variety of current and historical data, the accuracy of any on-screen digitized product is dependent on the spatial accuracy of the reference data. Therefore, it is important to understand and validate the accuracy of data used to create spatially referenced product, even though the data come with high spatial resolution.

One of the more popular and historical high spatial resolution data within most GIS labs is QuickBird's multispectral data at 2.44 × 2.44 m². Although there are current sensors available with a higher spatial resolution, the sometimes prohibitive expense of obtaining high spatial resolution data necessitates the need to utilize and assess historic data. Since the QuickBird has been the mainstay of high spatial resolution data since 2001, understanding the geometric accuracy of the DigitalGlobe's QuickBird user-defined panchromatic and multispectral image bundle product remains relevant. In this study, we assessed the positional accuracy of this product for its utility as an “off the shelf” base map for creating other spatially referenced products. The average Euclidean distance, RMSE (root mean square error), and RSME (root square mean error) between QuickBird-identified Universal Transverse Mercator (UTM) coordinates and coincident in situ GPS-collected UTM coordinates were calculated at 33 systematically selected locations throughout the city of Nacogdoches, Texas, USA. The average Euclidean distance, RMSE, and RSME between QuickBird's projected UTM coordinates and its corresponding GPS-collected UTM coordinates measured at 5.34 meters, 5.79 meters, and 4.05 meters, respectively. They were well within DigitalGlobe's stated RMSE positional accuracy of 14.0 meters for a panchromatic and multispectral QuickBird image bundle.

Keywords:

• QuickBird
• accuracy
• Euclidean distance
• RMSE
• GPS

Introduction

Starting in 1972 with the launch of the first Landsat Multispectral Scanner (MSS) satellite in a sun-synchronous orbit, the ability to quantify, qualify, map, and monitor earth's resources from a space-based platform on a routine basis was initiated (Campbell 2002). One of the primary or initial uses of satellite-based remotely sensed data was to classify a digital image into its respective land cover types. In addition to creating land cover map products, other uses of high spatial resolution sensors focus on large-scale quantitative assessments (e.g. vegetation indices, change detection), visual assessments at a large scale (e.g. urban forest health at the individual tree level), and using high spatial resolution data as a base map for on-screen digitizing spatially dependent vector products.

The traditional approach of converting a digital image into a land cover map as described by Jensen (2005) consists of a five-step sequential process; image acquisition, radiometric correction, geometric correction, image classification, and accuracy assessment. Geometric correction, the third step in the five-step sequential process of converting a digital image into a land cover map, is the process of projecting image data onto a two-dimensional (2D) plane to conform to a map projection. This step is essential in creating an accurate remotely sensed land cover map since any errors in the projection will result in scene-to-scene comparison errors and quantitative errors when calculating area and distance measurements within the final land cover map (Toutin and Chenier 2004; Turker and Gacemer 2004).

Traditionally, most remote sensing specialists have followed the classical approach when creating a land cover map using low- and mid-spatial resolution sensors including the Landsat MSS, Thematic Mapper (TM), and Enhanced Thematic Mapper (ETM+) data with spatial resolutions of 79 × 79, 30 × 30, and 30 × 30 m², respectively. With the launch of DigitalGlobe's (Longmont, CO, USA) QuickBird 2 satellite on 18 October 2001, delivering satellite imagery with spatial resolution of 60 × 60 cm² panchromatic and 2.44 × 2.44 m² multispectral the ability to use high spatial resolution imagery from satellite platforms to obtain data about the earth's surface at a large scale became apparent (Clark et al. 2004; Niu et al. 2004). The accuracy of high spatial resolution sensors have been evaluated and found to be effective relative to their ability for a pixel to accurately represent the true spatial coordinates of its representative area with a high degree of accuracy (Li et al. 2007; Qiao et al. 2010). High spatial resolution IKONOS data have been reported to result in high land cover classifications accuracy (Mehner et al. 2004), but were disputed in a study by O'Melveny (2004) that found land cover maps created using the Landsat ETM+ data at 30 × 30 m² were more accurate than land cover maps created from the SPOT 4 at 20 × 20 m² and the QuickBird data at 2.44 × 2.44 m² when compared across an urban and rural environment. O'Melveny (2004) also discovered that although increased spatial resolution capabilities of the QuickBird data facilitated visual assessment of surface features, the increased spatial resolution of the QuickBird data resulted in the classification of pixels into a land cover class of shadow that did not occur with the SPOT 4 or Landsat ETM+ data thereby decreasing overall map accuracy.

A QuickBird standard image bundle, which has been the mainstay of geographic information system (GIS) labs, consists of both the panchromatic and multispectral image data for a coincident geographic location and is designed for users with spatial science expertise and computer processing capability. The QuickBird's basic image product is delivered with a 16.5 km by 16.5 km area for a single scene and a 16.5 km by 165.0 km area for a strip. The data are radiometrically calibrated, corrected for platform sensor distortions, and come referenced to a user-defined cartographic projection (Toutin and Cheng 2002). Corrections of digital imagery of the QuickBird stereo imagery were examined by Oki et al. (2003), Noguchi, Iida, and Shimono (2003), and Noguchi et al. (2004), while geometric accuracy and feature compilation were examined by Atak and Altan (2006).

With the advent of high spatial resolution satellite imagery provided in a user-defined cartographic projection; users in the remote sensing community have the ability to use high spatial resolution data as a base map for on-screen digitizing spatially dependent products. Digital satellite data were historically supplied without a geographic reference allowing the user to perform the third step in the five-step sequential process of converting a digital image into a land cover map themselves. With the advent of satellite digital data being supplied with a geographic reference, the user can either assume the geometric correction process to have been completed correctly or validate an images geometric accuracy via a visual assessment. Thereby negating the need to physically perform the geometric correction process yourself that was inherent in traditional land cover classification methodology utilizing low- to mid-spatial resolution MSS and TM data (Campbell 2002).

A QuickBird panchromatic and multispectral image bundle has a stated root mean square error (RMSE) of 14.0 meters exclusive of terrain distortions. This 14.0 meters RMSE accuracy will affect the ability of the user to use the image as a base map and to derive any other spatially referenced products including on-screen digitized point, line and polygon features. This study evaluated the geometric accuracy of DigitalGlobe's user-defined panchromatic and multispectral image bundle product as an independent identity when compared to its stated RMSE of 14.0 meters and to evaluate its utility as an “off the shelf” base map for creating other spatially referenced products.

Methods

A cloud-free QuickBird panchromatic and multispectral image bundle encompassing the city of Nacogdoches, Texas, USA was purchased from DigitalGlobe. Nacogdoches, the county seat for Nacogdoches County in east Texas, encompasses 7386 hectares typified by minimal topographic variability within its city limits, with a minimum elevation of 74 meters, a high point of 152 meters and an average elevation of 111 meters. The image bundle, including both the 60 × 60 cm² panchromatic and 2.44 × 2.44 m² multispectral data, were purchased in GeoTIFF format and were requested to be projected to the Universal Transverse Mercator (UTM) coordinate system, zone 15, NAD83. The data were stacked together in ERDAS Imagine 10.0 (Intergraph Corporation, Madison, AL, USA) to create a panchromatic image and a multispectral image of the city. Once created, the coincident panchromatic and multispectral QuickBird images were merged to create a pan-sharpened multispectral image for analysis (Figure 1).

Figure 1. QuickBird pan-sharpened multispectral data encompassing the city of Nacogdoches, Texas, USA.

To identify locations to evaluate the geometric accuracy of the QuickBird data, the city of Nacogdoches was stratified into 64 equi-sized squares. Within each square, real world features that could easily be located on the ground and in the corresponding QuickBird image were identified. Traffic lines within the city limits were identified as the preferred surface feature to compare QuickBird-identified traffic line coordinates versus each traffic line real world coordinates, since traffic lines can be seen clearly on the QuickBird's high spatial resolution imagery, are easily accessible on the ground, and do not change over time.

Thirty-three individual traffic line locations were located within the city limits for analysis with no more than one traffic line per square to ensure a systematic sample. The predetermined QuickBird UTM easting and northing coordinates of all 33 traffic line locations were obtained by on-screen digitizing their locations using ERDAS Imagine 10.0. Real world UTM easting and northing coordinates of all 33 traffic line locations were obtained in situ with a Trimble GPS Pathfinder® Pro XRS unit (Figure 2). Each GPS coordinate was the average from 30 positions to ensure sub-meter horizontal accuracy observed in the field when using a Trimble GPS Pathfinder® Pro XRS unit (Kindra, Windholz, and Weber 2006). All traffic line digitized UTM easting and northing coordinates and their corresponding GPS-collected UTM easting and northing coordinates were compiled, post processed, and evaluated numerically (Table 1) and visually (Figure 3) to ascertain the difference between QuickBird's identified UTM coordinates and its GPS-derived UTM coordinates. The Euclidean distances between the purchased QuickBird UTM coordinates and the GPS-identified UTM coordinates, identified by calculating the straight line distance between on-screen digitized traffic lines and their corresponding GPS-identified locations, were calculated (Figure 4, Table 2).

Figure 2. Visual collection of QuickBird-derived UTM coordinates versus GPS-derived UTM coordinates for traffic line number 8.

Table 1. Digitized and GPS-collected UTM easting and northing coordinates for all 33 traffic line locations within the city of Nacogdoches, TX, USA.

	Digitized point		GPS point
Point (ID)	Easting (m)	Northing (m)	Easting (m)	Northing (m)
1	340,482.00	3,501,020.00	340,476.88	3,501,022.52
2	341,204.00	3,500,910.00	341,198.02	3,500,907.29
3	340,436.00	3,501,310.00	340,431.02	3,501,311.25
4	345,075.00	3,502,870.00	345,070.73	3,502,868.49
5	345,685.00	3,502,660.00	345,681.72	3,502,665.76
6	342,331.00	3,502,350.00	342,327.66	3,502,349.38
7	343,088.00	3,502,370.00	343,083.59	3,502,367.71
8	340,773.00	3,503,180.00	340,766.89	3,503,182.38
9	343,158.00	3,501,600.00	343,152.70	3,501,600.08
10	344,439.00	3,500,890.00	344,434.70	3,500,890.26
11	344,528.00	3,501,530.00	344,521.57	3,501,522.50
12	346,548.00	3,502,040.00	346,543.51	3,502,044.02
13	346,912.00	3,500,450.00	346,905.31	3,500,449.69
14	344,871.00	3,499,330.00	344,865.98	3,499,322.19
15	344,168.00	3,497,840.00	344,163.67	3,497,836.65
16	346,685.00	3,498,170.00	346,681.84	3,498,169.79
17	346,278.00	3,497,190.00	346,275.20	3,497,187.00
18	344,738.00	3,496,890.00	344,735.82	3,496,887.67
19	344,680.00	3,496,920.00	344,677.27	3,496,920.94
20	343,872.00	3,495,020.00	343,870.40	3,495,015.45
21	343,692.00	3,495,010.00	343,690.66	3,495,010.67
22	341,569.00	3,494,700.00	341,567.36	3,494,695.86
23	342,430.00	3,496,980.00	342,419.66	3,496,985.33
24	341,963.00	3,496,390.00	341,960.84	3,496,391.50
25	342,011.00	3,496,400.00	342,009.27	3,496,401.27
26	340,706.00	3,495,630.00	340,703.08	3,495,624.85
27	340,672.00	3,495,370.00	340,670.96	3,495,371.99
28	341,868.00	3,498,000.00	341,865.23	3,498,003.31
29	342,569.00	3,498,520.00	342,564.67	3,498,512.40
30	343,534.00	3,498,670.00	343,530.14	3,498,665.83
31	343,854.00	3,499,420.00	343,848.44	3,499,415.02
32	345,569.00	3,498,370.00	345,566.35	3,498,375.32
33	345,775.00	3,502,080.00	345,771.50	3,502,076.73

Figure 3. Visual comparison of QuickBird-identified UTM location and GPS-identified UTM location.

Figure 4. Calculating the Euclidean distance between the on-screen digitized UTM coordinates and the GPS-derived UTM coordinates for traffic line number 8.

Table 2. The Euclidean distance between the purchased QuickBird UTM coordinates and the GPS-identified UTM coordinates for all 33 traffic line locations.

	Pixel location minus GPS location
Sample point(ID value)	Easting difference (m)	Northing difference (m)	Euclidean distance (m)
1	5.12	–2.52	5.71
2	5.98	2.71	6.57
3	4.98	–1.25	5.14
4	4.27	1.51	4.53
5	3.28	–5.76	6.63
6	3.34	0.62	3.40
7	4.41	2.29	4.97
8	6.11	–2.38	6.56
9	5.30	–0.08	5.30
10	4.30	–0.26	4.30
11	6.43	7.50	9.88
12	4.49	–4.02	6.03
13	6.69	0.31	6.69
14	5.02	7.81	9.28
15	4.33	3.35	5.47
16	3.16	0.21	3.17
17	2.80	3.00	4.10
18	2.18	2.33	3.19
19	2.73	–0.94	2.88
20	1.60	4.55	4.82
21	1.34	–0.67	1.50
22	1.64	4.14	4.45
23	10.34	–5.33	11.63
24	2.16	–1.50	2.63
25	1.73	–1.27	2.14
26	2.92	5.15	5.92
27	1.04	–1.99	2.25
28	2.77	–3.31	4.32
29	4.33	7.60	8.75
30	3.86	4.17	5.68
31	5.56	4.98	7.46
32	2.65	–5.32	5.94
33	3.50	3.27	4.79
Mean	3.95	0.88	5.34
Standard deviation	1.92	3.74	2.28

To evaluate the geometric accuracy of the purchased QuickBird panchromatic and multispectral image bundle, mean errors, mean absolute errors, RMSE (root mean square errors), and RSME (root square mean errors) were calculated (Table 3). An error is defined by the difference between a QuickBird image coordinate to its corresponding GPS-derived coordinate, on easting and northing, respectively. And when the two are combined, it is the equivalent to Euclidean distance. An error on easting or northing could be positive or negative. Thus, calculation of the mean using absolute errors was also included. A RMSE was calculated by using Equation (1)(1) that squares each error to attain positive values only in calculating the average. It places more weight on large numbers as each error is squared. Another accuracy measurement is RSME calculated with Equation (2)(2) that will result in an absolute value of mean error on easting and on northing, but the combined one will be different from the average Euclidean distance. RSME is more closely comparable to results one may obtain when using a mapping-grade GPS for point data collection (Bettinger and Fei 2010).

Table 3. Geometric accuracy statistics of the QuickBird image based on 33 traffic line locations.

	Mean error	Mean absolute error	RMSE	RSME
	(m)	(m)	(m)	(m)
Easting	3.95	3.95	4.38	3.95
Northing	0.88	3.09	3.79	0.88
Distance	5.34	5.34	5.79	4.05

(1)

(2)

In the two equations above, x_i,image and y_i,image are the UTM coordinates identified on the QuickBird image, whereas x_i,gps and y_i,gps are the UTM coordinates attained from GPS that are considered true coordinates.

These statistics for geometrics accuracy assessment on the purchased QuickBird image bundle was compared with DigitalGlobe's stated RMSE of 14.0 meters for their panchromatic and multispectral image bundle to ascertain if data fell within the expected RMSE for “off the shelf” digital imagery.

Results and discussion

Results, seen visually in Figure 5, indicate that the errors are clustered on the east of the origin with a mean center at (3.95, 0.88) meters that is the mean errors for easting and northing found in Table 2. On average, a point found on the QuickBird image would have its coordinates 3.95 meters shifted to the east of its true location. Although the mean error for northing had a lower value at 0.88 meters because positive and negative values were averaged out, it had a standard deviation of 3.74 meters that is greater than that of the easting at 1.92 meters, indicating greater variation on accuracy for northing. Also found in Figure 5 is the median center at (3.73, 0.79) meters that is not far from the mean center. The median center is where the sum of Euclidean distances to each error point is minimalized. The directional ellipse represents the concentration of the errors centered on the mean center with one standard deviation for easting and northing as the semi-axes of the ellipse (Figure 5). It illustrates the central tendency, the spread, and the directional effect of the errors.

Figure 5. Visual comparison of all 33 QuickBird-identified UTM traffic line coordinates with their corresponding GPS-collected UTM traffic line coordinates.

Table 3 summarizes the geometric accuracy statistics including mean error, mean absolute error, RMSE, and RSME of QuickBird's UTM location compared to its corresponding GPS-collected UTM location. The mean errors are the same as the errors found in Table 2. However, when absolute values were used, the errors were averaged to 3.09 meters for northing, which is close to its RMSE (3.79 meters). The overall RMSE and RSME measured at 5.79 meters and 4.05 meters, respectively. As expected, the RMSE had the highest value among the accuracy statistics as more weights were placed on large errors. All of the values observed were well within DigitalGlobe's stated RMSE positional accuracy of 14.0 meters for a panchromatic and multispectral image bundle. In addition, they are also well within DigitalGlobe's stated RMSE positional accuracy of 6.2 to 15.4 meters for an orthorectified medium (1:12,000) to small scale (1:50,000) imagery product.

Oki et al. (2004) evaluated a DSM (Digital Surface Model) of the QuickBird imagery and found a consistent shift in all points with an average easting component shift of 13.13 meters and the northing component of 2.78 m. With the vectors having the same tendency, the shift was corrected by fixing one point as a ground control point and the easting component shift decreased to 0.74 meters and the northing component decreased to 0.79 m. The easting component shift reported by Oki et al. (2004) was validated in our study which can be seen in Figure 5.

The results of our study indicate that the data purchased were well within the stated positional error for a panchromatic and multispectral image bundle and were acceptable given DigitalGlobe's purchase specifications. However, if a user of high spatial resolution imagery will use the image data as a reference image or as a means to on-screen digitize other spatially referenced data such as point, line, and polygon features for use with nonanniversary spatial data, the image data may need to be registered together or will need to be geometrically corrected to the original data source to ensure a high level of geometric accuracy between data sources.