GEO-CEOS stage 4 validation of the Satellite Image Automatic Mapper lightweight computer program for ESA Earth observation level 2 product generation – Part 2: Validation

Abstract

ESA defines as Earth Observation (EO) Level 2 information product a multi-spectral (MS) image corrected for atmospheric, adjacency, and topographic effects, stacked with its data-derived scene classification map (SCM), whose legend includes quality layers cloud and cloud-shadow. No ESA EO Level 2 product has ever been systematically generated at the ground segment. To fill the information gap from EO big data to ESA EO Level 2 product in compliance with the GEO-CEOS stage 4 validation (Val) guidelines, an off-the-shelf Satellite Image Automatic Mapper (SIAM) lightweight computer program was selected to be validated by independent means on an annual 30 m resolution Web-Enabled Landsat Data (WELD) image composite time-series of the conterminous U.S. (CONUS) for the years 2006 to 2009. The SIAM core is a prior knowledge-based decision tree for MS reflectance space hyperpolyhedralization into static (non-adaptive to data) color names. For the sake of readability, this paper was split into two. The present Part 2—Validation—accomplishes a GEO-CEOS stage 4 Val of the test SIAM-WELD annual map time-series in comparison with a reference 30 m resolution 16-class USGS National Land Cover Data (NLCD) 2006 map. These test and reference map pairs feature the same spatial resolution and spatial extent, but their legends differ and must be harmonized, in agreement with the previous Part 1 - Theory. Conclusions are that SIAM systematically delivers an ESA EO Level 2 SCM product instantiation whose legend complies with the standard 2-level 4-class FAO Land Cover Classification System (LCCS) Dichotomous Phase (DP) taxonomy.

Keywords:

• Artificial intelligence
• binary relationship
• Cartesian product
• cognitive science
• color naming
• connected-component multi-level image labeling
• deductive inference
• earth observation
• land cover taxonomy
• high-level (attentive) and low-level (pre-attentional) vision
• hybrid inference
• image classification
• image segmentation
• inductive inference
• machine learning-from-data
• outcome and process quality indicators
• radiometric calibration
• remote sensing
• surface reflectance
• thematic map comparison
• top-of-atmosphere reflectance
• two-way contingency table
• unsupervised data discretization/vector quantization
• validation

Public Interest Statement

Synonym of scene-from-image reconstruction and understanding, vision is an inherently ill-posed cognitive task; hence, it is difficult to solve and requires a priori knowledge in addition to sensory data to become better posed for numerical solution. In the inherently ill-posed cognitive domain of computer vision, this research was undertaken to validate by independent means a lightweight computer program for prior knowledge-based multi-spectral color naming, called Satellite Image Automatic Mapper (SIAM), eligible for automated near-real-time transformation of large-scale Earth observation (EO) image datasets into European Space Agency (ESA) EO Level 2 information product, never accomplished to date at the ground segment. An original protocol for wall-to-wall thematic map quality assessment without sampling, where legends of the test and reference map pair differ and must be harmonized, was adopted. Conclusions are that SIAM is suitable for systematic ESA EO Level 2 product generation, regarded as necessary not sufficient pre-condition to transform EO big data into timely, comprehensive, and operational EO value-adding information products and services.

Acknowledgments

Prof. Ralph Maughan, Idaho State University, is kindly acknowledged for his contribution as active conservationist and for his willingness to share his invaluable photo archive with the scientific community as well as the general public. Andrea Baraldi thanks Prof. Raphael Capurro, Hochschule der Medien, Germany, and Prof. Christopher Justice, Chair of the Department of Geographical Sciences, University of Maryland, College Park, MD, for their support. Above all, the authors acknowledge the fundamental contribution of Prof. Luigi Boschetti, currently at the Department of Forest, Rangeland and Fire Sciences, University of Idaho, Moscow, Idaho, who conducted by independent means all experiments whose results are proposed in this validation paper. The authors also wish to thank the Editor-in-Chief, Associate Editor, and reviewers for their competence, patience, and willingness to help.

Competing interests

As a source of potential competing interests of professional or financial nature, co-author Andrea Baraldi reports he is the sole developer and intellectual property right (IPR) owner of the Satellite Image Automatic Mapper™ (non-registered trademark) computer program validated by an independent third party in this research and licensed by the one-man-company Baraldi Consultancy in Remote Sensing (siam.andreabaraldi.com) to academia, public institutions, and private companies, eventually free of charge. Throughout his scientific career, Andrea Baraldi has put in place approved plans for managing any potential conflict arising from his IPR ownership of the SIAM™ computer software.

Cover Image

Source: Satellite Image Automatic Mapper (SIAM) multi-level map generated automatically, without human-machine interaction, and in near real-time, specifically, in linear time complexity with image size, from the 30 m resolution annual Web-Enabled Landsat Data (WELD) image composite for the year 2006 of the conterminous U.S. (CONUS), radiometrically calibrated into top-of-atmosphere reflectance (TOARF) values. The multi-level map’s legend, shown at bottom left, is the SIAM intermediate discretization level, consisting of 48 basic color (BC) names reassembled into 19 spectral macro-categories by an independent human expert, according to a proposed hybrid (combined deductive and inductive) eight-step protocol for identification of a categorical variable-pair binary relationship, from a vocabulary of BC names to a dictionary of land cover (LC) class names. No discrete and finite vocabulary of color names, such as SIAM’s, equivalent to a set of mutually exclusive and totally exhaustive (hyper)polyhedra, neither necessarily convex nor connected, in a MS reflectance (hyper)space, should ever be confused with a symbolic (semantic) taxonomy of LC class names in the 4D geospace-time scene-domain. Black lines across the SIAM-WELD 2006 color map represent the boundaries of the 86 Environmental Protection Agency (EPA) Level III ecoregions of the CONUS, suitable for regional-scale statistical stratification required to intercept geospatial non-stationary statistics, typically lost when a global spatial average, e.g., at continental spatial extent, is superimposed on the local computational processes.

Notes

1. Figure 3. Six-stage hybrid (combined deductive and inductive) feedback EO image understanding system (EO-IUS) design, based on a convergence-of-evidence approach consistent with Bayesian naïve classification (Baraldi, 2017). Alternative to inductive feedforward EO-IUS architectures adopted by the RS mainstream, it supports a hierarchical approach to low-level (preliminary, general-purpose, sensor-, application- and user independent) EO image understanding followed by high-level (sensor-, application- and user-specific) EO image understanding (classification), consistent with the standard fully nested Land Cover Classification System (LCCS) taxonomy promoted by the Food and Agriculture Organization (FAO) of the United Nations (Di Gregorio & Jansen, 2000). For the sake of visualization each of the six data processing stages plus stage-zero for EO data pre-processing is depicted as a rectangle with a different color fill. Visual evidence stems from multiple information sources, specifically, numeric color values quantized into categorical color names, local shape, texture and inter-object spatial relationships, either topological or non-topological. An example of preliminary (low-level) general-purpose, user- and application-independent EO image classification taxonomy required by an ESA EO Level 2 Scene Classification Map (SCM) product is the 3-level 8-class FAO LCCS Dichotomous Phase (DP) legend, in addition to quality layers such as cloud and cloud-shadow. High-level EO image classification is user- and application-specific, where a thematic map product of Level 3 or superior is provided with a legend consistent with the FAO LCCS Modular Hierarchical Phase (MHP) taxonomy (Di Gregorio & Jansen, 2000); refer to Figure 3 in the Part 1 of this paper. Acronym SIAM stays for Satellite Image Automatic Mapper (SIAM), a lightweight computer program for MS reflectance space hyperpolyhedralization into a static vocabulary of MS color names, superpixel detection and vector quantization (VQ) quality assessment (Baraldi, 2017; Baraldi et al., 2006, 2010a, 2010b, 2010c; Baraldi, 2011; Baraldi & Boschetti, 2012a, 2012b; Baraldi et al., 2013; 2016; Baraldi & Humber, 2015).

2. Figure 9. Examples of geographic locations mapped as vegetation classes “Scrub/Shrub” (SS) or “Grassland/Herbaceous” (GH) in the USGS NLCD 2006 reference map (refer to Table 1) and predominantly as bare soil spectral categories (sbS_1, SmS_1, aS) in the SIAM-WELD 2006 test map (refer to Table 2), as pointed out in Table 8. The SIAM’s color names sbS_1, SmS_1, and aS mean that, from space, with a pixel size of 30 m × 30 m = 900 m², the contribution of sparse vegetation, rangeland, cheatgrass, dry long grass or short grass as foreground, mixed with a background of sand, clay, or rocks, like those shown in these pictures, becomes extremely difficult to detect, especially if a hard (crisp, defuzzified) label rather than a set of fuzzy class labels must be provided as the output product. (a) Sublette, WY, Rangeland, 42° 51ʹ 37” N, 109° 43ʹ 7” W. Copyright Ralph Maughan, Idaho State Univ. Reproduced with permission of the author. Acquisition date: 6/16/2011. [Online]. Available: http://www.panoramio.com (accessed on 24 February 2013). (b) Twin Falls, ID, Ripening cheatgrass infestation, 42° 23ʹ 52” N, 114° 21ʹ 9” W. Copyright Ralph Maughan, Idaho State Univ. Reproduced with permission of the author. Acquisition date: April 2010? [Online]. Available: http://www.panoramio.com (accessed on 24 February 2013). (c) Overton, NV, 36° 25ʹ 42” N, 114° 27ʹ 21” W. Copyright Ralph Maughan, Idaho State Univ. Reproduced with permission of the author. Acquisition date: 2/11/2009. [Online]. Available: http://www.panoramio.com (accessed on 24 February 2013). (d) San Juan, UT, 37° 16ʹ 43” N, 109° 40ʹ 27” W. Copyright Ralph Maughan, Idaho State Univ. Reproduced with permission of the author. Acquisition date: 3/4/2009. [Online]. Available: http://www.panoramio.com (accessed on 24 February 2013). (e) Springerville, AZ, Volcanic Field, 34° 15ʹ 6” N, 109° 21ʹ 9” W. Copyright Ralph Maughan, Idaho State Univ. Reproduced with permission of the author. Acquisition date: 3/3/2009. [Online]. Available: http://www.panoramio.com (accessed on 24 February 2013). (f) Esmeralda, NV, 38° 1ʹ 40” N, 117° 43ʹ 21” W. Copyright Ralph Maughan, Idaho State Univ. Reproduced with permission of the author. Acquisition date: 4/22/2010. [Online]. Available: http://www.panoramio.com (accessed on 24 February 2013).

Additional information

Funding

To accomplish this work, Andrea Baraldi was supported in part by the National Aeronautics and Space Administration (NASA) under Grant No. NNX07AV19G issued through the Earth Science Division of the Science Mission Directorate. Andrea Baraldi, Dirk Tiede, and Stefan Lang were supported in part by the Austrian Science Fund (FWF) through the Doctoral College GIScience (DK W1237- N23).

Notes on contributors

Andrea Baraldi

Andrea Baraldi (Laurea in Electronic Engineering, Univ. Bologna, 1989; Master in Software Engineering, Univ. Padua, 1994; PhD in Agricultural Sciences, Univ. Naples Federico II, 2017) has held research positions at the Italian Space Agency (2018-to date); Dept. Geoinformatics, Univ. Salzburg, Austria (2014–2017); Dept. Geographical Sciences, Univ. Maryland, College Park, MD (2010–2013); European Commission Joint Research Centre (2000–2002; 2005–2009); International Computer Science Institute, Berkeley, CA (1997–1999); European Space Agency Research Institute (1991–1993); Italian National Research Council (1989, 1994–1996, 2003–2004). In 2009, he founded Baraldi Consultancy in Remote Sensing. He was appointed with a Senior Scientist Fellowship at the German Aerospace Center, Oberpfaffenhofen, Germany (February 2014) and was a visiting scientist at the Ben Gurion Univ. of the Negev, Sde Boker, Israel (February 2015). His main interests center on the research and technological development of automatic near-real-time spaceborne/airborne image preprocessing and understanding systems in operating mode, consistent with human visual perception. Dr. Baraldi served as an Associate Editor of the IEEE TRANS. NEURAL NETWORKS from 2001 to 2006.