Land cover 2.0

ABSTRACT

Interest in knowing more about the Earth’s land cover and how it has changed over time motivated the mission and sensor design of early terrestrial remote sensing systems. Rapid developments in computer hardware and software in the last four decades have greatly increased the capacity for satellite data acquisition, downlink, dissemination, and end user science and applications. In 1992, Townshend reviewed the state of land cover mapping using Earth observation data at a pivotal point in time and in the context of years of research and practical experience with Landsat Thematic Mapper (TM), Satellite Pour l’Observation de la Terre (SPOT) High Resolution Visible (HRV) and Advanced Very-High-Resolution Radiometer (AVHRR) data, demonstrating the opportunities and information content possible with increased spatial, spectral, and temporal resolutions. Townshend characterized the state-of-the-art for land cover at that time, identified trends, and shared insights on research directions. Now, on the 25th anniversary of Townshend’s important work, given numerous advances and emerging trends, we revisit the status of land cover mapping with Earth observation data. We posit that a new era of land cover analysis – Land Cover 2.0 – has emerged, enabled by free and open access data, analysis ready data, high performance computing, and rapidly developing data processing and analysis capabilities. Herein we characterize this new era in land cover information, highlighting institutional, computational, as well as theoretical developments that have occurred over the past 25 years, identifying the key issues and opportunities that have emerged. We conclude that Land Cover 2.0 offers efficiencies in information generation that will result in a proliferation of land cover products, reinforcing the need for transparency regarding the input data and algorithms used as well as adoption, implementation, and communication of rigorous accuracy assessment protocols. Further, land cover and land change assessments are no longer independent activities. Knowledge of land change is available to inform and enrich land cover generation.

1. Introduction

Land cover is a critical descriptor of the Earth’s terrestrial surface and spatially explicit land cover information and summary statistics are a requirement for a range of natural resource management decisions at local, national, and international scales. Land cover informs the functional relationship between terrain, climate and soils, providing biophysical insights into the environment and drivers of change (DeFries et al. 2002; Townshend et al. 2012; Franklin and Wulder 2002; Andrew, Wulder, and Nelson 2014). Land cover is typically described as a set of hierarchical classes, each denoting the dominant biotic and abiotic assemblages occupying the Earths’ surface (e.g., Grekousis, Mountrakis, and Kavouras 2015). Spatially-explicit land cover data are important for characterizing anthropogenic activity and biogeographical and eco-climatic diversity (Loveland et al. 2000; Turner, Lambin, and Reenberg 2007).

The free and open availability of global coverage Earth observation data systematically collected and archived by space agencies, such as the European Space Agency (ESA) Sentinel series, and the U.S. National Ocean and Atmospheric Administration (NOAA) Advanced very-high-resolution radiometer (AVHRR), the National Aerospace Science Administration (NASA) Moderate Resolution Imaging Spectroradiometer (MODIS), and NASA and the United States Geological Survey (USGS) Landsat series, have enabled the rapid development of private and public data commons. In the last decade, rapid increases in computing power (and a significant reduction in associated costs), combined with free and open satellite data, have led to a migration of programs facilitating broad area land cover mapping from being exclusively the domain of public agencies and a small number of well-financed research groups. Cloud-based computing systems increasingly enable users to process data and develop land cover products without significant investments in computing infrastructure (e.g., Gorelick et al. 2017; Yang et al. 2017a). At the same time, there is increasing inclusion of these data (and derived products), especially Landsat, in national and international activities and programs (e.g., GCOS 2016; Gong et al. 2013; Hansen et al. 2013; White et al. 2014; Pettorelli et al. 2016).

The first terrestrial remote sensing data was provided by the Landsat (1972) and NOAA AVHRR (1978) programs. Initially, large area (i.e., regional to global coverage) land cover products were developed at coarse spatial resolutions using AVHRR data because the data were free and the small number of coarse spatial resolution (approximately 1.1 km data at nadir) bands associated with these data resulted in data volumes that could be stored and manipulated over large areas (Hansen et al. 2000; Loveland et al. 2000; Cihlar 2000; Franklin and Wulder 2002). The advent of the MODIS Terra and Aqua sensors with a design heritage from AVHRR and Landsat (Justice et al. 1998) lead to the development of a systematic global annual 500 m land cover product (Friedl et al. 2002), which is currently being reprocessed and updated for a fifth time using reprocessed MODIS data inputs. While AVHRR and MODIS afforded land cover classification at a global scale with freely available data, the relatively large pixels limited the applicability of the maps for management or planning activities (Cihlar 2000; Franklin and Wulder 2002), or to generate information at a level of detail informative to national or international programs (e.g., sub-hectare minimum mappable units, Patenaude, Milne, and Dawson 2005; Rosenqvist et al. 2003).

Land cover mapping research was undertaken from the first availability of Landsat data (Anderson et al. 1976), but was typically constrained to limited spatial extents because of data cost and (at the time) limitations to data storage (Cihlar 2000), with some notable exceptions including, for example, Landsat mapping of deforestation in the Amazon (Skole and Tucker 1993). As reviewed in Franklin and Wulder (2002), circa 2000 there were a number of large area land cover mapping projects implemented, with noteworthy examples including for the United Kingdom (Fuller, Groom, and Jones 1994), Europe (Bossard, Feranec, and Otahel 2000), the United States (Vogelmann et al. 1998), and Canada (Wulder et al. 2003; Wulder et al. 2008). Building upon these examples and experience, Cihlar (2000) summarised the state-of-the-art in the remote sensing of land cover at the turn of the century and offered an early understanding of how an end-to-end processing stream could be designed to support land cover mapping. Based upon experience in using MODIS data for monitoring, Townshend and Justice (2002) shared lessons learned (including science team involvement, reprocessing, algorithm evolution, systematic quality assessment, validation, and data systems) that continue to inform terrestrial monitoring activities and underpin concepts around the need for product data collections with periodic reprocessing, and the need for analysis-ready data products.

Despite the identification of end-to-end processing workflows, limitations associated with data costs and availability persisted and reduced the capacity and ambition of practitioners. Computing storage and processing power continued to increase as did the number, options, and functionality offered by commercial software packages and increasing acumen in image processing. The policy change by the USGS to provide all the Landsat data for free via the internet significantly enabled capacity to meet user ambition (Woodcock et al. 2008). Despite early issues with Landsat data access via the internet in less connected parts of the world (Roy et al. 2010a), the free data access policy was rapidly seen as a pivotal moment for land cover mapping and monitoring at a spatial resolution that was capable of capturing human influence (Wulder et al. 2012).

For planning and management purposes, knowledge of how humans use the land is often more important than knowledge of land cover. Land use describes the social, economic, and cultural utility of the land (Turner 1997), and is known to alter how ecosystems function (DeFries, Foley, and Asner 2004). Land use impacts ecosystem services including biodiversity (Tuanmu and Jetz 2014; Andrew, Wulder, and Nelson 2014), mediating greenhouse gas emissions (DeFries et al. 2002; Li, W. et al. 2017) and the temporal sequences of deforestation (Müller et al. 2016), as well as harvesting and regeneration (White et al. 2017). Critical to capturing and relating phenomena at human scales is the spatial resolution of the imagery (Townshend and Justice 1988; Small and Sousa 2016). Can the phenomena be captured using many pixels, or are the phenomena subsumed within a pixel? Strahler, Woodcock, and Smith (1986) outline the relationship between spatial resolution, object detectability, and information content. Commercial, high-spatial resolution, satellite data have only been available since 1999 and thus, despite the increasing number of sensors available acquiring data at a high (<10 m) and medium (10-100m) spatial resolution (Belward and Skøien 2015) no other satellite(s) program offers the temporal depth, radiometric calibration, and open access of the Landsat program (Roy et al. 2014; Wulder et al. 2016). As the Landsat-like ESA Sentinel 2 satellites (Drusch et al. 2012) acquire more data to archive, additional options emerge, especially given cross-calibration with Landsat (Zhang et al. In review) allowing for a shared archive of free and characterized multi-sensor data for time series analysis. The relative spatial and spectral compatibility of the Sentinel 2 and Landsat systems will enable a virtual constellation (Wulder et al. 2015) allowing the integration of their data to meet science and application needs and providing, for the first time, near weekly cloud-free medium resolution surface global coverage observations (Li and Roy 2017). The compatibility of Sentinel 2 with Landsat allows for new measures from Sentinel 2 to be supplemented by Landsat (or vice versa) and for utilization of the historic archive of Landsat to inform time series with Sentinel 2.

In this paper we review the key changes to land cover mapping over the past quarter century using Townshend (1992) as a benchmark. We reflect on these advances and the impacts on land cover generation, summarize the previous outlook for land cover, then offer insights on the current status, with some thoughts on outstanding issues and emerging trends. Building upon comparisons with conditions in the early 1990’s to present, we overview the current status and future opportunities while offering a conceptualization of a Land Cover 2.0.

2. Townshend: land cover

Townshend (1992) wrote a seminal paper describing the derivation of land cover information from Earth observation data. His paper was written at a time when image classification using visual photo interpretation was no longer considered a viable option, as a result of increasing labour costs, long turn-around times for outputs, and human subjectivity. At the same time, European remote sensing efforts were increasing with the launch of the first of a series of French SPOT satellites in 1986 (Courtois and Traizet 1986) and other satellite systems (Belward and Skøien 2015). At the time, land cover mapping efforts using remotely sensed data were disparate and independent. Townshend (1992) suggested that the provision of land cover information from remote sensing technology was in its infancy, and that the reliable provision of operational land cover information was a vision for the future. Through his synthesis, Townshend (1992) examined efforts that were being undertaken to generate more unified (via increasing standardization of common concepts and methods) land cover assessments, which in turn would herald an array of new opportunities globally.

Townshend (1992) highlighted factors that limited the derivation of land cover information from remotely sensed data. First and foremost were a lack of satellite data options, which included the number of satellites, the limited types of sensors and their related spectral and spatial characteristics. Limited data options resulted in an inability to distinguish land cover categories of interest due to a lack of spectral resolution (e.g., overly wide band-passes). Opportunities were noted regarding the use of multi-temporal analysis to derive phenological variation and reduce cloud impacts. Townshend forewarned that the assumption that a single correct class is appropriate for every pixel is not necessarily possible, especially at lower spatial resolution. He also suggested that radar data (specifically ERS-1) could help deal with issues such as persistent cloud cover and that advances in radar signal processing could result in improved land cover discrimination, especially over time. This role of radar for land cover mapping has generally been unfulfilled, with backscatter proving to be a complex response to understand and model (Dobson, Ulaby, and Pierce 1995). The provision of radar data in a systematically and consistently processed open access form such as by the recently launched Sentinel-1A and 1B C-band systems (Torres et al. 2012) is expected to lead to greater operational uptake. The second limiting factor identified by Townshend (1992) was image processing tools. New image processing tools were required that accounted for spatial variability, offering enhancements over per-pixel classifiers incorporating knowledge based information, contextual classifiers, and addressing the impact of mixed pixels.

Three main trends were noted by Townshend (1992). First, research into land cover mapping was undertaken by small, independent research groups with little integration between groups, and was constrained by limited data storage and computing power. Second significant data pre-processing (e.g., geometric and radiometric correction) and time consuming routines, such as reading data from tape archives, were required before land cover classification could begin. Lastly, and perhaps most importantly, Townshend (1992) highlighted a general need to refocus interest in the activity of land cover monitoring through time, and expanding beyond the assessment of land cover itself to include land cover dynamics and land use, with a special focus on the information needs and requirements of the end-user community.

Townshend (1992) identified that for land cover mapping and monitoring, the spatial resolution and related detail of Landsat TM (30 m) and SPOT-HRV (20 m) were essential. Recognising the increasing availability of higher spatial resolution from both North American and European programs, Townshend proposed that increased spatial resolution would not only improve discrimination between target land cover classes and reduce mixed pixels for certain land cover catagories, but also improve cloud screening (as more clouds would be visible, not subsumed within pixels), making inter- and intra-annual monitoring more feasible. Concurrent with increasing use of Landsat TM and SPOT-HRV data at that time was a resurgence in the use of coarse spatial resolution data, such as AVHRR, for land cover mapping. AVHRR had a daily revisit promoting compositing approaches (Holben 1986) leading to recognition of the value of having increased temporal frequency for land cover discrimination (Hansen et al. 2000). The linkage between easily accessible data and uptake is well known. For example, the number of images delivered in the best year prior to the Landsat archive being opened in 2008 was about 25,000 (Wulder and Coops 2014). In contrast, at present, over 1 million images are downloaded per month, for a total of over 68 million downloads since 2008 (USGS 2018). Arguably, the lack of uptake of SPOT imagery over the previous two decades is linked to data costs and the absence of a systematic collection to archive, even though the data is of high quality.

3. Land cover 2.0

3.1. Key concepts

Today, 25 years after Townshend’s (1992) review, the rationale for land cover mapping remains relevant and now includes clearly-defined information needs associated with understanding and mitigating climate change, monitoring habitats, deforestation, land abandonment, land conversion to agriculture, and urbanization. Users at all levels recognise that data used to address these needs should be accurate, synoptic, spatially and thematically detailed, and regularly updated in a consistent and systematic manner. In the past 25 years, significant changes in land cover mapping and monitoring have occurred. Here we draw a parallel with how the internet has evolved over time: from early development and exploration in the 1990s where users could view and download relatively static content, to today and the era of ‘Web 2.0’ where the content may change in near real time and users can interact directly with the content and participate in transparent and collaborative activities.

Various science and application advances have combined to form a newly integrated change detection and land cover mapping paradigm that we term Land Cover 2.0. This new paradigm for generating land cover products captures current understanding of automated data processing of analysis-ready and open-access data using flexible classification algorithms that result in timely and accurate land cover maps. Land Cover 2.0 can be characterized as empowering users to bring algorithms to analysis ready data, to access and include supplemental data to improve and validate classification results, and enable generation of classification maps with more user-specific legends given appropriate training data. In contrast to previous practices (Table 1), this emerging paradigm is enabled by open access to satellite imagery, new image processing techniques focused both on land cover classification, as well as land cover change that allows for increased flexibility in map training and production. Critical to Land Cover 2.0 is the capacity for integrated detection of change to inform and guide land cover mapping outcomes. The presence of change (e.g., wildfire), when combined with ecological knowledge or expectation of a given process post-change, can provide for a priori expectations of land cover. For example, treed areas that are burned by wildfire typically transition initially to non-treed herbaceous vegetation cover, followed by a succession of other classes, ultimately returning to a treed class, provided no change in land use or condition has occurred. As such, incorporation of change information into the classification process allows for insights regarding expected land cover class transitions related to successional process, and likewise provides a mechanism to identify illogical class transitions (e.g., tree to water). It should be noted that neither Land Cover 1.0 nor 2.0 are a singular set of methods, but rather an overarching approach, aimed at capturing and relating the state-of-art for different eras of land cover mapping.

Table 1. Comparison of general characteristics of previous and current land cover paradigms.

Land Cover 1.0	Land Cover 2.0
Limited data options in terms of spectral, spatial, and temporal resolutions	A broad range of data options
Limited data availability	Mass data availably (free, open archives)
Significant time and resources required to pre-process Earth observation data	Standardized, analysis-ready data products to greatly reduce or eliminate the need for pre-processing
Pre-processing and classification approaches developed on an ad hoc basis, often unique to the group developing the product or to the geographic area	Increasingly standardized and transparent approaches
Classification algorithms that are simplistic and make assumptions regarding the nature of the input data	Advanced algorithms that are flexible and make fewer assumptions regarding input data
High cost of data and processing	Low cost of data and processing
Single date representation, state not trends	Time series informed land cover, allows for both state and trends
Lack of agreement between land cover and land change products	Integration of land change information into land cover product development
Products not systematically generated	Systematically generated
Static products	More frequent products
Periodically updated, not necessarily temporally integrated	Frequently updated and temporally integrated
Difficulties in disseminating derived products	Ease of access and sharing of derived products

3.2. Elements of land cover 2.0

Land Cover 2.0 is a flexible and iterative approach to deriving land cover information. In Figure 1, we indicate the thematic elements of Land Cover 2.0; however, this conceptualization is not intended to be prescriptive or exhaustive, but rather illustrates the elements of an overall approach to land cover classification. These elements include: (i) a clearly defined information need; (ii) the definition of appropriate and realistic land cover legends; (iii) the acquisition of independent calibration and validation data; (iv) a standardised image analysis approach that incorporates image preprocessing innovations, derivation of spectral time series metrics, and integration of auxiliary data; (v) the use of classifiers able to deal with big datasets; (vi) post-classification error reduction and iterated accuracy assessment; and (v) innovative product delivery methods. Fundamentally, the element that most strongly distinguishes Land Cover 2.0 is the critical role of change information, which can be fully integrated into the land cover classification process.

Figure 1. Conceptual diagram contrasting Land Cover 1.0 and Land Cover 2.0 (see text for elaboration). Both paradigms are driven by clearly articulated information needs, which in turn are used to define desired products and traits. Chiefly, Land Cover 2.0 shows the integration of land cover and change detection, with opportunities to iterate and improve products prior to digital delivery.

3.2.1. A clearly defined information need

As a prerequisite to a successful land cover mapping project, it is important to be clear of the information needs of the end users. Is the product intended to support climate change mitigation or for monitoring habitats or land conversion to agriculture? Often land cover products fulfill a variety of information needs related to management, policy, and science (GCOS 2011; GCOS 2016). How a land cover product will be used and applied should guide decisions, ranging from the selection of land cover classes and the spatial and temporal extent of the map products, to how the outcomes are delivered. Usually the spatial and temporal extent is straightforward to define, although there may be limitations resulting from data availability. For example, outside of a few regions, the availability of historic Landsat data can be limited (Wulder et al. 2016). Also of regional concern is the presence and persistence of cloud cover, which can significantly reduce the number of surface observations available for land cover classification (Kovalskyy and Roy 2013).

3.2.2. Land cover legend

The information needs motivating the development of the land cover information should define the legend to be utilized. The land cover class definitions should be mutually exclusive and comprehensive (or include an unknown class) (Anderson et al. 1976). The classification scheme should reflect the information content of the data to be classified (Running et al. 1995; Tsendbazar, De Bruin, and Herold 2015). For example, even with comprehensive amounts of high quality training data, classifications become increasingly difficult to produce reliably as the number of classes increase. The spectral and spatial resolution of the satellite data may constrain the adoption of an appropriate class legend and when considering small areas and single images, it has been common practice to quantify class separability as part of the classification approach. The advent of large-area classifications, where the same land cover class may have quite different spectral signature geographically (Henderson 1976; Woodcock et al. 2001), and the use of multi-temporal satellite data to capture between-class phenological and other surface condition differences (DeFries, Hansen, and Townshend 1995; Zhang and Roy 2016), have reduced the applicability of this kind of class separability analysis. For many projects, there may be a programmatic and predetermined class legend, and the suitability of the classification scheme is inferred from class-specific accuracy assessment procedures. For example, the United States Department of Agriculture (USDA) 30 m Cropland Data Layer (CDL) provides an annual conterminous United States land cover product with more than 100 classes but with a high overall classification accuracy of about 85% due to the significant amounts of training data that are used (Boryan et al. 2011; Johnson and Mueller 2010). Standard classification schemes, such as the Land Cover Classification System (LCCS) (Di Gregorio 2005), may provide utility as pre-developed and portable classification scheme and be used when an information needs specific classification is not required.

The increasing availability of land cover products has logically led to some users comparing land cover maps as a means of better understanding their differences and strengths (Wulder et al. 2004; Herold et al. 2008) and to subsequently apply these combined products to meet science and application needs (Tuanmu and Jetz 2014; Jung et al. 2006). Map inter-comparisons however inform on the agreement and differences between products rather than provide insights on the classification accuracy. The degree of classification correspondence among land cover maps can be as much due to biases in the classification approaches, or data used, as much as it is a statement about accuracy. Despite this, general agreement between maps may highlight which classes are most likely to be reliably mapped or are dominant on a landscape (Herold et al. 2008). While inter-comparison of land cover maps does not provide insights into accuracy directly, it can be useful if map harmonisation is the goal. For example, to compare individual land cover classes in a given region derived from multiple sensor systems or at different spatial scales. These considerations highlight the utility of a robust accuracy assessment, while also buttressing the need for communication of actual methods applied in a given land cover mapping project.

3.2.3. Calibration and validation data

The majority of land cover classification approaches are supervised and require calibration (training) data composed of reference samples of known land cover classes. Ideally, classification accuracy is then quantified via comparison of the output land cover classification with an independent set of validation data. As calibration and validation data can be expensive and/or difficult to acquire, alternative approaches such as cross-validation (Friedl et al. 2010) or bootstrapping have also been applied (Champagne et al. 2014). Calibration and validation data may be acquired via field sampling (Nusser and Klaas 2003), visual assessment of higher spatial resolution remotely sensed data (Morisette et al. 2003; Wulder et al. 2006; Pengra et al. 2015; Midekisa et al. 2017), pre-existing, yet scale appropriate, non-remote sensing sources (Inglada et al. 2017), or from other land cover classification products that are subjected to filtering (Hansen et al. 2008; Gray and Song 2013; Wessels et al. 2016; Zhang and Roy 2017; Hermosilla et al. 2018). New ways of obtaining calibration and validation data are currently under investigation, including adaptive learning based training data collection and classification (Egorov et al. 2015) and use of volunteered geographic information (VGI; Foody et al. 2015) such as open street maps (Yang, D. et al. 2017), crowdsourcing (Geo-Wiki, Fritz et al. 2012; 2017), or volunteered photographs (Tracewski, Bastin, and Fonte 2017). Approaches, protocols and sampling designs for the collection of calibration and validation data have matured over time (e.g. Janssen et al. 1994; Foody 2002, Stehman 2009; Stehman 2013) with increased emphasis on calibration data needs relative to specific data inputs (i.e. time series, Pelletier et al. 2017) and classification approaches (e.g. Millard and Richardson 2015; Mellor et al. 2015; Scott et al. 2017).

3.2.4. Standardised image analysis approaches

3.2.4.1. Data and processing

Satellite data processing and land cover classification can be undertaken on single desktop computers, on local networks of computers, on high performance computing systems that enable parallel processing, and on cloud computing systems that use a network of remote servers accessible via the internet. Systematically generated large-area land cover products such as the global MODIS land cover product are generated conventionally on dedicated non-public high performance government funded computing systems (Justice et al. 2002). In recognition of the need for accredited science users to ‘bring their application to the data’, many government agencies are building high performance computing systems that host large satellite data collections, such as the NASA Earth Exchange (NEX) (Nemani et al. 2011) and the Australian Geoscience Data Cube (AGDC) (Lewis et al. 2017), to enable large volume processing. Initiatives such as the Committee on Earth Observation Satellites (CEOS), have suggested portable open-access data cube architectures (e.g., CEOS 2017) with the aim of facilitating the use of EO data and sharing knowledge across the EO global community.

With the advent of well calibrated image data, analysis ready data, and higher-level derived products, the need to undertake pre-processing steps has been significantly reduced. For large area land cover classification, the satellite data should be defined in a common map projection and tiling scheme, requiring that the data are reprojected and resampled, which raises issues concerning appropriate projections and resampling methods (Eidenshink and Faudeen 1994; Roy et al. 2010b; Roy et al. 2016a; Li, Z. et al. 2017). Correction for atmospheric effects is required when classification or change detection involve large areas or multi-temporal data (Song et al. 2001). Previously, relative normalization of reflectance or radiance from different image was often undertaken, and based typically on statistical calibration among images at pseudo-invariant features identified in spatially overlapping acquisitions (Schott, Salvaggio, and Volchock 1988; Schroeder et al. 2006). The use of surface reflectance derived from calibrated sensor data has largely reduced the need for relative normalization approaches. Atmospheric correction of the at-sensor reflectance or radiance to surface equivalents has evolved from an ad hoc and largely empirical pre-processing step (Chavez 1988), to an automated standard process using radiative transfer algorithms and atmospheric characterization data (Vermote et al. 2016). Despite this, some researchers do not advocate the classification of atmospherically corrected data and prefer not to use the shorter wavelength visible bands (e.g. Hansen et al. 2011), which are particularly sensitive to atmospheric effects (Ju et al. 2012). Bi-directional reflectance variations imposed by variations in the viewing and solar geometry occur over most terrestrial surfaces and for classification purposes are considered as a source of noise unless sensors with specific directional sensing capabilities are used (de Colstoun and Walthall 2006). Typically, bi-directional reflectance variations are removed from wide field of view data, such as MODIS, before classification (Friedl et al. 2002). In narrow field of view data, such as Landsat or Sentinel-2 (Gascon et al. 2017), these effects have been minimized using statistical correction methods (Toivonen et al. 2006; Hansen et al. 2008) and more recently using more physically-based methods (Roy et al. 2016b, 2017). Optically thick clouds, cirrus, haze, and smoke can be difficult to detect reliably and may be detected as part of the land cover classification process (Hansen et al. 2008; Lindquist et al. 2008) or removed prior to classification using cloud masking algorithms that are labelled directly in the input data (Frey et al. 2008; Foga et al. 2017; Zhu and Woodcock 2014a; Zhu, Wang, and Woodcock 2015). Temporal composites that attempt to select the ‘best’ available pixel sensed over a given time period have been developed (Holben 1986; Roy 1997; Roy et al. 2010b; Griffiths et al. 2013; White et al. 2014) to reduce the influence of atmospheric contamination, clouds and in some cases phenological variations (Frantz et al. 2017).

3.2.4.2. Time series metrics

Dimensionality reduction techniques have been applied in an attempt to maximize the information content and minimize noise prior to classification, including linear techniques such as principal component analysis (Collins and Woodcock 1996) and more recently non-linear techniques (Journaux, Foucherot, and Gouton 2006; Yan and Roy 2015). However, these approaches cannot usually handle missing data and are computationally expensive (Yan and Roy 2015). More commonly, spectral band indices and spectral bands are classified. The current state of the practice for large area multi-temporal land cover classification is to derive spectral band and band ratio metrics from the image time series. The choice of time series metrics, usually the maximum and quartile values of spectral indices and spectral bands are often used, as these metrics are robust to missing data and phenological variations (DeFries, Hansen, and Townshend 1995; Friedl et al. 2010). Data fusion approaches, such as those based upon blending of MODIS and Landsat data (Gao et al. 2006; Hilker et al. 2009) may be implemented to produce synthetic gap-free composites for classification purposes (Senf et al. 2015). As demonstrated by Azzari and Lobel (2017), time series based temporal metrics (e.g., change magnitude; persistence of change, among others) that inform on pixel level trends in spectral values can be informative to land cover classification. Similarly, Hermosilla et al. (2015b) used time series metrics as inputs to classify a range of forest change classes (fire, harvest, among others).

3.2.5. Integration of ancillary data

Ancillary data such as slope and aspect derived from an elevation model can improve image classification outcomes (Strahler, Logan, and Bryant 1978; Rogan et al. 2003) with recent examples summarized in Zhu et al. (2016). Midekisa et al. (2017) demonstrated the use of night time lights as an ancillary data input, to help classify urbanization and extent of built-up lands. GIS-derived information, such as distance to nearest road (or populated place), also provide valuable discriminatory information for land cover mapping (Rogan and Chen 2004). Additional ancillary data options are evident in the land cover literature (e.g., as reviewed in Franklin and Wulder 2002; Khatami, Mountrakis, and Stehman 2016) and are often straightforward to incorporate given the current processing and classification environment. We note that early classification algorithms often could not handle ancillary data due to assumptions regarding the distribution or nature (i.e. categorical versus continuous) of the data inputs.

Given an increase in cumulative evidence from decades of satellite measures and products, it is expected that there will be a greater use of a priori information to inform land cover classifications and aid with pixel screening activities. The judicious application of such a priori information can be useful for reducing class confusion in environments that are known to be problematic (e.g., mountainous areas) and for improving classification accuracy.

3.2.6. Advanced classification algorithms

Twenty five years ago, unsupervised algorithms such as k-Means, ISODATA, and supervised parametric classifiers (e.g., Parallelepiped, Minimum Distance, Maximum Likelihood, Linear Discriminant) were commonly used for land cover classification (Cihlar 2000; Franklin and Wulder 2002; Grekousis, Mountrakis, and Kavouras 2015). Unsupervised classifiers required minimal user intervention except for specification of how many classes were desired and then manual assignment of class labels to automatically generated output pixel clusters. Early supervised classifiers such as parallelepiped and minimum distance were easy to implement but not particularly reliable; the former could only classify pixels whose values fell in specific ranges, and the latter assumed spectral variability was the same across all spectral bands. Other early supervised classification algorithms were parametric and made assumptions regarding the statistical distribution of the input data, typically assuming a normal distribution (Swain and Davis 1978). Notably, the maximum likelihood classifier assumes a normal distribution, requires sufficient ground truth data to enable the estimation of the mean vector and variance/co-variance matrix of the population, and also has a need for sufficient training data to invert the covariance matrices which makes maximum likelihood unattractive for large data sets. Satellite data is not expected to have values that are normally disturbed nor are classes expected to have entirely distinct features spaces or the same variance structure (within or between spectral bands). As a result, classifiers that are robust to these known data characteristics are preferred.

Over the past 25 years non-parametric supervised classifiers have grown in utility to accommodate complex feature space relationships among classes, including artificial neural networks (ANN), support vector machines (SVM), and decision tree classifiers, and have been shown to be more efficient and accurate than parametric classifiers (Pal and Mather 2005; Breiman et al. 1984; Mannan, Roy, and Ray 1998). Non-parametric classifiers make no assumptions regarding the distribution of the input data, but can be sensitive to over-fitting of the classifier to the training data. More recently ensemble non-parametric classifiers that use a different random subset (bootstrap) of the training data and are applied many times were developed and have been widely adopted for satellite classification (Doan and Foody 2007; Hansen et al. 2008; Freidl et al. 2010). Random Forests is emerging as a commonly applied algorithm for land cover classification (Gisalson et al. 2006; Rodriguez-Galiano et al. 2012; Belgiu and Drăguţ 2016), particularly for large area applications (e.g., Pelletier et al. 2016; Zhang and Roy 2017). Random Forests are a form of decision tree classifier, operating in an ensemble, that use randomly selected predictor variables for each tree, as well as randomly selected training data subsets (Breiman 2001). Other supervised classifiers are also emerging, notably the use of deep learning algorithms that are being applied for land cover classification (e.g., Azzari and Lobell 2017). All non-parametric classifiers require appropriate configuration prior to their application and the optimal configuration of these classifiers has itself become the subject of investigation (e.g., Pelletier et al. 2016; Pacifici et al. 2008).

3.2.7. Post-classification error reduction

Post-classification approaches to improve classification accuracy by application of median filters to replace isolated noisy (salt-and-pepper) classified pixels are commonly implemented. The examination of posteriori class probabilities also provide insights to improve the image classification results (Beaubien et al. 1999). Moreover, when time series land cover maps are generated, post-classification error reduction approaches that incorporate knowledge of land cover dynamics and ecological processes are helpful (Gómez, White, and Wulder 2016). For example, rules restricting land cover transitions have been commonly applied (e.g., Liu and Zhou 2004). Other methods rely on the spectral evidence of land cover types provided by temporal trends and changes (e.g., Gutiérrez-Vélez and DeFries 2013) and initial classification probabilities (e.g., Cai et al. 2014; Gong et al. 2017). Abercrombie and Friedl (2016) proposed a post-processing framework to mitigate spurious land cover changes in classification time series based on the use of a Hidden Markov Model (HMM). Using this framework, initially assigned posteriori class probabilities are temporally analyzed and modified based on a transition matrix that specifies the likelihood of a land cover class transitioning into another. Hermosilla et al. (2018) integrated time-series derived changes and HMM to control class transition likelihoods by combining disturbance presence information with knowledge on ecological succession.

3.2.8. Accuracy assessment

Accuracy assessment is a critical element of Land Cover 2.0. Accuracy assessments that are statically defensible and transparent are essential to ensure the integrity of the products developed and enable end user confidence and uptake. Depending on the accuracy outcome (i.e. the quality of mapping of a given category of interest), it is possible to revisit the calibration data or classification approach to produce revised land cover products. Given the increasing ease with which remotely sensed land cover classifications can now be generated, accuracy assessments will be critical for the user community to differentiate between the available products.

Accuracy assessments are only meaningful when they are designed in a transparent and statistically defensible manner (Stehman 2009) and when they make use of independent validation data that were not used in model development. Since the early 1990s, extensive recommendations and good practice guidance have been put forward in the literature regarding approaches for accuracy assessment of land cover and land cover change (e.g, Foody 2002; Wulder et al. 2006; Olofsson et al. 2014). Agreement upon standards and definitions across the remote sensing land cover community for creation of an open database for calibration and validation would aid in providing independent unbiased validation capacity. Researchers could access, and contribute to, this database to verify their data products (Olofsson et al. 2012; Stehman et al. 2012). This has appeal as interest in citizen science initiatives grow and the capacity to obtain very large datasets from users could aid in accuracy assessment endeavours (e.g., Comber et al. 2016). A common repository of reference data would be very advantageous for land cover science. First, such a repository will potentially increase accuracy due to the huge amounts of training data provided from users around the globe. Second, it promotes the science behind land cover mapping and continues to build a well-informed user community on the benefits and cautions involved in using land cover information. This notion of a global accuracy assessment database for land cover is not new (e.g., Loveland et al. 2000; Morisette et al. 2003; Pengra et al. 2015), with much of the background for the design and implementation of such a data base already established in the literature (Olofsson et al. 2012, Stehman et al. 2012; Khatami, Mountrakis, and Stehman 2017). If mapping to an existing and established set of land cover categories this allows for synergies with, and access to, existing reference datasets. Custom maps will require purpose collected and documented reference data. Robust product validation follows through a number of general stages of sophistication, from a small number of spatially limited samples (not recommended), through to the recommended systematic assessment of new products using independent, well-distributed, reference data following a probability sample (using a design-based approach) relating uncertainties in a statistically robust fashion (Strahler et al. 2006).

3.2.9. Digital delivery and open access for resulting land cover products

Despite the variability in investment and capability, the overarching notion of Land Cover 2.0 is to allow users to bring their algorithms to the data. Commercial cloud services such as Google Earth Engine, Amazon Web Services, Microsoft Azure, and Descartes Labs are proliferating, offer differing computing environments, infrastructure, and business models (as outlined in Yang et al. 2017a) while offering significant analysis and data storage potential. Co-location of data and computational capacity with common analysis tools in cloud systems have significant potential for enhancing collaboration and community consensus, as well as the transparency of land cover product development. Outstanding issues related to these services include the development of interfaces and middleware to allow users to address data compatibility, processing, and interoperability (Jayaraman et al. 2017), as well as concerns regarding data egress from the cloud, intellectual property, and costs for cloud service use.

The capacity to digitally deliver land cover products allows for many user options that did not previously exist. Web Mapping Services (WMS), for instance, allow users to view, access, and/or download data over the internet. Additional functionality can also be added to allow users to create custom products, based upon refined sets of land cover classes for instance, to meet particular information needs. The services can either be simple based upon stored layers or hierarchal rules, or more complex, requiring cloud computing-enabled analyses for custom queries via a web processing service (Chen et al. 2012).

3.3. Examples of land cover 2.0

There are several recent examples of land cover projects exemplifying Land Cover 2.0 principles. Midekisa et al. (2017) utilized the Google Earth Engine cloud computing platform to map land cover change over continental Africa. Fifteen years of Landsat data were assessed to better understand changes in land cover, land use, and impervious surfaces. Training data were obtained by visual inspection of Google Earth imagery and a random forest supervised classification model was applied. The nation of France was mapped in a systematic fashion by Inglada et al. (2017) using Landsat imagery. The authors aimed for the procedures developed to be portable to other regions and amenable to development, to allow for changes in nomenclature or update frequency. An automated approach using all available image data was implemented using a parallel workflow for rapid map production and reprocessing as required. A reference training data set based upon existing datasets, not visual interpretations, was utilized. Using a spatially partitioned implementation of a random forests classifier, a map for 2014 was generated along with accuracy statements and confidence maps. Similarly, Zhang and Roy (2017) generated a 30 m land cover map of North America using 3 years of publically available higher processed Landsat data (GWELD 2017) with training data derived by automated filtering the standard MODIS 500 m land cover product and a spatially partitioned random forests classification approach. Using a multi-decadal series of annual Landsat image composites representing the forested area of Canada, Hermosilla et al. (2018) created a time series informed data cube of land cover. A random forests classification input with Landsat reflectances, vegetation indices, and supplemental data layers (elevation and related derivatives), insights on land cover dynamics for Canada were produced. A transition matrix in conjunction with independently derived spectral change information was also used to avoid illogical changes. A Hidden Markov Model allowed for the class likelihoods for all pixel-series to be considered allowing for incorporation of temporal information in reducing the presence of spurious transitions.

4. Outlook for land cover 2.0

4.1. Integrated land cover and land cover change monitoring

In the context of Land Cover 2.0, land cover mapping and change detection are no longer seen as independent, disconnected activities. Previously, land cover products were generated to represent a certain date or period in time and land cover change detection was undertaken as an entirely separate activity, or as a post-classification exercise (Figure 1). For many users, change in land cover is of paramount interest. The conversion of forests to non-forest land use or of native grasslands to agricultural land use are of wide interest for considerations encompassing carbon consequences, biodiversity, provision of ecosystem services, and habitat suitability.

In the context of Land Cover 2.0, change information now becomes a key input to land cover classification. That change information may take several forms. For example, mapped disturbances can be integrated into the land cover product, either to aid in labelling the current cover type, or to provide a priori information for likely land cover types at a given location. Change metrics, derived from image time series, can likewise be used as inputs to classification algorithms. New approaches capture change information as a component of data preparation (e.g., cloud infill, haze detection; Hermosilla et al. 2015a; 2016), and thus knowledge of if and how a pixel series has changed through time is captured before land cover classification begins. In a time series context, this spectral change information can be used to aid in identifying when and where land cover class transitions occur (i.e., to inform carbon monitoring and reporting, Boisvenue et al. 2016). Moreover, errors commonly associated with post-classification change procedures, whereby the difference between land cover maps is used to determine change, is also mitigated (Fuller, Smith, and Devereux 2003).

4.2. Temporal aspects of accuracy assessment

An emerging issue with regards to accuracy assessment is the difficulty in obtaining validation data when generating a time series of land cover (Gómez, White, and Wulder 2016). Visual techniques using the high spatial resolution imagery in Google Earth are problematic prior to ~2000 when high spatial resolution satellite data became increasingly common at the global scale. Even post-2000 when there is imagery available, the amount and distribution freely available on platforms such as Google Earth may not support rigorous accuracy assessment protocols; however, agencies and governments may be able to procure data to support accuracy assessments (e.g., Morisette et al. 2003). Some jurisdictions may have monitoring plots (such as from forest inventory programs) but these too will be small in number and limited in distribution. An outstanding and overarching question relates to how to characterize the accuracy of a multi-year (i.e. annually over decades) land cover product. We can expect data to be sparse in the early years with more likelihood of data getting closer to the present time. If the same methods are applied to produce the time series of land cover, do all years need to be assessed, or can accuracy be reported on for a representative year? Is a robust single year assessment preferred over a multi-year assessment that cannot fulfill sample design requirements? These time series related validation questions remain; framing of the issues and initial developments have been shown, with scope remaining for notable future improvements.

4.3. Error reduction, confidence building, opportunities

As previously described in the section concerning post-classification error reduction (4.2.7), the combined analysis of posteriori class probabilities and ecological knowledge of land cover transitions serve to improve both map accuracy and the temporal consistency of land cover products (Abercrombie and Friedl 2016; Hermosilla et al. 2018). Ensemble decision tree classifiers such as Random Forest generate class-conditional probabilities from the number of votes, or class assignment probabilities, for each class from each decision tree (McIver and Friedl 2001; Zhang and Roy 2017). In the training phase, Random Forest uses the arithmetic mean of the class assignment probabilities from each tree to determine the final class assignment (Belgiu and Drăguţ 2016). In the classification phase however, each tree in the ensemble will vote for class membership for an unlabelled pixel, and the class with the most votes, is assigned to the pixel. Mapping these posteriori class probabilities can provide spatially explicit information useful for refined classification of land cover, offering further local information for specific classes with unique and enhanced modeling opportunities possible beyond purely categorical classification outcomes (Figure 2).

Figure 2. Example of class membership likelihoods for select classes found in Canada. High confidence is shown in warmer (red) shades. The spatial distribution of dominant classes (e.g. water, shrubs, coniferous, broadleaf) are shown. Confusion between spectrally similar classes can also be evident in lower likelihoods.

Additionally, these posteriori class probabilities enable relative confidence maps to be derived, enriching the land cover product suite. In this sense, layers such as distance to the second class based on the post-classification probabilities (Figure 3; Mitchell et al. 2008) or per-pixel predicted accuracy maps (Khatami, Mountrakis, and Stehman 2017) facilitate confidence building for the derived products, increasing the transparency of the mapping process and enabling regions of interest to be highlighted. Moreover, class probabilities provide the opportunity for end users with different information needs to combine or reassess classes allocated and/or probabilities assigned for a given year or over time (Comber, Law, and Lishman 2004). The uncertainty emerging from a mapping process can be used as information to alter categories or to alert map users or producers where issues may be present. A land cover map remains a single interpretation of reality (where many possible interpretations may exist), mediated by the information needs and application of the end user (Comber, Fisher, and Wadsworth 2005). Given current practices and algorithms, it is possible to carry and convey the uncertainty between classes rather than leaving pixels unclassified or classified without reference to confidence (e.g., Foody 1996; Mitchell et al. 2008). For additional examples for incorporating per-pixel information to improve map results or better understand uncertainties, see McIver and Friedl (2001) and Friedl et al. (2010).

Figure 3. Example land cover classification (a) with distance to second class heat map (b). The example is located near Kotcho Lake in northwest British Columbia, Canada (Lat/Long: 59.06816°N/121.14950°W). High confidence classes (e.g., water) have large distances to second class based on the post-classification probabilities. Large distances to second class are also noted for homogeneous extents of forest. Lower distances to second class, denoting less confidence in the allocated category, are found for more rare and spatially limited classes (e.g., bryoids) or classes that have spectral similarity (e.g., herb, shrub, and broadleaf).

4.4. Trends, opportunities, and directions in land cover

Given the contemporary context of Land Cover 2.0 in juxtaposition to the institutional and scientific limitations outlined by Townshend (1992) where will the science and application of land cover be 25 years from now? Remote sensing is a rapidly changing and highly dynamic field. There is an increasing number of satellites capturing data at scales relevant to human interactions with the landscape. The large missions that are the purview of national space agencies are no longer the only sources of data, with commercial data options proliferating. The niches currently filled by government and commercial satellite operators are diverging, with commercial vendors availing upon new, lower cost technologies and launch opportunities to place large numbers of small-satellites into orbit. What these small satellites lack in radiometric consistency, spectral bands, and orbital stability, is somewhat offset by their high spatial resolution and frequent temporal revisit period. Determination of how to integrate data across the range of space-based measures will provide additional monitoring and land cover mapping opportunities. In the future, land cover applications that are not reliant on a specific data stream will likely emerge (e.g., Yu, Wang, and Gong 2013; Chen et al. 2015), increasing the flexibility and speed with which land cover information can be generated (e.g., Zhang and Roy 2017).

A key facet of Land Cover 2.0 will be the use of data from different satellites to provide more complete spatial mosaics and/or richer time series for land cover classification purposes. Arguably, differences between different sensor data can be overcome if large amounts of training data are used, although, as discussed above, appropriate training data collection can be challenging. Alternatively, the different sensor data may be processed to be more compatible for classification purposes. To achieve this, the different sensor data may be combined to provide a sensor fusion product, or may be kept separate but harmonized so that the processed data may be used more reliably together. The former fusion approach strictly requires that the data from each sensor are precisely co-registered, calibrated, spectrally normalized to common wavebands, atmospherically corrected (using the same atmospheric characterization), and corrected for surface anisotropy. Sensor data fusion has been undertaken using physically based approaches, for example, by inverting Bidirectional Reflectance Distribution Function (BRDF) models using reflectance data from different sensors (Jin et al. 2002; Schaaf et al. 2002). Sensor data harmonization can be undertaken in a more empirical manner, for example, by computation of spectral vegetation indices derived from different sensors, and/or by applying statistical transformations between similar bands of different sensors (Brown et al. 2006; Miura, Huete, and Yoshioka 2006; Roy et al. 2016c; Zhang et al. In review). Ideally, these approaches will enable interchangeable and combined sensor data use as part of an integrated workflow (or elements of a virtual constellation, Wulder et al. 2015). However, this becomes challenging if the satellite data and resolution characteristics and quality differ significantly. For example, research on the use of MODIS and Landsat data for classification purposes is well established (Hansen et al. 2008). Integration of these data with newer commercial remote sensing systems, which may be of lower spectral quality but higher spatial and temporal resolution, offers unique opportunities but additional investigation is required (e.g., Houborg and McCabe 2018).

A greater integration of modeling with remote sensing could also be expected. Over terrestrial ecosystems, deterministic processes are at play. For example, disturbed forests return in a relatively consistent sequence of cover types, and agricultural cropping patterns are likewise not random. Models projecting a given land cover condition can be statistically confirmed or denied, thereby reducing reliance on satellite observations to provide a unique information outcome, and offering opportunities to increase product temporal density or spatial detail. Furthermore, data assimilation has been shown to offer new opportunities for integrating remotely sensed data and models beyond land cover, such as in the generation of other forest inventory information (Nyström et al. 2015). Integration of data from different sensors with disparate spectral and spatial resolutions may also be enabled through a combined modeling and remote sensing approach. This integrated approach could serve to reduce the need for radiometric uniformity across an envisioned data stream (opening opportunities for CubeSat, for instance). Caution must be exercised with regards to the radiometry and spectral consistency of CubeSat data (Houborg and McCabe 2018). Large numbers of CubeSat images are required to represent areas of the extents discussed herein (regional, continental, to global). Differences in view angles and illumination conditions can be expected, which serves to reduce spectral comparability, as well as the science and applications possible. Non-calibrated data sources may be useful for capturing change but will be limited for the more broad suite of attributes desired of remote sensing (land cover, biophysical estimates, or capture of subtle change or trends). Here, the distinction between the use of images as pictures versus the use of images as calibrated observations must be made; the two products service different information needs and have different utility in a long-term land cover monitoring context.

The capacity to generate land cover information in near-real time is a logical progression from the aforementioned prospects for integrating different data sources and modelling approaches into land cover product development. Many stakeholders would benefit from reduced latency and increased consistency in obtaining information on when, and where, land cover change is occurring (e.g., deforestation, illegal logging, wildfire, disaster response). While the need is evident with methods currently under development (Zhu and Woodcock 2014b), the nature, timing, and detail of what is reported in near-real time remains to be determined. Given the data storage and download capacity now possible, data can increasingly be collected in an ‘always-on’ mode. This would provide the regular, consistent within-year measures that are required for near-real time algorithms. Moreover, increasingly sophisticated automated ground systems reduce the latency in end-user data access. The fusion of data from a number of complementary sensors is also poised to become increasingly common to inform near real-time applications.

5. Conclusions

It is evident that much progress has been made in addressing both the data and methodological limitations identified by Townshend (1992). It can also be stressed that many of the institutional or programmatic barriers to the ‘provision of reliable operational land cover information from satellite data’ identified by Townshend (1992), have also been removed or have ceased to be the barriers they once were. Today, many national or regional programs rely on land cover information generated from Earth observation data to support monitoring efforts and inform science and policy (e.g., Bossard, Feranec, and Otahel 2000; Wickham et al. 2014). Ultimately, the greatest need for reliable land cover information has not changed since Townshend (1992): that of improving our understanding of drivers and processes of global change.

The work of Townshend (1992) provides an important benchmark in the evolution of remote sensing science. Advances towards reliable operational land cover products have been made on many fronts in the past 25 years. Land cover and land cover change are now a fully integrated information output. Today, land cover information can be operationally generated in an increasingly automated, systematic, and rigorous fashion. As a concept, Land Cover 2.0, brings together disparate advancements in data sources and policies, algorithms, and computational capacity. Open data has fueled the rapid development in land cover science and application, and set the stage for future innovations in land cover products and applications. An increasingly diverse set of data are at the core of the Land Cover 2.0 concept, with quality reference data remaining an important priority.

The manner in which reference data are acquired, standardized, and shared will be a critical to the quality of products generated through Land Cover 2.0. The use of reference data for independent validation of land cover products will be critical, with transparent and robust methods to be implemented, and a broader suite of confidence building layers of interest to modellers, data users, and policy makers. Building on the insights and experiences of the remote sensing community, land cover 2.0 provides an opportunity to meet the information needs of a number of stakeholders and to approach the vision of early scientists and practitioners. The rapidly developing technological (satellite, communications, computing) and applications landscape will continue to evolve and further empower science in support of pressing interests of society.

Acknowledgments

This research was undertaken as part of the “Earth observation to inform Canada’s climate change agenda (EO3C)” project jointly funded by the Canadian Space Agency (CSA), Government Related Initiatives (GRI), and the Canadian Forest Service (CFS) of Natural Resources Canada. Support was also provided by a Natural Sciences and Engineering Research Council of Canada (NSERC, RGPIN 311926-13) grant to Coops. Indication of commercial products or services does not indicate or imply endorsement. The Editor and Reviewers are thanked for valuable insights and guidance that improved the quality and content of this paper.

Disclosure statement

No potential conflict of interest was reported by the authors.