Storage and processing of massive remote sensing images using a novel cloud computing platform

Abstract

In recent years, the rapid development of remote sensing technology has proliferated high-quality images that occupy larger and larger storage spaces. Video has become widespread for environmental observation. Hence, digital data is growing exponentially, and geographic information systems must determine how to manage and process images and video effectively. Researchers cannot limit themselves to desktop PCs due to computational and storage limits. The aim of this article was to propose and implement an architectural design for a novel cloud computing platform based on two Web Coverage Service and Web Map Service interfaces from the Open Geospatial Consortium (OGC), cloud storage from Hadoop Distributed File System (HDFS), and image processing from MapReduce. Results are presented on tablet computers (Asus transformer pad) and websites. Within this framework, we implemented image management as well as simple WebGIS and created an experiment in read/write performance with four kinds of data sets (normal distribution, skew to left, skew to right, and peak in left and right). For write/read performance with HDFS, the proposed system outperformed a local file system for large files (most files ranged from 8 MB to 10 MB), with many concurrent users (simulated threads equal to 40 or 50). An observer on the ground with a touchscreen can identify central points (man-made centroids) of real-time images by tapping the tablet with a finger. A second experiment revealed that the convergence for human intervention was better than convergence for random centroids in two kinds of cloud computing environments.

Keywords:

• cloud computing
• Hadoop
• K-means
• remote sensing

1. Introduction

The term “remote sensing” refers to recording, observing, perceiving (sensing) objects or events remotely, and image processing (Weng 2010; Richards and Jia 2006; Jensen 2006).

Modern image processing technology helps scientists extract meaningful regions from acquired images (De Jong and Van Der Meer 2004; Chitade and Katiyar 2010; Wang et al. 2012). For example, Breckenridge and Dakins (2011) used fixed-wing and helicopter unmanned aerial vehicles (UAVs) to measure bare ground in a sagebrush steppe ecosystem. After collecting data with digital imagery, they ran the image analysis using SamplePoint software (Booth, Cox, and Berryman 2006). However, as time goes on, high resolution images will occupy more hard disk space, and image processing will require more time; therefore, we must have a well-designed platform to store remote sensing images and we must process images effectively in a distributed environment. Hardin and Jensen (2011) provided an introduction for small-scale unmanned aerial systems in environmental remote sensing in a special issue of GIScience & Remote Sensing. As to computational driven solutions for remote sensing data, Filippi et al. (2012) proposed a power of high-performance cluster computational approach (40× speed-ups versus sequential architecture) that was used to compute hyperspectral above-surface remote sensing reflectance.

Recently, the information sciences jargon term “cloud” has been discussed in many books (Sosinsky 2011; Cafaro and Aloisio 2011; Mahmood and Hill 2011). We think “cloud” is not just an adjective; it should be a real, workable infrastructure and platform. In a cloud computing environment, managers and engineers can have large pools of resources that are both usable and accessible, i.e., hardware usage, development platforms, and related services. These resources can be dynamically reconfigured to ensure optimal resource utilization. Further, resources are typically exploited by a pay-per-use model where long-term guarantees are offered and followed by customized service-level agreements (SLAs) from providers. Researchers can observe surface deformations, develop land-cover classifications, and detect events in given regions by relying on multitemporal remote sensing images from selected areas (Canty and Nielsen 2006; Ficheral et al. 2012; Codella et al. 2011). These large amounts of real-time geodata are difficult to process, especially with limited storage space; to improve performance, cloud computing provides appropriate tools.

MapReduce proposed by Google (Dean and Ghemawat 2004) has been widely used for large-scale data analysis in the cloud. This kind of programming model simplified data processing on large clusters. In the past 6 years, the Hadoop project realized the mass data storage Hadoop Distributed File System (HDFS)/analysis (MapReduce). Lv et al. (2010) introduced MapReduce and parallel K-means clustering for remote sensing images. Wang et al. (2012) proposed a classification algorithm for high resolution remote sensing data based on mapping mechanisms. Almeer employed the MapReduce framework to implement parallel processing of remote sensing images and built an experimental 112-core high-performance cloud computing system in the Environmental Studies Center at the University of Qatar (Almeer 2012). These studies focused on computational ability using MapReduce. However, we focus on total solution based on our UAV product.

Therefore, the main contributions of this article are to propose a framework and implementation of a system that can store and process massive remote sensing images on a cloud computing platform. Similar image storage and processing infrastructure can be performed in MapReduce programming model tools (LV et al. 2010; Zhao et al. 2009; Zhang et al. 2012, Almeer 2012; Li et al. 2010). The uniqueness of our works are to focus on clustering results shown in a front-end interface by app in Asus transformer pad and presented on MapServer with OpenLayers, real-time image processing using MapReduce-based novel K-means architecture, Hadoop ecosystem integration with proposed data processing module, and image query from HDFS. The framework requires you not to worry about your operations department with diagnosing and fixing hardware issues. Therefore, the framework refers you to have mid-level rack servers with dual sockets, and Serial Advanced Technology Attachment (SATA) drives optimized for Redundant Array of Independent Disks (RAID) storage (Holmes 2012). In this article, HDFS cluster primarily consists of a NameNode and three DataNodes (four machines in total), and each node is equipped with an [email protected] GHz CPU, 12 GB or 4 GB RAM (NameNode or DataNodes), and 1 TB disk space. Besides the prerequisite software of Hadoop, you also need to install Apache Tomcat and MySQL for UAV images management (Web Apps), MapServer and OpenLayers for image clustering results, and our developed app (apk) in the pad.

The remainder of this article is organized as follows. Sources of remote sensing images are introduced in Section 2. Section 3 discusses the processing of remote sensing images on a single machine and in a cloud computing environment. We propose a suitable platform for remote sensing image processing in Section 4. Section 5 presents a prototype implementation of the WebGIS embedded image post-processing presentation layer (website and pad interface). Section 6 explains the design of four types of data sets used to test write/read performance of local file systems and HDFS. Section 7 is the conclusion.

2. Sources of remote sensing images

To collect aerial images, the GIS Research Center of Feng Chia University (FCU) developed a UAV (named AS-4) that integrates aerial engineering and electronics, wireless transmission, image post-processing, and geographic information system (GPS) coordination with Open Geospatial Consortium (OGC)-Sensor Web Enablement (SWE) compatible software for comprehensive cross-platform environmental investigation. The AS-4 is a helicopter, and its detailed specifications are as follows: maximum weight capacity is 15 kg; maximum altitude is 10,000 feet (about 3000 m); it can fly for 1 hour with a standard fuel tank; range is a 30-km radius; and the highest speed is 100 km/hour. The flight mode can be controlled by remote manually or by an automatic flight system at the ground control station.

Since the AS-4 is configured with the relevant modules (microwave image transmitter, high-resolution camera, and real-time monitoring system), it can collect high-resolution pictures or Hard Drive (HD) movies, and operators on the ground can run monitoring software installed on personal computers. One important application of the UAV is that the scientists can depict the flight path of a UAV that took off from a given location (for example, disaster areas), then they can collect images/videos after the completion of the task (images have been shot). In general, these data sets are massive and will be processed in batch; therefore, they can be mosaicked into a large high-resolution image by commercial software (Esri ArcGIS, Erdas Imagine) so that researchers can get a panoramic picture of the environment. The system can also process metadata for each aerial photograph (e.g., *.jgw of *.jpg, and *.tfw of *.tiff).

3. Processing of massive remote sensing images

There are two main image classification methods: supervised classification and unsupervised classification. In this article, we apply unsupervised classification by K-means algorithms that are highly flexible, efficient, and easy to use. Many unsupervised classification algorithms (De Jong and Van Der MEER 2004; Canty and Nielsen 2006) have been widely used in the classification of remote sensing images. Two of the most usual algorithms are K-means (Chitade and Katiyar 2010; LV et al. 2010; Zhao et al. 2009) and ISODATA (Li et al. 2010).

The K-means method was first used by MacQueen in 1967. It is one of the most extensive cluster analysis methods to have been developed. It relies on random centroids, but in this article, we propose a new version of this algorithm where initial centroids are man-made instead of random. Each user can select K initial centroids, and the system can calculate the distance between each pixel and each centroid; each pixel will be assigned to its own cluster such that the distance from the pixel to the centroid of its own cluster is minimized. After that, the algorithm will redefine the center of each cluster until the termination condition is met.

In this section, we study remote sensing image processing on single machines and in cloud computing environments in the following two subsections.

3.1. Remote sensing images processed on single machines

In this article, the input for image processing is a multispectral remote sensing image, and each pixel of this image is represented by an RGB-value. We apply a transformation to each pixel (Chitade and Katiyar 2010). This transfers each pixel from an RGB-value to a L*a*b*-value where L is a brightness layer and can be ignored, a* indicates where color falls along the red-green axis, and b* indicates where color falls along the blue-yellow axis.

In this section, we express how to process these images with a single machine. First, a researcher can retrieve one image from the repository returned by AS-4. Then, this article implements the normal steps (Chitade and Katiyar 2010) of processing clustering for one image as follows:

1.	Read image from aerial image repository.
2.	Image transformation (RGB→L*a*b*).
3.	Extract a pair (a*, b*) of each pixel in the step above and keep them in an array.
4.	Apply clustering algorithm (K-means is used here).
5.	Label every pixel in the image using K-means.
6.	Clusters are rendered in a graphics layer.

On a personal computer, users can obtain the results of image processing by Matlab (2011b). First, users can load images using the imread() function and then complete the image transformation by means of the Matlab function makecform (“srgb2lab”). The transformation and clustering algorithm uses the K-means method by coding (France and Hlavac 2004).

Implementing this process requires some technical skills; researchers can also use commercial software to render the result visually. Such commercial software does not require as much technical skill as Matlab; users only need to load the image and set the parameters. Users can load one aerial city photograph and run the image classification of a K-means (K = 5) algorithm in Environment for Visualizing Images (ENVI) 4.7. Different colors illustrate different clustering results, i.e., red and green indicate the trees and lawns, deep blue indicates the roads, yellow indicates the playgrounds, and light blue indicates buildings.

3.2. Remote sensing images processed in cloud computing

Remote sensing image processing can be done by standalone software, such as ENVI. However, computational power and disk storage space are two important limits that often act as bottlenecks (Zhao et al. 2009). Therefore, researchers have developed variants of the algorithm that use parallel execution, and most of them can be implemented with the Message Passing Interface (MPI) protocol. However, programs in MPI require sophisticated skills. Fortunately, convenient cloud-based platforms let ordinary users create and manage Virtual Machines (VMs).

Besides the scholarly efforts of MapReduce and Hadoop, MapReduce enabled clustering implementations in Apache Mahout (Owen et al. 2012). It offers users the opportunity to create scalable machine-learning algorithms, including K-means, fuzzy K-means, Canopy, Dirichlet, and Mean-Shift.

MapReduce is the most crucial technology in a Hadoop-based cloud environment; it can process images in distributed environments. One image will be split into several files (with an RGB value of each pixel in the given image) based on block size and the system will save the files on multiple data nodes. The Map function computes the cluster each pixel belongs to, and then generates an input data structure with key values to reflect an output key value. The Reduce function gathers all points and classifies them into different clusters depending on their attributes. We implement the concept of K-means algorithms and complete image clustering. A comprehensive picture of our proposed approach is shown in Figure 1.

Figure 1. MapReduce-based novel K-means architecture.

4. Integrated system based on cloud computing environment

The limited capacity of image processing with Matlab or ENVI on a single machine is offline image processing. It is convenient for staffs to process imagery in the cloud. A single surveillance operation can generate many gigabytes of imagery in a single day, and a standard interface can access such data over the web (by means of a web service). There are four major issues: (1) it is difficult to expand hardware installations and integrate with existed open source software; (2) software and hardware are costly; (3) efficiency is reduced when multiple users access the system; and (4) it can be challenging to ensure the reliability of the system.

Therefore, based on the above issues and requirements, this article proposes an integrated system based on a cloud computing environment that is suitable for storing and processing remote sensing images. This framework consists of six major parts and is described in Figure 2: Part A: users can upload and retrieve aerial images by UAV image management; Part B: WebGIS is used to render aerial images and generate a dynamic layer on the web page; Part C: data analysis and processing modules store and process images; Part D: the cloud computing environment is a basic infrastructure; Part E: observers can upload UAV images and download clustered images promptly; and Part F: uploaded images will be sent into the cloud by WebHDFS and image processing can be done by the web service MapReduce in a data-processing module. We implement all these features in this article; we show the interface of parts A, B, and E in Section 5 and test the performance of parts C, D, and F in Section 6.

Figure 2. Integrated system based on cloud computing environment.

4.1. Part A: UAV image management

UAV image management is a system for users to upload images and manage them. This system includes image retrieval and an image manager on a website. This system's image retrieval part is a subsystem for browsing, searching, and retrieving images from the database of aerial images. Most image retrieval methods utilize metadata; thus relevant images can be retrieved from websites. Manual operation is time-consuming, laborious, and expensive. To address this, much research has been done on automatic image annotation. Additionally, users can upload related images, and these uploaded images will be stored in the HDFS of Hadoop.

4.2. Part B: WebGIS service interface

Users can import parameters from WebGIS into the system, and relevant map information will be added to different layers in OpenLayers (Hazzard 2011). The presentation can be accessed by OGC interfaces, that is, Web Coverage Service (WCS) and Web Map Service (WMS).

•	OpenLayers – Google Maps provides a quick and easy way to add maps to your website, but when you are using Google's API, your ability to display other data is limited. If you have your own data apart from Google to display, OpenLayers, an open source JavaScript library, can give you other options. OpenLayers can work with data on your server and can integrate other services such as GeoServer, MapServer, or data published by WMS.
•	WCS – The OGC WCS allows for the publication of “coverages” – digital geospatial information representing space-varying phenomena.
•	WMS – A WMS is a standard protocol for serving georeferenced map images over the Internet that are generated by a map server using data from a GIS database.

4.3. Part C: data analysis and processing module

The image retrieval subsystem allows browsing, searching, and retrieving images from a large database of digital images. MapServer (Kropla 2005) is a popular Open Source project whose purpose is to display dynamic spatial maps over the Internet. Our proposed HDFS adapter can mount HDFS as a file system.

4.4. Part D: cloud computing environment

Hadoop is an open-source software framework from the Apache Software Foundation, where HDFS and MapReduce are two important core products. We can store UAV images and detailed information by HDFS and deal with images using the MapReduce framework. In our cloud computing environment, we clustered two-dimensional points (the values of a* and b*) discussed above and applied our proposed image classification module.

4.5. Part E: upload UAV images and download clustered image

An operator can collect images by means of a UAV and choose desired images for which the K-means approach will be computed in the cloud. These images will be sent from the SD card of the camera on the UAV and analyzed in the cloud.

4.6. Part F: data processing module

The data processing subsystem is a communicating interface that connects the service entrance module and the Hadoop environment. When the service entrance module inputs UAV remote sensing images and receives requests from users, it assigns tasks to the data processing module. The data processing subsystem includes three processing modules:

1.	Image preprocessing module. The RGB value of each UAV image is analyzed by the image preprocessing module, and these processed items of information are stored in HDFS.
2.	K-means module. Users can employ our proposed K-means module to classify images according to their RGB values by means of the MapReduce framework. It can identify images with uncommon speed and store them in HDFS by WebHDFS in the Hadoop environment.
3.	Computation result. We apply HDFS to store the computational results of the K-means module and display them visually on personal tablets by calling web services.

5. Implementation

After uploading images to the repository in HDFS, researchers can query images by year, country, town, and village. These images can also be fully downloaded as a zip file in our prototype. Furthermore, the image processing modules execute as a back-end that can be integrated with MapServer (Kropla 2005); the front-end interface shows layers of overlapping effects. In the following, we demonstrate the implementation of Parts A, B, and E of Figure 2.

5.1. Part A: user interface

Metadata is used to present “semantic descriptions of the images” here (it records images by year, location, and description), and the manager can retrieve related images from metadata by the “Submit” button shown in Figure 3. Then the results of query are shown on the right side; each has an “Open” button that can access images photographed at the same time and place by thumbnail in Figure 4.

Figure 3. Implementation of Part A in Figure 2.

Figure 4. Query results of UAV photographed in Alishan area of Taiwan.

Figure 4 shows aerial photographs from the UAV repository in the cloud computing environment built by the GIS research center of Feng Chia University. These photographs were taken in Fengshan Village, Alishan Town, Chiayi County, Taiwan, after Typhoon Morakot on 17 August 2009. Researchers can also download these images as a zip file using the “Download” button shown in Figure 4.

5.2. Part B: presentation by MapServer with OpenLayers

To facilitate processing of aerial images, we built a simple user interface by OpenLayers (Hazzard 2011). An original image is shown in Figure 5(a), and overlapping layers present the result of combining the original layer and clustered layer (70% transparency) in Figure 5(b).

Figure 5. Implementation of Part B in Figure 11 by OpenLayers. (a) Disabled checkbox shows original image layer; (b) enabled checkbox combines two layers (original image layer and clustered result layer).

5.3. Part E: UAV remote sensing image processing in Asus transformer pad

In this section, we discuss the implementation of the UAV remote sensing image processing (UAV-RSIP) on Asus transformer tablet computers. Lin et al. (2013b) demonstrated the screen interfaces of UAV-RSIP, where users can choose any image from the SD card of the UAV camera that will be sent and analyzed in the cloud. To test a real case, we took pictures and selected one in Longtan of Taoyuan, Taiwan. This picture consists of 11,197,444 pixels and 6.79 MB. Third, the user directly selects K original centroids in UAV images, and then the UAV-RSIP module will call the K-means module to process in the cloud; eventually, the computational results module will display a clustered image on the tablet computer.

When the user clicks several points (man-made centroids) on the tablet, an app will keep the values of XY coordinates and RGB values for the centroids and send them into cloud. To demonstrate the proposed system, we set up K = 4 and K = 5 and made experiments as shown in Figure 6(b) and 6(c). There are many pixels that keep the same information (duplicated RGB values); therefore, we can just hold unduplicated RGB information (eliminate duplication) by a new record to speed the convergence rate of the K-means algorithm. In some situations, cloud service may be provided by independent vendors such as International Data Corporation (IDC). The current research built two kinds of cloud environments (the first had four VMs managed by Hyper-V in one physical machine; the second was a cluster with three physical machines). The experimental environments are illustrated in Figures 10 and 11.

Figure 6. The results of the experiment by UAV-RSIP in the Part F of Figure 2. (a) The selected image after completing the uploading step, (b) the clustered result (K = 4), and (c) the clustered result (K = 5).

6. Experimentation

We confronted two difficulties during our experiment (Lin et al. 2013a). The first one was the storage of large number of aerial images; the second one was image processing. This section describes performance tests of storage and access of image files by a local file system on a single machine and an HDFS (replication = 1) in Hadoop under Ubuntu environment. We simulated four types of data sets (100 files were included in each data set to test the performance of Part C in Figure 2), as depicted in Figure 7: (1) normal distribution with average of around 5MB and a standard deviation of around 1MB; (2) skew to left with 80% of files measuring 1–3MB; (3) skew to right with 80% of files measuring 8–10MB; and (4) peaks at left and right with 40% of files measuring 1–3MB and 40% of files measuring 8–10MB. Figure 7 presents the data distribution of the above types of data sets, and the Y-axis is frequency within the range of generated data. These four kinds of data sets were all tested in a local file system and the cloud environment. We simulated concurrent users by various numbers of threads (10, 20, 30, 40, and 50 shown on X-axis of Figures 8 and 9) used for writing and reading.

Figure 7. Distributions of different data sets. (a) Data set 1: normal distribution, (b) data set 2: skew to left, (c) data set 3: skew to right, and (d) data set 4: peak in left and right.

Figure 8. Write performance in different data sets. (a) Write performance in data set 1, (b) write performance in data set 2, (c) write performance in data set 3, and (d) write performance in data set 4.

Figure 9. Read performance in different data sets. (a) Read performance in data set 1, (b) read performance in data set 2, (c) read performance in data set 3, and (d) read performance in data set 4.

To test the performance of Part D in Figure 2, our cluster environment includes one NameNode and three DataNodes (four machines in total), and each node is equipped with an [email protected] GHz CPU, 12 GB or 4 GB RAM (NameNode or DataNodes), and 1 TB disk space. Additionally the local file system on a single machine was tested on NameNode with system functions (fread/fwrite) that are used for reading/writing data from/to the file opened by fopen function.

Figures 8 and 9 show the results for read/write performance with four kinds of data sets. HDFS has lower writing performance than the local file system, as shown in Figure 8(a) and 8(b). HDFS has lower reading performance than the local file system, as shown in Figure 9(a) and 9(b). Therefore, the local file system outperformed HDFS for normal distribution and skew to left data.

However, it is noteworthy that the write/read performance of HDFS increases as concurrent users increase. Overall, the results have been very positive; in particular, HDFS can serve more threads and have better performance than the file system for data set 4. Fortgeschrittene indicated HDFS writing performance scales well on both small and large data sets in Hadoop performance evaluation (Dinh, Hadoop report). Our experiments also confirm this trend, especially for numbers of threads greater than 40. The write/read performance of HDFS is better than the local file system if files are large (most files range from 8 MB to 10 MB) or if there are many concurrent users (simulated threads equal to 40 or 50).

In Figures 10 and 11, to test the performance of random centroid by system and proposed man-made centroid by an operator, we compared K = 5 and K = 10 in computational time. We also compared the computational time between the original file by map 1 and the chunk numbers of the original file by map 12. From the results, we can observe three facts: (1) the man-made centroid approach requires less computation than the random centroid method for two kinds of cloud environments (four VMs and three machines); (2) computation time for K = 10 is greater than that for K = 5 due to complexity; (3) more maps in the MapReduce framework can result in less computation time for the random centroid case ; and (4) and the man-made centroid is a good start for K-means algorithm for MapReduce.

Figure 10. Results of the experiment in a cloud computing environment with four virtual machines.

Figure 11. Results of the experiment in a cloud computing environment with three machines.

7. Conclusions and future work

Based on the remote sensing images collected by the UAV flown by the GIS research center of Feng Chia University, we proposed a cloud-based framework for massive remote sensing image storage and processing. We also demonstrated how to process real remote sensing imagery with Matlab and ENVI on a single machine and by a proposed novel K-means method using a cloud computing environment. This framework also implemented image management by communicating with metadata units and data management units; furthermore, users can pass parameters to the WebGIS prototype, and this system will show the clustered result with OpenLayers in MapServer. The benefits of proposed platform/system are as follows: (1) it is easy to increase storage capacity; (2) it adds new capabilities (computational power); (3) it integrates with open source software; (4) it accesses UAV imagery or other types of remote sensor data; and (5) it processes the imagery immediately.

Finally, we generated four types of data distributions and simulated concurrent users by threads to test the system. The test involved writing and reading massive image files while observing the performance levels of the local file system and the Hadoop HDFS. The simulation results show that cloud distributed architecture has better efficiency than a traditional server.

When environmental disasters happen, an observer can immediately classify images by a tablet computer without cumbersome post-processing. This article explored a novel K-means image classification and implemented it by means of cloud architecture. In future research, we will focus on the accuracy of the classifications (automatic determination of the number of clusters) and propose a new approach to generate initial centroids automatically. It is to be hoped that this will accelerate the convergence time of the algorithm.

Acknowledgment

This work is supported by the National Science Council of Taiwan, under grant NSC100-2625-M-035-003, NSC100-2119-M-035-001, NSC101-2625-M-035-002, and NSC101-2119-M-035-003. We also thank reviewers for their comments.