Earth observation brain (EOB): an intelligent earth observation system

Abstract

Since the twenty-first century, with the rapid development of high-resolution earth observation satellites, the earth observation satellite system has developed from the initial single satellite observation model to the current satellite constellation formed by light and small satellites observation model. All-weather and all-directional fine earth observation can now be realized. In the future, the satellite constellation, communication satellites, navigation satellites, and aircrafts are linked through dynamic linking network to form an air-space information network to realize real-time services of intelligent air-space information. To further enhance the perception, cognition, and quick response ability of the network, we propose the concept and model of the Earth Observation Brain (EOB) − the intelligent earth system based on events perception in this paper. Then, some key technologies needed to be solved in the EOB are also described. An application example is illustrated to show the process of perception and cognition in the primary stage of the EOB. In the future, EOB can observe what change of what object, the when and where to push these right information to mobile terminal of right people at the right time and right place. Global users can obtain any data, information, and knowledge in real-time through the EOB.

Keywords:

• Earth Observation Brain (EOB)
• intelligent earth observation system
• space-air-ground co-observation
• spatial perception and cognition
• emergency response ability

1. Introduction

In 1957, the Soviet Union launched the world’s first man-made earth satellite, “Sputnik 1,” officially opened the human “sky age.” This changed the history of human beings, in that up to this time, mankind could only observe the earth’s surface from the ground or air for thousands of years. Since 1957, human beings have launched weather, resources, ocean, and other satellite sequences. A variety of spatial information that human beings need can now be obtained by optical and radar remote sensing satellites (Teillet et al. 2002; Loiacono, Germain, and Smith 2004). Therefore, these satellites are collectively referred to as earth observation satellites.

Over the past 20 years, with the rapid development of aerospace and aeronautical remote sensing technology, the spatial resolution, spectral resolution, and temporal resolution of earth observation satellite have greatly improved. In particular, in 1999, the successful launch of the first commercial high-resolution remote sensing satellite, IKONOS (US Space Imaging company with I meter spatial resolution) opened a new era of commercial high-resolution remote sensing satellite (Li 2001; Goward et al. 2003). As such high-resolution satellites brought great military and civilian benefits, they have attracted much attention in the fields of civil and military applications (Cichy, Davies, and Tran 2005; Cordeau and Laporte 2005; Fritz et al. 2008; Li 2008). Countries around the world have now developed such high-resolution remote sensing satellites. The United States, France, Russia, ESA, Japan, India, and several other countries and institutions have launched a large number of high-resolution remote sensing satellites, such as World-View series, GeoEye-1, Peidase-1A/B, SPOT series, ALOS, and Cartosat-1/2. China has also launched a series of high-resolution resource satellites of resources series (Li 2003).

On the basis of continuous improvements in satellite performance, the observation of earth observation has changed from the traditional single satellite model to satellite constellations formed by light and small satellites to meet shorter re-visit period, wider range of observation coverage and quick response, continuous dynamic monitoring and other data access needs, based on specific task target requirement (Baysal 2004; Li and Shen 2005). With the successful launch of the SPOT-7 in 2014, the Airbus Defense and Space Company’s planned four-satellite constellation consisting of SPOT 6 & 7 and Pleiades 1A & 1B is finally completed. The US DigitalGlobe company has built a constellation consisting of WorldView-4 and WorldView-3. In addition, the SkySat satellite constellation has been developed by SkyBox Imaging. The Dove series satellites were developed by Planet Labs to construct the Flock-1/1b/1c/1d/1e constellation. The Kestrel Eye satellite constellation has been developed by the US Army’s Space Missile Defense Command. Additionally, the United Kingdom Surrey Space Center and United States NASA are on the mobile satellite (the combination of smart phones and microsatellites) and chip satellite (a single integrated circuit contains all the spacecraft functions) research demonstration and preliminary experiments, marking construction of light and small remote sensing satellite constellation as the focus of the development of earth observation in the world.

Observations by on-board multi-satellite constellation, at the same time, will obtain hundreds of TB of images per day (Zhang 2011). Taking China as an example, by 2020, dozens of new high-resolution remote sensing satellites will be developed to achieve 30 to 50 remote sensing satellites at the same time in orbit. A conservative estimate of the amount of data acquired will reach hundreds of TB per day. The model of traditional satellite remote sensing data acquisition and processing which is data acquisition on-board, data transmission between satellite and ground and receiving, processing and distributing data on the ground processing center cannot meet the application requirements nowadays (Feng et al. 2011). Therefore, the task-driven satellite-ground co-processing mechanism, which takes the user’s task demand as the core, becomes the development direction of remote sensing image data acquisition and processing. In this task-driven satellite-ground co-processing mechanism, the following functions would be carried out: optimizing the allocation of satellite and ground data acquisition resources, computing resources, storage resources, transmission resources and receiving and processing resources, and making full use of various algorithms of satellite-ground co-processing, such as high-quality real-time imaging algorithm of multi-source sensors, high-precision real-time geometric positioning algorithm, intelligent data compression algorithm, typical target intelligent detection algorithm, and change detection algorithm. According to the different terrestrial task information (geographical location, observation area size, and target type), model, and process of satellite-ground co-processing is intelligently planned to realize automation and intelligent satellite-ground co-processing. Thus, it can quickly provide high-precision, high-quality, and highly reliable information needed for the decision-making mission (Mouginis-Mark et al. 1991). The earth observation system will enter a new era of intelligence (Li 2012).

With the development of aeronautics and astronautics technologies, a kind of space network composed of satellite-ground coordinated intelligent earth observation system, communication satellites, navigation satellites, and aircrafts is gradually formed. Through the dynamic link networking, the space network can do real-time access, transmission, and processing of massive data to realize air-space information application (Li et al. 2014). This space network is called an air-space information network. The global surface sub-meter level spatial resolution, hourly time resolution of the data acquisition, and meter-level precision navigation and positioning services can be achieved using such an air-space information network. With the support of space time big data technology, cloud computing technology and an air-space information service intelligent terminal, and using a space and earth integration communication network, these will enable one to achieve seamless global interoperability, to provide positioning, navigation, timing, remote sensing, and communication for all sectors of the national economy, the industry, and the majority of mobile phone users (Li 2016).

The twenty-first century is the era of intelligent information. Accordingly, combining with brain science, cognitive science, psychology, statistics and computer science, and knowledge of other fields, human hope to create a machine similar with the human’s brain to promote the human analysis and make a decision, through automatically and intelligently accessing, processing, and analyzing the data. In the field of geo-spatial information science, taking advantage of the theoretical knowledge in the fields of brain science with cognitive science, it will enhance the perception, cognition, and decision-making ability of air-space information network, and hence further enhance the intelligence level of the air-space information network. In short, the air-space information network has similarities with intelligent perception, and the cognitive ability of the brain to form the Earth Observation Brain (EOB).

2. The EOB

2.1. Development of brain science and cognitive science

Artificial intelligence, as one of the three cutting-edge technologies in the twenty-first century, is a simulation of information processes of human’s consciousness and thought. Artificial intelligence involves the theoretical knowledge in the areas of brain science, cognitive science, psychology, statistics, and computer science (Li 2009). Especially, the improvement of brain science and cognitive science, that is the origin of artificial intelligence, has a significant role in promoting the development of artificial intelligence. Intelligence is the core concept in the field of brain science and cognitive science. Since 1979, Cognitive Science Society was established in the United States, many venerable institutions have setup their own brain science and cognitive science research teams, which greatly promoted the development of brain science and cognitive science (Hall 2013). In order to occupy the summit of the artificial intelligence field, countries around the world have carried out a variety of research projects on brain science and cognitive science.

In 1990, the United States pioneered the “Decade of the Brain” project. Europe followed the United States to launch “The European Decade of Brain” project in 1991. Japan also planned to invest two billion to launch the “Brain Science Era” project to achieve the goal of “Understand the Brain,” “Protect the Brain,” and “Create Brain” in 1996 (Edelman 2006). In 2013, the United States launched “BRAIN Initiative” project (The White House 2013). Then the European Union also launched “The Human Brain” Project (He and Liu 2016).

China has also attached great importance to the development of brain science and cognitive science in recent years. In the National Medium and Long-Term Science and Technology Development Plan (2006−2020) , the “Brain Science and Cognitive Science” is, as China’s long-term science and technology development planning, of one of the eight cutting-edge sciences. In 2012, the Chinese Academy of Sciences launched a strategic pilot science and technology special (Type B), the “Brain Function Linkage Map” project. On this basis, China’s “Brain Science” Project is being planned. In the future, the project implementation will promote the development of brain science, brain disease prevention, and artificial intelligence to achieve significant research results and strengthen the research team of brain and cognitive science in China.

Over the past two decades, scientists in the field of brain science and cognitive science have cooperated and communicated with each other. Scientists have cooperated and integrated multidisciplinary theories and experiments to promote the understanding of the nature of the human mind, and to obtain many theoretical and practical achievements. These achievements provide an important guiding ideology and theoretical basis for the intelligent development of geo-spatial information science. Perception and cognition abilities of brain science and cognitive science are integrated into the air-space information network system to enhance the perception and cognitive abilities of the system, and enhance the intelligence level of the air-space information network system. The system can quickly acquire data, extract useful information, and drive the corresponding application to achieve the perception, cognition, and action of the brain, and thus to form the EOB.

2.2. Concept of EOB

The EOB is an intelligent earth observation system which simulates the brain perception and cognition process. Under the air-space information network environment, the EOB can integrate measurement, calibration, target perception and cognition, service users and other abilities. The EOB is, by means of the remote sensing satellite constellations and communication navigation constellations, airships and airplanes, able to obtain the spatial data and information of the Earth’s surface, and then to use on-board image processing technology, data analysis technology, and other technologies to obtain knowledge and information to service users decision-making. The whole process realizes the real-time intelligent earth observation with satellite and ground co-processing.

The EOB is just like the human’s brain that possesses visual, auditory, and other functions. Information on the environment around the human can be obtained using visual, auditory, and other abilities, and then be delivered into the left and right hemispheres using neurons. The left and right hemispheres can analyze information to lead the human’s behavior (Morrison and Hof 1997; Illes 2010). Similarly, the EOB also has the abilities of vision, auditory, and brain analysis. The schematic diagram of the EOB is shown in Figure 1.

Figure 1. Schematic diagram of the earth observation brain model.

The remote sensing satellite constellation, the navigation satellite constellation, and the airborne-spacecraft network serve as the visual function of the EOB (Li and Shao 2009). A series of observation data, such as the image of the target area, the geographic spatial coordinates, and so on, can be obtained in real-time through the visual function of the EOB. Communication satellite constellation is the auditory function of the EOB. The communication among the satellite observation, the navigation satellite constellation and the airborne-spacecraft network can be realized through the auditory function. In addition, the communication and information transmission among the EOB, the ground processing center, the individual user, and the user units can be also realized. In the EOB, remote sensing satellites, navigation satellites, communication satellites, airborne-spacecraft network, and other equipment, also act as the brain cells of the EOB to realize the analysis function, which are similar with the brain cells in the human’s brain. According to user’ needs, the observed data are processed and analyzed by the brain cells of the EOB to obtain the information of users requirements (Ling et al. 2013).

The end-to-end integration for customer service is realized by the EOB. The ground clients, such as individual users and user units send a demand signal to the ground control center. The ground control center performs the task coding as per the demand signal, and then to send the task coding to the EOB. As per the task coding, the EOB schedules the existing remote sensing satellites, navigation satellites, and the airborne-spacecraft network, to obtain the target area image and position. The brain cells of the EOB are used to perform the target detection and information extraction on the on-board observation data, to obtain the information requested by the user. This user information is then transmitted to the user through the auditory function in real-time, to realize the synergistic real-time earth observation (Li et al. 2012). The EOB can answer When, Where, and What changes of What objects (4 W), and to push these Right information to the Right people at the Right time and Right place (4R), achieving 4 W real-time information to push the 4R users (Li and Shao 2008). Global users can obtain any data and information they need in real-time. The whole observation process, from the receiving task instruction, the target area data acquisition, data processing analysis, and information download to the user is achieved by the EOB in real-time.

The air-space information network provides Positioning, Navigation, Timing, Remote Sensing, and Communication (PNTRC) for users in different fields.

(1)	Positioning & Navigation: Providing better than meter-level high-precision real-time navigation and positioning information for the various types of users (including terrestrial mobile phone users) (Li 2005).
(2)	Timing: Providing high-precision time information and time synchronization information.
(3)	Remote Sensing: At all-day and all-weather, real-time acquire and process data and video of the remote sensing to obtain information users need. Subsequently, through users’ mobile phones and various types of intelligent mobile terminals, information is pushed to users.
(4)	Communication: Providing security, reliability, high-speed integrated communication, and data transmission services for global users to overcome the limitations of the terrestrial communications network coverage.

The air-space information network is then intelligently upgraded to become the EOB, which greatly enhances perception, cognition, and guiding the users’ action abilities of earth observation system. Except enjoying the basic PNTRC services, users can also enjoy Perception and Cognitive (PC) services using the EOB. The users put forward the application demand to the EOB. According to the users’ needs, the EOB can perceive the target area to obtain useful information, and then push useful information to users to help users make decisions. According to the decision-making, users can take appropriate actions.

3. Key technologies in the EOB

In the processing of providing Positioning, Navigation, Timing, Remote Sensing, Communication, and Perception and Cognitive (PNTRC + PC) services, in addition to the research application of navigation, remote sensing, communications, and other fields, some key technologies such as on-board image processing and information extraction, satellite distributed collaborative computing, intelligent perception and cognitive, and other functions need to be solved.

3.1. Real-time data processing and rapid generation of information on satellite

Due to the large amount of satellite imagery and video and limited data storage and transmission on the satellite, data acquisition, processing, and analysis functions are integrated into the EOB to achieve real-time data processing and rapid generation of information. In the whole processing chain from on-board data acquisition to information extraction, it is necessary to carry out image preprocessing operations such as radiometric correction and geometric positioning on the satellite data in real-time (Aumann 1994). The on-board intelligent detection and extraction of the typical targets are carried out on the preprocessed images. Subsequently, images are intelligently compressed on the satellite. Compressed images are transmitted to maximize the effective data down-stream under limited bandwidth. The whole process needs to break through the real-time radiometric correction and geometric positioning of on-board image, on-board intelligent detection and extraction of the typical target, intelligent compression of on-board image, and other technical problems.

3.2. Distributed collaborative computing technology of data on the satellite

The whole system realizes the end-to-end service. All the data acquisition, information extraction and intelligent perception, and cognition are done on the server terminal, the EOB. Therefore, the robust calculation and processing ability on the satellite is needed, so as to put forward higher requirements on the hardware. First, the intelligent interfaces on the satellite need to be researched, so that the task command of the ground processing center can be accurately received by the on-board processing platform. Second, the lighter body, larger storage capacity, and stronger heat resistance of the disk storage arrays need to be developed. Third, the on-board computing processors which have a better performance than current processors, need to be researched, like the current Digital Signal Processing (DSP) or Field-Programmable Gate Array (FPGA). Finally, due to the implementation of distributed collaborative computing of satellites, it is necessary to study the technology of cooperative computing for all satellites.

3.3. Data intelligent processing technology

Because of the perception and cognitive abilities of the EOB, it is necessary to investigate more intelligent algorithms for processing and analysis of the data to obtain useful information and timely feedback to the user. In this way, the user can understand the target area and make decisions. The research on efficient storage and query technology of multi-source spatial data makes the system obtain the required data efficiently and accurately in the process of perception and cognition. The research on the intelligent spatial data mining technology makes the system obtain the hidden knowledge and information from images and geospatial coordinates, to achieve the system’s perception and cognitive ability (Di, Li, and Li 1999). The research on geospatial data-driven application technology makes codes for data and information perceived by the EOB. These codes are accurately transmitted to the client. And then the user’ action is guided.

4. Application examples of the EOB in the primary stage

This paper gives an example to illustrate the basic process of detecting and locating the time sensitive targets in the primary stage of the EOB, as shown in Figure 2. After receiving a request for the detection of the ship from the user, the EOB uses the vision function − optical remote sensing satellites on-board imaging. Then, the radiometric correction, target detection, and geometric positioning processing of the image are done on the on-board processing platform to extract the effective information. The effective information is transmitted to the ground processing center using the auditory function of the EOB, communication satellites (Li and Li 2001). According to the needs of the user, the ground processing center will quickly push the effective information to the user to help the user make a decision. During the whole process, the user can quickly and accurately detect and locate the target on the ocean, and then make a decision.

Figure 2. Ocean target detection and location of intelligent observation architecture.

In the process of obtaining the effective information in the primary stage of the EOB, the effective information the user needs is extracted using the acquisition and data analysis abilities of the vision function of the EOB. The effective information is transmitted to the ground processing center using the auditory function of the EOB. Subsequently, the ground processing center will push the effective information to the user. The whole process realizes the automatic information extraction of the specific target in real-time and the intelligent perception and cognitive ability of the EOB. The research results have been applied in practice.

5. Conclusions and prospects

After more than 10 years of rapid development of high-resolution remote sensing satellites, the earth observation satellite system has been developed from the initial single satellite observation model to the current satellite constellation formed by light and small satellites observation model. All-weather and all-directional earth observation can be realized to make revisiting period shorter, and coverage greater of earth observation and rapid response, continuously dynamic monitoring and other actions on task specific target demands. Nowadays, the earth observation system has a good ability of measurement, calibration, and data acquisition, but there is still a lack of perception and cognitive ability. For the development of the future intelligent earth observation system with the perception and cognitive ability, the concept of the EOB is put forward. The connotation and the key technologies needed to be solved for the EOB have been introduced in detail, to point out the research directions for the realization of the EOB.

The EOB is the intelligent earth observation system based on events perception and cognitive ability, to provide positioning, navigation, timing, remote sensing, communication, and intelligent perception, and cognitive servers for users in different fields. In the future, through the EOB, global users can obtain any information of the world at any time and any place using terminal equipment, which will undoubtedly promote the popularization and commercialization of the EOB. However, we should realize that many of the techniques and equipment in the EOB are not yet realized, and that much more scientific research is required to realize the EOB.

Funding

This work was substantially supported by the National Natural Science Foundation of China [grant number 91438203] and the National Basic Research Program of China (973 Program) [grant number 2014CB744201].

Notes on contributors

Deren Li is currently a professor and chair of the Academic Committee of the State Key Laboratory for Information Engineering in Surveying, Mapping, and Remote Sensing (LIESMARS) with Wuhan University, Wuhan, China. He was elected as the Academician of Chinese Academy of Sciences in 1991, and Chinese Academy of Engineering and Euro-Asia Academy of Sciences in 1995. His research interests include photogrammetry and remote sensing, global navigation satellite system (GNSS), and geographic information system (GIS), their innovation integrations and applications.

Mi Wang is currently a professor at the State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing (LIESMARS), Wuhan University. His research interests include the measurable seamless stereo orthoimage databases, geographic information systems (GIS), and the integration global navigation satellite system (GNSS), remote sensing, and GIS.

Zhipeng Dong is currently a PhD student with the State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing (LIESMARS), Wuhan University. His current research interests include artificial intelligence, pattern recognition and remote sensing image processing and information extraction.

Xin Shen is currently an associate researcher with the Collaborative Innovation Center of Geospatial Technology, Wuhan University. His current research interests include orbit design of remote sensing satellite and imaging task planning.

Lite Shi is currently a deputy director of the Administration Office of the State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing (LIESMARS), Wuhan University. She is in charge of international exchange in LIESMARS.