Web-system for processing and visualization of meteorological data for Siberian environment research

Abstract

At present, mainly GIS software is used for exploration and visualization of spatially distributed data. At the same time an approach based on the combination of advanced web technologies along with the standard rules of information-computational system development is more suitable for performing tasks requiring more sophisticated data analysis.

In this report, functional capabilities of the information-computational system created for meteorological and climatic data processing and online visualization are introduced. The system represents a dedicated web-interface, which allows performing of mathematical and statistical operations on the diverse observational and model data and to determine characteristics of global and regional climate changes.

Currently such datasets as NCEP/NCAR Reanalysis, ECMWF ERA-40 Reanalysis, etc. are available for processing. In particular, the system allows calculating of temporal average and extremum values, time trends, etc. at arbitrary spatial and temporal ranges for different meteorological parameters. The possibility of online intercomparison of meteorological characteristics calculated for different datasets is also realized in the system. The final version of the system being developed is supposed to find application in meteorological and climatological investigations and should help researchers to save time during performing routine analytical tasks by simplifying handling of huge arrays of spatially distributed meteorological data.

Keywords:

• climate change
• geoinformatics
• global environmental change
• visualization

Introduction

It is well known that specifics and regional environmental applications of basic Earth system sciences make them multidisciplinary. This requires involving into studies a number of nationally and internationally distributed research groups. Such teams of specialists should perform cooperative work, exchange data and knowledge, and co-ordinate activities optimizing the usage of information-computational resources, services and applications. Also the community acknowledged that stronger involvement of information-computational technologies is required to properly understand dynamics of regional environment and to perform its assessment on the base of monitoring and modeling. This should lead to development of information-computational infrastructure as an inherent part of such investigations (Abel 1998, Gordov 2004a, Gordov 2004b, Frans 2005, Descamps et al. 2006). In particular, recently it was stressed that management of multidisciplinary environmental data coming from large regions requires new data management structures and approaches (Parson and Barry 2006). Thus the contemporary challenge is to save efficiency of such efforts via development of a platform/mechanism providing a collaborative working environment for scientists engaged, as well as giving an access to and preservation of scientific information resources, such as environmental data collections, models, results, etc. As observational and model output datasets in the Earth and Space Sciences increase in availability and resolution, there is an increasing demand from the community for efficient tools simplifying data processing and analysis. In the present Web era such tools should be accessible in the Internet and allow the user to remotely handle huge data arrays and get knowledge hidden in them. Moreover, those tools should be combined into web-based information-computational system forming important elements of Advanced informatics infrastructure that requires the multidisciplinary knowledge integration (see EGU-ESSI Position Paper http://sites.google.com/a/imaa.cnr.it/egu-essi).

This activity is stimulated by strong demand of the community investigating dynamics of Siberia environment under Global Change impact. Really, the regional aspect of science for sustainability and of international global change research is becoming even more important nowadays. It is clear now that regional components of the Earth System may reflect significantly different Earth System dynamics and changes in regional biophysical, biogeochemical and anthropogenic components and also may produce considerably different consequences for the Earth System at the global scale. Regions are ‘open systems’ and the interconnection between regional and global processes plays a key role. Some regions may function as choke or switch points (in both biophysical and socio-economic senses) and small changes in regional systems may lead to profound changes in the ways in which the Earth System operates. International Geosphere-Biosphere Programme (IGBP) suggested (IGBP 2002) to develop integrated regional studies (IRS) of environment in selected regions, which would represent a complex approach to reconstruction of the Earth System dynamics from its components behavior. There are two examples of existing IRS: a matured large biosphere-atmosphere experiment in Amazonia (LBA) (http://lba.cptec.inpe.br/lba/indexi.html) and a recently started ESSP (Earth system Science Partnership) Monsoon Asia Integrated Regional Study (MAIRS) (http://www.mairs-essp.org/). Also IGBP reported that the circumboreal region including Northern Eurasia is one of the critical ‘Switch and Choke’ points of the Earth system, which may generate minor changes in regional systems potentially leading to profound changes in the ways in which the Earth System operates (Groisman 2007).

As a response, a joint program of the Russian Academy of Sciences (RAS) and NASA ‘Northern Eurasia Earth Science Partnership Initiative’ (NEESPI) now has transformed into the international Program and quite recently it was adopted as one of the external projects of IGBP. Currently the Siberia IRS (SIRS) is a NEESPI mega project coordinating national and international activity in the region in line with ESSP approach. The SIRS (http://sirs.scert.ru) current state of the art (Gordov and Begni 2005, Gordov et al. 2006b) is characterized by the appearance of a number of large-scale projects on Siberia environment investigations in line with SIRS objectives and the very beginning their clusterization. Now SIRS as a large-scale multidisciplinary investigation of huge region environment dynamics badly needs an information-computational infrastructure supporting this activity. Due to the efforts of the Siberian Branch of Russian Academy of Sciences (SB RAS) that provided its scientific centers distributed over the region with stable high speed communication channels (Shokin and Fedotov 2003) SIRS became the testbed for the information-computational infrastructure under development and provides major user community for it.

Below yet developed elements of the web-system will be discussed in more detail. To illustrate the system potential, dynamics of some regional climatic characteristics will be analysed and discussed.

The state-of-the-art and approach applied

At present there is a large community dealing with geo-referenced data, which hopes that a backbone for the infrastructure required should be formed on the base of GIS (Gupta et al. 1999, Dragicevic et al. 2000, Peng and Tsou 2003, Frans 2005). In spite of some remarkable achievements (Vatsavai et al. 2000, Yakubailik 2007, http://gis.ict.nsc.ru) we consider using of GIS combined with computational resources required to support modern models and sharing of huge data archives not very promising. The approach relying upon web technologies potential was chosen for development of the information-computational infrastructure required. This approach is based on suggestion (De Roure et al. 2003) that each separate computational task (it also might be a data assimilation task as well or combination of both of the above) can be represented as an information system employing the three-level model – data/metadata (Mccartney and Jones 2002), computation and knowledge levels. Usage of this approach for development of Internet-accessible information-computational systems for chosen thematic domains and organization of data and knowledge exchange between them looks as a quite perspective way to construct distributed collaborative information-computational environment to support investigations, especially, in multidisciplinary area of Earth regional environment studies.

The web-based environmental informatics system (with accompanying applications) representing distributed collaborative information-computational environment to support multidisciplinary investigation of Siberia will form a powerful tool for better understanding of the interactions between the ecosystem, atmosphere, and human dynamics for large Siberia region under the impact of global climate change. On this initial stage major attention was paid to components aimed at handling/processing different datasets coming from monitoring and modeling of regional meteorology, atmospheric pollution transformation/transport and climate, that is important for regional environment dynamics assessment under climate change.

At present only a few web-based information systems devoted to handling of geophysical data are known. The first one is Goddard Earth Sciences Data and Information Services Center (GES-DISC) Interactive Online Visualization ANd aNalysis Infrastructure (GIOVANNI). It is created in NASA and dedicated to organizing access to data processing mechanisms (Acker and Leptoukh 2007, http://daac.gsfc.nasa.gov/techlab/giovanni/). However, it is based on satellite observations and mainly performs visualization of spatiotemporal data. Another one is dissemination system based on Coupled Climate Model of Institute for Numerical Mathematics of the RAS (INM RAS). It is dedicated to visualization of modeling results for different climatic scenarios developed in INM RAS. Processing and analysis as well as access to data from other sources are not supported (http://ksv.inm.ras.ru/gcm_data_plotting/gcm_inm_data.html). The third one is Climate explorer by Royal Netherlands Meteorological Institute (KNMI) (http://climexp.knmi.nl/). It has plenty of different historical observational data, reanalysis data, climatic indices, modeling results and seasonal weather forecasts. Visualization of data is implemented as time series 1-D plots. Visualization of fields of spatial distribution of data as well as processing and analysis are not supported. The model of distributed informational-analytical system (Shokin et al. 2007) for searching, processing and analysis of spatial data based on the combination of GIS and Web technologies currently being developed at the Institute of Computational Technologies of the SB RAS (ICT SB RAS) might also be mentioned. Thus, in spite of the presence of some efforts made in Earth and Space Sciences Informatics research area there is still a lack of powerful tools combining various capabilities to process, analyse and visualize multiple sources data collections using unified web-interface for integrated study of global and regional climate change.

The first step of creation of key elements of demanded information-computational infrastructure was done in the course of development of bilingual (Russian and English) scientific web portal ATMOS (Gordov et al. 2004c, Gordov et al. 2006a) (http://atmos.iao.ru/). ATMOS is designed as an integrated set of distributed but co-ordinated topical web sites, combining standard multimedia information with research databases, models and analytical tools for on-line use and visualization. The main topics addressed are from Atmospheric Physics and Chemistry domain. It should be noted that in spite of the fact that the portal middleware employs PHP scripting language (http://www.php.net/), it has quite flexible and generic nature, which allows one to use it for various applications. Currently on this basis specialized web portals are developed and launched, providing distributed collaborative information-computational environment to organizations/researchers participating in execution of EC FP6 projects ‘Environmental Observations, Modeling and Information Systems’ (http://enviromis.scert.ru/) and ‘Enviro-RISKS – Man-induced Environmental Risks: Monitoring, Management and Remediation of Man-made Changes in Siberia’ (Baklanov and Gordov 2006) (http://risks.scert.ru/). The portals are also powerful instruments for dissemination of the projects results and open free access to collections of regional environmental data and education resources.

Data available

The data of the first and second editions of Reanalysis of National Center for Atmospheric Research (NCAR)/National Centers for Environmental Prediction (NCEP), JRA-25 Reanalysis of Japan Meteorological Agency (JMA)/Central Research Institute of Electric Power Industry (CRIEPI) and ERA-40 Reanalysis of European Centre for Medium-Range Weather Forecasts (ECMWF) are currently used for processing and analysis. These data represent file-based archives and stored on the local high-performance server. The data of the NCEP/NCAR Reanalysis (http://www.cdc.noaa.gov/cdc/reanalysis/reanalysis.shtml) cover the years from 1950 to 2002 and stored in GRIB file format. The more recent data of NCEP/DOE (Department of Energy) Reanalysis AMIP II (http://www.cdc.noaa.gov/cdc/data.ncep.reanalysis2.html) represent a set of NetCDF files and span 25 years from 1979 to 2003. Japan JRA-25 Reanalysis (http://jra.kishou.go.jp/JRA-25/index_en.html) consists of a set of GRIB files and covers the date range from 1979 to 2008. Finally, ECMWF ERA-40 Reanalysis (http://www.ecmwf.int/products/data/archive/e4/index.html) represents a set of NetCDF files covering 48 years from 1957 to 2002. It was converted from GRIB-2 files in order to simplify data access and processing. It is planned to add remote (satellite instruments) and local (meteorological stations) observational data in the future.

Web-system description

The system being considered represents a specialized web-application aimed at mathematical and statistical processing of huge arrays of meteorological and climatic data. Grid Analysis and Display System (GrADS) (http://www.iges.org/grads/) and Interactive Data Language (IDL) (http://www.ittvis.com/idl/) software is used for processing and visualization of the results obtained.

The system consists of graphic user interface, set of software modules written on script languages of GrADS or IDL and structured meteorological datasets. The graphic user interface is a dynamic web form for setting calculation and visualization parameters and designed using DHTML, PHP and JavaScript languages (Figure 1).

Figure 1.  Main window of the system.

The set of software modules consists of independent modules implementing analytical algorithms required for meteorological data processing, switched on with assistance of PHP and executed by GRADS/IDL system, which generates a graphical file containing the results of calculations performed. The latter along with corresponding metadata is passed to the system kernel for subsequent visualization at the web site page. The planned step forward is to provide a user with a data file containing results of analysis along with necessary metadata. This file will comply with ISO 19115 standard in order to provide interoperabilty with other similar systems (http://www.opengeospatial.org/standards/as). The structured meteorological data are stored at the specialized server and they are available only for processing by the system so that the user cannot access data files directly but can freely get results of their analysis.

At present the following climatic characteristics have been chosen for subsequent analysis: atmosphere temperature, pressure and humidity at different altitudes; soil temperature and wetness at different depth levels; precipitation water, water equivalent of accumulated snow depth and geopotential height (Figure 2). Due to the middleware used the system structure is rather flexible and this set can be easily expanded by adding new features to the interface as well as to the executable software.

Figure 2.  Visualizations of different characteristics.

Currently the system allows performing the following mathematical and statistical operations for chosen spatial and time ranges, which are key for the climate assessment: calculation of maximum, minimum, average, variance and standard deviation values, number of days with the value of the chosen meteorological parameter within given range, time trends (Figure 3), as well as time smoothing of parameter values using moving averaging window.

Figure 3.  Temperature trend.

It is also possible to calculate correlation coefficient for an arbitrary pairs of parameters (Figure 4); linear regression coefficients between some pair of characteristics; to determine first (last) cold (warm) period (such as day, week, month) of the year (Figure 5); to calculate 12 climate change indices developed by the Commission of Climatology, Climate Variability and Predictability Project and the Joint WMO-IOC (world Meterological Organization-Intergovernmental Oceangraphic Commission) Technical Commission for Oceanography and Marine Meteorology (CCl/CLIVAR/JCOMM) Expert Team on Climate Change Detection and Indices (ETCCDI) (http://cccma.seos.uvic.ca/ETCCDMI/indices.shtml) (Figure 6).

Figure 4.  Correlation coefficients.

Figure 5.  First warm day.

Figure 6.  WMO: monthly maximum value of daily minimum temperature.

Since there are several datasets containing the same meteorological characteristics within the same time and spatial scales used in the system, one could want to compare datasets with each other. There is such functionality in the system (Figures 7 and 8).

Figure 7.  Dataset comparison parameters.

Figure 8.  Dataset comparison results.

The user interface allows one to choose geographic domain, time interval, characteristic of interest and visualization parameters (Figure 9). For example, a pull down menu ‘Region’ includes the following options: Siberia, Europe, Asia, Eurasia, the whole Earth and that is defined by the user. The last option choice leads to appearance of ‘Longitudinal range’ and ‘Latitudinal range’ fields in which the user should enter coordinates of the geographical domain of interest. The user can also choose type of statistical characteristic, interval for averaging, altitude level, and so on. While calculating averages with moving window, which width can be specified as week, month, three months, half of the year and year, one gets the smoothed sequence of spatial distributions for the characteristic of interest. This set is displayed as an animation, which can be viewed either in an automatic or in a controlled regime. Several screenshots from such sequence are shown including the control bar specifying the viewing mode (e.g. in Figure is 6 and 8).

Figure 9.  Parameters window.

The system will be useful for regional meteorological and climatic investigations aimed at determination of trends of the processes taking place. Also it will simplify the work with huge archives of the spatially distributed data. It would allow scientific researchers to concentrate on solving of their particular tasks without being overloaded by routine work and to guarantee the reliability and interoperability of the results obtained.

Case study: analysis of recent dynamics of regional climatic characteristics

As seen from above, only a few elements of the information-computational infrastructure have been elaborated by this time. However it has proved its efficiency. To illustrate it, recently reported (Melnikova et al. 2007) analysis of dynamics of some regional climatic characteristics is discussed. It should be noted that since the system currently processes fields of meteorological characteristics obtained from different Reanalysis datasets, conclusions derived will characterize these datasets mainly. The latter are not very well correlated with reality for this region due to poor observational network taken into account.

Due to multi-factor forming of atmospheric processes, statistical methods are the most reliable approach for quantitative assessment of characteristics of climatic dynamics and the reported study deals namely with those. In particular, statistical properties of recent variability of precipitation and temperature at two meters in West Siberia were analysed. To support this analysis, a set of functions built-in into the web system was employed. Among those are determination of the first warm/cold day, week and month in a year (variability of the warm season duration); determination of a number of days with daily mean precipitation amount from the chosen interval (variability of the precipitation amount); determination of a number of days with daily mean temperature from the chosen interval (variability of warming); calculation of correlation coefficients for different pairs of meteorological parameters (degree of their linear dependence). Both, parametric and non-parametric statistical criteria were used for analysis. The analysis performed shows that precipitation amount is slightly decreasing during warm seasons and slightly increasing during cold seasons. However homogeneity of precipitation datasets allows one to consider these differences during the last 50 years as insignificant. Analysis of daily temperature dynamics for each season reveals their insignificant difference within the obtained series. However the mean temperatures for larger intervals (weeks, two weeks and month) for each season form inhomogeneous series. Thus one can state that according to NCEP/NCAR Reanalysis and NCEP/DOE Reanalysis-2 data in West Siberia significant increase of mean temperatures takes place during spring and summer. For winter season results obtained reveal more complicated dynamics. Namely, weekly mean temperatures are decreasing, while two-weeks and monthly mean temperatures are increasing.

Conclusions

As for further development of the system firstly we plan to add new meteorological and environmental datasets into the existing data archive as well as to implement new functionality of the web systems under development. Among those a functionality to work with maps, satellite images and RS data should be added. At the second stage we plan to interrelate the developed elements thus arriving at the Distributed Information-Computation System (DICS) based on the approaches described in this paper. It will accumulate data also by means of applications developed for thematic domains and generate thematic data and knowledge, including those ready for computer processing. Sources of this knowledge are tasks and problems of the thematic domain integrated into information systems as applications. The system will be provided with tools, which allow multidisciplinary users to perform calculations correctly and to avoid redundant activity. During the process of user work in the system thematic domain objects and processes in which they are participating will be followed by the data specified by task ontology and domain ontology. Software tools, which allow programmers and knowledge engineers to integrate thematic applications into the system and provide DICS users with supportive work environment, will also be developed and employed by the system. This system must give additional value to existing data by providing archive databases with relevant APIs and descriptions, in order to enable independent agents to discover, query, retrieve results in usable/understandable formats, and transfer them to other agents for further processing. Being generic, it should provide researchers with a reference open platform (portal plus tools) that may be used, adapted, enriched or altered on the basis of the specific needs of particular applications for different regions.

As a whole, this activity will lead to the appearance of a powerful instrument for better understanding of the interactions between the ecosystem, atmosphere and human dynamics in this large region under the impact of global climate change. It will also provide national and international teams carrying out regional environment researches with powerful tools to perform multidisciplinary investigations of Siberia environment under the Global Change. Extended information resources on the state of Siberia environment available to regional environment managers and decision makers via the Internet might raise their concern on regional environmental issues.

Notes on Contributors

Alexander Titov is a researcher at the Siberian Center for Environmental Research and Training, Institute of Monitoring of Climatic and Ecological Systems SB RAS, Tomsk, Russia. His major research areas are Semantic Web technologies and software development for meteorological and climate investigations. His current work is devoted to the development of information-computational framework for environmental data management.

Evgeny Gordov is a Principal Researcher of the Institute of Monitoring of Climatic and Ecological Systems of the Siberian Branch of Russian Academy of Sciences and Director of the Siberian Centre for Environmental Research and Training, the leading organization developing organizational and information-computational infrastructure of the Siberian Integrated Regional Study. He also serves as deputy Chair of Russian National Committee for International Geosphere-Biosphere Program and heads its Siberian Branch. His major research areas lie now in the fields of development of environmental information-computational systems and regional climate dynamics assessment. He is co-editor of Man-induced Environmental Risks: Monitoring, Management and Remediation of Man-made Changes in Siberia (2008) and he has recently contributed to Heiko Balzter (ed.), Environmental Change in Siberia (2009).

Igor Okladnikov is a Senior Researcher at the Institute ofMonitoring of Climatic and Ecological Systems (Siberian Branch of theRussian Academy of Sciences), Tomsk,Russia.Hismajor interest is in the unification and informatization of scientific processes, particularly those intended for global and regional climate investigations. His more recent work focuses on development of websystem for processing and visualization of meteorological and climatic data.

Tamara Shulgina is a PhD student at the Institute of Monitoring of Climatic and Ecological Systems (Siberian Branch, Russian Academy of Sciences). Her major scientific interests include research of climate change processes using methods of mathematical statistics, assessment of climate change impact on the environment, scientific programming.

Acknowledgements

Support by a number of national and international projects is appreciated, partial support by SB RAS Basic Research Program 4.5.2 (Project 2) and Integrated Projects 34 and 86 being specially acknowledged. The project is also partially supported by FP6 EC Projects ENVIROMIS-2 (INCO-CT-2006-031303) and Enviro-RISKS (INCO-CT-2005-013427), as well as Russian Foundation for Basic Research (RFBR) grants 05-05-98010, 06-07-89201-a.