First extensive and cost-effective quality check of Crisis Maps: presentation of assessment parameters and results

Abstract

The digital representation of disaster situations into maps, mainly based on remotely sensed observations, is becoming a widely used instrument for emergency management. Thousands of maps are being produced all over the world and big attention is paid by international institutions, such as the World Bank, the United Nations and the European Commission to these tools. The quality of crisis maps is a crucial element to ensure effectiveness in the disaster response chain, but it is often neglected with respect to the need for a rapid delivery. In this paper a sample of crisis maps produced between 2005 and 2010 by world leader providers has been evaluated through around 40 parameters assessed by visual analysis and extracted from the validation protocol designed at the Joint Research Centre (JRC) of the European Commission. The maps turned out to be in most cases clearly readable, but some gaps and inconsistencies have been singled out, due to the lack of international standard references. The results are analysed in detail and some remarks are presented.

Keywords:

• emergency maps
• quality assessment
• readability
• metadata
• cartographic standards

1. Introduction

Crisis maps derived from satellite observations are digital representations of the earth surface for situations related to disaster events. In the last decades crisis maps have contributed significantly in disaster management, reaching different actors, from fire brigades, to national civil protections, to over-national institutions, such as the World Bank, the United Nations and the European Commission. This has been evident in particular for dramatic events like the Indonesian tsunami of December 2004 and the Haiti earthquake of January 2010. During crises the exchange of information among the different users, which are independent one from the other, is crucial to allow everyone to contribute to the emergency operations. An effective data exchange is possible only if the information is clear, complete and supplied together with metadata. When fundamental details are missing from geographic data, such as scale and information sources’ resolution, or when it is incomplete, for example when the legend is not clear enough, often it is not possible to exploit them, e.g. for integration with other layers into Geographic Information Systems (GIS).

The aim of this paper is to analyse the quality of a representative number of crisis maps, produced between 2005 and 2010 by five world leader service providers and available for Internet download, identifying and computing a measure of their shortcomings, usually consequence of the compromises due to time constraints. The paper also examines if the weaknesses are acceptable from a cartographic point of view, since it can happen for example that the level of detail of the information sources is not compatible with the map scale.

In general the quality of data is defined as ‘fitness for use’ and some parameters commonly used to measure it are positional and thematic accuracy, consistency and completeness [see US National Committee Digital Cartographic Data Standards Task Force, (DCDSTF 1988)]. According to Borrough and McDonnell (2000) the factors which affect spatial data quality are: currency, completeness, consistency, accessibility, accuracy and precision, sources of errors in data (e.g. choice of original data model or processing) and sources of errors in derived data and in the results of modelling and analysis. These quality parameters have been taken into account in the validation protocol (Broglia et al. 2010, Corbane et al. 2011a) for Crisis Maps that has been developed at JRC and applied in the framework of SAFER FP7 (Services and Applications For Emergency Response, http://safer.emergencyresponse.eu) project. The protocol is based on a set of quantitative and qualitative parameters that have been grouped into four categories: reliability of the information content, consistency of the information support, usability of the product and efficiency of the service. To be applied and to be able to assess all its parameters, the validation protocol requires available reference data (e.g. ground truth), in order to compute, for example, thematic and positional accuracy, and the knowledge of the framework in which the map has been developed, in order to know the specifications that the map had to fulfil. The protocol has been fully applied for small numbers of crisis maps for single crisis events (Corbane et al. 2011a, 2011b). The purpose of this paper is to explore the crisis map production over five years, through the analysis of a large number of maps, considering an extensive and cost-effective approach instead of an in-depth approach. With such large number of maps, distributed all over the world, it would have not been possible to assess spatial and thematic accuracy for a significant number of maps and to know the specific requirements for which they had been produced. The aim of this paper is to assess a subset of quality parameters and to explore the minimal requirements which are necessary in a map to allow its use and moreover its integration into a GIS. Some examples of these minimum requirements are the possibility to understand the geographic position of the disaster, its temporal reference and the map basic metadata, such as scale and accuracy. To verify that these elements are present on the map is fundamental in particular if one considers that during crises there is a huge exchange of data among users and that these maps are available on the Internet for free download. In this perspective a checklist including around 40 entries has been specifically designed, deriving it from the JRC validation protocol.

The novelty of this analysis is that no measure of the quality of crisis maps is available on a large sample of maps. Only the evaluation of the accuracy of single products can be found. In scientific literature the important role of crisis mapping based on satellite products is described. In Allenbach et al. (2005) the huge map production realised in the framework of the International Charter ‘Space and Major Disasters’ is presented. In particular the Charter's operational procedure, the different types of maps that can be produced with respect to the available satellite platforms are treated. In addition the sensitive matter of the time constraints related to rapid mapping are discussed. The same topic is examined in Voigt et al. (2007); in this paper, in addition to considerations about rapid map production and the importance of the International Charter, some examples of applications are presented. In particular some constraints in the choice of processing algorithms for rapid mapping are addressed: whenever time is an issue, an algorithm which allows the best compromise between reliability and processing time must be chosen. For some of the examples it is explained how the positional accuracy has been improved with image ortho-rectification to meet the accuracy requirements. In van den Broek et al. (2009) different approaches for damage assessment for satellite based crisis maps are described.

In Schöning et al. (2009) the topic moves towards the media used to improve the communication of spatial information in crisis response, thanks to the joint use of mobile digital devices and paper maps. Dymon (2003) discusses the critical issue linked to emergency maps interpretation, due to the non-availability of standards for their production. In particular the paper focuses on the non-homogeneity of map symbols, proposing the development of a set of shared symbols. From a different point of view the same topic is discussed in Konečný et al. (2011), where an experiment to assess the map usability with respect to different map backgrounds is described.

From this short review of scientific publications about crisis maps it is possible to see that there are papers about the map production and about their readability, but there are no overall assessments of the quality of maps. The accuracy of single maps or algorithms is discussed, but there is no overview of the quality assessment of the characteristics of a large number of maps.

It is well known that the main characteristics of topographic maps are standardised in terms of legend content and representation and in terms of accuracy with respect to the scale, at least at local level (American Society for Photogrammetry and Remote Sensing (ASPRS) Specifications Standards Committee 1990), and, at European level, the INSPIRE directive (http://inspire.jrc.ec.europa.eu/) has already defined the specifications for base maps. However the same specifications are not yet available for thematic maps, in particular for satellite based thematic maps. At European level, in the next future, an important contribution will be provided by the INSPIRE directive, with the Annexes II and III¹ specifications (concerning for example land cover, land use, natural risk zones), but for the time being no explicit references are available for thematic maps legend definition. In addition to this, crisis maps are a specific subset of thematic maps which are produced with critical time constraints, to allow end-users, such as National Civil Protections, to have in the shortest lapse of time a ‘measure’ of the disaster extent or impact. Some compromises have to be made to be able to deliver data in a very short time. In general these compromises are accepted by the users (Carrion et al. 2011), if the lack in quality does not overcome a certain threshold, which is difficult to set and which can be dependent from the case and from the user.

2. The sample of Crisis Maps produced over 2005–2010

As a starting point for considering the crisis map production, the Indonesian Tsunami of December 2004 as been taken into account and, as an end point, the Haiti earthquake of January 2010. These events are two catastrophic crises that have caused a very big international effort to provide help to the affected population.

Five world leader service providers have been taken into account. These providers, through their websites, make their map production publicly available: those public maps, covering the time period under study, have been downloaded and considered as the population of crisis maps under evaluation for this paper. The authors decided not to analyse the differences among the service providers, but to focus on the population of maps itself.

The total amount of downloaded maps is 2009. The maps have been classified with respect to the type of crisis and the year of production (see Figure 1).

Figure 1. Population of crisis maps considered in this study (January 2005–January 2010).

There are only four maps regarding the Indonesian Tsunami event that have been produced already in December 2004, the most part has been issued at the beginning of 2005. The 2004 maps have not been included in the graphs by year (Figure 1, 2, 3 and 9). On Figure 1 it is possible to see that the Indonesian Tsunami maps represent almost half of 2005 production. It is also possible to notice that the type of event that induced the greatest map production is Flood (38% of downloaded maps), followed by Earthquake and Hurricane (16% and 14% of downloaded maps, respectively).

For the analysis presented in the following it is important to take into account that, for 2010, the Haiti Earthquake maps only have been downloaded, so the 2010 map production is not fully considered.

Out of the population of downloaded maps, a sample of 500 maps (Figure 2 and 3) has been randomly extracted: this has been considered a significant sample (25% of total) to perform a quality analysis of the crisis map production. The random extraction has been performed with the R software, considering a uniform distribution within the population of maps. The uniform distribution has been chosen because the interest of the analysis was to explore the population of maps, considered as a whole.

Figure 2. Sample of maps considered in the study.

Comparing Figure 1 (showing the whole population of maps) with Figure 2 (showing only the sampled maps) it is possible to notice that the sample reflects the population behaviour.

Looking at the spatial distribution (Figure 3) of the sampled maps it is possible to see that there is no homogeneous behaviour over time: in 2005 and 2009 there is a predominance of maps regarding Asia (58% and 50% of the maps of the year respectively), which is not shown in other years. Asia is in general represented in most of the maps: 37% of total, over the five years.

Figure 3. Distribution per continent of the sampled maps over time.

3. The quality assessment checklist

The quality of spatial data is related to their ‘fitness for use’ and can be affected by several factors, which include reliability of the information content, consistency of the information support, usability of the product and efficiency of the service, according to JRC validation protocol. The purpose of this paper is to explore the minimal set of requirements needed to allow the map interpretation and its integration into a GIS. The work consists in a visual evaluation of the main characteristics of the maps, the thematic and positional accuracies have not been computed, for the lack of reference data and for the impossibility to know the context and the specific requirements for each map.

To reach this aim an ad hoc checklist has been designed for a rapid and cost-effective investigation of the quality of maps over time. Among all the parameters presented in the JRC validation protocol, only the ones that can be evaluated by a visual analysis, hence assessing the maps without any ground truth, have been taken into account and eventually adapted to the purpose. In particular, the attention has been focused on map readability and usability.

In Table 1, the complete list of the fields that have been checked on each map is presented.

The map evaluation, based on the checklist, has been performed by 12 trained experts, inside the team. The maps have been explored in digital format, on desktop personal computers. To ensure homogeneity in terminology and to reduce mistakes in data entry, most of the fields have been filled using a drop-down list, through a web interface, and a guide has been provided to the evaluators and discussed with them.

4. Quality check of 500 crisis maps and results

The 500 maps sample has been distributed to the 12 evaluators, who have filled the 43 entries checklist (see Table 1). The evaluators have different backgrounds (five have experience in GIS, five in Remote Sensing, one in Urban Geography and one in Human Perception), all working in a team dealing with crisis events and damage assessment. To ensure a consistent evaluation, the checklist parameters have been preliminarily presented and discussed with the evaluators to agree on a common approach. Moreover the whole assessment has been revised by two evaluators (one being author of this article) to avoid mistakes and possible inconsistencies; nevertheless from Table 1 it is possible to acknowledge that the majority of parameters is not liable to the subjective judgment of the evaluator. Preliminary results had already been presented in (Carrion et al. 2010). In the next paragraphs the results of the assessment are presented, to explore the main map characteristics.

Table 1. List of 43 parameters considered for the map quality check. The parameters in bold have been considered for the global evaluation of the map quality (see Section 4.6.).

Validation protocol category	Parameter
Usability	Service provider
	Type of crisis event
	Type of map
	Date of crisis event (month and year)
	Date of map production (month and year)
	Date of crisis event (day, if present)
	Date of map production (day, if present)
	Place of crisis event – continent
	Place of crisis event – country
	Place of crisis event – town
	First language of the map
	Second language of the map
Reliability of the information content	Information on occlusion of EO sources (clouds, artefacts)
	Time gap between crisis event and crisis image – value
	Time gap between crisis event and crisis image – unit
	Legend semantic definition for thematic data (e.g. Corine Landcover)
Consistency of the information support	Spatial resolution of EO source (in meters)
	Consistency between map and legend symbols
Usability/readability	Contrast between background and thematic entities
	Symbols easily differentiable
	Scale bar
	Declared scale
	Presence of overview map
	Coordinate graticules/grid and its labels
	Presence of interpretation text
	Presence of map title
	Completeness of title: information on geographical area, date of event, thematic content
	Type of map background
Usability	Type of sensor used for information extraction (e.g. flood mask)
	Printing size
	Presence of name of producer
	Information on conditions related to access, use and information sharing
	Responsibility assumption (on a dataset or information sources)
	Consistency between declared scale and resolution of the images used to produce the map
Usability/metadata	Metadata – description of data sources used
	Metadata – description of processing steps
	Metadata – information on quality control procedure used
	Metadata – information on known sources of error
	Metadata – information on spatial accuracy
	Metadata – information on thematic accuracy
	Metadata – point of contact
	Metadata – reference datum
	Metadata – reference projection

4.1. Analysis of the completeness of information describing the event

One of the most important aspects of map readability is the possibility to clearly understand the main information regarding the event. An effective and very short summary of the event information should be shown in the title and eventually in the subtitle: the geographical area of interest, the date of event or of the image used as information source, and the type of thematic content. In this study the information of the title has been considered complete when all three above mentioned parameters are present. The title is complemented by the interpretation text, which contains further details about the event and about the map production. The contingency table representing the completeness of title and the presence of the interpretation text is shown in Table 2.

Table 2. Contingency table representing the completeness of title and the presence of the interpretation text on the map.

	Completeness of title: information on geographical area, date of event, thematic content
Presence of interpretation text	No	Partial	Yes	Total
No	3%	25%	15%	43%
Yes	4%	22%	31%	57%
Total	7%	47%	46%	100%

For only 31% of the maps, the information in the title is complete and the interpretation text is present. The percentage of maps goes to 53% when the interpretation text is present and the information in the title is at least partially complete. It is surprising that in general 43% of maps miss the interpretation text. This could be explained by the fact that some service providers consider the map itself as self-explanatory. According to the opinion of end-users from humanitarian field (World Food Program, oral communication) often during emergencies the time pressure does not allow reading carefully the text displayed on a map, hence it is very important that the map is self-explanatory, even if the added value of the interpretation text is undisputed. For best practice both aspects should be taken into account: the biggest effort should be put in place to allow the reader to understand the main map content just in a glance, e.g. working on the representation of map layers and legend, and the possibility to deepen the understanding should be given providing a short but exhaustive interpretation text.

4.2. Analysis of map readability: contrast and symbology

The map readability involves many aspects; it has been possible to explore only a few of them in the study presented in this paper, such as the quality of contrast and the differentiation among symbols. Usually crisis maps are characterised by a main thematic layer representing the impact area (e.g. flooded area or burnt area), together with some reference layers, such as roads and cities, and, on the background, in most of the cases, there is a topographic map or a satellite image, while in a few specific cases a digital elevation model.

More than half of the sample of maps (53%) presents a good readability in terms of quality of contrast between background and thematic entity and ease in differentiating symbols and 89% of maps presents fair or good values (see Table 3). In general it seems that the map readability is quite high.

Table 3. Contingency table representing the quality of contrast between background and thematic entities with respect to the ease in differentiating the symbols.

	Contrast between background and thematic entities
Symbols easily differentiable	Bad	Fair	Good	Total
Bad	5%	3%	2%	10%
Fair	1%	18%	12%	31%
Good	0%	6%	53%	59%
Total	6%	27%	67%	100%

Often crisis maps display a satellite image as a background. This can be convenient because most of the maps are produced from satellite images and often no other layers, such as topographic maps, are available, in particular in developing Countries. Nevertheless, when possible, the use of a topographic map is preferable, because it has an interpreted content, where only the pertinent elements to each representation scale are shown.

In view of the importance of the type of background for map readability this aspect has been checked. In general, only 23% of the 500 checked maps displays a Topographic map as a background, while most of maps (61%) display a satellite image as a background. Considering the type of map background with respect to the type of event (Figure 4) it is possible to observe that for fires almost all maps (90% of maps of fires) display a satellite image as a background, the same occurs for maps of tsunamis (89%). The presence of DEMs as a background is significant only for flood events, namely they are used in 18% of the flood maps.

Figure 4. The use of different map backgrounds (topographic maps, satellite images or Digital Elevation Models) in the sample of maps, with respect to the type of event.

In Figure 5 the type of map background with respect to the place where the event occurred is shown. It is possible to notice that the type of layer used as background does not change significantly with respect to the place of the event. One could expect that in some Countries, for example in Europe or in North America, the availability of topographic maps is much higher and that this could facilitate their use. On the contrary, in Figure 5 it is possible to see that in Europe and North America the use of satellite images as background is even higher than in other continents.

Figure 5. The use of different map backgrounds (topographic maps, satellite images or Digital Elevation Models) in the sample of maps, with respect to the place of event.

4.3. Analysis of consistency of map scale and resolution of EO information source

The spatial accuracy is of great importance when evaluating a map. Every user, using a map can expect to go in the field with a GPS in hand and to find in a specific location the same coordinates that he can read on the map. For sure it is impossible to avoid a positional error: this error should be compatible with the map scale and should be acceptable for the user.

The positional error can be quantified by assessing the positional accuracy with a number of points with known coordinates. This kind of assessment is not applicable for the evaluation presented in this paper, because of the large number of maps considered and because of the lack of reference data.

However if the map accuracy is not declared (see section 4.5) its level of detail can be deduced from the value of the declared scale. Indeed, according to cartographic standards, the declared scale should correspond to a given planimetric accuracy (USGS 1947, American Society for Photogrammetry and Remote Sensing (ASPRS) Specifications Standards Committee 1990). This has been defined for topographic maps, but since there are no different definitions regarding thematic maps, these standards should be considered as a reference or guideline for any kind of map, at least respecting the order of magnitude of the accuracy required for a specific scale. If, for special reasons (e.g. explicit users' requirements, problems due to data availability or time constraints), these guidelines cannot be fulfilled, it should be clearly mentioned on the map, otherwise a misuse of the map can occur. Besides – of course – it has been checked if the declared scale and/or the scale bar are shown.

The criterion chosen to check the consistency between scale and information sources resolution is the following: the half of the image pixel size must be less or equal than the planimetric accuracy required by NMAS standards (USGS 1947). It is in agreement with the assumption that it is possible to orthorectify the images with an accuracy equal to half the image spatial resolution (Congalton and Green 2009). Anyway, assuming that this is always possible is quite optimistic. This approach has been chosen to use cartographic standards as reference but also to take into account that they have been conceived for topographic map production and not for satellite based thematic maps' production. Images with pixel size lower than 5 m have not been taken into account since their positional accuracy can vary because of many different factors (Congalton and Green 2009), so in this case it is not possible to apply a general rule of thumb to relate pixel size and orthorectification spatial accuracy. Other maps have been excluded, if no satellite images were used or if the image resolution information was not available. As a result, only 169 maps have been checked for the consistency between the resolution and the scale: it has been found that for 22% of maps (out of 169 maps), the declared scale was too large with respect to the resolution of the image used to produce them. This means that, for the subset of 169 maps for which this evaluation has been performed, a user going in the field can find a positioning error greater than the one expected, for one map out of five.

Moving to a fundamental level, considering the whole sample of 500 maps, 12% of the maps miss the declared scale, however the scale bar is practically always present. The declared scale is necessary to quantify the level of detail of the map, if no indication of the spatial accuracy is provided on the map.

Out of 500 maps, nine maps (2%) miss both the declared scale and the scale bar, among these, four maps have at least a complete graticule of the coordinates, but the other five miss also the graticule, therefore on those maps it is impossible to measure any spatial item.

4.4. Analysis of semantic definition of the legend

The semantic definition of thematic maps is an open issue, the only available references for land use are FAO land cover classification system [LCCS (Di Gregorio and Jansen 2005)] and, for Europe, Corine Land Cover (http://www.eea.europa.eu/publications/COR0-landcover) and the specifications of INSPIRE directive (http://inspire.jrc.ec.europa.eu/) which will become operational in the next future (Annex I specifications, regarding mainly base maps, have already been issued, but the ones of Annexes II and III ² regarding for example land cover, land use and natural risk zones are still under test, but already publicly available for download). For the time being the choice of the names of the items for crisis maps is up to service providers' experience and it can be subjective. On maps it is possible to see different definitions of thematic layers which are not coded or standardised and which, if not properly explained, can lead to misunderstandings. For example the meaning of ‘damaged area’ or ‘flood traces’ are not obvious and they can be differently interpreted if they are not accompanied with clear definitions (e.g. impact area: the area in which the percentage of damaged buildings with respect to the total number of buildings per map unit exceeds a given threshold). The presence of a clear definition of the map legend semantic has been checked on the sample of maps. The results are shown in Figure 6.

Figure 6. Presence on the sample of maps of semantic definition of the legend with respect to the type of event.

In 54% of the maps the legend semantic definition is missing, this means that the meaning of the legend items has not been considered as straightforward, while only in 30% of cases the semantic information has been considered complete. More attention should be paid on an explicit definition of the legend items, where possible referring to publicly available definitions or giving ad-hoc definitions or quantitative measures, even if approximate. A big effort should be made at international level to overcome this issue. INSPIRE directive will play a key role in defining and giving the guidelines for the implementation of general map standards at the European level.

4.5. Presence of metadata

The presence of some information regarding the metadata (see Figure 7) has been checked, since the user should be aware about the quality and the characteristics of the product that he has in hand. From the cartographic point of view, the presence of the declaration of the reference datum and projection is fundamental, together with the possibility to read the coordinates on the map thanks to the proper graticule. On the 500 sampled maps it has been assessed that:

• none of the maps presented all the metadata elements that have been checked and only 1% of the maps misses all the elements;

• 6% of the maps misses reference datum or reference projection;

• almost all maps miss information about processing steps, information on quality control procedure used, information on known sources of error, thematic and spatial accuracy;

• almost all maps present information about a point of contact.

It is fundamental from a cartographic point of view to declare on the map its thematic and positional accuracies. Of course satellite based crisis maps present big constraints to the accuracy computation: the significant added value of satellite products is the possibility to map the area without visiting it and the only way to correctly compute the accuracy, in particular thematic accuracy, is to check a certain number of points in the field. This activity cannot be performed, in most of the cases, in the few days or hours dedicated to the map production and without considering additional costs. However an indication on map accuracy or reliability should be provided to the user. This indication could be supplied for example on the basis of previous validations performed on analogous products or on the basis of the service provider's experience. In the framework of SAFER FP7 project, the JRC validation team, external quality control, led by CNES (Centre National d'Etudes Spaciales), the user representatives and the service providers had a long discussion on this topic, which came to the proposal of showing on maps a ‘level of confidence’ (e.g. Low/Medium/High confidence) that the service provider can assign to its product. This is a compromise which, although far from fulfilling completely cartography principles, can be considered a step forward in communicating to the user information on maps' quality and reliability.

Figure 7. Presence of some information regarding the metadata.

4.6. Evaluation of the quality of maps and of its evolution over time

The presented results explore different map characteristics, but do not give a global idea of the quality of the maps. To provide a measure of quality, based on the performed analysis, therefore considering only visual checks, the number of parameters fulfilled by the 500 maps has been computed. For this purpose 25 parameters have been selected from the checklist (the ones in bold in Table 1), taking into account those describing the quality of the maps, that can be considered as fulfilled or not fulfilled (e.g. ‘Completeness of title’ is ‘fulfilled’ if it has ‘Yes’ value). Then for each map the number of fulfilled parameters has been counted. Figure 8 shows the percentage of maps fulfilling at least a given percentage of the chosen parameters. It is possible to observe that almost all maps (94%) fulfil at least 50% of the twenty-five considered parameters, but the number of maps drops dramatically when looking for higher percentages of fulfilled parameters: 24% of maps fulfils at least 70% of parameters and only 3% fulfils at least 80%.

Figure 8. Percentage of maps fulfilling (at least) a given percentage of the parameters (shown in bold in Table 1).

It has been presented in (Carrion et al. 2010) for a first sample of 255 maps that it was not possible to single out a significant evolution of fulfilment of the explored parameters over time. This result has been confirmed on the enlarged sample of 500 maps, in particular considering the evolution of the single parameters results on the examined maps. For this reason in the previous paragraphs the evaluation of parameters has been provided considering the whole explored period and not by year.

To give e general idea on the evolution over time of the evaluated maps, from the perspective of the parameters considered in this paper, the percentage of fulfilled parameters has been computed by year. In Figure 9 the percentage of maps fulfilling a given percentage (50%, 60%, 70% and 80%) of parameters by year, together with the regression line, is shown. A very slight positive trend is visible in all lines, a bit higher for 60% and 70% of parameters (it is important to note that the 2010 sample is the less numerous, see Figure 2).

Figure 9. Trends over time of the percentages of maps fulfilling (at least) 50%, 60%, 70% and 80% of parameters (shown in bold in Table 1).

The trend does not show a significant improvement, at least regarding the parameters explored in this study. Since research is being performed and new sources of information are available, one can expect that the accuracy and the number of typologies of provided products have increased, but from this study it seems that no growing attention has been paid to the quality parameters that have been checked.

4.7. Summary of the results

The main results presented in the previous paragraphs can be summarised in the following. In general, the main map content has been considered easily readable, in terms of layers and symbology interpretation, but not all the details regarding the represented event were always clearly shown and 43% of maps missed the interpretation text. It is not common (23% of maps) that maps display a topographic map as a background, instead of, for example, a satellite image (61%), even if the topographic map is preferable as an interpreted content, specifically designed at a certain scale. For a subset of the map sample (169 maps) it has been possible to check the consistency between the map scale and the resolution of the information sources: one map out of five turned out to be produced from images with a too coarse resolution with respect to map scale.

The legend semantic definition, which is not yet adequately standardised for thematic maps and in particular for crisis maps, has not been considered clear enough for half of the 500 maps.

Metadata regarding some basic cartographic information, such as reference datum and projection and information on the used data sources are almost always present, while more detailed information on the map production (processing steps) and quality (thematic and positional accuracy, known sources of error, quality control procedure used) are almost never present.

5. Conclusions

In this paper five years' crisis map production, from 2005 to 2010, has been explored through a sample of 500 maps out of 2009 downloaded from five world leader service providers' websites. The maps have been visually checked by a trained group, considering a subset of the validation protocol for crisis maps that has been developed at the JRC.

The aim of this work was to give an overview of crisis maps production evaluating a large sample of maps with a set of visually assessed parameters. It is evident that many of the shortcomings that have been singled out can be related to the time constraints which crisis maps service providers have to face, or to specific agreements with the end-users for whom the maps were designed, but in some cases probably more attention could be paid to the map itself, as a cartographic product, to avoid for example inconsistencies between image resolution and map scale.

Metadata are crucial to allow the user understanding if the map is suitable for the purpose, where as ‘user’ not only the user for whom the map has been designed should be considered, but every user who can access the map (available for Internet download), in particular during a crisis, when a significant data exchange occur among different actors. At least a qualitative idea should be given about the map accuracy and reliability (e.g. spatial and thematic accuracy), since it can be impossible to obtain quantitative estimates of those parameters at the time of map production.

The results presented in this paper should raise the attention of the map providers to the cartographic value of crisis maps and encourage them to focus more efforts in trying to provide exhaustive information to the users to allow an efficient use and integration of the maps into their own GIS.

Nowadays data are exchanged in digital format, a big effort is being done to embed metadata into data (e.g. GML or shapefiles); the same attempt could be done for raster images representing maps, enlarging for example the information embedded in the file header: many details regarding maps can be very useful for an experienced user (e.g. processing steps), if not all the information is worth it to be put on the map, it should nevertheless be included in accessible metadata.

In view of the increasing interest that the international community involved in crisis management has in crisis maps, it is desirable that some effort should be put in the improvement in the map as a cartographic product and not only in the map content.

Acknowledgements

The authors wish to thank the ISFEREA team (GlobeSec Unit, IPSC, JRC of the European Commission) for its contribution to the map assessment, in particular, in alphabetical order: Arend De Haas, Christiane Weber, Daniela Poli, Emanuele Angiuli, Giovanna Trianni, Ivano Caravaggi, Lauren Rizzo, Patrizia Tenerelli and Roberta Castoldi.

Notes

1. INSPIRE Annex I, II and III specifications: http://inspire.jrc.ec.europa.eu/index.cfm/pageid/2/list/7, URL Accessed on March, 2012.

2. See note 1.