Development and implementation of a validation protocol for crisis maps: reliability and consistency assessment of burnt area maps

Figures & data

Figure 1.  (a) Subset of the map of affected urban areas (1:15,000); (b) subset of the map of affected forest cover (1:25,000). North direction is upward in both maps.

Figure 2.  Workflow of the validation methodology used for reliability and consistency assessments of the two crisis maps.

Table 1. Reference data used for the validation of the two rapid crisis maps.

Reference data	Map of affected urban areas	Map of affected forest cover
Remote-sensing data
ALOS AVNIR-2 pre- and post-event images	X
SPOT-4 post-event image	X
Manually digitized forest cover
Two different layers performed by two analysts		X
In-situ data
Perimeter of burnt area	X
Field observations/GPS locations	X

Table 2. Assessment of the total area of forest cover based on visual interpretation of Google Earth data and performed by two analysts; comparison of the total burnt areas obtained from remote-sensing data to the total burnt area estimated by ONF.

Total area of affected forest cover (Km^2)
Map	18.7
Analyst #1	20.0
Analyst #2	18.0
Total burnt area (Km^2)
ALOS AVNIR-2	28.0
SPOT-4	30.0
ONF	30.0

Table 3. Inter-comparison of the masks of burnt areas derived from SPOT-4 (reference) and ALOS images.

Inter-comparison of burnt areas extracted from SPOT-4 (reference) and ALOS (%)
Overall agreement	Commission error	Omission error	Producer accuracy	User accuracy
86.89	0	13.1	86.9	100.0

Figure 3.  Perimeter of burnt areas derived from SPOT-4 (yellow outline) and ALOS AVNIR-2 images (red outline).

Figure 4.  Ground photographs taken in situ and highlighting the limitations of nadir looking remote-sensing data for the detection of understory burns.

Table 4. Error matrix computed with the 38 surveyed sample units. Each unit is representative of 50×50 m² (or 5×5 pixels).

Reference	Burnt	Non-burnt	Total
Map
Burnt	23	0	23
Non-burnt	5	10	15
Total	28	10	38
Overall accuracy	86.84%
Kappa	0.7