An automatic counter for aerial images of aggregations of large birds

Abstract

Capsule Counts with precision comparable to those of an expert observer can be achieved automatically.

Aims To develop an open-source freeware program that automatically extracts and counts the number of birds in aerial images of dense aggregations.

Methods Software was developed and validated using aerial photographs of Greater Flamingo colonies from France, Turkey and Mauritania. We used a stochastic approach based on object processes. The objects representing flamingos were defined as ellipses, and the software detected these and produced a count. Details of the procedure are provided in electronic supplementary material. The energy is extracted by a birth-and-death algorithm, leading to the total number of birds.

Results This procedure gave counts with precision comparable to manual counts of the aerial photographs by an expert (<5% difference). The procedure does not need any supervision of the extraction by an operator, thus reducing the processing time required to get the estimate of bird numbers in the image from >4 h to 20 min on average.

Conclusion This open-source freeware should be useful for ornithologists and wildlife managers.

Monitoring the size of bird populations has become a routine activity for many ornithologists and wildlife managers. This allows managers to detect population trends and highlight possible environmental or anthropogenic changes which negatively affect species. Whereas capture–mark–recapture methods have gained enormous importance in diagnosing the underlying causes of population declines, adding census data in an integrated modelling approach has proved to improve predictions of population trajectories greatly (Besbeas et al. 2002, Gauthier et al. 2007). Hence, estimating the size of bird populations remains a major prerequisite for the conservation and management of bird species.

For several gregarious bird species, wildlife managers take the opportunity of breeding and migrating aggregations to monitor variations in population size. This task is accomplished using photographs to comprehensively record the birds present in the given area. Birds are then counted on the photographs, either projected or printed, which avoids bias resulting from real-time visual estimation (Leonard & Fish 1974, Boyd 2000, Béchet et al. 2004). However, counting individual birds, even in photographs, can be time-consuming, particularly for dense congregations. This is the case for geese, penguins and flamingos, which often breed in colonies of several thousand individuals and sometimes up to over 1 million. Although it is feasible to count predefined sample areas and use interpolation procedures, this complicates the detection of temporal trends due to the variance factor associated with the resulting estimate (Béchet et al. 2004). The recent development of high-resolution image scanners and digital photography enables automated counting methods based on computerized image analysis (Bajzak & Piatt 1990, Trathan 2004).

General models for object detection based on classical image processing tools such as mathematical morphology, template matching, or colour threshold segmentation (Gougeon 1995, Pollock 1998, Larsen 1999) appears to be unsatisfactory when there is a high degree of overlap between birds in the flocks. Other approaches may be available in commercial software solutions (e.g. ecognition; Milton et al. 2006), yet they are too expensive for most wildlife managers and scientists, especially those working in developing countries. Our aim was to develop a tool to count large numbers of birds automatically from images which would have the following properties: (i) provide accurate counts on a wide range of photograph types that would be most comparable with counts from an expert; (ii) be user-friendly; (iii) be free and open-source. The tool was developed and validated on aerial photographs of Greater Flamingo Phoenicopterus roseus colonies. We developed an approach embedding a geometric model of the bird silhouette in the image and some constraints on their spatial repartition. We considered an ellipse as our reference object to model flamingos. Indeed, on aerial photographs, incubating flamingos look like ellipses. The back of incubating flamingos is mainly covered with white feathers which gives a feature to evaluate the contrast between the background and the flamingos themselves. The extraction of flamingos on the image then relies on an energy minimization routine, where the energy term is decomposed into a data term to locate flamingos on the image, and a prior term which introduces constraints between the objects of the configuration.

Here, we present the model rationale, its validation, and discuss the minimal image quality required to get reliable estimates from the open-source freeware that we have developed.

MATERIALS AND METHODS

In the Mediterranean basin and West Africa, the size of several Greater Flamingo breeding colonies is monitored annually by aerial surveys at the peak of the laying season (Cézilly et al. 1995, Balkız 2006, Diawara et al. 2007, Béchet & Johnson 2008). The proposed image-processing model was developed on a set of aerial photographs from colonies in the Camargue (France), Tuz lake (Turkey) and Kiaone Islands (Mauritania). In France, flamingos breed on an artificial island in a commercial salt pan. Flamingo nests are dense, so that, from the air, incubating birds overlap. In Turkey, Greater Flamingo nests are spread out with no overlap between birds. In Mauritania, flamingos do not erect a nest but lay their single egg on the rocky top of the island. Incubating birds can overlap and there is little contrast between the flamingos and their background. Photographs were taken at 300 m asl either with 35 mm film (100 iso) or digital (5 M pixels) reflex cameras. Basic and high-resolution true 2400 dpi optical scanners were used to digitize the slides from the film cameras as high-resolution scanners were not available to us before 2005 (Table 1).

Table 1. Image type, size, spatial resolution and approximate number of pixels covering an incubating flamingo on the photographs used to validate the model.

Image	Type	Size (px)	Spatial resolution(px/m)	px/flamingo
Camargue 2002	Low-resolution scan	5028 × 3408	20	10
Camargue 2005	High-resolution scan	5664 × 2752	25	13
Tuz 2004	High-resolution scan	3000 × 2037	20	10
Tuz 2006	Digital photograph	3504 × 2336	10	5
Kiaone 2005	Digital photograph	3008 × 2000	8	4

The marked point process model

Model rationale

We model aerial images composed of flamingos whose positions and attributes are some realization of a marked point process X (see Van Lieshout 2000 and the supplementary material, available online via the Supplementary Content tab of the article's online page at dx.doi.org/10.1080/00063657.2011.588195). X is also a random variable whose realizations are random configurations of objects x belonging to a state space. We define an energy U(x) that takes into account some interactions between the geometric objects (the prior energy U _p (x)), and the way they fit to the data (the data energy U _d (x)):

Extracting flamingos from the image consists in minimizing this energy. In our case, the model used to extract flamingos consists of a marked point process of ellipses (see online supplementary material). As we aim to detect individuals in dense populations, we model flamingos as possibly slightly overlapping ellipses. Then, the prior energy U _p (x) that introduces interactions penalizes configurations according to the overlapping objects area (see Perrin et al. 2006 and online supplementary material). Each object is penalized depending on the maximal overlapping it exhibits with neighbouring ellipses (Fig. 1).

Figure 1. Variations of the overlapping coefficient used to penalize ellipse configurations resulting in overlapping areas.

Data energy

On the image, a flamingo can be modelled as a bright ellipse surrounded by a darker background. To evaluate the contrast between the ellipses and the background, we calculate a distance between the reflectance distributions of the object and its boundary. The data energy U _d (x) associated with the object x is then defined as a quality function which gives some positive value to small Bhattacharyya distance (weakly contrasted objects) and negative value (well-located objects) otherwise (Bhattacharyya 1943; Fig. 2 and online supplementary material).

Figure 2. An ellipse and its boundary used to calculate the adjustment of ellipse location by the Bhattacharyya distance.

Parameter estimation

This quality function depends on a threshold parameter d ₀, which defines the minimum contrast between the object and the background for a detected bird. To estimate this parameter, we first consider an initial value, low enough and empirically set to 10, and compute the data term obtained with this value on each pixel for a disk whose radius is the minimal size of a bird. This allows us to compute a predetection map where pixels belonging to birds are favoured because their data term is good (low data energy U _d (x); see online supplementary material).

Flamingo colour is often heterogeneous within a given image because of the properties of the camera (focus, focal length, distortion and colour uniformity). Thus, we use a local parameter estimation method to estimate the mode of flamingo colour from the initial image in small squared areas (whose size is proportional to the size of the flamingo on the picture). d ₀ is then computed by taking into account this flamingo radiometry estimation (see online supplementary material).

Birth and death dynamics

In order to optimize the model, we consider a simulated annealing based on a birth and death process (Descombes et al. 2008). The algorithm simulating the process starts by initializing the temperature parameter and the discretization step and then alternates birth and death steps, by adding new objects to the current configuration or removing objects from the configuration, respectively. The convergence is obtained when all the objects added during the birth step, and only these, have been killed by the death step (see online supplementary material).

The software was developed in C++ and packaged to be executable on GNU Linux or Microsoft windows platforms.

Model validation

The set of images used to validate the model consists of two film photographs from the Camargue colony, one digital photograph from the Kiaone colony and two photographs from the Tuz colony in Turkey (one film and one digital). On the resulting digitized images a flamingo is represented by at least four pixels (Table 1). The model was validated by comparing counts from the model to those of an expert (AA) on three square samples, selected randomly over each image. Sample sizes were adjusted for each photograph to contain a sufficient number of flamingos to be counted (Fig. 3). These samples were also counted by five non-experts. We compare the mean difference of these non-expert counts to the count of the expert in order to classify the samples as Easy, Medium or Hard to count. Three measures are used to evaluate the difference between the model results and those of the expert: (i) the percentage of birds missed by the model, (ii) the percentage of false alarms, and (iii) the percentage of difference between the two estimates.

Figure 3. Configuration of the square samples used to validate the models on the Camargue colony (top), the Tuz colony (middle) and the Kiaone colony (bottom).

Finally, we present the comparison between the counts given by the model and the expert on the whole image for each photograph. We also provide the time required by a computer (Intel® Core duo™ CPU 2.53 GHz) to provide a count of the whole colony, and the time required by an expert to count the same image by hand.

RESULTS

The false alarm rate varied from 0 to 14.7%, and less than 12.9% of the birds were missed by the model (Table 2). The overall error was <5% in 11 of 15 samples, and <2% in 9 cases out of 15. The performances of the model were independent of our a-priori classifications of the difficulty to count the image.

Table 2. Counting difficulty (mean difference with expert counts), number counted by the expert, number counted by the model, rate of missed birds, false alarms and overall error of the model compared to the expert.

Image	Difficulty	Expert	Model	Missed (%)	False alarm (%)	Overall error (%)
Camargue 2002
Sample 1	Hard (11)	226	224	7.0	6.7	–0.9
Sample 2	Medium (4)	175	177	1.7	2.8	1.1
Sample 3	Easy (2)	128	126	3.1	1.6	–1.6
Camargue 2005
Sample 1	Medium (8)	153	164	1.9	8.5	7.2
Sample 2	Medium (7)	152	156	2.6	7.7	2.6
Sample 3	Medium (8)	111	129	2.7	16.3	16.2
Tuz 2004
Sample 1	Easy (2)	101	102	1.9	3.9	0.9
Sample 2	Easy (3)	84	83	0	0	–1.19
Sample 3	Easy (1)	55	51	7	0	–7.3
Tuz 2006
Sample 1	Easy (2)	213	215	0	0.9	0.9
Sample 2	Medium (7)	314	313	1.3	1.6	–0.3
Sample 3	Medium (7)	326	324	0.6	0.3	–0.6
Kiaone 2005
Sample 1	Hard (13)	317	338	5.4	12.4	6.6
Sample 2	Hard (15)	362	377	7.2	12.2	4.1
Sample 3	Hard (11)	443	436	12.9	14.7	–1.6

When considering the whole image, the model performed well in most cases (Figs. 4 & 5) and gave counts with good precision compared to manual counts by an expert (<5% difference; Table 3). Computation took between 11 and 35 min on a whole image, depending on both the number of objects to be extracted and the size of the image. Hence, as this approach does not need any supervision of the extraction by an operator, it reduces from >4 h to 20 min on average the processing time required to get the estimate (Table 3). Even though the Kione Island photograph was difficult because it had low contrast and was not completely vertical, which decreases the number of pixels per flamingo, the model extracted the ellipses correctly.

Figure 4. Top: initial image of the Camargue colony. Bottom: sample of the extraction.

Figure 5. Sample of the extraction on the Kiaone colony. Only the ellipse centre is pointed.

Table 3. Estimated number of birds from the model, time needed by the software to provide an estimate, estimated number of birds from the expert, time required from the expert to provide an estimate, and percentage of difference between the two estimates.

Image	Model	Time (min)	Expert	Time (min)	%
Camargue 2002	9,696	29	10,068	420	–3.69
Camargue 2005	9,934	25	9,582	420	3.67
Tuz 2004	3,597	11	3,684	120	0.14
Tuz 2206	2,379	23	2,375	105	0.17
Kioane 2005	13,845	10	13,650	390	1.43

DISCUSSION

Our model performed well in extracting flamingos on several types of aerial photographs and is thus a cost-efficient tool to count aggregations of various bird species rapidly. Differences between counts from the expert and from the model were in the range (<5%) of other image processing methods (i.e. 3–6%; Trathan 2004). In most cases, the disagreement between the expert and our extraction in the validation samples corresponds to ambiguous cases where no decision can reasonably be taken from the image alone (Table 2). The efficiency of the detection remained good on low-resolution images, where it is even difficult for the human eye to distinguish flamingos one from another.

The major advantage of the extraction model is that no user supervision is needed. In comparison, methods based on colour threshold filters (Gilmer et al. 1988, Bajzak & Piatt 1990, Trathan 2004) need both pre-processing and pre-analysis of bird colour characteristics on the image. Furthermore, this approach does not provide good results when birds overlap. Finally, the execution time to obtain flamingo population extractions is quite reasonable because of the proposed birth and death process, which outperformed classical RJMCMC schemes (Descamps et al. 2009). Execution time also outperformed the expert. Furthermore, even if the method had taken longer than a human expert, but did a comparable job, it would still remain extremely useful, as it is very easy to run the computation overnight, or on multiple computers.

The software built upon the model, flamingo, is distributed under the CECILL-C licence (a French derivative of the LGPL; http://www.cecill.info/) and is available for download at http://flamingoatlas.org/dwld_flamingo.php. It does not need any third-party software (e.g. mapinfo, arcgis) to run. Applications of this new tool are numerous. We believe it could be used to make exhaustive counts on photographs of bird aggregations which were previously restrained to samples because of large population sizes (e.g. Flamingo census in the African rift valley; Woodworth et al. 1997). The software is very flexible regarding the type of image material provided. It gives reliable results from digital images made by modern, easily affordable high resolution digital cameras as well as from low resolution scans of 35 mm films. It may thus be advantageously used to count black and white photograph archives after they are scanned. Also, as satellite resolution gets better (e.g. QuickBird satellites; Sasamal et al. 2008) it should be possible to count large bird colonies directly from these kinds of images. The model is not restricted to nesting flamingos as it has been elaborated for ellipses. Hence, it should perform well for any large aggregations of birds producing this kind of shape on a picture, such as nesting gannets, wintering and moulting ducks or geese. As shown by the Kiaone Island photograph presented here, the picture does not need to be perfectly vertical for the model to provide reliable counts. Nevertheless, we expect results to be better in the case of vertical photos. The model also performs well on images where birds are darker than the background (e.g. flamingo fledglings in a creche; Descamps et al. 2009) or show dark bird species such as scoters (Melanitta spp.). In this case, it is necessary to indicate to the software at the start that the background is lighter than the objects to be counted. In both cases, a sufficient contrast between the ellipse and the background is nevertheless required. However, as the algorithm relies on the extraction of ellipses, it provides poor results when attempting to count birds in flight (e.g. large flocks of Snow Geese Chen caerulescens) or large penguin colonies (e.g. King Penguins Aptenodytes patagonicus) photographed from a vantage point. Hence, the software is not suitable for this type of survey. Finally, there should be no problem in counting vertical photographs of large flocks which necessitate several overlapping pictures, as the software allows interactive delineation of the part of the flock to be counted in each picture.

Because integrated modelling promises to make increased used of bird population counts (Besbeas et al. 2002, Gauthier et al. 2007), we believe this software should be useful for increasing synergies between field ornithologists and scientists in providing accurate count data.

ACKNOWLEDGEMENTS

The authors would like to thank Lucie Schmaltz, Mathieu Berson, Benjamin Phéline and Christophe Germain from Tour du Valat for their participation in model validation as non experts. We acknowledge Lisa Ernoul for her revision of the last version of the manuscript. Images used for model validation were provided by Özge Balkız from Doğa Derneği (Turkey) and by Yelli Diawara (Parc National of Banc d'Arguin, Mauritania) through projects funded by Tour du Valat and Mava Foundation as part of the Mediterranean and West Africa Greater Flamingo Network. This work has been partially supported by the COLOR ‘Flamant’ funded by INRIA Sophia Antipolis.