Evaluating an image analysis system for mapping white clover pastures

Abstract

We have developed an image analysis system for mapping white clover pastures. The information from digital colour photographs is processed by software (Trifolium.exe) specially designed for the purpose. The software estimates the coverage of clover, weeds and bare ground, and the unidentified remainder of the total area is regarded as covered by grass. To evaluate the reliability of the estimates of clover content, the clover on paper printouts of non-processed images were marked manually and analysed by a photo scanner and commercially available software. The outputs from Trifolium.exe and the estimates from scanned manual markings were highly correlated (r ²=0.81). A sensitivity test was conducted to quantify the impact of changes of six user-adjustable parameters of Trifolium.exe. The software output of clover coverage was sensitive for changes in three, soil coverage for changes in one, and weed coverage for changes in all parameters. The fact that the digital image acquisition and analysis produce nearly continuous and exactly positioned data, implies further that it is a very appropriate tool for analyses of spatial dynamics in grass-clover pastures.

Keywords:

• clover coverage
• image analysis
• Trifolium repens
• two-dimensional data

Introduction

The proportion of clover in grass-white clover swards fluctuates both temporally and spatially (Steele & Shannon, 1982; Rickard & McBride, 1986; Cain et al., 1995; Loiseau et al., 2001), and the systems are regarded as inherently unpredictable and difficult to manage (Frame & Newbould, 1986; Kessler & Nösberger, 1994; Frame et al., 1998). Several models for the interaction between the two species have been developed in order to understand the mechanisms behind the fluctuations (Thornley et al., 1995; Schwinning & Parsons, 1996a,b). Further theoretical analyses and improved understanding depend, however, on more detailed recordings of what takes place in the field in space and time. Hitherto, the proportion of clover in the swards has been determined by sorting of samples from bulk harvests, which to a certain extent masks clover patchiness, or by laborious counting of plants on rather small areas.

Image analysis is a means for extracting quantitative information (i.e. numerical data) from an image and involves conversion of the original picture to grey level images, thresholding, edge extraction (Jain, 1989) and different morphological operations (Soille, 1999). Image analysis systems (i.e. tools for gathering data from an image) have been used in plant science to estimate the chlorophyll content of leaves (Kawashima & Nakatani, 1998), measuring wheat senescence (Adamsen et al., 1999) and to quantify turfgrass cover (Richardson et al., 2001). With in ambition a desire to scale up the investigations in grass-clover fields with regard to efficiency and accuracy, we have developed an image analysis system enabling us to map such fields semi-automatically. The method involves obtaining and processing information from digital colour photographs, and the outcome is pixel classification files with a resolution of at least 1600×1600 pixels m⁻² and maps of the field with a resolution of at least 400×400 pixels m⁻². The design, reliability and applicability of the image analysis system is presented and discussed in the present paper.

Material and methods

Image acquisition and analysis

The images analysed in the present investigation were taken in June 2001, 10 days after cutting of the grass-clover swards of an experimental field established in an eight-year-old pasture at Mære agricultural school (63°56′ N, 11°25′ E, 40 m above sea level). The species sown in 1993 were smooth meadow grass, Poa pratensis L.; white clover, Trifolium repens L.; timothy, Phleum pratense L. and meadow fescue, Festuca pratensis Hudson. At the time of image acquisition, smooth meadow grass and white clover dominated the pasture. The timothy and meadow fescue had nearly disappeared and there was a low weed content. The area to be photographed was shielded from sunlight by an opaque parasol, and the internal blitz of the camera was used to eliminate shadows and ensure even illumination. A 0.5 m×0.5 m² movable metal frame placed on the ground delineated the squares of the field which were photographed in sequence and later combined into a complete picture of the investigated pasture. The camera was mounted on a rack on a metal frame such that the distance from the camera to the ground was kept constant at 80 cm. A digital Olympus Camedia C-3040 Zoom camera was used, and the images were stored in a compressed format that gave a spatial resolution of approximately 1600×1600 pixels m⁻².

The first step of the image analysis by the software especially developed for the purpose (Trifolium.exe) is extraction of the metal frame by thresholding the image intensity, defined by:

where R, G and B are the red, green and blue colour components of the image, and x and y are the pixel coordinates. This converts the original colour image to a grey level image with values between 0 and 255, where 0 corresponds to black and 255 corresponds to white. The threshold was set to 170, as this removes the majority of the pixels which do not belong to the frame. This gives an image where the frame is represented by ones and the background is represented by zeros. The position of the frame is then determined by computing column and row sums of the binary frame image and finding the points where these sums exceeded a certain threshold, defined as 1.2 times the mean value of the row/column sum. That part of the image which is within the frame is the region of interest, and is processed further.

The coverage of bare soil and dead plant material is estimated by thresholding the image defined by:

where r and g are the red and green colour components divided by the image intensity, I. The 1 is added to avoid dividing by 0 and does not affect the classification of soil as long as an appropriate threshold is chosen. Thresholding I _RG gives a binary soil image where pixels classified as soil have value 1 and all other pixels are 0. The default Soil Threshold value is 0.6 because this value gave the most correct classification of the images used during the development of the software, but the user of the programme may change the value if the red/green balance of the images differs among the data sets. Decreasing the Soil Threshold gives fewer pixels classified as bare soil and dead plant material. Pixels with very low intensity are also classified as soil, but very small separate regions (less than 4×4 pixels) are set to zero because they are most probably noise or leaf shadows. This method does not make it possible to distinguish between bare soil and dead plant material.

The clover coverage is estimated by making use of the fact that clover leaves are larger and more circular than grass leaves. First, the colour image is transformed to a monochrome grey level image by carrying out principal component analysis (Gonzales & Woods, 1993) and retaining only the first component. This transform selects the image component with the largest variance (Fig. 1b). The edges of the grey level image are then computed by the Sobel method (Jain, 1989) with a user-defined Gradient Threshold. By choosing a suitable threshold, the clover leaves will appear as open areas, whereas areas with grass will have many edges (Fig. 1c). For the image in Fig. 1a, a Gradient Threshold of 21 was applied. The edge image and the binary soil image were combined to avoid classifying areas as both soil and clover at the same time. Erosion and dilation are morphological operators that shrink and dilate an object (Soille, 1999). An object is here defined as a set of connected pixels with value 1. After computation of the edges, most of the grass leaves and clover leaves are separate objects. By eroding and dilating the combined edge image an appropriate number of times, the thin grass leaves will be removed while the larger and rounder clover leaves will be left. Before the dilation, single pixels are removed because they are more likely to be grass leaves than clover leaves. The number of Erosions depends on the size of the clover leaves and the resolution of the image. The larger the clover leaves, the higher number of Erosions may be applied. Increasing the image resolution also requires an increase in the number of Erosions. For our images with 1600×1600 pixels m⁻², four Erosions were chosen, but the user may change this parameter. The resulting areas may also include large weed leaves. In order to remove these, all areas above a specific area threshold were marked as weeds. Areas marked as weeds may, however, consist of clover leaves that grow so close together that they are detected as one area. This may to some extent be avoided by requiring that area-to-perimeter ratios of the areas marked as weeds are higher than a certain threshold. The initial value of the Weed Area/Perimeter threshold was chosen to be 6.5. The Smallest Weed Area and the Largest Weed Area depend on the size of the weed leaves, so these parameters must be adjusted according to the size of the weeds that are likely to occur in the images. Initial values are 7000 and 13000 pixels, which correspond to approximately 27 cm² and 51 cm².

Fig. 1 Outputs at sequential steps in the analysis of a digital colour image (a) by the software Trifolium.exe, (b) grey level image, (c) edge image, (d) classified image, where red indicates clover, blue indicates grass and black indicates soil.

The output of the programme is a file with the classification of each pixel in the images of the field, and a map of the field where each classification image is one quarter the size of the original image. The programme output also includes a file with the coverage of clover, grass, soil and weeds for each image in the field, and a pixel classification file and map of the field with a resolution of 100×100 pixels m⁻².

Software test

To test the software Trifolium.exe, six series of 10 images each were randomly selected from the 1536 images from the acquisition of the previously described experimental field in June 2001. None of them were used during the development of the software. Firstly, the 60 selected images were analysed by Trifolium.exe using the default parameter settings that were assessed during the development of the software. Thereafter, for each of the 60 images, the Trifolium.exe output from the run with default settings was qualitatively checked against the original images. If there were visible deviations, the parameter settings were changed image-wise until the software analysed the clover coverage as correctly as possible. Then, for a final analysis of the 60 images with Trifolium.exe, the parameters were changed to their average values from the run with image-wise adjustments of settings.

As a basis for a statistical evaluation of the software output, all the 60 test images were printed on a black and white laser printer. On the printout the clover leaves were manually covered with black ink. The ink soaked the sheet, and on the other side of the sheet we got distinctly marked black ‘clover leaves’ on a white background. The black-white sheets were then scanned using an EPSON Expression 1600 scanner (www.epson.com), and WinRhizo software (www.regent.qc.ca) was used to determine the area of the black ‘clover leaves’. Since there was no substantial overlap of clover and grass leaves, this method gave a very accurate estimate of the clover coverage. Both a deviation based statistics, root mean square error (RMSE), and linear regressions were used to quantify the relationship between clover coverage as estimated from manual markings and from the different runs of Trifolium.exe.

Furthermore, 10 other images not used in the previously described tests were selected for a sensitivity analysis to evaluate the impacts of changes in the parameter settings (Pannell, 1997). As a base-case, the default parameter values of the adjustable parameters were used: Erosions, 4; Gradient Threshold, 21; Soil Threshold, 0.6; Smallest Weed Area, 7000 (27 cm²); Largest Weed Area, 13000 (51 cm²); and Weed Area/Perimeter, 6.5. By varying the setting of one parameter at a time, the coverage of clover, bare soil and dead plant material, and weeds were re-estimated and related to the tested parameter and the base-case output. Based on the result of the sensitivity analysis a range of 30 combinations of Erosions (E) and Gradient Threshold (GT) were established. The 60 test images were then run by Trifiolium.exe for all the 30 combinations. A plane of the squared deviations (SD) between the Trifolium.exe output of clover coverage and the manually marked clover coverage was determined using the model procedure of the programme package SAS (SAS Institute Inc. 1999). The model was:

The combination of E and GT that minimized SD was then calculated using:

Results and discussion

In the 60 images where clover coverage was estimated both from manual markings on paper printouts and by Trifolium.exe at default settings, the coverage was assessed as 7.2% and 6.0% respectively. The underestimation by the software increased with increasing clover coverage (Fig. 2a, Table 1). The estimates from the software and the manual analyses were, however, highly correlated (r ²=0.81) (Table 1), and the software distinguished well between low and high clover coverage in the range investigated (0.0–22.3%) (Fig. 2a). In the same run of the software, the average coverage of bare soil and plant material was as high as 34.5%, reflecting that the images were taken 10 days after cutting of the swards. The average coverage of weeds was 0.2%. The mismatch between software estimated and manually estimated clover coverage was not related to the coverage of bare soil, dead plant material and weeds in the images.

Fig. 2 Clover coverage (%) in digital images estimated by the software Trifolium.exe run with different parameter settings plotted against coverage estimated by scanning of manually marked printouts of the same images.

Table 1. Goodness of fit between clover coverage estimated by scanning of manually marked paper printouts and by the software Trifolium.exe run with four different parameter settings, E=Erosions, GT=Gradient Threshold, SWA=Smallest Weed Area

Parameter settings	RMSE ^1	α ^2	β ^2	r ^2,3
Default	2.77	0.88	0.71	0.81
Adjusted image-wise (variable)	3.34	0.30	0.64	0.89
Average (E=4.1, GT=20.4, SWA=6500)	4.41	−0.03	0.55	0.79
Optimized (E=3.4, GT=22.17)	2.49	1.23	0.75	0.81
^1 RMSE: root mean square error of the clover coverage (%) estimated by Trifolium.exe.
^2 Fitted parameters of the linear regression clover coverage estimated by Trifolium.exe and the coverage of the manually marked clover. The parameter ‘α’ represents intercept and the parameter ‘β’ represents the slope.
^3 r ^2: coefficient of determination for the relationship between clover coverage estimated by Trifolium.exe and the coverage of the manually marked clover.

To improve the fit between the coverage estimated by Trifolium.exe and the coverage assessed by manual markings, the software output was qualitatively checked against the original images. If there were visible deviations, the parameters of the software were adjusted such that all the clover leaves were marked with a red colour. This adjustment resulted in an even larger underestimation, although a lower scattering (r ²=0.89) (Fig. 2b, Table 1). It thus seems that Trifolium.exe systematically misses small parts of the clover, and it is probably the outer edges (pixels) of the leaves that are not classified correctly (Fig. 1a, d).

In line with this, the use of average values of the parameter settings based on the image-wise adjustments did not result in any increments in assessed clover area either (Fig. 2c, Table 1).

On the assumption of a more correct estimation of clover coverage by manual markings and WinRhizo than by Trifolium.exe, we wished to optimize the parameter settings to achieve as good an agreement as possible. Trifolium.exe was designed with six adjustable parameters because it was regarded as likely that the resolution of the images, their red/green balance, the size of the clover leaves and the weed leaves varied between series of images. To select the appropriate parameters to optimize, a sensitivity analysis was conducted. This analysis revealed that for the selected images the estimation of clover coverage was sensitive for changes in Erosions, Gradient Threshold and Soil Threshold. The higher the settings for Gradient Threshold were, and the lower the settings for Soil Threshold and the number of Erosions were, the higher became the estimates for clover coverage (Fig. 3). The estimates for clover coverage were, however, not influenced by changes in the weed parameters (Smallest Weed Area, Largest Weed Area, Weed Area/Perimeter). The estimates for coverage of bare soil and dead plant material were only sensitive to changes in the Soil Threshold parameter. Minor changes in the Soil Threshold settings changed the estimates for coverage of bare soil and dead plant material radically. In addition, an increase of the Soil Threshold value also resulted in a steep decrease in estimates for clover and weed coverage. The estimation of weed coverage was also affected by changes in the settings for Gradient Threshold and the number of Erosions. The main trend was similar to that of the estimation of clover coverage. An increase in the parameter value Largest Weed Area gave higher estimated values for weed coverage, while an increase in the parameter values of Smallest Weed Area and Weed Area/Perimeter resulted in a lower coverage.

Fig. 3 Relative changes in software estimates for coverage of clover, soil and dead plant material, and weeds in digital images after changes in settings of the parameters: Erosions, Gradient Threshold, Soil Threshold, Largest Weed Area, and Weed Area to Perimeter. Means ±SE for analyses of 10 images (seven with weed coverage) are given.

Based on the results of the sensitivity analysis, a parameter optimization procedure as described in the Materials and methods section was conducted on the Erosions and the Gradient Threshold. The Soil Threshold parameter was omitted due to its very strong influence on the estimates of coverage of soil and dead plant material. The weed parameters were also omitted since changes in them did not affect the estimates for clover coverage. This method resulted in optimum values of the Erosion and Gradient Threshold parameters of 3.4 and 22.17, respectively. Use of those parameter settings gave a closer fit to the 1:1 line than by the use of the default settings (Fig. 2d, Table 1), and the normalised root mean square error for clover coverage, calculated as root mean square error divided by the average value of the attribute, was 0.32. This is slightly lower than 0.36 obtained by Fagerberg & Sundquist (1994) in their analysis of red clover content in organic leys by the dry weight-rank method (Mannetje & Haydock, 1963). Furthermore, our method is faster and has the capability of a complete and very detailed spatial mapping. When fields can be mapped at a rate of approximately 40 m² per h at a resolution of 1600×1600 pixels m⁻², the dimension of future investigations can be radically scaled up compared to what has been possible with manual recordings (e.g. Cain et al., 1995).

The nearly continuous, and two-dimensional, data that the analysed images and maps provide represent a flexible source for various kinds of spatial pattern analysis in plant ecology as stated by experts in this research area (Dale, 1999; Perry et al., 2002). Thus, our image analysis system is an applicable tool for data acquisition in an ongoing study of two-dimensional patterns in grass-clover pastures. Temporal fluctuations in clover coverage might also be recorded with high resolution by the current method, as the images are taken at positions in the field that are well and consistently defined. The accuracy and reliability of the assessment of the spatial distribution of clover between and within images are so far not thoroughly analysed. Visual evaluation of the images confirms that the software is capable of determining the distribution also at pixel level (Fig. 1a, d).

The fact that the image acquisition has to be conducted before the grass and clover leaves overlap each other, is a limitation of the software. It might, however, be developed further to interpret the information presently hidden in the contours of the images. Clausen et al. (2000) did this in order to identify partly occluded weed plants in images from cereals fields. Nevertheless, for our purpose, which is to record dynamics in clover coverage in space and time, a mapping before substantial leaf overlap occurs is satisfactory. As a necessary standardization of the stages of regrowth and phenological development for image acquisition, we have chosen 10 days after cutting. At such an early stage there is a limited disturbance and damage in the fields due to trampling; our impression after six complete mappings of two experimental fields during the years 2001–2003 is that differences among the experimental conditions can be identified at that stage.

Conclusion

Image acquisition and analysis by means of Trifolium.exe appears to be an efficient and accurate method for mapping clover coverage in grass-clover pastures. The present investigation revealed a high agreement between the clover coverage as estimated by our image analysis system and by scanning of manually marked paper printouts of the original images. The fact that the digital image acquisition and analysis produce nearly continuous and exactly positioned data for whole fields implies further that it is a very appropriate tool for analyses of spatial dynamics in grass-clover pastures. The sensitivity test, involving stepwise changes away from the default settings of the parameters, revealed that the software outputs were very sensitive to adjustments. This sensitivity indicates that Trifolium.exe might be flexible enough to analyse images acquired under conditions and in environments different from those for which it was originally was developed, e.g. red clover pastures containing grass species with more erect growth.

Acknowledgments

Senior research technician Anne Stine Ekker is acknowledged for her contribution to this work.

Additional information

Notes on contributors

Helge Bonesmo *

Bonesmo, H., Kaspersen, K. and Bakken, A. K. (The Norwegian Crop Research Institute, Kvithamar Research Centre, NO-7500 Stjørdal, Norway, and SINTEF Electronics and Cybernetics, PO Box 124, Blindern, NO-0314 Oslo, Norway). Evaluating an image analysis system for mapping white clover pastures.

Notes

Bonesmo, H., Kaspersen, K. and Bakken, A. K. (The Norwegian Crop Research Institute, Kvithamar Research Centre, NO-7500 Stjørdal, Norway, and SINTEF Electronics and Cybernetics, PO Box 124, Blindern, NO-0314 Oslo, Norway). Evaluating an image analysis system for mapping white clover pastures.