Determination of Tangerine Volume Using Image Processing Methods

Abstract

In this study, two image processing techniques; namely, segmentation and ellipsoid approximation, were used to estimate the volume of tangerines. The proposed system consisted of two CCD cameras, capture cards, an appropriate lighting system and a computer. The paired t-test showed the estimated volume using segmentation method was not significantly different from the volume determined by water displacement method (p > 0.05). The difference between the volumes estimated by ellipsoid approximation and water displacement method was statistically significant (P < 0.05). The Bland–Altman approach showed the size of tangerines has no effect on the accuracy of volume estimation by segmentation method.

Keywords:

• Tangerine
• Volume
• Segmentation method
• Ellipsoid approximation
• Image processing
• Fruit sorting

INTRODUCTION

Fruit size, dimensions and geometrical attributes are important geometric properties of agricultural products. More specifically, the estimation of mean size and volume of agricultural materials is needed in many applications such as meeting quality standards, increasing market value,[1] monitoring growth, shrinkage, predicting yields and assessing optimal levels of fertilization and irrigation, as well as the design and development of size sorting mechanisms. Size estimation is also helpful in planning food packaging system, transportation and marketing operations. Among the physical attributes of agricultural materials, volume, mass, density and surface areas are the most important ones in sizing systems.[2–5] There are some situations in which it is desirable to determine relationships among geometric dimensions; for example, fruits are often graded by size, but it may be more economical to develop a machine vision system which grades by weight or volume. The size of an agricultural produce is frequently represented by its weight because it is relatively simple to measure. However, volume-based sorting and growth monitoring may provide a more efficient method than weight sorting. In addition, the weight of agricultural produce can be estimated from volume if the density of the produce is known, since one would expect that the weight of a piece of fruit would be directly proportional to its volume. This is because fruit density is almost constant within a batch (with any density variations being small and unpredictable). Simple model equations to estimate the volume of various agricultural products such as potato chips,[5] bell pepper,[6] and kiwifruit[7] from are available. However, measuring dimensions using a digital caliper is subject to human error and may not be an efficient and practical approach to estimate volume, particularly in sorting large quantities of agricultural products indoors or in monitoring yield during harvesting.

In recent years, image processing techniques are found increasingly useful in the food industry, especially for applications in quality inspection and shape sorting. Researches in this area indicate the feasibility of using machine vision systems to improve product quality while freeing people from the traditional manual inspection of agricultural materials. Currently, machine vision is the most effective tool for external feature measurements such as color intensity, homogeneity, bruises, size, shape, shrinkage and stem identification.[8–23] Lan et al.[16] used a machine vision system to obtain images of fissured rice grains. Their system was consisted of a CCD (Charge Coupled Device) black/white camera, an image frame grabber, and a computer. The machine vision system used in their study was able to reveal 94% of all the fissure lines detected in medium rice samples by a human expert and 100% in long rice samples. The use of machine vision is gaining interest for the volume determination of fruit and irregular-shaped objects, because it is a non-destructive method requiring image analyses and image processing operations. A machine vision algorithm using neural networks was developed for estimation of pear volume from 2D digital images.[17] The RMS percentage error using a single digital image was 3%. This percentage was reduced to 1.9% when the volume was estimated from a set of four images. Lorestani et al.[18] developed a fuzzy logic based algorithm for sorting golden delicious apples. Features such as color and size were measured by a machine vision system consisted of apple's sorter, an illumination chamber, a webcam, and a computer. Grading results obtained from the fuzzy expert system showed 91.2% and 95.2% agreement for off-line and online methods, respectively, with the results from by a human expert. Hahn and Sanchez[19] developed an imaging algorithm to measure the volume of non-circular shaped agricultural products, such as carrots. Sabliov et al.[20] used an image-processing algorithm to determine the volume and surface area of axi-symmetric agricultural products. These authors reported that the method could successfully estimate the volume and surface area of eggs, lemons, limes, peaches, and tomatillos. They created a representation of the produce with a set of elementary cylindrical objects of unit pixel height and estimated the volume by summing the elementary volumes of individual cylinders. Wang and Nguang[21] designed a low-cost sensor using the methodology developed in Sabliov et al.[20] to measure the volume and surface area of agricultural products. Bailey et al.[22] demonstrated an image processing approach which estimated the weight of agricultural products accurately. More recently, Koc[23] determined the volume of watermelon using ellipsoid approximation and image processing. Here we extend the ideas presented in Koc[23] in two ways: (1) extend it to the case of tangerine and other axi-symmetric fruits and agri-poducts; and (2) use a more sophisticated image processing method. The objective of this study was to develop an efficient algorithm for volume determination of tangerines based on machine vision. The proposed image processing technique turned out to be quite general and may provide alternative means for estimating volume of other axi-symmetric agricultural products such as melon, kiwifruit, pomegranate, pear, eggs and limes.

MATERIALS AND METHODS

Fifty randomly selected tangerines of various sizes were purchased from a local market. The weight of each tangerine was measured by a digital balance, with an accuracy of ± 0.01 g. The minimum and maximum weights were 59.8 and 99.4 g, respectively (Table 1). In order to determine the volume of tangerines a machine vision system was designed, developed and tested. The proposed machine vision system was consisted of two CCD cameras, two capture cards, an appropriate lighting system, and a personal computer (Fig. 1). A white cardboard was placed on a table to provide a white background. The cameras were placed at right angles to each other in order to capture two perpendicular images on each tangerine. The light source and cameras were mounted on a frame which was attached to the measurement table. A program was developed to capture and record the surface images of the tangerine. Algorithms were all implemented using Visual Basic 6.0 programming language. Each tangerine was placed at the center of the cameras' field of view and two RGB color images were captured. To extract the fruit image, the acquired image was subtracted from the background by calculating RGB values of pixels of fruit image. Obtained RGB values were compared with background RGB values. If the difference between the two was less than 5 (≈3σ), where σ is the standard deviation, the pixel would be regarded as background else it is part of the fruit. To calibrate the system, a spherical object with known dimensions was selected. The acquired dimension by the machine vision system was then compared with the actual dimension of the spherical object.

Table 1 Weight and volumes of tangerines used in this study

		Volume* (cm^3)					Volume* (cm^3)
Sample No.	Mass (g)	WDM	EAM	SM	Sample No.	Mass (g)	WDM	EAM	SM
1	75.6	86.4	83.1	87.4	26	74.7	86.7	79.2	90.4
2	60.9	67.0	59.1	63.2	27	65.9	72.6	69.3	71.8
3	84	100.2	93.6	103.8	28	59.9	71.3	64.1	72.0
4	72.6	80.2	72.9	79.3	29	73.0	82.7	73.4	85.9
5	64.5	73.8	70.7	76.0	30	88.3	99.7	92.0	99.7
6	62.6	69.1	63.8	69.2	31	76.8	85.2	72.6	81.9
7	60.6	64.6	58.7	65.8	32	94.3	98.7	95.3	96.7
8	64.8	72.1	64.7	72.9	33	75.2	82.4	74.8	82.0
9	72.9	82.3	72.7	85.3	34	67.8	74.2	67.5	74.5
10	86.3	96.8	90.1	101.2	35	60.5	67.9	60.5	68.9
11	72.5	80.1	77.2	83.0	36	74.6	81.9	71.7	79.6
12	72.5	79.7	71.2	85.2	37	87.0	100.7	90.9	101.0
13	64.1	69.8	61.0	70.3	38	69.9	77.8	77.5	76.3
14	90.9	107.6	99.9	110.4	39	59.8	63.4	54.8	59.0
15	93.9	109.8	98.7	117.2	40	80.6	88.1	82.1	83.2
16	69.9	75.5	73.1	75.2	41	77.0	95.6	88.6	93.2
17	79.7	95.2	87.4	100.0	42	99.4	113.2	105.0	113.0
18	69.2	73.6	67.4	73.0	43	69.3	79.0	70.5	72.1
19	79.6	92.5	74.9	96.6	44	81.6	89.3	80.7	87.9
20	84.7	92.9	82.8	94.2	45	75.3	81.0	74.3	82.7
21	85.7	95.7	87.9	96.0	46	67.1	73.2	66.5	76.3
22	71.6	77.3	74.3	76.9	47	67.6	76.9	68.6	80.1
23	61.8	64.6	55.3	63.3	48	83.4	95.4	87.1	98.5
24	62.3	68.9	60.2	68.7	49	78.6	87.4	84.0	91.2
25	67.9	76.2	68.5	76.9	50	64.8	70.0	61.4	69.9
*WDM: water displacement method; EAM: ellipsoid approximation method; and SM: segmentation method.

Figure 1 Schematic of developed machine vision system.

Volume Evaluation using Ellipsoid Approximation

Each tangerine was approximated as a uniform ellipsoid. The volume estimation formula for an ellipsoid, a modification of sphere volume Eq. (1), is given by Eq. (2):

(1)

and

(2)

where r is the radius of sphere; and a, b and c are the three perpendicular diameters of ellipsoid shown in Figs. 2 and 3. The three diameters were calculated from the two perpendicular images of the tangerine (Fig. 2). Dimensions a and b were obtained from the first image and c from the second image. Since the three perpendicular diameters of a sphere are equal, the volume of the sphere can be calculated easily by Eq. (1), provided r is known. Similarly, the volume of an ellipsoid can be calculated using Eq. (2), provided the three perpendicular diameters (a, b, and c) are known.

Figure 2 (a) front view: acquire two perpendicular images of tangerine and calculation of b and c diameters; top view: acquire one image with camera 1 and calculation of a diameter.

Figure 3 The dimensions of an ellipsoid.

Volume Evaluation by Segmentation Method

By removing the background and dividing the fruit image into a number of distinct sectors or frustums (Fig. 4 a,b), the overall error was decreased. Surface area of each sector can be calculated using the two perpendicular diameters as in Fig. 4c. The surface area of each sector, (A_i , i = 1,2, …, n), is given by:

(3)

where and are the two perpendicular diameters of lateral surface. The accuracy of estimated surface area was depended on the position of minimum and maximum diameters of the tangerine's surface. The volume of each sector (V_i , i = 1,2, …, n) is then calculated by (Fig. 4d):

(4)

where A_i and A_{i +1} are the surface areas of i and i + 1 segments, respectively; and δ is the thickness of a segment (assuming all segments are of equal thicknesses). Once the volume of all sectors was obtained, the volume of the tangerine (V_t ) can be readily calculated by adding them up:

(5)

where n = 8 is the number of sectors.

Figure 4 (a) Tangerine fruit; (b) segmentation of tangerine; (c) extraction of two diameters of one surface of one sector; (d) thickness of sector and its surface areas.

Actual Volume

The actual fruit volume and its density were determined using the water displacement method.[24,–25] For this purpose, a tangerine was submerged into the known water and the volume of water displaced was measured. Water temperature was kept at 25°C. Specific gravity of each tangerine was calculated by the mass of tangerine in air divided by the mass of displaced water.

Statistical Analysis

The paired t-test and the mean difference confidence interval approach were used to compare the volume determined from the two approaches (ellipsoid approximation and segmentation technique) with that of water displacement method. The paired t-test was used here for testing whether the difference between the two measurements was significantly different. The important feature of this test was its ability to compare the measurements within each subject. Furthermore, the Bland and Altman[26] approach was used to plot the agreements between the tangerine volume measured by water displacement method (actual) and the estimated volumes, using ellipsoid approximation or segmentation method. The statistical analyses were performed with Analysis-it, a Microsoft® Excel add-in program (Microsoft Corporation, Redmond, WA).

RESULTS AND DISCUSSION

Comparison of Ellipsoid Approximation with Water Displacement

The volume determined by the ellipsoid approximation was compared with the mean volume measured by water displacement method. The results are shown in Table 1 and Fig. 5 a. The mean volume difference between the ellipsoid approximation and water displacement was d ₁ = −7.3 cm ³ (95% confidence interval: −8.1 and −6.47 cm ³) with an underestimation by ellipsoid approximation compared to water displacement. The standard deviation of the volume differences was sd ₁ = 2.87 cm ³. The paired t-test results showed that the tangerine volume measured with water displacement method was significantly different than the volume estimated by ellipsoid approximation (p < 0.0001) (Table 2). The volume differences between these two methods were normally distributed and 95% of the volume differences were expected to lie between d ₁ − 1.96sd ₁ and d ₁ + 1.96sd ₁, known as 95% limits of agreement.[23,26] The 95% limits of agreement for comparison of volumes measured with water displacement method and ellipsoid approximation were calculated at −12.91 and −1.66 cm ³ (Fig. 5b). The results showed that volumes determined using the ellipsoid approximation was less than the one measured with the water displacement method. When the ellipsoid approximation method was chosen for tangerine volume estimation, a perfect ellipsoid (such as the one shown in Fig. 3) was used to describe tangerines. However, in reality, the tangerines were randomly selected from an unsorted source, and their shapes were not perfect ellipsoids. The polar radii of tangerines were unequal and were greater than the polar radii of a perfect ellipsoid. Therefore, the volume estimated by the ellipsoid approximation was less than the actual volume measured by the water displacement method. The accuracy of the ellipsoid approximation method was highly depended on the uniformity of the fruit having the presumed shape.

Figure 5 (a) Tangerine volume measured using water displacement method (WDM) and ellipsoid approximation method (EAM) with the line of equality; (b) bland–Altman plot for the comparison of tangerine volumes measured with water displacement and ellipsoid approximation methods; outer lines indicate the 95% limits of agreement (–1.66 ;–12.91 cm ³) and center line shows the average difference (−7.3 cm ³).

Table 2 t-test analyses on comparing volume measurement methods

Paired t-test*		95% confidence interval for the mean difference
WDM & EAM	WDM & SM	WDM & EAM	WDM & SM
Different (P < 0.0001)	Same (P = 0.1032)	−8.1; −6.47	−0.14; 1.45
*WDM: water displacement method; EAM: ellipsoid approximation method; and SM: segmentation method.

Comparison of Segmentation with Water Displacement

A plot of the volumes measured by the segmentation and water displacement methods is shown in Fig. 6a. The mean volume difference between the two methods was d ₂ = 0.66 cm ³ (95% confidence interval: −0.14 and 1.45 cm ³). The standard deviation of the volume differences was sd ₂ = 2.8 cm ³. In this case, the paired t-test results showed that the volume measured with the segmentation method was not significantly different than the volume measured with the water displacement method (P = 0.1032) (Table 2). Further more, the Bland-Altman plot[26] (Fig. 6b) indicated the volume differences between the two methods were normally distributed. The 95% limits of agreement in comparing these two methods were calculated at −4.83 and 6.14 cm ³ (Fig. 6b). Fig. 6b shows that tangerine's size showed no effect on the estimation of volume. Koc[23] reported that as the size of the watermelon increased, the image processing method overestimated the volume. The improved accuracy in the current study can be attributed to a more sophisticated image processing method (two CCD cameras instead of one) used here. In summary, the average percentage differences for estimating the volume of tangerines with the ellipsoid approximation and the segmentation method were 8.9% and 2.6%, respectively.

Figure 6 (a) Tangerine volume measured using water displacement method (WDM) and segmentation method (SM) with the line of equality; (b) bland–Altman plot for the comparison of tangerine volumes measured with water displacement and segmentation methods; outer lines indicate the 95% limits of agreement (−4.83; 6.14 cm ³) and center line shows the average difference (0.66 cm ³).

CONCLUSION

In this paper, two methods for estimating the volume of tangerines were presented. Both methods were quite general and may be readily applied for volume computation of other ellipsoidal agricultural products such as eggs, lemons, limes, peaches, onion, melon, kiwifruit, pomegranate, and pear. The volumes computed showed good agreement with analytical and experimental results. The difference between the volumes estimated by ellipsoid approximation and water displacement method was statistically significant (P < 0.05) while the difference between the volumes estimated by segmentation method and water displacement method was not statistically significant (P > 0.05). The Bland–Altman approach[26] showed that the size of tangerine had no effect on the estimation of volume. The developed method proved to be accurate, precise, and easy to be used. It should be pointed out that the method used here was rotationally invariant and did not require fruit alignment on the conveyor. The method used here for background segmentation was not based on threshold values, and therefore it can be used with other fruits. The present work may be extended in several ways. For instance, the characterization results of citrus fruits showed that the computed volume by segmentation method and measured mass parameters are highly correlated. Therefore, the same algorithm presented here may be used to grade the fruits based on the mass by using the estimated volume information that has already been computed by segmentation method. In fact, the authors have recently extended the algorithms for estimating the mass and surface areas of orange[27] and other citrus fruits.[28] However, further studies are still needed to integrate estimated volume, surface area and mass of fruits with already available color information (RGB or HSI) in devising a fully automatic multi-product sorting system for grading citrus fruits. Research in this direction is currently being continued and the results will be published in near future.

ACKNOWLEDGMENTS

The financial support provided by the Research Department of University of Tehran, Iran, is gratefully acknowledged. Also, the authors would like to thank Mr. T. Bagherpour and Mr. M. Aliakbari for their valuable helps and assistants.