Classifying Healthy and Fungal Infected-Pistachio Kernel by Thermal Imaging Technology

Abstract

Thermal imaging has been considered as a new beneficial technique for inspectional aims in agriculture and also in food safety and food control. In this research, fungal infection caused by KK11 and R5 isolates of Aspergillus flavus fungi was detected by thermal imaging technique. Seven infection stages were considered to be identified by the technique. The features from acquired thermograms from healthy and infected pistachio kernels were extracted, selected, and then classified based on quadratic discriminant analysis and artificial neural network methods in MATLAB 2010 environment. The results showed that thermal imaging successfully can classify healthy and fungal-infected pistachio kernels without considering the isolate type with 99.00% accuracy. This technique separated infection stage one from other stages with accuracy of 86.30%.

Keywords:

• Thermal imaging
• Pistachio
• Fungal infection
• Classification
• Quadratic discriminant analysis
• Artificial neural network

INTRODUCTION

Food safety is one of the most important concerns in agricultural engineering to protect human health. In this regard, food control techniques should be applied to detect undesirable and unhealthy products. Pistachio is one of the best fruits considering its food components and economical rolls. Beside of mentioned advantages, pistachio is more important in Iran due to high production value. One of the main problems in this valuable material is the existence of aflatoxin (AF), a displeasure in food and also as an instability economically.^[1,2]

AF is an undesirable food contamination that is produced mainly by Aspergillus species, in special, A. flavus and A. parasiticus.^[3–7] These fungi can grow on agricultural materials, and also nuts, under satisfying conditions and produce AF after reaching the maturing stage.^[8,9] The presence of these fungi in agricultural material is not a disease, but they are dangerous due to their ability for producing AF contamination^[10] that is a hazardous mycotoxin for human beings.^[11] So, AF-producing fungi must be detected for further diagnostic and protective measurements in order to possibly eradicate fungi, to interdict generation of AF, and/or to prevent infection of other healthy foods.

Conventional techniques such as gas and mass spectrometry, optical spectrometry,^[12] gas chromatography combined with mass spectrometry (GC-MS),^[13] and new techniques such as sensory analysis^[14] and electronic noses^[13,15,16] were applied to recognize fungal infection in diverse materials. Despite reliability of these techniques, they are time-consuming for sample preparation and testing.^[17] Thermal imaging (TI) technology is a new method of non-contact thermometry giving thermal image or thermogram of the targets. In thermograms, the color of each pixel represents the corresponding temperature of that.

This technique has the vast applications for condition monitoring in almost all engineering fields. In agriculture, the TI technique is a useful method with high speed and vast applications beyond temperature determination, such as nondestructive assessing of food quality and reliability in the food industry.^[18]

In the field of food safety, Chelladurai et al.^[17] used the TI technique to classify healthy and fungal-infected wheat kernels. Manickavasagan, Jayas, and White,^[19] detected infestations of Cryptolestes ferrugineus inside wheat kernels. The goal of this article was to classify healthy and fungal-infected pistachio kernels using the TI technique. Also two isolates of Aspergillus flavus fungi and different infection stages were considered to be identified by the technique.

MATERIALS AND METHODS

This research was managed in Mechanical Engineering of Agricultural Machinery Department, and Department of Plant Protection, University of Tehran, Karaj, Iran. Pistachio samples were randomly selected from Akbari variety of Rafsanjan region, Kerman, Iran.

Preparing Healthy and Infected Specimens

First, pistachio kernels were sterilized using an autoclave machine at 121°C for 20 min. Some of the sterilized kernels were separated as healthy specimens and the rest were taken for infecting by Aspergillus flavus fungi. These fungi have many of the most capable isolates needed to produce afflation in oilseeds. This toxin can be formed in oilseed crops in farms and stores.^[20] To study the ability of TI in detection of fungi isolates, around half of the kernels were infected by R5 and the remaining kernels by KK11 isolate. R5 is an AF-producing fungus, whereas KK11 is unable isolate to produce afflation. Several layers of sterilized paper towels and then many infected pistachio kernels were put in each sterilized Petri dish. Each of the five Petri dishes were put in a freezer plastic bag and then were placed in an incubator at 30°C for one week. To investigate on the different fungal infection stages, samples were taken to acquire thermal images after each day, i.e. infection stage one (one day after putting infected samples in incubator), infection stage two, … . and infection stage seven.

Thermogram Acquisition

The implemented system for detecting fungal infection in pistachio kernels has been shown in Fig. 1. To obtain thermograms of the samples, a TI camera (model TI160, Zhejiang ULIRvision Technology Co, Zhejiang, China) with resolution of 160×120 pixels was used (Fig. 1). The spectral range of the camera was 8000–14,000 nm. A Pinnacle card was installed on a personal computer (PC) to capture pseudo-image stream given by the camera. Finally, thermograms were obtained using video-to-frame software.^[21]

FIGURE 1 Thermal imaging system to classify healthy and fungal infected pistachio.

In this research, more than 20 replications^[22] were investigated for each pistachio class. First, the emissivity of the TI camera adjusted to 0.95,^[23] and then three thermograms were acquired for each kernel to study the rate of thermal changes of the kernels; (a) before heating, (b) immediately after heating for 90 s at 90°C, and (c) after cooling for 10 s at room temperature. Three different thermograms have been shown in Fig. 2 for a healthy pistachio.

FIGURE 2 Thermal images of a healthy pistachio kernel, a) before heating, b) after heating, and c) after cooling.

Feature Extraction

An algorithm was programmed in MATLAB 2010 software to extract some statistical features. The features were maximum, mean, minimum, standard deviation, coefficient of variation, median, and mod. As mentioned before, the rate of thermal changes of the kernels was compared. So, the extracted features from thermogram a, b, and c were used to calculate new features. The new features were obtained by subtracting the features of thermogram (a) and (c) from corresponding features of thermogram (b). The new features were used as the inputs of classifier models.

Thermogram Classification

Chelladurai et al.^[17] classified the healthy and fungal-infected wheat kernels by linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA) method. They reported that the QDA classifier had higher performance than LDA classifier. So, in this article, a model was developed in MATLAB 2010 to classify the new features based on the QDA method. Another model was programed based on artificial neural network (ANN) method in the software. This classifier was a feed-forward back propagation network with a hidden layer including 14 neurons. The transfer functions were logsig and purelin. For classification, the data were divided to three different parts for train, cross validation, and test, respectively.

RESULTS AND DISCUSSIONS

Classification results to categorize healthy and fungal-infected pistachio kernels at various fungal infection stages were shown in Table 1. In fact, there were eight different classes; 1-healthy pistachio kernels, 2-infected pistachio kernels at stage one (one day after infection), 3-infected pistachio kernels at stage two, … and 8-infected pistachio kernels at stage seven. But, here only four classification types were considered as follows:

TABLE 1 Classifications results for different infection stages

	CCR (%)
	KK11		R5		All
Classification type	QDA	ANN	QDA	ANN	QDA	ANN
Two-way	98.33	95.60	100.00	95.60	99.00	97.70
Three-way	75.00	85.00	98.33	85.60	69.00	86.30
Four-way	48.33	60.60	43.33	65.00	31.00	64.70
Eight-way	30.00	28.10	38.33	38.80	36.00	31.30

1. Two-way classification: Detecting; 1-healthy pistachio kernels and 2-infected pistachio kernels at all stages of fungal infection.

2. Three-way classification: Detecting; 1-healthy pistachio kernels, 2-infected pistachio kernels at stage one, 3-infected pistachio kernels at stage two to seven.

3. Four-way classification: Detecting; 1-healthy pistachio kernels. 2-infected pistachio kernels at stage two, 3-infected pistachio kernels at stages two and three, and 4-infected pistachio kernels at stage four to seven.

4. Eight-way classification: Detecting all of eight pistachio classes in their own classes.

In Table 1, different stages of fungal infection by KK11 and R5 isolates, separately and together, were classified. For two-way classification, the correct classification rate (CCR) of both classifiers was higher than 95%. In this classification type, the QDA model gave a better result than the ANN model. The CCR of the QDA model for classification of healthy pistachio kernels and infected kernels by KK11 isolate was 98.33%. This amount for detecting infection by R5 isolate was higher than that of KK11 (100.00%). The CCR of the QDA model for detecting infection in pistachio kernel without considering isolate type was a medium result of 99%. These results show that the TI technique can classify healthy and fungal-infected pistachio kernels with high classification accuracy.

Considering three-way classification, the ANN classifier showed higher performance than the QDA model for detecting fungal infection by KK11. Also the ANN model gave better result than the QDA model for classifying the kernels without considering the isolates. So, the ANN model can detect infection stage one from other infection stages with accuracy of 86.30%. The CCRs of both classifiers for four- and eight-way classification types were decreased. Also for these classification types the results of the ANN model were higher than the QDA model. For four-way classification, the CCRs of the ANN model for detecting fungal infection with and without considering the isolates were around 65%. The corresponding CCRs for eight-way classification types were less than 39%.

In Table 1, it can be seen that the difference between four- and eight-way classification types is more than that between three- and four-way classification types. This can be due to low differences between infection stages four to eight. Classification results of different fungi isolates with and without considering infection stages were shown in Table 2. In this table, five types of classifications were considered:

TABLE 2 Results of classifying different fungi isolates

	CCR (%)
Classification type	QDA	ANN
Two-way	52.00	60.7
Three-way	51.00	58.00
Five-way	31.00	52.00
Seven-way	26.00	35.00
Fifteen-way	22.00	19.00

1. Two-way classification: Detecting; 1-pistachio kernels infected by KK11 isolate and 2-pistachio kernels infected by R5 isolate, without considering infection stages.

2. Three-way classification: Detecting; 1-healthy pistachio kernels, 2-pistachio kernels infected by KK11 isolate and 3-pistachio kernels infected by R5 isolate, without considering infection stages.

3. Five-way classification: Detecting; 1-healthy pistachio kernels, 2-pistachio kernels infected by KK11 isolate at stage one, 3-pistachio kernels infected by KK11 isolate at other infection stages, 4-pistachio kernels infected by R5 isolate at stage one, and 5-pistachio kernels infected by R5 isolate at other infection stages.

4. Seven-way classification: Detecting; 1-healthy pistachio kernels, 2-pistachio kernels infected by KK11 isolate at stage one, 3-pistachio kernels infected by KK11 isolate at stages two and three, 4-pistachio kernels infected by KK11 isolate at other infection stages, 5-pistachio kernels infected by R5 isolate at stage one, 6-pistachio kernels infected by R5 isolate at stages two and three, 7-pistachio kernels infected by R5 isolate at other infection stages.

5. Fifteen-way classification: Detecting all 15 pistachio classes, separately; 1-healthy pistachio kernels, 2-pistachio kernels infected by KK11 isolate at stage one, 3-pistachio kernels infected by KK11 isolate at stages two, …, 9-pistachio kernels infected by R5 isolate at stages one, … . and 15-pistachio kernels infected by R5 isolate at stages seven.

In this step, the ANN model showed better ability than the QDA model for classification of the kernels. The highest CCR for classifying the different kernels was obtained for two-way categorization. The CCR was less than 61%. Also, for other classifications, the CCRs of both classifiers were more decreased. According to these results, it can be told that TI technology cannot perfectly classify the infecting isolates. The thermal images of fungal-infected pistachio kernels at the seventh growing stage were classified by the models. The CCR of the QDA model was 59.85%, whereas that of the ANN model was higher (75.00%). This is a promising result to do more research on detection of AF contamination in pistachio kernels.

CONCLUSIONS

The TI technique was applied to detect fungal infection in pistachio kernels. Two classifier models were developed based on the QDA and ANN method to classify the acquired thermal images from healthy and fungal-infected pistachio kernels. It can be concluded that TI technology can successfully classify healthy and fungal-infected pistachio kernels by the developed QDA model without considering fungus isolate and infection stages. After detecting infection, the TI technique, accompanied with the ANN model, can approximately classify infection stage one from later infection stages (CCR = 86.30%). The use of CCR of the ANN model to classify the investigated isolates at final the growing stage is a promising result to manage more research for detection of AF contamination in pistachio kernels, and also in other agricultural products using the TI technique. Finally, the technique can be used as an industrial food safety machine to detect fungal infection in pistachio kernels and to be more developed to detect other diseases in agricultural materials.

ACKNOWLEDGMENTS

Thanks to Mechanical Engineering of Agricultural Machinery Department and Department of Plant Protection, University of Tehran, Karaj, Iran. Special thanks to Eng. Alizadeh, Nazari, Rahimi, and Heidarbeigi for their valuable assists.