Automated prediction of sensory scores for color and appearance in canned black beans (Phaseolus vulgaris L.) using machine vision

Figures & data

Figure 1. Experimental procedure for automatic quality evaluation of black beans.

Figure 2. Color chart for black beans representing five typical categories found in commercial canned beans (note that the color categories were reproduced using the same bean image with different tones).

Figure 3. Appearance chart for black beans after drained representing five typical quality categories found in commercial canned beans.

Figure 4. Original and preprocessed images of canned black beans and brine. A: Original color image after draining and washing, B: segmented beans from the original image (with black background), and C: brine.

Figure 5. Relationship between CIE L*a*b* values (CIELAB D₅₀ 2° observer) for ColorChecker and CDC camera after transformations using a polynomial 20-by-3.

Figure 6. Distribution of visual appearance and color rates of canned black beans using the proposed color charts.

Table 1. Statistics of the data sets for color and appearance rates of the different types of canned black beans.

Types of canned black beans	Number of samples	Color			Appearance
		Minimum	Maximum	Mean	Minimum	Maximum	Mean
Reduced sodium	13	1.9	5.0	3.5 (±0.3)	1.8	4.3	3.0 (±0.2)
No salt added	9	2.5	3.5	3.0 (±0.2)	3.2	4.1	3.4 (±0.4)
Organic	8	2.8	4.7	3.4 (±0.3)	2.1	3.5	2.8 (±0.2)
Low calorie	16	2.6	4.4	3.4 (±0.4)	2.1	3.3	3.0 (±0.2)
Regular^2	23	1.5	4.4	3.2 (±0.2)	1.1	4.5	3.1 (±0.2)

¹: Standard Error of the Mean, .²Samples without any special indication or label were referred to a “regular.”

Table 2. Overall agreement and Kappa analysis among 17 panelists for visual color and appearance of canned black beans at different categories.

Feature	Parameters	5 Categories 1– 2– 3– 4– 5	3 Categories (1, 2) – (3) – (4, 5)	2 Categories (1, 2) – (3, 4, 5)
Color	Overall agreement (%)	38.5	49.6	74.3
	Multi-rater *	0.23	0.28	0.49
Appearance	Overall agreement (%)	32.3	45.4	68.6
	Multi-rater *	0.21	0.23	0.41

*Kappa rates: <0 = less than chance agreement; 0.01 – 0.2 = slight agreement; 0.21 – 0.40 = fair agreement; 0.41 – 0.60 = moderate agreement; 0.61 – 0.80 = substantial agreement; 0.81 – 0.99 = Almost perfect agreement.

Table 3. Calibration and prediction results for color of canned black beans using the average pixel intensity data and standard deviation, and their texture image information extracted from R, G, B, L*, a*, b*, H, S, and V color scales.

Set of features
Color (full seed)	3	0.896	0.33	0.881	0.36	2.1^a
Color (segmented seed)	6	0.917	0.29	0.882	0.36	2.1^a
Color (brine)	5	0.893	0.33	0.873	0.42	1.9^b
Texture (full seed)	6	0.938	0.26	0.900	0.33	2.3^c
Color (full seed + brine)	6	0.941	0.25	0.893	0.36	2.1^a
Color (segmented seed + brine)	6	0.939	0.25	0.896	0.36	2.2^a,c
Color (full seed) + Texture (full seed)	5	0.943	0.25	0.875	0.35	2.2^a,c
Color (segmented seed) + Texture (full seed)	6	0.953	0.22	0.903	0.33	2.3^c
Color (full seed + brine) + Texture (full seed)	6	0.965	0.20	0.929	0.28	2.7^d
Color (segmented seed + brine) + Texture (full seed)	7	0.970	0.18	0.937	0.26	2.9^e

Avg. fact: Average number of features required for the partial least square model after optimization. and : Average correlation coefficients of calibration and prediction, respectively, over four calculations. and : Average standard error for calibration and prediction, respectively, over four calculations.: Ratio of sample standard deviation to .^a,b,c: values in the same column with different letters are statistically different (p < 0.05).

Table 4. Calibration and prediction results for appearance of canned black beans using textural image information (i.e., contrast, correlation, energy, and homogeneity) and their combination with color information extracted from R, G, B, L*, a*, b*, H, S, and V color scales.

Appearance
Texture (full seed)	6	0.856	0.37	0.806	0.47	1.6^a
Color (full seed) + Texture (seed)	6	0.885	0.34	0.822	0.43	1.8^b
Color (segmented seed) + Texture (seed)	7	0.882	0.34	0.822	0.44	1.7^a,b
Color (full seed + brine) + Texture (seed)	7	0.927	0.27	0.834	0.41	1.8^b
Color (segmented seed + brine) + Texture (seed)	8	0.934	0.26	0.871	0.38	2.0^c

Avg. fact: Average number of features required for the partial least square model after optimization. and : Average correlation coefficients of calibration and prediction, respectively, over four calculations. and : Average standard error for calibration and prediction, respectively, over four calculations.: Ratio of sample standard deviation to .^a,b,c: values in the same column with different letters are statistically different (p < 0.05).

Figure 7. Predictions for visual color and appearance rates A and B, respectively, using color (segmented seed + brine) and image textural features (contrast, correlation, energy, homogeneity) extracted from RGB, L*a*b*, HSV, and gray intensity channels.

Table 5. Classification results of commercial canned black beans into “acceptable” and “unacceptable” qualities based on color, appearance, and their combination using color and textural image features.

Quality attributes	Selected variables^1	Accuracy (±std.; %)^2
Color^3	Segmented grain, Segmented grain , Energy scale	97.2 (3.9)
Appearance^3	Segmented grain , Homogeneity scale, Brine ,	93.9 (3.0)
Color/appearance^3	Segmented grain, Segmented grain , Segmented grain , Brine ,	89.7 (5.7)

¹Variables or image features selected by a sequential forward selection method (SFS), and used by the support vector machine model over five replications.

²Average classification performance or accuracy (in bold) of the support vector machine models over five replications.

³The threshold quality values for both color and appearance were set to 3.2 rate when these attributes were tested independently and in combination.

Figure 8. Example of using the two best selected image features by the sequential forward selection method in sorting by color and appearance commercial canned black beans into “acceptable” and “unacceptable” qualities. The decision line was carried out using a support vector machine classifier.