MS-YOLOv5: a lightweight algorithm for strawberry ripeness detection based on deep learning

Abstract

The existing ripeness detection algorithm for strawberries suffers from low detection accuracy and high detection error rate. Considering these problems, we propose an improvement method based on YOLOv5, named MS-YOLOv5. The first step is to reconfigure the feature extraction network of MS-YOLOv5 by replacing the standard convolution with the depth hybrid deformable convolution (Ms-MDconv). In the second step, a double cooperative attention mechanism (Bc-attention) is constructed and implemented in the CSP2 module to improve the feature representation in complex environments. Finally, the Neck section of MS-YOLOv5 has been enhanced to use the fast-weighted fusion of cross-scale feature pyramid networks (FW-FPN) to replace the CSP2 module. It not only integrates multi-scale target features but also significantly reduces the number of parameters. The method was tested on the strawberry ripeness dataset, the mAP reached 0.956, the FPS reached 76, and the model size was 7.44M. The mAP and FPS are 8.4 and 1.3 percentage higher than the baseline network, respectively. The model size is reduced by 6.28M. This method is superior to mainstream algorithms in detection speed and accuracy. The system can accurately identify the ripeness of strawberries in complex environments, which could provide technical support for automated picking robots.

KEYWORDS:

• Cross-scale feature fusion
• attention mechanism
• deformable convolution
• YOLO
• object detection

1. Introduction

Strawberries are very popular with consumers because of their high nutritional value (Kowalska et al., 2018). Nowadays, strawberry acreage is increasing dramatically and the problem of how to pick them efficiently has become a pressing issue. In most places, strawberry picking is done manually, which requires pickers to use a variety of techniques to ensure that the strawberries they pick meet certain quality standards. This work has undoubtedly driven up labour and material costs. With the development of smart agriculture, robotic picking (Zhou et al., 2022) has attracted much attention due to its efficiency, time-saving and cost-saving. Picking robots need to be equipped with relevant object detection methods (Huang & Wu, 2023; Mai et al., 2020; Qiao et al., 2021; Wahjuni & Nurarifah, 2023) to achieve the detection of strawberry ripeness. However, these detection methods cannot reach a balance between detection accuracy and speed. The phenomenon of low detection speed and high accuracy (Zhao et al., 2022) or high detection speed and low accuracy (Zhang et al., 2022) usually occurs, so it is necessary to design a detection method that can combine speed and accuracy.

The lightweight MS-YOLOv5 model proposed in this paper can take into account both model lightness and high detection speed. Our contributions can be summarized as follows:

1. By reconfiguring the feature extraction network, expand the receptive field to extract multi-scale information and reduce the number of parameters to some extent.

2. By building a double mechanism of cooperative attention to strengthen the discriminability of object and background information.

3. We propose a fast fusion network with cross-scale feature weights to adapt to multiscale feature fusion and improve detection accuracy.

The rest of this paper is organized as follows: Section 2 describes the background content related to this research, such as the introduction of traditional object detection and deep learning-based algorithms; Section 3 introduces the method used in our proposed MS-YOLOv5 and describes related details. Section 4 lists the experimental operating environment, experimental parameter settings, comparison results, and visual analysis of the experimental results; Section 5 summarizes and looks at the text.

2. Related work

Object detection methods are generally divided into traditional image detection methods and deep learning detection methods. Deep learning-based detection methods are divided into one-stage and two-stage object detection methods. The primary one-stage methods are YOLOv3-YOLOv8 (Chen et al., 2023; Hurtik et al., 2022; Li, Li, et al., 2022; Vats & Anastasiu, 2023; Xing et al., 2022; Zhang et al., 2023). The primary two-stage detection methods are Faster RCNN (Zhong et al., 2022), Libra RCNN (Pang et al., 2019), Cascade RCNN (Cai & Vasconcelos, 2018), and so on.

Inspired by the traditional image detection method, (Anraeni et al., 2021) used RGB feature extraction and the k-nearest neighbour algorithm (k-NN) to identify a strawberry fruit’s ripeness. However, there is still a need to improve the classification accuracy of ripeness. (Pérez-Zavala et al., 2018) the HOG (Histogram of Oriented Gradients) and LBP (Local Binary Pattern) features were used to detect fruit clusters and achieved an accuracy of 88.61 percentage points. The precision was not high due to the size of the fruit and the biometric characteristics. (Arefi et al., 2011) used the threshold-based analysis method to extract tomato combination features from RGB, HIS, and YIQ space respectively to locate the target. Still, the detection was poor when the tomato fruit was small. (Lu & Sang, 2015) used the canny edge algorithm to detect the edge of the chromatic aberration map, enhancing ripened citrus fruits in complex environments. In conclusion, the traditional extraction methods can solve the problem of fruit recognition to a certain extent, the features extracted by these methods need to be artificially designed, which makes it difficult to automatically extract the target information and cannot be balanced regarding the time and effectiveness of the feature extraction.

Inspired by deep learning object detection methods (Li, Wang, et al., 2022), used deep convolution and mixed attention mechanism methods to improve YOLOv4 and effectively reduce the size of the model. The mAP of the DAC-YOLOv4 is 72.7 percentage points, but the detection speed is only 20 FPS. (Gai et al., 2023) combined DenseNet interlayer density with CSP-Darknet53 to improve feature extraction for precision detection of small targets and masked fruit. (Latha et al., 2022) used the YOLOv4-tiny model to implement various fruits and vegetables for detection, achieving a mean inference speed of 18 ms but an accuracy of 51 percentage points. (Chen et al., 2022) improved the detection rate by adding attention and depth-separable convolution modules to the detection head and neck, respectively. Although the detection time of this model is 0.06 s, the model size of 187 MB is not suitable for use in embedded devices. (An et al., 2022) proposed a YOLOX-inspired SDNet model. They improved the convolution module in the YOLOX feature extraction network while adding an attention module and replacing the SIoU loss function. The SDNet achieved a precision of 94.26 percentage points and a recall of 90.72 percentage points. (Wang & He, 2021) used a channel pruning method for the YOLOv5s and refined the pruned model for rapid fruit detection, which can simplify the model while ensuring accuracy. (Li et al., 2023) used the YOLOv5 for parasite detection while introducing an attention mechanism to improve the detection accuracy of the model, with 96.51 percentage points accuracy and detection times up to 7.7 ms. Although these methods can realize the detection of fruit, they still have certain limitations to the detection of multi-target fruit in natural planting environments. Therefore, it is necessary to design an algorithm with high detection accuracy and speed for multiple targets in a complex environment.

3. MS-YOLOv5 detection method

Strawberry images can be particularly challenging to analyze due to the small size of the objects and the potential for target occlusion. To address these challenges, this study proposes an MS-YOLOv5 strawberry ripeness detection method with depth hybrid deformable convolution, double cooperative attention mechanism, and fast fusion of cross-scale feature weights. The network structure of MS-YOLOv5 is shown in Figure 1.

Figure 1. Improved diagram of the framework network structure for MS YOLOv5.

There are three enhancement strategies on the baseline YOLOv5 network: (1) Reconfiguration of the lightweight CSP-Darknet-53 feature extraction network in the MBS module by mixing multiple depth deformable convolution kernels (Ms-MDconv) into one convolution operation to replace the original standard convolution, thereby taking full advantage of deformable convolution. This makes the network lighter by quickly extracting multi-scale features from the target and obtaining more feature information about small objects. (2) Since the target features are not obvious due to the target occlusion and complex background, to improve the detection accuracy of the model, we add BC-attention behind the depth hybrid deformable convolution in module CSP2. It can enrich the target feature information as well as the localization information to obtain richer semantic information about the target. (3) We incorporate a fast-weighted feature pyramid network (FW-FPN) in the Neck part of MS-YOLOv5, which is designed to fuse target information from different feature layers to obtain desirable results.

3.1. Reconfiguration of feature extraction networks

The input feature map is sampled at a fixed location by a standard convolutional kernel, and the max-pooling layer continuously reduces the size of the feature map to generate a spatially location-constrained region of interest (RoI). This results in the same receptive field size at different locations, making it difficult to extract features at different target scales and with different deformation targets. Therefore, we use deep deformable convolution for feature extraction. Compared with the original feature extraction network, it does not need to extract the feature sequentially according to the stride but can automatically adjust the scale and receptive field for the feature according to the feature shape of the target. Deep deformable convolution feature extraction is more targeted, most of the irrelevant feature information is eliminated, and the number of parameters can be reduced. Deep deformable convolution can be adapted to a specific scale and receptive field, but it cannot cover all scales. On this basis, we propose depth hybrid deformable convolution. By changing the offset, multiple deformable convolution kernels are mixed on the same convolution to achieve strawberry image extraction, so as to obtain more scale target features. Although the depth hybrid deformable convolution increases the number of parameters compared to a single deformable convolution, it is much less than the original feature extraction network. The depth hybrid deformable convolution is shown in Figure 2.

Figure 2. Depth hybrid deformable convolution.

From Figure 2, a depth deformable convolution with a mixture of different offsets can automatically adapt to scale and rotation transformations of the target based on its size and shape. The input feature map (H × W × N) is passed through a convolution kernel to produce a feature map of (H × W × 2N), where 2 means that there are offsets in both x and y directions. The ordinary convolutional network operation is shown in Equation (1), and the depth hybrid deformable convolution adds an offset to the ordinary convolutional network as shown in Equation (2). (1) $\begin{array}{rl}\text{y}\left(L_0\right)& =\sum _{L_n\in R}w\left(L_n\right)x(L_0+L_n)\end{array}$(1) (2) $\begin{array}{rl}\text{y}\left({\hat{L}}_0\right)& =\sum _{L_n\in R}w\left(L_n\right)x(L_0+L_n+\Delta L_n)\end{array}$(2) In the equation above, x is the input value, y(L₀) and y(${\hat{L}}_0$) are the output value resulting from the calculation of L₀, w is the weight matching the input, $\mathrm{\Delta }{L}_n$ is an offset made to $L_n$, R is a grid of acceptance surface, as shown below. (3) $\begin{array}{r}R=\left\{\left(-1,-1\right),(-1,0),\dots ,(0,1),(1,1)\right\}\end{array}$(3)

3.2. Double cooperative attention mechanism

The background in the strawberry images is so complex that the features cannot be distinguished, especially unripe strawberries and leaves both with the same colour, which are likely to be misidentified. We introduce an attention mechanism that enriches the features of the strawberry itself, weakens the background information, and enhances the feature representation of the target. SENet (Hu et al., 2018) can adaptively learn the channel weights and focus on the more useful channel information. However, SENet’s attention mechanism typically focuses only on the channel dimension and cannot effectively capture attention in the spatial dimension. Inspired by SENet’s attention mechanism, a double cooperative attention mechanism is constructed based on SENet’s enhancement, which cooperatively performs feature weighting operations through channel and spatial dependencies to improve the perception of target features. Firstly, the attention mechanism compresses spatial information and then adopts an incentive channel strategy to obtain rich information about the target channel; secondly, it adds a PA module that conveys more precise spatial information about the target in the spatial dimension by obtaining attention in the spatial dimension. The double mechanism of cooperation is shown in Figure 3.

Figure 3. Double cooperative attention mechanism.

Figure 3 shows that X and U are the inputs and outputs of F_tr, respectively, the module that improves on SENet is the network structure after U. We do the spatial information compression operation on U (i.e. F_sq(.)) and then, in concert with Position Attention (PA), an enhancement of the localization information. Due to the scale being applied to the entire channel, the target important channel features are enhanced by controlling the scale size and weakening the background information, making the extracted features more informative. The spatial information is averaged into a single value before a global average pooling precedes the F_sq(.) operation to filter the spatial distribution information to exploit the correlation between channels. The scale is then calculated based on the global channel information, making the calculation more accurate. The 1 × 1 × C data is output from the spatial information compression operation. The channel information excitation is applied (i.e. F_ex(.)) and finally normalized using a Sigmoid function and multiplying the value as scale onto the C channels of U as the input to the following module, where U is calculated as follows. (4) $\begin{array}{r}{U}_n=v_n\times X=\sum _{s=1}^{C^{\mathrm{\prime }}}{v}_n^S\times X^s\end{array}$(4) The compressed channel information function is a unitary mapping of computation by mapping the characteristics of the input X∈R _H0×W0×C0 to those of U∈R _H1×W1×C1. In Equation (4), v denotes the set of learned feature filter kernels, where v_n refers to the parameters of the n filter and then outputs as U = [u₁, u₂, … , u_n][n] shown. z∈R_n compresses the dimension of the space U by average pooling, H is the height of the feature map, W is the width, and F_sq(.) of z is computed in the following manner. (5) $\begin{array}{rl}{\text{z}}_n& =F_{sq}\left(U_n\right)=\frac{1}{H\times W}\sum _{i=1}^H\sum _{j=1}^W{U}_n\left(i,j\right)\end{array}$(5) (6) $\begin{array}{rl}S& =F_{ex}\left(z,W\right)=\sigma \left(g\left(z,W\right)\right)=\sigma \left(W_2\delta \left(W_{1}z\right)\right)\end{array}$(6) The extraction operation equation is shown in (6), where $W_1\in R^{\frac{C}{T}\ast C}$ (vector multiplication to obtain a C/T dimensional vector, dimensionality reduction), $W_2\in R^{\frac{C}{T}\ast C}$ (vector multiplication to obtain a C dimensional vector, dimensionality increase), δ is the Relu function, while σ is the Sigmoid function and the σ(.) represents the second fully connected layer. Finally, the output ${\tilde{X}}_n$ is obtained by multiplying the scalar S_n with the feature U_n, as shown in the following equation. (7) $\begin{array}{r}{\tilde{X}}_n=F_{scale}\left(U_n,S_n\right)=S_n{U}_n\end{array}$(7) Whereas the augmented SENet can gain richer information about the characteristics of the target channel, this does not increase the spatial information of the target. Meanwhile, the ability to locate the target is poor, which tends to cause the detector to learn more background information and not make good use of the detector’s performance. Therefore, enriching target features through position and channel attention can yield more information on the channel and position, hence improving the network’s detection performance. PA (Position Attention) is a positional attention module capable of embedding spatial information about the target in the pathway and generating a coordinate perception attention map. The structure of the PA attention mechanism is shown in Figure 4.

Figure 4. PA attention mechanism module.

Consistent with the discussion above, this paper includes the location information of the attention mechanism embedded in the attention of the channel to compensate for the location information lost by the global pooling of the network. This module generates two independently oriented feature maps using two one-dimensional global pooling of the input features along the vertical direction Y and the horizontal direction X, respectively. Two feature maps with location information are separately weighted and encoded into two attention maps, each of which captures the long-range correlation dependencies of the input feature map along a spatial direction. The coordinate information is integrated into the channel attention maps to store the location information in the generated channel attention maps. The two one-dimensional global pooling operations gave the input X. The module uses two spatial extents of the convolution kernel of the pooling layer (H, 1) or (1, W) to encode each channel along the horizontal coordinate and the vertical coordinate, respectively. The output equation for the channel q at h is shown below. (8) $\begin{array}{r}{G}_q^h\left(h\right)=\frac{1}{W}\sum _{0\le i<W}{x}_q\left(h,i\right)\end{array}$(8) Similarly, the following equation gives the channel output q at w. (9) $\begin{array}{r}{G}_q^w\left(w\right)=\frac{1}{H}\sum _{0\le j<H}{x}_q\left(j,w\right)\end{array}$(9) This module generates directional feature maps by clustering features along two spatial directions. It enables the network to localize the target of interest more accurately by retaining valid information about the precise location. In this paper, the double mechanism of cooperative attention proposed is that the channel attention mechanism and the spatial attention mechanism work together. Thus, not only the channel information of the target can be noticed, but also the target’s location information can be strongly perceived.

3.3. Fast-weighted fusion of cross-scale feature pyramid networks

In response to the variable size of strawberry scales and the fact that unripe strawberries are typically difficult to detect, we constructed a pyramid network of fast-weighted features (FW-FPN). By transferring rich semantic information from higher layers to lower layers, resulting in a high-resolution feature map with strong semantics, we gain more information about the characteristics of small targets to improve the correct detection rate of small targets. To fuse features of different resolutions, the usual method for fusing multi-scale target features is to make their resolutions equal in terms of the number of channels, then perform a summation calculation over the channels and assign the correct weight value to the different input features. The fast-weighted feature pyramid network is shown in Figure 5.

Figure 5. Fast-weighted feature pyramid network.

C1-C5 shown in Figure 5 are the feature maps at different scales extracted from the original network by CSP-Darknet53. The C4 module modified the feature channels by 1 × 1 convolution, with the number of modified channels equivalent to C5. We fused the C5 module with the C4 module for feature fusion to obtain P5, the C3 module was fused with the P5 module by 1 × 1 convolution to obtain P4, and the C2 module was fused with the P4 module by 1 × 1 convolution to obtain P3. During the top-down feature extraction process, it is highly likely that the feature information from the bottom layer will be absent. Thus, to make up for this, we incorporate three cross-layer connections (illustrated by the red arrows). The K2 module is obtained through a 1 × 1 convolution of the C2 and K1 modules and fused with the P3 module. The K3 module is obtained through a 1 × 1 convolution of the C3 and K2 modules and fused with the P4 module. The K4 module is obtained through a 1 × 1 convolution of the C4 and K3 modules and fused with the P5 module.

The initial SiLu network utilizes a Sigmoid activation function which computes the contribution of each input feature for boosting computational speed, quicker model convergence, and more stable training. The computational formulas for SiLu and Hard-Swish are presented below. (10) $\begin{array}{rl}& SiLu\left(x\right)=x\times Sigmoid\left(x\right)\end{array}$(10) (11) $\begin{array}{rl}& Hard-Swish\left(x\right)\{\begin{array}{c}0,x\le -3\\ x,x\ge 3\\ {\displaystyle \frac{1}{6}}x(x+3),other\end{array}\end{array}$(11)

4. Experimental results and analysis

4.1. Experimental environment

The experimental operating system for this study was Ubuntu 20.04; the CPU was Intel® Xeon® CPU E5-2678 v3 @ 2.50 GHz * 2, Python was 3.9, the deep learning framework was Pytorch 1.11.0, CUDA version 11.1, the training network size was 640 × 640, and the number of iterations was 300. A training batch size of 16, a momentum of 0.9, a learning rate of 0.01, and a weight decay of 0.0001.

4.2. Dataset acquisition

The dataset used in this paper was 640 × 640, consisting of 3468 strawberry images. It used the LabelImg tool in VOC format, which contains information about the target’s category and location. Table 1 shows the division of the dataset in the experiments.

Table 1. Division of the dataset in the experiments.

Dataset category	Training set	Validation set	Test set
Ripe and Unripe	6676	952	1908

The original images were subjected to light and dark transformations, flips, and scale transformations to make the model more robust and generalized. To avoid overfitting to some extent, we increased the dataset to 9536 images. The effect of data augmentation is shown in Figure 6.

Figure 6. Data enhancement of original images.

We have defined samples of ripe and unripe strawberries to help readers understand our dataset more intuitively, as shown in Figure 7.

Figure 7. Display of ripe and unripe strawberries in the dataset.

4.3. Results analysis

4.3.1. Indicators of experimental evaluation

One way to better assess the detection accuracy of the model is to evaluate the performance of each category in the model using the Average Precision (AP), which is often closely related to Recall and Precision. This experiment divides maturity into ripe and unripe, and the two AP categories are first summed and then averaged to get the mean Average Precision (mAP). The recall and precision formulas are given below. (12) $\begin{array}{rl}Recall& =\frac{TP}{TP+FN}\end{array}$(12) (13) $\begin{array}{rl}Precision& =\frac{TP}{TP+FP}\end{array}$(13) Where TP indicates the number of positive samples detected correctly, FN is the number of negative samples detected as errors, and FP presents the number of negative samples detected correctly. The equations for AP and mAP are given below. (14) $\begin{array}{rl}AP& ={\int }_0^{1}P\left(R\right)dR\end{array}$(14) (15) $\begin{array}{rl}mAP& =\frac{{\sum }_i^{N}A{P}_i}{N}\end{array}$(15) Finally, Frame Per Second (FPS), model complexity (FLOPS), and model size were introduced to evaluate the model’s superiority.

4.3.2. Ablation experiments

To test the effectiveness of the model enhancement, we perform ablation experiments using YOLOv5 as the base network for the enhancement. As ablation comparison experiments, we use a reconfiguration feature extraction network, double cooperative attention, and a fast-weighted feature pyramid network. The ablation results are shown in Table 2. We have designated depth hybrid deformable convolution as module A, double cooperative attention as module B, and fast-weighted feature pyramid network as module C.

Table 2. Ablation experiments of MS-YOLOv5.

A	B	C	F1-score%	Recall%	mAP%	FPS	Parameters	GFLOPS	Model Size (M)
–	–	–	86	86.0	87.2	75	9014256	18.0	13.72
√	–	–	86	88.5	90.6	82	7025023	12.0	11.0
√	√	–	87	89.9	93.3	77	7502506	16.0	15.2
√	√	√	91	93.2	95.6	76	7015519	12.9	7.44

The ablation experiments in Table 2 show that the improved depth hybrid deformable convolution module, double cooperative attention, and fast fusion of cross-scale feature weights can efficiently improve the detection accuracy of the model as follows:

1. Depth hybrid deformable convolution: by replacing the standard convolution of the original network with depth deformable convolution and mixing depth deformable convolution of different scales acting in the same convolution kernel. As shown in Table 2, the mAP was 90.6 percentage points and the recall rate was 88.5 percentage points, an improvement of 3.4 and 2.5 percentage points, respectively, over the baseline network. The model size was also reduced by 2.72M, indicating that the use of mixed-depth deformable convolution can effectively expand the receptive field, extract more information from the target features, and reduce the number of parameters by a certain amount to reduce the model size and improve the detection speed.

2. Double cooperative attention mechanism: to help the detector distinguish the target from the background, feature fusion is achieved by channel attention in cooperation with spatial attention. When comparing the ablation experiments, several parameters are introduced that can improve the performance of the detector. 2.7 percentage points can still be improved based on the introduction of the depth hybrid deformable convolution module, and the experiments show that the detector’s performance can be fully exploited.

3. Fast-weighted fusion of cross-scale feature pyramid networks: since strawberries contain large, medium, and small scales, the original network could not be better for multi-scale feature fusion, so fast fusion of cross-scale feature weights is proposed. Experimental results show that this module can improve the mAP by up to 2.3 percentage points, improving the target detection rate, and is lighter than the feature pyramid network of the original network.

4.3.3. Comparing different attention mechanisms

It is also worth noting that the results of using different attention mechanisms within the algorithm are different; therefore, to test the superiority of our proposed augmented attention mechanism, the results of the comparison with the SENet, CA (Hou et al., 2021), and mainstream CBAM (Woo et al., 2018) attention mechanisms are shown in Table 3.

Table 3. Comparison table of YOLOv5 with different attention mechanisms.

Methods	mAP%	Model Size (M)	FPS
YOLOv5 (baseline)	87.2	13.72	75
YOLOv5 + SE	89.1	16.0	73
YOLOv5 + CA	91.3	14.2	76
YOLOv5 + CBAM	90.6	17.1	72
YOLOv5+ BC-attention (ours)	92.6	15.5	74

Table 3 shows that YOLOv5 combined with BC-attention improves the detection accuracy of the model. The model is better at detecting the ripeness of strawberries, demonstrating the superiority of our method. Visualizations of the different attention mechanisms are shown in Figure 8, which may reflect where the model should focus more.

Figure 8. Visualization of the effects of applying different attentional mechanisms to the features of YOLOv5.

4.3.4. Comparison experiments of different advanced methods

To quantify the detection and misdetection rates of this network, we plotted a confusion matrix according to the detection outcome, as shown in Figure 9. It can reflect missed and inaccurate detection rates in the model predictions. The confusion matrix shows that the background and target false detection rates for this network have improved, which fully validates the nature of this network in more detail.

Figure 9. Confusion matrix comparison between MS-YOLOv5 and YOLOv5.

To qualitatively analyze the superiority of this network model, Precision_Recall curves were plotted against the log of the training process log and compared based on the baseline network, as shown in Figure 10. The area enclosed by the Precision_Recall curve is more significant than that of the baseline network, indicating that the MS-YOLOv5 network is superior in the accuracy of detection.

Figure 10. Precision_Recall comparison between MS-YOLOv5 and YOLOv5.

The results of this method are compared with other state-of-the-art detection methods on the strawberry ripeness dataset. The evaluation of MS-YOLOv5’s comparative tests is mainly compared from multiple perspectives, such as detection accuracy and detection speed, and the table of comparative tests is shown in Table 4.

Table 4. Table comparing MS-YOLOv5 with state-of-the-art methods.

Methods	mAP%	Recall%	Unripe Strawberry%	Ripe Strawberry %	Model Size (M)	FPS
YOLOv3	79.9	75.6	71.4	88.4	120.7	55.0
YOLOv5	87.2	86.0	89.3	85.0	13.72	75.0
Faster RCNN	88.5	92.3	89.9	87.1	458.2	0.12
YOLOv4	89.4	85.3	85.3	93.5	250.2	69.0
Cascade RCNN	92.1	93.4	91.5	92.7	671.6	0.48
YOLOv6	92.8	90.2	89.2	96.5	39.3	61.0
YOLOv7	93.7	91.6	92.1	95.2	71.4	72.0
Libra RCNN	93.7	91.0	92.5	94.9	366.0	–
YOLOv8	95.2	92.2	93.6	96.7	18.1	75.0
MS-YOLOv5	95.6	93.2	94.4	96.9	7.44	76.0

As can be seen in Table 4, the YOLOv5 method incorporates depth hybrid deformable convolution, double cooperative attention mechanism, and fast fusion of cross-scale feature weighting, achieving the best results in many aspects, such as detection accuracy and speed. In the strawberry ripeness dataset, the mAP of MS-YOLOv5 was 1.9 and 0.4 percentage points higher than YOLOv7 and YOLOv8. MS-YOLOv5 has a detection rate of up to 76, slightly higher than the YOLOv5, which can meet the requirements of real-time control. In summary, MS-YOLOv5 shows a significant performance improvement in terms of detection accuracy and speed.

4.4. Visualization of MS-YOLOv5 model detection results

The results of the detection effect under different conditions between YOLOv5 and MS-YOLOv5 are shown in Figure 11. As can be seen in Figure 11, the MS-YOLOv5 model is more accurate than the YOLOv5 model in detecting objects with occluded objects, multiple targets, and small target objects in the datasets over the course of training. The improved MS-YOLOv5 model, when tested with YOLOv5, significantly reduces the occurrence of false detections, which can improve the statistics of strawberry ripeness.

Figure 11. Detection effect under different conditions between YOLOv5 and MS-YOLOv5.

5. Conclusion

In this experiment, the MS-YOLOv5 model is proposed for strawberry ripeness detection to improve the detection accuracy and speed. To highlight the difference between the background and target features, our method further enhances the target features and localization information. For the detection of strawberry ripeness, the improved model achieves a mAP of 95.6 percentage points, an accuracy improvement of 8.4 percentage points over the original method. Here, we carried out ablation and comparison experiments on the improved method, which showed that it is superior to state-of-the-art detection methods such as YOLOv7 and YOLOv8 regarding model size, detection accuracy, and detection speed. The MS-YOLOv5 model offers strong technical support for the real-time identification of strawberry ripening. However, its precision and speed in detection suffer when numerous categories and targets are present in the dataset. This demonstrates that our detector is amicable for the identification of a limited number of targets, which suggests an area for enhancement.

Availability of data and materials

The data used to support the findings of this study are available from the corresponding author upon request.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by National Natural Science Foundation of China [grant number 62001009]; R&D Program of Beijing Municipal Education Commission [grant number KM202210009003].