Deep learning for crop yield prediction: a systematic literature review

Figures & data

Table 1. Differences in related work.

Study	Review Type	Focus	Short Description
(van Klompenburg et al. 2020)	Systematic	Crop yield prediction with Machine Learning	Analysis of Machine Learning techniques for crop yield prediction in literature. Analysis of most used features for crop yield prediction with Machine Learning. Found that CNN is the most widely used Deep Learning algorithm.
	Non-systematic	Fruit detection and counting methods for yield mapping	Semantic segmentation for fruit counting and detection. Comparative analysis against widely used techniques. Found that Gaussian Mixture Models outperform several deep learning models.
(Koirala et al. 2019)	Non-systematic	Fruit detection for yield estimation	Overviewed the application of Deep Learning in machine vision, fruit detection, and yield estimation. Recommended using common public image sets to allow the implementation of Transfer Learning.
(Lee et al. 2019)	Non-systematic	Crop yield prediction using Deep Learning	Usage of weather and farm status information with crop disease diagnosis to predict crop yield. Experimented with several Deep Learning algorithms on the diagnosis of crop disease and the prediction of crop yield. CNN had the best performance for crop disease diagnosis. ANN with ReLU activation function showed the best accuracy for crop yield prediction.
(Chlingaryan et al. 2018)	Non-systematic	Crop yield prediction and nitrogen status estimation with Machine Learning	Analyzed Machine Learning techniques for crop yield prediction and nitrogen status estimation. Concluded that technological development contributes to cost-effective and comprehensive solutions.
(Zhang et al. 2020)	Non-systematic	Employing Deep Learning for dense scenes in agriculture	Analyzed the application of Deep Learning in agricultural dense scenes. Reviewed application of Deep Learning in recognition, classification, detection, counting, and yield estimation tasks. Showed that Deep Learning outperforms dense scenes in agriculture.
(Dharani et al. 2021)	Non-systematic	Crop Prediction Using Deep Learning Techniques	Analyzed the usage of Deep Learning techniques in agriculture, with a focus on crop yield prediction. Reviewed the application of different Deep Learning algorithms for crop yield prediction with both images and tabular data. They resulted that CNN outperforms other networks when image data are used RNN, LSTM, and hybrid networks outperform other networks on tabular data. RNN, LSTM algorithms are more capable of dealing with time-series data.

Figure 1. Conducting the review in the SLR process.

Figure 2. Quality score distribution of the selected studies. (a): Box plot with quantiles, median, and mean values. (b): Histogram with density, median, and mean values.

Figure 3. The year-wise distribution of papers.

Figure 4. The distribution of papers per publisher.

Table 2. Top-5 values of keywords occurrence.

Keyword	Occurrence	Percentage of Occurrences
Deep learning	18	7.53%
Machine learning	8	3.35%
Yield prediction	8	3.35%
Yield estimation	7	2.93%
Agriculture	5	2.09%
LSTM	4	1.67%
Crop yield prediction	4	1. 67%
Remote sensing	4	1. 67%
UAV	3	1.26%
CNN	3	1.26%
Crop yield	3	1. 26%
Neural network	3	1. 26%
Winter wheat	3	1. 26%
Yield	3	1. 26%
Feature selection	3	1. 26%

Figure 5. The distribution of papers per key motivation.

Table 3. Top-5 values of occurrence for key motivation.

Motivation	Occurrence	Percentage of Occurrences
To process multiple arrays format in nonlinear modules	23	20.18%
To integrate accurately multiple parameters	18	15.79%
Superior results compared to other methods	15	13.16%
To automate and/or simplify tasks	15	13. 16%
To capture time dependencies	14	12.28%
Capability of generalisation	10	8.77%

Figure 6. The trend of the top-3 values for the key motivation behind applying deep learning.

Figure 7. The distribution of papers per crop.

Table 4. Top-5 values of occurrence for crops.

Crop	Occurrence	Percentage of Occurrences
Maize	14	23.33%
Wheat	8	13.33%
Soybean	7	11.67%
Rice	6	10.00%
Apples	5	8.33%

Figure 8. The trend of the top-5 crop values for applying deep learning in crop yield estimation.

Table 5. Top-5 values of occurrence for data features.

Feature	Occurrence	Percentage of Occurrences
Images	27	7.58%
Precipitation	25	7.02%
Actual yield	15	4.21%
Maximum Temperature	14	3.93%
Minimum Temperature	13	3.65%
Mean Temperature	13	3.65%

Figure 9. The distribution of papers per group of data features.

Table 6. Values of occurrence for data features groups.

Feature	Occurrence	Percentage of Occurrences
Yield Data	24	19.20%
Water Conditions	23	18.40%
Weather Conditions	23	18.40%
Satellite and Aerial Data	15	12.00%
Vegetation Indices	14	11.20%
Soil Conditions	10	8.00%
Crop and Plant Data	6	4.80%
Time	5	4.00%
Degree Days	5	4.00%

Table 7. Values of occurrence for datasets.

Data Source	Occurrence	Percentage of Occurrences
Images	27	21.09%
USDA-NASS	10	7.81%
MODIS	9	7.03%
Not mentioned publicity	8	6.25%
ImageNet	7	5.47%
Microsoft COCO	4	3.12%
Cropland Data Layer	4	3.12%
Daymet	4	3.12%

Figure 10. The distribution of papers per implementation framework.

Table 8. Values of occurrence for all implementation frameworks.

Implementation Framework	Occurrence	Percentage of Occurrences
Not mentioned	17	33.33%
TensorFlow	17	33.33%
Keras	12	23.53%
PyTorch	3	5.88%
Theano	1	1.96%
Caffe	1	1.96%

Figure 11. The distribution of papers per category.

Table 9. Values of occurrence for all categories.

Motivation	Occurrence	Percentage of Occurrences
Supervised	24	42.86%
Semi-supervised	15	26.79%
Unsupervised	15	26.79%
Reinforcement learning	2	3.57%

Figure 12. The trend of the category values in estimating crop yield with deep learning.

Figure 13. The distribution of papers per algorithm.

Table 10. Top-5 values of occurrence for algorithms.

Algorithm	Occurrence	Percentage of Occurrences
CNN	12	14.29%
LSTM	11	13.10%
DNN	7	8.33%
ANN	7	8.33%
Faster R-CNN	5	5.95%
MLP	3	3.57%

Figure 14. The distribution of papers per algorithm’s performance.

Table 11. Values of occurrence for algorithms’ performance.

Algorithm	Occurrence	Percentage of Occurrences
CNN	11	24.44%
DNN	5	11.11%
LSTM	4	8.89%
Faster R-CNN	3	6.67%
CNN-LSTM	2	4.44%
CNN-RNN hybrid	2	4.44%

Figure 15. The trend of the top-3 values of best performing algorithm crop yield estimation.

Figure 16. The distribution of papers per evaluation approach.

Table 12. Values of occurrence for evaluation approaches.

Evaluation Approach	Occurrence	Percentage
Root Mean Squared Error (RMSE)	25	17.99%
Determination Coefficient (R2)	19	13.67%
Mean Absolute Percentage Error (MAPE)	14	10.07%
Mean Absolute Error (MAE)	13	9.35%
Mean Squared Error (MSE)	10	7.19%
Precision	7	5.04%
Recall	7	5.4%
Accuracy	7	5.04%

Table 13. Values of occurrence for challenges and proposed solutions reported in primary studies.

Challenge	Possible Solution	Occurrence	Percentage
Limitation of the large training dataset to avoid overfitting and achieve higher model performance	Data augmentation with random cropping, rotation, fancy principal analysis, changing colour channels, adding filters, random increase in scale, random rotations, vertical and horizontal inversion (mirroring), and colour distortion OR Employing Transfer Learning between similar tasks OR Collecting and recording data digitally	11	5.21%
Capture linear and non-linear effects of weather, soil, climatic and remote sensing data, and other features, even without specifying the non-linear model in advance for weather estimation, while reducing dimensions, including features’ complexity, and extracting intricate information effectively through the process of transforming raw inputs (i.e satellite and climate data) gradually towards a higher level of information (i.e crop yields)	Usage of neural networks, i.e CNN OR Usage of a CNN-LSTM network, consisting of 2-Dimensional Convolutional Neural Networks (Conv2D) and LSTM networks OR Using a semiparametric variant of a deep neural network for yield modelling OR Consider Artificial Neural Networks (ANNs), especially Stacked Sparse Auto-Encoder (SSAE) OR Using Machine learning algorithms OR Using a feed-forward Deep Neural Networks with architectures of multiple fully connected layers	10	4.74%
Determine the hyperparameters like learning rate, the number of hidden units, optimiser, activation function, and the dropout values	Based on the degree to which the training process and cost function reduce the test error OR Evaluate the model with a validation set OR Employing a combination of Bayesian Hyperparameter Optimization (BHO) and early stopping, and forming an ensemble of models trained in this fashion (i.e. bootstrap aggregation) to further improve performance OR Using grid-search with ten-fold cross-validation	6	2.84%
CNN and RNN have been used independently without taking full advantage of their methods integrating time-series data, constant data, and deep learning for yield prediction	Applying a Multilevel Deep Learning Network (MLDL-Net) consisted of several modules, including a CNN to extract spatial features from imagery and weather data, an LSTM to extract temporal features from time-series data, and a CNN to extract features from soil property data OR Usage of Recurrent Neural Network (RNN) OR Usage of LSTM networks	6	2.84%
No Challenge specified	No possible Solution specified	4	1.90%
Reduce the overhead of labelling the training samples for object counting problem	Creating a synthetic dataset for training OR Collect a subset by random subsampling OR Sub-sectioning the image into smaller sections OR Employing a Weakly-Supervised Deep architecture that relies only on an image level binary classifier	3	1.42%
Improve the efficiency of depicting temporal patterns at various frequencies	Usage of LSTM	3	1.42%
The black box property of deep learning models makes them less explainable	Performing feature selection on the trained model using the backpropagation method	2	0.95%

van Klompenburg T, Kassahun A, Catal C. 2020. Crop yield prediction using machine learning: a systematic literature review. Computers and Electronics in Agriculture. 177:105709. doi:10.1016/j.compag.2020.105709.

 Web of Science ®Google Scholar

Koirala A, Walsh KB, Wang Z, McCarthy C. 2019. Deep learning–method overview and review of use for fruit detection and yield estimation. Computers and Electronics in Agriculture. 162:219–234.

 Web of Science ®Google Scholar

Lee S, Jeong Y, Son S, Lee B. 2019. A self-predictable crop yield platform (SCYP) based on crop diseases using deep learning. Sustainability. 11(13):3637.

 Web of Science ®Google Scholar

Chlingaryan A, Sukkarieh S, Whelan B. 2018. Machine learning approaches for crop yield prediction and nitrogen status estimation in precision agriculture: A review. Computers and Electronics in Agriculture. 151:61–69. doi:10.1016/j.compag.2018.05.012.

 Web of Science ®Google Scholar

Zhang Q, Liu Y, Gong C, Chen Y, Yu H. 2020. Applications of deep learning for dense scenes analysis in agriculture: a review. Sensors. 20(5):1520.

 PubMed Web of Science ®Google Scholar

Dharani M, Thamilselvan R, Natesan P, Kalaivaani P, Santhoshkumar S. 2021. Review on crop prediction using deep learning techniques. Paper presented at the Journal of Physics: Conference Series.

 Google Scholar