Artificial intelligence analysis of contributive factors in determining blackleg disease severity in canola farmlands

Abstract

Canola (Brassica napus L.) production is threatened by blackleg disease caused by Leptosphaeria maculans. Disease outcome is determined by interactions among pathogens, plants, farming practices, and environmental factors. Although the gene-for-gene interactions between the pathogen and its plant host are relatively clear, how precisely the pathogen interacts with the environment and farming practices is still poorly understood, making disease forecasting challenging for commercial farmlands. In recent years, artificial intelligence (AI) has been successful in forecasting disease risks based on environmental factors. In this study, we evaluated two AI methods and a data augmentation method to forecast disease risk using a dataset collected from 116 farmlands in Alberta in 2021 and 2022. We first assessed a machine learning model (support vector machine or SVM) and a deep-learning model (convolutional neural network or CNN) to predict blackleg severity based on five weather variables, flea beetle damage, root maggot damage, and crop-rotation variables. Both SVM and CNN predicted the disease risk with an accuracy of over 66%. The data augmentation method did not improve model performance. Flea beetle feeding and maggot damage contribute little to the model’s performance, and omitting these data did not appear to affect the results. In contrast, crop rotation contributes substantially to model performance. The five weather variables contribute roughly equally to the model’s performance, and removing any of the individual weather variables did not impact prediction ability for both models.

Résumé

La production de canola (Brassica napus L.) est menacée par la nécrose du collet causée par Leptosphaeria maculans. L’issue de la maladie est déterminée par les interactions entre l’agent pathogène, la plante, les pratiques culturales et les facteurs environnementaux. Bien que les interactions gène pour gène entre l’agent pathogène et sa plante hôte soient relativement claires, celles mettant précisément en cause l’environnement et les pratiques culturales ne sont pas bien comprises, ce qui constitue un défi quant à la prédiction de la maladie sur les terres agricoles commerciales. Récemment, l’intelligence artificielle (IA) a réussi à prédire les risques de maladie en se basant sur les facteurs environnementaux. Dans cette étude, nous avons évalué deux méthodes d’IA et une technique d’augmentation des données pour prédire le risque de maladie avec un jeu de données collectées, en 2021 et 2022, sur 116 terres agricoles de l’Alberta. Nous avons d’abord évalué un modèle d’apprentissage automatique (machine à vecteur de support ou MVS) et un modèle d’apprentissage profond (réseau neuronal à convolution ou RNC) pour prédire la gravité de la nécrose du collet en nous basant sur cinq variables climatiques, le dommage causé par l’altise, le dommage causé par la mouche des racines et les variables associées à la rotation des cultures. La SMV et le RNC ont prédit le risque de maladie avec une précision de plus de 66%. La technique d’augmentation des données n’a pas amélioré la performance du modèle. Les attaques d’altise et les dommages causés par la mouche des racines ont faiblement contribué à la performance du modèle et l’omission de ces données ne semble pas avoir influencé les résultats. En revanche, la rotation des cultures contribue substantiellement à la performance du modèle. Les cinq variables climatiques contribuent à peu près également à la performance du modèle, et la suppression de quelque variable que ce soit n’a pas influencé la capacité de prédiction des deux modèles.

Keywords:

• blackleg
• crop rotation
• deep learning
• disease forecast
• flea beetle
• machine learning

Motsclés:

• altise
• apprentissage automatique
• apprentissage profond
• prévision des maladies
• nécrose du collet
• rotation des cultures

Introduction

Canola (Brassica napus L.) contributes significantly to the Canadian economy but its production is under constant threat of blackleg disease caused by Leptosphaeria maculans (Desm.) Ces. & de Not., a hemibiotrophic fungal pathogen (Fitt et al. 2006). A noticeable increase in blackleg incidence has been reported in recent years, although this trend varied depending on location and year (Peng et al. 2020). Blackleg infection and disease development can be influenced by many factors, such as climate conditions, farming practices, pathogen and host interactions, as well as interactions between these factors (Sosnowski et al. 2004). For example, both temperature and moisture affect the maturation of pseudothecia on canola stubble (Petrie 1994), and ascospore release can be triggered by rainfall or dew (Barbetti 1975). Wind can carry the released spores to canola cotyledons and/or leaves, and subsequent spore germination can be triggered by at least four hours of continuous wetness and temperatures above 20°C (Hall 1992). When temperatures remain between 15 and 20°C, pathogen infection can be favoured by at least 48 hours of leaf wetness, and symptoms can develop on cotyledons and leaves within a week (Biddulph et al. 1999; Toscano‐Underwood et al. 2001). Abundant pycnidia can be produced from leaf lesions when day/night temperatures range between 18/11°C and 24/15°C (Barbetti 1975). Rain splash and wind help spread the inoculum to other parts of the plant and neighbouring plants (Sosnowski et al. 2004). Long periods of high humidity increase the rate of infection (Mcgee and Emmett 1977; Xi et al. 1991). In general, climate conditions affect life cycle, infection, as well as spread of blackleg disease. The interactions between environmental conditions and blackleg outbreaks have been studied previously (Mcgee and Emmett 1977; Xi et al. 1991; Biddulph et al. 1999; Toscano‐Underwood et al. 2001; Sosnowski et al. 2004; Ghanbarnia et al. 2009). However, these studies either concerned only a specific event in disease cycle or involved limited environmental conditions. Analysis using a more comprehensive set of conditions encompassing weather and other disease-associated events can be very helpful to control this disease.

Flea beetle (Phyllotreta spp.) feeding wounds may also facilitate infection as the pathogen can infect cotyledons readily through fresh wounds in the complete absence of leaf wetness (Huang et al. 2022). Typically, overwintered flea beetle adults feed extensively on canola seedlings during that time of year (Westdal and Romanow 1972; Lamb 1989; Cárcamo et al. 2008), leaving severe feeding wounds on seedling cotyledons or true leaves. Usually, infection of intact leaves through stomata requires a long period of leaf wetness under warm conditions (Hammond et al. 1985; Biddulph et al. 1999; Toscano‐Underwood et al. 2001; Huang et al. 2005), but these conditions are uncommon on the Canadian prairies in spring. It is highly possible that the blackleg pathogen infects plant cotyledons through feeding wounds. However, the impact of the wounds on blackleg disease development is largely unknown and hypothetical. In a recent study using plot trials, controlling flea beetles with in-crop insecticide failed to show consistent reduction of blackleg disease severity (internal communication). The association between flea beetle feeding and blackleg infection has not been thoroughly examined using data collected from real-world commercial fields.

Models have been developed to identify the relative contribution of different elements to blackleg disease and to forecast blackleg disease development. For example, the concentration of Leptosphaeria spp. in air samples was identified as important information for predicting the disease (Jedryczka et al. 2008; Poslušná and Plachka 2014). Lô-Pelzer et al. (2010) built the SIPPOM system to predict disease based on a comprehensive set of data which involved the genetic structure of pathogen populations, weather data, soil characteristics, cropping systems, and their spatial distribution. This model integrated five sub-models to predict (1) primary inoculum production, (2) ascospore dispersal, (3) crop development, (4) plant infection, disease severity and yield loss, and (5) changes in the genetic structure of pathogen populations over time. In this model, R² was 56% for the variation in the predicted leaf spots per plant and ranged from 75% to 85% for the disease index and relative yield loss. However, the parameters used in the prediction model such as the coefficient of interception of ascospores and fungicide persistence are relatively hard to be collected in real world.

The collective contribution from many sources makes it challenging to forecast blackleg risk with traditional methods. In recent years, there has been a growing exploration of artificial intelligence (AI) and the large-scale data to analyze contributing factors to disease and to provide support in disease prevention (Fenu and Malloci 2021). For example, using an long short-term memory (LSTM) network, Xiao et al. (2018, 2019). found that moderate temperature, humidity, low wind speed, and rainfall in autumn and winter contributed to the occurrence of cotton pests and diseases. Khaki et al. (2020) employed a convolutional neural network (CNN) in conjunction with a recurrent neural network (RNN) to construct a CNN-RNN model. This model was employed to forecast corn and soybean yield by considering weather patterns, soil conditions at different depths, and various agricultural management approaches. This study also explored the ability of the CNN-RNN model in explaining how factors such as weather conditions, management practices, and soil conditions contribute to crop yields. Zhang et al. (2019) explored the contribution of different factors to the final maize yield. In the study, environmental, fluorescence, optical and thermal satellite data were used. Besides its high explanation of yield variation (75%) with the multi-source data, the analysis based on deep learning also found that the silking stage, high temperature, and vapour pressure deficit during the reproductive period were more important factors than others, while the contribution of soil properties and management factors cannot be fully explained. The Gaussian mixture model developed by Skelsey (2021) reached an accuracy of 97% when predicting the outbreak of potato late blight using weather data.

While AI has proved its ability in data analysis, data selection and collection per se from commercial fields present a huge challenge. For blackleg, weather information, cultivar resistance, crop rotation history, pathogen population (Fernando et al. 2018) and even flea beetle damage may all be relevant to risk assessment. Some of the data are difficult to collect precisely. For example, both qualitative (major gene) and quantitative (multi-gene) resistance may exist in many Canadian canola varieties. After years of planting, pathogen population in fields may vary under selection pressure, resulting in breakdown in qualitative resistance and increased blackleg rating in some fields. Gathering these data involves extensive on-site surveys, which can be costly. Often smaller datasets are collected, but they are usually insufficient for in-depth analyses. Data augmentation methods may be used in these cases. For example, through random sampling, Zhong et al. (2018) augmented the dataset of soybean variety trials with less than 35 samples to make their model more stable in predicting yield potential.

In this study, we analyzed a small dataset for canola blackleg disease with the intention of answering the questions of (1) what is the capability of AI methods, specifically deep learning and machine learning, in predicting the risk of blackleg disease using a dataset that is both complex and restricted in size, (2) does data augmentation enhance both machine learning and deep learning in the accuracy and F1-score, and (3) to what extent do different factors influence blackleg risk assessment. The dataset was collected in 2021 and 2022, and included weather data, crop rotation history, flea beetle and root maggot damage, and blackleg severity. The initial analysis focused on (1) AI model training and comparison, (2) data augmentation, and (3) relative contributions of different factors.

Materials and methods

The workflow of this study is shown in Fig. 1. First, the data collected from 116 canola fields were preprocessed and divided into model training and testing datasets. The training dataset was augmented first before model training. Both support vector machine (SVM) and CNN were trained, and the trained models were validated with the testing dataset.

Fig. 1 The framework in data pre-processing, dataset separation, model development, and model evaluation. The dataset was first preprocessed, which involved data cleaning, imputing missing data, normalization, classification based on blackleg incidence, and dataset separation into training and testing datasets. Only the training dataset was augmented to evaluate data augmentation effects. Models were trained with non-augmented or augmented datasets, and the results were evaluated for accuracy and F1-score.

Data collection

We collected four-year crop rotation history over 116 commercial canola fields in Alberta, Canada with the consent of participating producers. The assessment of flea beetle damage was conducted at the cotyledon stage (BBCH 10–13) by visually inspecting the percentage of cotyledon area affected by feeding, employing a rating scale outlined by the Canola Council of Canada (2020). In a subsequent visit to the fields, between the ripening (BBCH 80) and senescence (BBCH 90) stages of crop growth, the plants were evaluated once more for blackleg severity and root maggot damage. Pest damage and blackleg severity were collected through assessing 20 plants at five locations per field, following a W-shaped transect pattern. Root maggot injury was estimated using a 0–5 scale, where 0 = no damage to roots; 1 = slight feeding, < 10% of tap root surface damaged; 2 = 10–25% of root surface damaged; 3 = 26–50% of root surface damaged, tunnelling into interior; 4 = 51–75% of root surface damaged; extensive tunnelling; and 5 = 76–100% of root damaged, or root severed (Dosdall et al. 1994). Blackleg severity was also estimated using a 0–5 scale where: 0 = no disease in a cross-section of the stem base; 1 = decay on < 25% of the cross-sectional area of the crown; 2 = decay on 25%–50% of the cross-section; 3 = decay on 51%–75% of the cross-section; 4 = decay on > 75% of the cross-section; and 5 = death of the plant (Guo and Fernando 2005). Weather conditions for each field were obtained from the nearest weather station in the Alberta Climate Information Service network (https://agriculture.alberta.ca/acis/, 2020). The weather data included daily maximum temperature (°C), minimum temperature (°C), mean relative humidity (%), rainfall (mm), and wind speed (kmph). The 58 fields for 2021 data were covered by 22 different weather stations, while the 58 fields for 2022 were covered by 29 different weather stations (Table 1).

Table 1. Weather stations and the number of covered fields for each station.

Location	2021 classes		2022 classes
	0 = < Sev ≤ 0.23	Sev > 0.23	Location	0 = < Sev ≤ 0.23	Sev > 0.23
	A	B		A	B
Rosalind	5	0	Viking	0	1
Bellshill	1	0	Kinsella	1	3
Forestburg	1	1	Rosalind	0	4
Kinsella	1	0	Ferintosh	0	1
Morrin	3	7	St Mary Reservoir	0	2
Beiseker	1	0	Fairview	0	2
Hespero	1	0	Wainwright	0	1
Olds College	1	0	Foremost	1	0
Neir	2	0	Bow Island	1	0
Rosemary	1	1	Three Hills	0	1
Thorsby	3	0	Linden	0	1
Edmonton	1	0	Morrin	0	5
Wimborne	3	0	Vegreville	0	1
Divide Hills	2	2	Rosemary	0	2
Delburne	4	4	Brooks	1	0
Craigmyle	0	2	Alliance	0	1
Stettler	3	2	Edmonton Stony Plain	1	1
Three Hills	2	2	Edmonton South Campus	1	1
Oliver	3	0	Priddis Obs	1	0
Busby	2	0	Beiseker	1	0
Gleichen	1	1	Savanna	2	0
			Jean Cote	2	0
			Oliver	3	1
			Abee	3	2
			Sandy Lake	0	1
			Vermilion	2	0
			Busby	2	2
			Pibroch	1	0
			Gleichen	1	1

(1) ‘Sev’ refers to ‘blackleg severity’. Blackleg disease and related pest variables and crop-rotation variables were collected from 58 fields for the 2021 dataset and 58 fields for the 2022 dataset. The weather variables were collected from the nearest weather station for each of the fields. For the 2021 data, five fields could be covered by two weather stations, thus each of the five fields were separated as two independent data entries with weather variables from either of the two weather stations. Thus, there are 63 data entries for 2021 data.

We used a blackleg severity of 0.23 to group the data entries into two categories, and they are A (low) and B (high). There are two basic considerations: (1) 0.23 disease severity (or ~ 4% in yield loss as indicated by Hwang et al. (2016)) is noticeable in profit loss, and (2) this value (or 0.23) could separate the blackleg disease data into two classes with each of the classes having a similar number of data points (65 for class A and 56 for class B).

Data processing

For each field, blackleg severity serves as the dependent variable, while weather data, pest damage data, and crop-rotation data served as the independent variables. The weather data were collected spanning a time frame of 182 days from March 3rd to August 31th in either 2021 or 2022. Additionally, pest damage variables include flea beetle and root maggot feeding. Furthermore, the crop rotation history for the years 2017–2020 or 2018–2021 was incorporated, preceding canola sowing in 2021 or 2022. Each rotation crop was assigned a corresponding number: barley = 1, wheat = 2, pea = 3, potato = 4, flax = 5, fall rye = 6, grassland = 7, corn = 8, canola = 9, oat = 10, alfalfa = 11, durum = 12, and lentil = 13. Thus, there were 182 columns to include 182 days from March 3rd to August 31st 2021 or 2022 in the dataset. In each column, there were 12 rows representing the five weather variables, flea beetle incidence, root maggot incidence and severity, and crop codes for four years. Therefore, there are 182 × 12 or 2,184 independent variables for data entry.

For the 2021 data, there were five fields which each had similar distances to two separate weather stations, and thus two different sets of weather variables could be assigned to each of the five fields. Thus, each of these five fields were treated as two independent data entries, producing a total of 121 entries in the final dataset. There were 11 entries that had no root maggot data while one entry lacked flea beetle information; they were filled with the mean value of non-missing values.

Data augmentation

To enrich the dataset, we selected the weighted form of the dynamic time warping barycentric averaging (DBA) technique (Petitjean et al. 2011, 2014, 2016). Dynamic Time Warping (DTW) is a method used to measure the similarity between two time series that may have different lengths and exhibit temporal distortions. The DTW Barycentric Averaging (DBA) technique is a way to iteratively find a representative time series, called the barycentre, for a given set of time series data which is the case for our dataset. By varying the weights in DBA, an infinite number of new data entries can be created from a given dataset. Briefly, the data augmentation process started with a random selection of a data entry from the training dataset, and this entry was assigned a weight value of 0.5. Then five nearest neighbours, based on the dynamic time warping distance, were searched and two of them were randomly selected, with each being assigned a weight value of 0.15. The remaining three neighbours would share the rest of the weight, 0.2. It should be noted that the data augmentation process was only applied to the training dataset. The testing dataset remained intact to validate trained models. To evaluate how data augmentation scale impacts AI performance, we augmented the dataset into scales of 120, 320, 519, 719, and 1,119 data entries.

Prediction models

As our dataset is relatively small, the over-fitting problem may occur. For this reason, we chose SVM as the specific model to represent the machine learning method for its robustness to the over-fitting problem (Chi et al. 2008). For a similar reason, we designed a CNN model architecture containing a small number of hidden layers (Pasupa and Sunhem 2016) to avoid over-fitting. The detailed CNN architecture is illustrated in Fig. 2. The CNN was trained for 600 epochs with a learning rate of 0.001. The loss between the prediction output and the ground truth was calculated with the CrossEntropyLoss function, while the Adam optimizer was used to optimize model parameters. In the end, the testing dataset was used to validate the developed model. We used the k-fold cross-validation method to minimize the random effect, and thus the final accuracies and F1-scores were the averaged values of the k-fold datasets. Also, to have a proper number of data entries for model validation in each fold scheme, we trained the models with 3-, 4-, or 5- fold cross-validation methods. When dividing the dataset, each class was distributed proportionally in both training and testing datasets to avoid bias. The numbers of entries for training and testing datasets are shown in Table 2. The CNN model development used Pytorch (version 1.11.0) and Python (version 3.8.3), while the SVM model was developed using Python (version 3.8.3).

Fig. 2 (Colour online) The architecture for the convolution neural network (CNN) model. The input matrix contains data from 12 weather variables spanning across 182 days. It undergoes various stages within the CNN architecture. These stages include convolution, followed by pooling and dropout, along with linear layers. Eventually, the softmax function is applied to transform the model’s output into classification predictions.

Table 2. Dataset separation in the k-fold cross-validation method.

k-fold	Dataset	Dataset 1	Dataset 2	Dataset 3	Dataset 4	Dataset 5
3-fold	Training	80	81	81	n/a	n/a
	Testing	41	40	40	n/a	n/a
4-fold	Training	90	91	91	91	n/a
	Testing	31	30	30	30	n/a
5-fold	Training	96	97	97	97	97
	Testing	25	24	24	24	24

Three separations (refers to 3, 4, and 5-fold cross-validation methods) were used to avoid inappropriate dataset separation in the k-fold cross-validation method. As the dataset was separated in different ways, the number of data entries varies in each separation. When separating the dataset, each class is distributed proportionally in training and testing datasets to avoid bias.

We used prediction accuracy and F1-score to evaluate the performance of SVM and CNN models. Accuracy is to measure the percentage of correctly predicted objects out of the total objects in our dataset. It is computed using the formula: accuracy = (number of correctly classified objects)/(total number of objects). While accuracy is a widely used and valuable metric, it may not always provide a complete picture of the model’s performance, especially when dealing with imbalanced datasets. The F1-score is a metric that takes into account both precision and recall evaluating our model’s performance. Precision refers to the ratio of true positive predictions to the total positive predictions, and recall is the ratio of true positive predictions to the total actual positive instances. The F1-score is estimated as: F1-score = 2 × (precision × recall)/(precision + recall). The F1-score is particularly useful when the data are imbalanced.

Results

Dataset distribution across weather stations

The dataset consisted of a total of 121 data entries which were classified as low (A) or high (B) based on a disease severity threshold of 0.23. Blackleg severity across Alberta is summarized in Fig. 3. The 2021 weather data were collected from 22 weather stations, and the 2022 weather data was collected from 29 weather stations. While the weather data for class A was recovered from 36 weather stations, weather data for class B was recovered from 23 weather stations. Although several data entries shared the same weather variables, crop rotation history was unique for each of the data entries. The distribution of each data entry across the weather stations is shown in Table 1.

Fig. 3 (Colour online) Summary of disease severity across Alberta in 2021 and 2022. (A) Blackleg disease severity in 2021. (B) blackleg disease severity in 2022.

Model performances without data augmentation

The prediction models were trained, and the SVM model reached average testing accuracies of 66.08 ± 4.50%, 66.94 ± 7.09%, and 57.8 ± 10.50%, and average F1-scores at 0.66 ± 0.04, 0.66 ± 0.07, and 0.58 ± 0.10 for the 3-, 4-, and 5-fold datasets respectively (Table 3). Meanwhile, the CNN model reached 66.19 ± 4.81% in accuracy and 0.65 ± 0.03 in F1-score when using the 3-fold dataset, 64.28 ± 5.63% and 0.64 ± 0.06 for the 4-fold dataset, and 62.79 ± 10.35% and 0.61 ± 0.09 for the 5-fold dataset, respectively. When comparing these two models, the CNN and SVM models showed similar predictive accuracies (P = 0.89, 0.52 and 0.23 for 3-, 4- and 5-fold cross-validation datasets, respectively). The F1-scores were also similar between the CNN and SVM models.

Table 3. Model prediction accuracies for the datasets that deleted different variables.

	Variables	Model	Fold scheme	Model performance					Average	Average
				Dataset 1	Dataset 2	Dataset 3	Dataset 4	Dataset 5
Accuracy (%)	All	SVM	3	70.73	67.50	60.00	n/a	n/a	66.08 ± 4.50	62.91 ± 9.30
			4	67.74	76.67	66.67	56.67	n/a	66.94 ± 7.09
			5	64.00	41.67	70.83	50.00	62.50	57.8 ± 10.50
		CNN	3	72.97	62.35	63.27	n/a	n/a	66.19 ± 4.81	64.14 ± 7.93
			4	68.33	60.95	56.87	70.95	n/a	64.28 ± 5.63
			5	51.27	61.65	76.96	52.04	72.02	62.79 ± 10.35
	Pest variables deleted	SVM	3	75.61	62.50	67.50	n/a	n/a	68.54 ± 5.40	66.29 ± 7.01.
			4	67.74	63.33	66.67	70.00	n/a	66.94 ± 2.40
			5	68.00	66.67	70.83	45.83	70.83	64.43 ± 9.44
		CNN	3	71.18	65.77	71.84	n/a	n/a	69.59 ± 2.72	67.95 ± 6.15
			4	69.52	70.34	65.10	76.94	n/a	70.48 ± 4.23
			5	57.80	59.78	79.08	64.88	63.18	64.94 ± 7.49
	Rotation variables deleted	SVM	3	63.41	62.50	52.50	n/a	n/a	59.47 ± 4.94	56.57 ± 7.57
			4	51.61	60.00	53.33	66.67	n/a	57.90 ± 5.95
			5	48.00	45.83	66.67	45.83	62.50	53.77 ± 8.96
		CNN	3	65.41	68.32	58.93	n/a	n/a	64.22 ± 3.92	59.25 ± 10.38
			4	61.03	44.49	63.95	74.35	n/a	60.95 ± 10.72
			5	52.33	36.90	70.83	54.85	59.61	54.9 ± 11.01
F1-score	All	SVM	3	0.70	0.68	0.60	n/a	n/a	0.66 ± 0.04	0.63 ± 0.09
			4	0.68	0.76	0.66	0.56	n/a	0.66 ± 0.07
			5	0.63	0.42	0.71	0.50	0.62	0.58 ± 0.10
		CNN	3	0.68	0.62	0.63	n/a	n/a	0.65 ± 0.03	0.63 ± 0.07
			4	0.64	0.62	0.58	0.74	n/a	0.64 ± 0.06
			5	0.50	0.59	0.72	0.52	0.71	0.61 ± 0.09
	Pest variables deleted	SVM	3	0.75	0.62	0.67			0.68 ± 0.05	0.66 ± 0.07
			4	0.68	0.63	0.67	0.70		0.67 ± 0.03
			5	0.68	0.66	0.71	0.46	0.71	0.64 ± 0.09
		CNN	3	0.68	0.64	0.69			0.67 ± 0.02	0.67 ± 0.06
			4	0.67	0.69	0.66	0.76		0.69 ± 0.04
			5	0.58	0.61	0.76	0.60	0.71	0.65 ± 0.07
	Rotation variables deleted	SVM	3	0.62	0.63	0.53			0.59 ± 0.05	0.56 ± 0.08
			4	0.51	0.59	0.52	0.66		0.57 ± 0.06
			5	0.45	0.44	0.67	0.44	0.62	0.53 ± 0.10
		CNN	3	0.66	0.68	0.56			0.64 ± 0.05	0.59 ± 0.09
			4	0.60	0.53	0.62	0.72		0.62 ± 0.07
			5	0.48	0.41	0.68	0.54	0.64	0.55 ± 0.10

Pest variables or crop rotation variables were deleted from the dataset, and SVM and CNN were trained with the datasets lacking different variables. Model performances were evaluated with the averaged accuracy or F1-score.

Model performances with augmented datasets

Data augmentation did not apparently affect the model performance (Fig. 4), and there was no association between the model performance and levels of dataset augmentation. For a straightforward comparison of model performances before and after data augmentation, averaged F1-socres and accuracies were examined across different augmentation levels. After augmentation (Fig. 5), SVM performance changed only slightly in average accuracies and F1-scores. The performance of the CNN model was not improved with the data augmentation (P = 0.052, 0.072, and 0.831 for 3-, 4-, and 5-fold cross-validation methods, respectively).

Fig. 4 (Colour online) Model performances with the dataset after data augmentation. The k-fold cross-validation method was used to have a comprehensive model evaluation. In each cross-validation scheme, the training datasets were augmented to five levels (120, 320, 519, 719, and 1,119 entries for each level). With the augmented datasets, SVM and CNN models were trained and tested. Their performances were evaluated with prediction accuracy (%) and F1-score for the five levels. (A) Model accuracies for different separation ways and augmentation levels. (B) F1-scores for different separation ways and augmentation levels.

Fig. 5 (Colour online) Averaged model performances across the five augmented datasets. The k-fold cross-validation method was used to have a comprehensive model evaluation. The dataset was divided into 3, 4, and 5 folds. In each separation, the training datasets were augmented to five levels (that is 120, 320, 519, 719, and 1,119 entries for each level). With the augmented datasets, SVM and CNN models were trained and tested. Their performances were evaluated with the averaged prediction accuracy (%) and F1-score for the five levels. (A) prediction accuracies for different models and separation ways. (B) F1-scores for different models and separations.

Evaluating candidate variables contributing to blackleg

Prediction models involved several sets of variables, including weather, insect pests, and crop-rotation history. Using weather data as the basic information, we tested the relative contribution of insect-feeding and crop-rotation variables to the blackleg severity. When flea beetle incidence and root maggot incidence and severity were deleted from the dataset (Table 3), SVM and CNN performances, based either on accuracy or F1-score, increased slightly when compared to the analyses using full dataset.

When crop rotation variables were removed from the dataset, the model performance decreased remarkably compared to the models trained with the dataset containing these variables (Table 3). The lowered average accuracies and F1-scores indicated the importance of crop rotation history to blackleg.

At the same time, it is intriguing to examine the extent to which individual weather variables contribute to the accuracy and F1-score of the models. To do so, we performed an experiment where each weather variable was individually removed from the dataset, resulting in five different datasets. Subsequently, we trained CNN and SVM models on these modified datasets. We found that, compared to the full dataset, the model performances including accuracy and F1-score did not show an apparent decrease (Table 4). Also, the differences among the five models in the perspective of accuracy and F1-score were minor.

Table 4. Analysis of relative contributions of different weather variables.

	Deleted Variables	Model	Fold-type	Model performance					Average	Average
				fold 1	fold 2	fold 3	fold 4	fold 5
Accuracy	Daily minimum temperature deleted	SVM	3	75.61	70.00	60.00	n/a	n/a	68.65 ± 6.46	64.98 ± 8.04
			4	67.74	73.33	70.00	60.00	n/a	67.77 ± 4.91
			5	64.00	41.67	79.17	50.00	58.33	58.63 ± 12.75
		CNN	3	65.70	63.27	64.34	n/a	n/a	64.44 ± 1.00	62.56 ± 5.96
			4	68.93	64.97	56.94	71.56	n/a	65.60 ± 5.52
			5	50.86	47.87	77.64	48.89	63.01	57.65 ± 11.37
	Daily maximum temperature deleted	SVM	3	73.17	70.00	57.50	n/a	n/a	66.89 ± 6.76	65.26 ± 8.45.
			4	67.74	76.67	73.33	60.00	n/a	69.44 ± 6.31
			5	64.00	41.67	79.17	54.17	58.33	59.47 ± 12.29
		CNN	3	70.08	63.37	64.95	n/a	n/a	66.13 ± 2.87	65.09 ± 5.41
			4	62.28	60.88	61.36	70.20	n/a	63.68 ± 3.80
			5	60.00	67.09	80.87	51.87	67.52	65.47 ± 9.58
	Daily mean relative humidity deleted	SVM	3	75.61	70.00	57.50	n/a	n/a	67.70 ± 7.57	64.70 ± 8.63
			4	67.74	73.33	66.67	56.67	n/a	66.10 ± 6.01
			5	64.00	41.67	79.17	54.17	62.50	60.30 ± 12.32
		CNN	3	68.19	60.56	68.47	n/a	n/a	65.74 ± 3.66	64.40 ± 6.55
			4	66.36	58.10	71.09	69.18	n/a	66.18 ± 4.96
			5	54.37	57.74	81.12	49.15	64.03	61.28 ± 11.03
	Daily rainfall deleted	SVM	3	75.61	70.00	60.00	n/a	n/a	68.54 ± 6.46	64.70 ± 8.18
			4	70.97	70.00	66.67	53.33	n/a	65.24 ± 7.06
			5	60.00	45.83	79.17	54.17	62.50	60.33 ± 11.03
		CNN	3	66.20	61.84	65.00	n/a	n/a	64.35 ± 1.84	62.98 ± 5.12
			4	63.33	56.53	65.17	73.20	n/a	64.56 ± 5.94
			5	54.53	54.00	72.62	54.00	65.05	60.04 ± 7.57
	Daily wind speed deleted	SVM	3	70.73	70.00	57.50	n/a	n/a	66.08 ± 6.07	64.45 ± 7.13
			4	64.52	70.00	70.00	60.00	n/a	66.13 ± 4.19
			5	64.00	45.83	79.17	54.17	62.50	61.13 ± 11.12
		CNN	3	67.40	64.69	67.24	n/a	n/a	66.45 ± 1.24	65.80 ± 4.38
			4	67.41	67.62	63.13	71.63	n/a	67.45 ± 3301
			5	58.29	61.48	77.30	51.53	68.96	63.51 ± 8.88
F1-score	Daily minimum temperature deleted	SVM	3	0.75	0.70	0.60	n/a	n/a	0.68 ± 0.06	0.65 ± 0.08
			4	0.68	0.73	0.69	0.59	n/a	0.67 ± 0.05
			5	0.64	0.42	0.79	0.50	0.58	0.58 ± 0.13
		CNN	3	0.66	0.62	0.65	n/a	n/a	0.64 ± 0.02	0.62 ± 0.06
			4	0.67	0.66	0.57	0.71	n/a	0.65 ± 0.05
			5	0.55	0.49	0.76	0.46	0.63	0.58 ± 0.11
	Daily maximum temperature deleted	SVM	3	0.73	0.70	0.58	n/a	n/a	0.67 ± 0.07	0.65 ± 0.08
			4	0.68	0.76	0.73	0.59	n/a	0.69 ± 0.07
			5	0.63	0.42	0.79	0.54	0.58	0.57 ± 0.12
		CNN	3	0.69	0.62	0.64	n/a	n/a	0.65 ± 0.03	0.64 ± 0.06
			4	0.62	0.63	0.60	0.72	n/a	0.64 ± 0.05
			5	0.60	0.62	0.78	0.52	0.70	0.64 ± 0.09
	Daily mean relative humidity deleted	SVM	3	0.75	0.70	0.58	n/a	n/a	0.67 ± 0.07	0.64 ± 0.09
			4	0.68	0.73	0.66	0.56	n/a	0.66 ± 0.06
			5	0.63	0.42	0.79	0.54	0.62	0.60 ± 0.12
		CNN	3	0.66	0.61	0.66	n/a	n/a	0.65 ± 0.02	0.64 ± 0.05
			4	0.65	0.58	0.68	0.68	n/a	0.65 ± 0.04
			5	0.53	0.61	0.79	0.53	0.68	0.63 ± 0.10
	Daily rainfall deleted	SVM	3	0.75	0.70	0.60	n/a	n/a	0.68 ± 0.06	0.65 ± 0.08
			4	0.71	0.70	0.66	0.53	n/a	0.65 ± 0.07
			5	0.60	0.46	0.79	0.54	0.63	0.60 ± 0.11
		CNN	3	0.66	0.62	0.66	n/a	n/a	0.64 ± 0.02	0.63 ± 0.05
			4	0.63	0.59	0.61	0.74	n/a	0.65 ± 0.06
			5	0.55	0.56	0.75	0.52	0.68	0.61 ± 0.09
	Daily wind speed deleted	SVM	3	0.70	0.70	0.58	n/a	n/a	0.66 ± 0.06	0.64 ± 0.07
			4	0.65	0.69	0.69	0.60	n/a	0.66 ± 0.04
			5	0.63	0.46	0.79	0.54	0.63	0.61 ± 0.11
		CNN	3	0.68	0.66	0.65	n/a	n/a	0.66 ± 0.01	0.66 ± 0.04
			4	0.66	0.68	0.63	0.71	n/a	0.67 ± 0.03
			5	0.58	0.62	0.74	0.53	0.68	0.63 ± 0.07

Each of the five weather variables were deleted from the dataset, and SVM and CNN were trained with the datasets lacking different weather variables. Model performances were evaluated with the averaged accuracy or F1-score.

We further investigated the significance of different growth stages (or earlier/later weather) on the model’s performance. For this purpose, we conducted an experiment in which we removed either earlier (from March 3rd to June 1st) or later (from June 2nd to August 31st) data from the dataset, resulting in two distinct datasets. The results revealed that the SVM performance remained consistent across both stages, while the CNN model exhibited reduced predictive capability when utilizing the later weather variables, as evidenced by lower accuracy and F1-score values (Table 5). This suggests that SVM might offer greater robustness than CNN when dealing with smaller datasets, and it also implies that later weather conditions could exert a more significant influence on blackleg disease development.

Table 5. Analysis of relative contribution of early and later growth stages.

	Days	Model	Fold-type	Model performance					Average	Average
				fold 1	fold 2	fold 3	fold 4	fold 5
Accuracy	March 3rd to June 1st	SVM	3	75.61	70.00	65.00	n/a	n/a	70.20 ± 4.33	63.88 ± 7.21
			4	67.74	66.67	66.67	53.33	n/a	63.60 ± 5.94
			5	60.00	41.67	75.00	50.00	62.50	57.83 ± 11.35
		CNN	3	66.90	65.92	59.23	n/a	n/a	64.02 ± 3.41	62.56 ± 5.96
			4	64.91	59.39	62.72	67.41	n/a	63.61 ± 2.95
			5	54.69	62.24	76.87	52.04	72.53	63.68 ± 9.70
	June 2nd to August 31th	SVM	3	78.05	67.50	55.00	n/a	n/a	66.85 ± 9.42	63.32 ± 9.41
			4	67.74	73.33	56.67	56.67	n/a	63.60 ± 7.21
			5	60.00	45.83	79.17	50.00	62.50	59.50 ± 11.60
		CNN	3	58.74	56.63	64.59	n/a	n/a	59.99 ± 3.37	54.31 ± 4.93
			4	43.98	53.95	55.51	38.91	n/a	48.09 ± 6.90
			5	49.06	53.40	62.76	56.04	53.06	54.86 ± 4.53
F1-score	March 3rd to June 1st	SVM	3	0.75	0.70	0.65	n/a	n/a	0.70 ± 0.04	0.64 ± 0.07
			4	0.68	0.66	0.66	0.53	n/a	0.63 ± 0.06
			5	0.60	0.42	0.75	0.50	0.63	0.58 ± 0.11
		CNN	3	0.66	0.65	0.58	n/a	n/a	0.63 ± 0.04	0.63 ± 0.05
			4	0.63	0.58	0.63	0.70	n/a	0.64 ± 0.04
			5	0.54	0.62	0.74	0.56	0.70	0.63 ± 0.08
	June 2nd to August 31th	SVM	3	0.78	0.68	0.55	n/a	n/a	0.67 ± 0.09	0.63 ± 0.09
			4	0.68	0.73	0.57	0.57	n/a	0.64 ± 0.07
			5	0.60	0.46	0.79	0.50	0.63	0.59 ± 0.12
		CNN	3	0.60	0.57	0.63	n/a	n/a	0.60 ± 0.02	0.55 ± 0.07
			4	0.42	0.49	0.64	0.35	n/a	0.48 ± 0.11
			5	0.53	0.53	0.74	0.53	0.61	0.59 ± 0.08

Canola growth period was divided into two stages. The first stage is from March 3^rd to June 1st and the second stage is from June 2^nd to August 31^th. Both SVM and CNN were trained to evaluate the relative contribution of the two growth stages.

Discussion

Blackleg severity can be accurately predicted using weather, insect feeding, and crop-rotation history information

On the prairies, many factors such as weather conditions, farming practices, insect feeding wounds, cultivar resistance, and pathogen population collectively influence the occurrence of blackleg infection (Ghanbarnia et al. 2011; Huang et al. 2022). In this study, we employed AI models to make predictions about blackleg severity using weather, insect feeding, and crop-rotation variables, achieving high accuracies and precisions. The results showed that these variables contributed significantly to the levels of blackleg disease within commercial canola fields. This aligns with conclusions drawn from prior plot trials (Guo and Fernando 2005; Guo et al. 2005).

We also conducted assessments to determine the relative significance of insect feeding and crop-rotation concerning blackleg severity. This was accomplished by removing these factors from AI models individually. The results indicated that insect feeding may not be a crucial factor, as the models’ predictive ability actually improved slightly when relevant variables were omitted for analysis. This outcome aligns with similar findings from plot trials, wherein a reduced flea beetle feeding through in-crop insecticide application failed to reduce blackleg severity at crop maturity (Huang et al. 2022). However, caution is required in interpreting the data as they were collected from only 116 fields from 2021 and 2022; possible interactions among pathogen inoculum, host resistance, insect feeding wounds, and environments might not have been fully captured. Data from more fields and years across different environmental conditions will be helpful for more robust analysis. In contrast, when rotation variables were removed from the dataset, the model performances were remarkably poorer for both SVM and CNN models (Table 3). This finding is consistent with previous plot trials (Guo et al. 2005), in which the severity of blackleg was significantly lower when canola was rotated with wheat or flax.

When removing the individual weather factors, the overall model performance in terms of accuracy and F1-score did not exhibit a significant decrease compared to using the full dataset. This suggests that the models are relatively robust to the absence of any single weather variable. Additionally, we observed that the differences in accuracy and F1-score among the five models were relatively minor. These findings imply that the weather variables in our dataset might have a certain level of redundancy or might not individually contribute to the predictive power of the models substantially. It is important to consider that machine learning and deep-learning models often exploit correlations among features, and the impact of removing a single feature can be mitigated by the presence of other correlated features.

AI may help assess additional management practices

Cultivar resistance, R-gene stacking, R-gene rotation, and crop sequence all play a role in influencing the outcome of blackleg disease. However, determining the most effective practice or combination of practices for managing blackleg remains an ongoing question. Data are currently being collected for some of these management practices in real-world commercial fields on the prairies. Integrating these new datasets into an AI model holds potential for enhancing blackleg risk assessment. Results from plot trials, with limited number of entries or variables, may also be used as supplements in the model, although machine learning will be much more powerful in handling huge datasets involving a large number of data entries and multiple variables. In some instances, data augmentation techniques can be employed to expand the dataset size. Utilizing big data allows for more comprehensive analyses and potentially more accurate predictions. Findings from this study emphasize that AI serves as a novel potential tool for assessing additional management practices aimed at mitigating blackleg disease within commercial fields. Traditional statistical approaches may struggle to handle the complexity in some datasets, especially when multiple interacting factors are at play. Machine learning, especially deep learning techniques, can effectively integrate disparate data sources and analyze them cohesively. Although this study only employed three sets of variables, the ability to evaluate the relative significance for each factor and integrate various data sources underscores the value of AI models in comprehensively analyzing complex disease data.

Practical implications

The successful utilization of AI models, fuelled by weather, insect feeding, and crop-rotation variables to predict blackleg severity presents a promising starting point. The results are encouraging for further development of practical tools aimed at enhancing blackleg management strategies within commercial canola fields. The predictive accuracy attained was notably high (reaching up to 70%, as indicated in Table 3), and precision was also noticeable (with F1-score of up to 0.69 as listed in Table 3). This suggests a strong potential for utilizing machine learning and deep learning for disease management decisions

It is important to acknowledge that this analyses, while yielding favourable outcomes, remain in a preliminary stage. The dataset employed encompassed only 116 canola fields (or 121 data entries) surveyed in Alberta during 2021 and 2022. Despite the relatively robust prediction accuracy and precision, there is room for refinement. One potential avenue for improvement involves incorporating multi-year data spanning all prairie provinces. This expansion could potentially identify fields exhibiting a wider spectrum of blackleg severity, enabling the classification of data into more distinct categories for more accurate regional predictions. The current study did not include factors such as variety resistance and pathogen population, which could potentially impact the model’s accuracy. However, the relative resistance level of cultivars in different fields could not be measured as the pathogen population may be different in these fields (Soomro et al. 2021). Going forward, pathogen Avr genes may be monitored over crop districts in each province, which may be more useful for defining cultivar resistance levels based on R-gene stacking. It is possible that including the resistance and pathogen virulence factors in AI models can further enhance disease risk assessment.

Author contributions

W.G.D.F., G.P., M.W.H, and R.L. initiated the project. L.Z. performed all the analysis work and wrote the manuscript. M.W.H. collected field data. G.P. and S.W. provided input in manuscript preparation. W.G.D.F. and L.Z. wrote the ms, revised and made corrections.

Acknowledgments

We thank Dr. Michael W. Harding’s lab for collecting the data sets that helped us do the analysis. We thank Ms. Merrit Kennedy and Mr. Greg Daniels for assistance in collecting field data and express our appreciation to cooperating canola producers for providing information and allowing access to their fields.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

All datasets generated for this study are included in the article/Supplementary Material.

Additional information

Funding

This research was supported by Canola CAP that was awarded to Dilantha Fernando, through Government of Canada, SaskCanola, MB Canola Growers and Alberta Canola Growers Commission.