Development of a predictive tool for rapid assessment of soil total nitrogen in wheat-corn double cropping system with hyperspectral data

ABSTRACT

Precision farming based on soil total nitrogen status is crucial to informed fertilization strategies while enhancing environmental quality of agricultural ecosystems. Hyperspectral technology was applied for rapid assessment of soil total nitrogen content in a long-term fertilization experiment of wheat-corn rotation system. The first-order derivative of squared spectra and Soil Adjusted Spectral Index transformation provided the highest correlation with soil total nitrogen contents. Inversion modeling results involving multiple linear regression, back propagation neural network (BPNN), and partial least square regression indicated that BPNN provided the best performance with coefficient of determination (R²) reaching 0.8 and 0.9, respectively, for wheat and corn seasons. Model performance with wheat and corn season combined was poor due to seasonal difference in hyperspectral patterns and different decomposition of straws returned to the field. This study provided a quantitative tool for rapid and accurate diagnosis of soil total nitrogen content for precision agriculture.

KEYWORDS:

• Soil total nitrogen
• hyperspectral pattern
• wheat-corn rotation
• BPNN

1. Introduction

Nitrogen (N) is one of the most essential nutrients in soil, for supporting the productivity and functioning of agriculture ecosystems. Soil total N content (TN) is an important indicator of soil N supply capacity to sustain crop yield [1–3]. Rapid and accurate diagnosis of soil TN is of great significance for farmland management and efficient use of N fertilizer. Traditional measurements of soil TN rely on soil digestions such as the Kjeldahl method or instrumental analysis [4]. While these methods offer high precision in measurements, they are costly, time consuming, and complicated in operations, thereby limiting their suitability for rapid analysis of large quantity of samples. The development of precision agriculture requires innovative technology for rapid assessment of soil TN status to achieve precise and accurate adjustment of various field management measures, optimize the highest yield and maximum economic benefits while reducing environmental risks [5–7].

The laboratory hyperspectral method is a non-destructive technique that requires only a few seconds to scan each sample [1,8]. The method identifies substances in soils and determines their chemical composition according to the reflection characteristics of the materials and the physical and chemical properties of the soil. Once a soil sample is scanned, the same spectra can be used for estimation of a variety of soil quality parameters such as soil organic matter, nitrogen, phosphorus, potassium, and soil water content. Its rapid and non-destructive nature also provides a research tool to elucidate mathematical means of analyzing the spectra data for rapid monitoring of soil TN content [9]. Several authors have used statistical models to establish relationships between soil reflectance and soil TN content. Wu et al. optimized the prediction of soil TN using in the hyperspectral sensitive bands and inversion characteristics of a subtropical red soil through a stepwise multiple linear regression modeling procedure [10]. Qiao et al. compared various pretreatment methods of reflection data and used partial least squares regression method to establish the prediction model of calcareous cinnamon soil [11]. They identified the optimal wavelength of the inversion model as well as the optimal combination of soil nutrients and pretreatment methods. Guy Serbin et al. used hyperspectral remote sensing to diagnose soil nutrients under straw mulching and X-ray diffraction analysis technology to determine the influence of soil minerals on the spectrum [12]. These authors did not find any effect of crop stubble on soil functional groups. However, Alabbas et al. found that several functional groups of organic compounds in soil such as hydroxyl and carboxyl groups responded to soil spectra [13].

The current literature clearly shows that the quantitative relationships between soil spectra and soil nutrient properties are confined to specific regions and time in association with changes in cropping systems and regional hydrological conditions. Few studies have established hyperspectral analysis models for soil TN content under the long-term effects of controlled-release fertilizer applications for crop rotations. In this study, we selected experimental plots that have received the same fertilization treatments with controlled release N fertilizer for 10 years in the North China Intensive Agro-ecosystem Experimental Station (Shandong, China). The objectives of this study were to: (i) elucidate the relationships among TN content, organic functional groups, and hyperspectral patterns of soil in different harvest season under wheat and corn rotation; (2) compare and contrast the accuracy of three types of quantitative modeling procedures for prediction of soil TN using soil hyperspectral characteristics at wheat and corn harvests.

2. Materials and methods

2.1 The study area

The North China Intensive Agro-ecosystem Experimental Station (117°21ʹ59”E, 36°57ʹ75”N) is located in Huantai County Shandong Province (Figure 1). The soil belongs to the fluvo-aquic soil subgroup, a light lime moist embryonic soil in Chinese Soil Taxonomy (Carbonatic Ochri – Aquic Cambosols). Planting system is winter wheat-summer corn rotation. After wheat and corn harvest, straw is returned directly to the field by mechanized crushing every year with a depth of returning about 20 cm [2]. Soil samples were collected at wheat maturity stage (June) and corn maturity stage (September) in 2014, 2015 and 2017, respectively. The diagonal five-point sampling method was employed. All soil samples were air-dried and passed through 0.25 mm sieve for further analysis.

Figure 1. Research area showing the experimental plots and soil sampling locations.

2.2 Hyperspectral measurements

The reflectance measurements were accomplished with the FieldSpec4 spectrometer produced by Analytical Spectral Devices Inc. The wavelength range was 350–2500 nm. The spectral sampling interval was 1.4 nm in 350–1000 nm wavelength with a spectral resolution of 3 nm. The spectral sampling interval was 1 nm in 1000–2500 nm wavelength with a spectral resolution of 2 nm. Total outputs were 2115. In order to avoid the interference of natural light, the measurement was conducted in the darkroom where all light sources were cut off. The height of halogen lamp was 70 cm and the angle was 15 degrees. Whiteboard calibration was performed after 20 min of preheating. Every time when the sample spectrum was collected, the white board calibration was carried out. The field of view angle of the probe was 25 degrees. The distance between optical head and sample or reference panel was 16 cm, and 10 spectral curves were obtained within the field of view angle range [14]. Ten spectral curves of each sample were averaged and corrected splice. The spectral curve of each soil sample removes 350–399 nm and 2451–2500 nm wavelength with high noise. Soil TN content was determined with Vario Micro cube manufactured by Elemental Analyzer.

2.3 Data pre-processing and representative bands

Soil reflectivity data were pre-processed with operations involving reciprocal, square, square root, logarithm, first-order differential, first-order differential of reciprocal, first-order differential of square, and first-order differential of square root. Sensitive bands were identified by correlation analysis between soil TN content and processed spectral reflectance obtained at each incremental wavelength of 1 nm. The absolute values of Pearson correlation coefficient (r) were compared for identification of the most efficient pre-processing method.

In order to further improve the correlation, the spectral parameters were further transformed after selecting the best pre-processing method [15]. Parameter transformation is suitable in areas where soil brightness varies greatly due to differences in moisture, roughness, shadows or organic matter. In this study, the Soil Adjusted Spectral Index (SASI) was chosen to transform the soil reflectance with Equation (1) [16] such that

(1) $\mathrm{S}\mathrm{A}\mathrm{S}\mathrm{I}=\left(2{\mathrm{R}}_1+1-{\left({\left(2{\mathrm{R}}_1+1\right)}^2-8\left({\mathrm{R}}_1-{\mathrm{R}}_2\right)\right)}^{1/2}\right)/2$(1)

where R₁ and R₂ are consecutive bands in the sequential analysis. After transformation, the characteristic bands of the soil samples were screened based on the combined spectral reflectance absorption rules and collinearity diagnosis [17,18]. The bands with absolute r values larger than 0.55 were firstly selected. Collinearity diagnosis was further conducted in IBM SPSS Statistics 22 with selection criteria of tolerances >0.1 and the variance expansion factor <10.

2.4. Quantitative modeling procedures

Three modeling methods were used: Multiple Linear Regression model, Back Propagation Neural Network model, and Partial Least Squares Regression method. The independent variables were characteristic bands and the dependent variables were measured total nitrogen content.

Multiple Liner Regression (MLR), also known as inverse least squares method, uses the least squares method to estimate the coefficient matrix [19]. A linear regression model of soil TN content and characteristic bands was constructed using multiple linear regression models. The pre-processed data were used as the independent variable, and the measured values were used as dependent variables to perform multiple linear regression analysis using IBM SPSS Statistics 22.

Back Propagation Neural Network (BPNN) is a distributed processing mathematical model that simulates the neural activity of human brain. The BPNN generally includes an input layer, a hidden layer and an output layer. The implicit layer transfer function, the number of nodes and the network, different settings of the training function strive to achieve the purpose of predictive optimization [20]. The response values of the respective wavelengths are inputs at the input layer, and the estimated values of the concentrations to be measured are obtained by the nonlinear function in the hidden layer [21]. In this study, MATLAB R2016 was used to build BPNN model. The nonlinear logsig function was used to train the data with a network learning rate of 0.05.

Partial Least-Square regression (PLS) is a comprehensive modeling method using principal component analysis, multiple linear regression analysis and least squares regression [22]. It uses the extracted principal components to screen for variables with high contributions. PLS features with abilities to interpret multiple autocorrelation between multiple hyperspectral features and construct regression models using valid data [23]

In this study, two principal components were extracted for subsequent least squares analysis. This analysis was conducted using IBM SPSS Statistics 22.

The samples were divided according to the growth seasons of different crops with twice inversions being carried out. Specifically, soil samples in the wheat maturation stage collected in June 2014 and June 2015 were used for model calibration and the soil samples in the wheat season of June 2017 were used for model validation. Similarly, soil samples in the corn maturation stage collected in September 2014 and September 2015 were used for model calibration and soil samples collected in September 2017 were used for model validation. For comparison purposes, the same procedures were applied for data with wheat and corn seasons combined for model calibration and validation.

The performance of the model was evaluated using the determination coefficient R², root mean square error (RMSE), and relative percent deviation (RPD). Typically, large R² and small RMSE indicate that the model has high estimation accuracy. In addition, values of RPD > 2 are indicative of ‘good’ model predictive ability while RPD < 1.4 suggest ‘poor’ model predictive ability [24]. RPD values in between suggest that the model could be improved further by model optimization.

2.5 Infrared spectroscopy

Infrared spectroscopy is a method commonly used for identification and structural analysis of organic compounds in soil [25]. These spectral patterns of double and combined frequencies of molecular vibration have been linked to organic compounds containing C-H, N-H, O-H groups [26]. In this study, samples collected at the same location during wheat harvest and corn harvest were analyzed with Fourier transform infrared (FTIR) spectroscopy. The sample was ground into fine powder, and a small amount of sample was mixed with potassium bromide at the ratio of 1:100 and pressed into thin sheets. Since most of the basic absorption bands of organic compounds occur in the mid-infrared region, the absorption peaks ranging from 400 to 4000 cm⁻¹ were selected.

3. Results

3.1 Soil total nitrogen

Soil TN contents are summarized in Table 1 according to the modeling and validation procedure for each crop. For samples taken during wheat harvest, soil TN content averaged 1.27 g/kg (range: 0.94 to 1.49 g/kg) for the modeling data set, and 1.32 g/kg (range: 0.99 to 1.51 g/kg) for the validation data set. For samples taken during corn harvest, soil TN content averaged 1.23 g/kg (range: 0.93 to 1.49 g/kg) for the modeling data set, and 1.30 g/kg (range: 1.11 to 1.43 g/kg) for the validation data set.

Table 1. Total nitrogen content (g/kg) in soil samples.

	Sample sets	Sample size	Maximum value	Minimum value	Mean value	Standard deviation
Wheat harvest	Modeling data set	29	1.493	0.9388	1.2734	0.1544
	Validation data set	18	1.510	0.9720	1.3214	0.1267
Corn harvest	Modeling data set	28	1.491	0.9269	1.2285	0.1417
	Validation data set	17	1.430	1.1070	1.3042	0.0931

Overall, soil TN contents in the wheat season were only slightly higher than in the corn season, possibly due to the difference in N demand, N fertilization, and soil management between the wheat and corn seasons [17]. Meanwhile, compared to soil TN contents of modeling set with soil TN contents of verification set, the conclusion is drawn that the inter-annual difference was not significant (Table 2).

Table 2. T-test of total nitrogen content of the validation set and the modeling set.

				T-test for Equality of Means
		Levene’s test for equality of variances							95% Confidence Interval of the Difference
		F	Sig.	t	df	Sig. (2-tailed)	Mean Difference	Std. Error Difference	Lower	Upper
	Equal variances assumed	2.267	0.136	1.196	90	0.235	.0347	.0291	−0.023	0.092
	Equal variances not assumed			1.199	89.330	0.234	.0347	.0290	−0.023	0.092

3.2 Soil reflectance

The general trend of soil spectral reflectance in the wavelength of 400–2400 nm was similar for wheat season and corn season is shown in Figure 2 with samples signed as different colors. Features of baseline translation and tilt between the spectra, which may be caused by scattering effects of different soil particles [18], occurred at roughly the same bands of wavelength. However, variations among samples at a particular wavelength were generally larger for the wheat season than for the corn season. The average reflectance of all the samples was also larger for the wheat season than for the corn season.

Figure 2. Difference original reflectance of soil samples in wheat (a) and corn (b) seasons.

The pre-processing procedures performed on the spectral data reduced noise in the data to varying degrees. Among all the pre-processing methods used, the first-order differentiation after squaring had the best effect. The correlation coefficient reached the highest level in both wheat and corn seasons (Figure 3). After the SASI transformation, the correlation coefficients for the wheat season and corn season were further improved (Figure 4), affirming that this method was suitable for the transformation of hyperspectral data in this study. The characteristic wavelengths of soil samples for both wheat and corn seasons were further screened and selected. For the wheat season, the characteristic wavelengths were 602, 720, 775, 926, and 2215 nm. For the corn season, the characteristic wavelengths were 442, 553, 998, 1082, 1087, 1285, 1314, 1969, and 2377 nm. When the two data set were combined, the sensitive wavelengths were 926, 935, 1082, 1087, and 1491 nm. These wavelengths are consistent with those of other studies [26,27].

Figure 3. Correlation degree between soil reflectance and soil total nitrogen content after pretreatment of soil reflectance in wheat (a) and corn (b) seasons.

Figure 4. Comparison of correlation coefficients between wheat season (a) and corn (b) season soil reflectance before and after SASI calculation.

3.3 Modeling results

The modeling analysis results are summarized in Table 3 with the goodness of fit visually displayed in Figure 5 for the wheat season and Figure 6 for the corn season. For the wheat season, BPNN provided the best model for both the modeling and validation data sets. For example, the R² values of the modeling data sets were 0.73, 0.80, and 0.52, respectively, by MLR, BPNN and PLS. The corresponding RMSE were 0.0807, 0.0708, and 0.0815, respectively. The performance in the validation period was similar with RPD of 2.219 for BPNN, suggesting a good model predictability. Values of RPD were <2 for both MLR and PLS.

Figure 5. Measured vs. Predicted values of MLR, BPNN, PLSR modeling data (a), validation data (b) of wheat seasons.

Figure 6. Measured vs. Predicted values of MLR, BPNN, PLSR modeling data (a), validation data (b) of corn seasons.

Table 3. Model analysis accuracy of different modeling methods for wheat-corn double cropping system.

Model	Wheat harvest					Corn harvest					Wheat harvest+Corn harvest
	Modeling samples		Verification samples			Modeling samples		Verification samples			Modeling samples		Verification samples
	R^2	RMSE	R^2	RMSE	RPD	R^2	RMSE	R^2	RMSE	RPD	R^2	RMSE	R^2	RMSE	RPD
Multiple Linear Regression	0.727	0.081	0.475	0.093	1.946	0.690	0.084	0.780	0.051	1.385	0.498	0.106	0.410	0.082	0.834
Back Propagation Neural Network	0.801	0.071	0.782	0.092	2.219	0.895	0.054	0.887	0.035	1.970	0.692	0.107	0.370	0.087	0.940
Partial least Squares Regression	0.721	0.082	0.472	0.092	1.927	0.682	0.078	0.550	0.056	1.491	0.495	0.089	0.400	0.084	0.816

For the corn season, model performance was similar to that for the wheat season with BPNN being the best. For example, for the modeling data set, the R² values of MLR, BPNN, and PLS were 0.6900, 0.8951, 0.6819, respectively (Table 3). The corresponding RMSE were 0.084, 0.054, and 0.087, respectively. The validation performance was comparable though the RDP value for BPNN was only 1.93, possibly due to the lower number of validation data than for the wheat season. Nevertheless, the most suitable hyperspectral inversion model for both the wheat and corn seasons was obtained with BPNN.

For data combining wheat and corn seasons, the model performance was not as good as that for either wheat season or corn season in spite of a larger data set. While BPNN was still the best among the three quantitative procedures during the model calibration stage, its performance was slightly worse than MLR and PLS. The RDP values there all less than 1, indicating poor model predictability.

3.4 Infrared spectra

Infrared spectra of soil samples displayed similar trends between the wheat season and the corn season with obvious absorption peaks at 469, 777, 1030 and 3427 cm⁻¹ while differences among samples were much less in the wheat season than in the corn season (Figure 7). The absorption intensity of characteristic peaks was more dynamic in the corn season than in the wheat season, especially for the carbonyl absorption peaks (1900–1650 cm⁻¹). The absorption peaks below 900 cm⁻¹ are overlapped peaks of protein NH and carbohydrate sugar ring vibration [28]. Absorption peaks around 1031 and 3426 cm⁻¹ are more prominent. The peaks of 1050 − 1000 cm⁻¹ are mainly carbohydrate absorption bands. The absorption peaks of 1031 cm⁻¹ represent Si-O stretching vibration in silicate minerals and C-O stretching vibration in carbohydrate and polysaccharide structures [29]. The absorption peaks at 3426 cm⁻¹ manifest the existence of hydroxyl groups in soil clay. These groups are linked with hydrogen bonds, representing stretching vibration of -OH forming hydrogen bond and the absorption of N-H stretching vibration in amino acids, cellulose, hemicellulose, starch and monosaccharide [30,31].

Figure 7. Infrared spectrum of corn (a) and wheat (b) seasons soil.

4 Discussion

4.1 Model comparison

The excellent model performance exhibited by BPNN suggests that the relationship between soil TN and spectral wavelengths was non-linear in this study. The essence of multivariate linear regression is to estimate the coefficient matrix by least square method while the partial least square method incorporates the principal component analysis into least square regression [32]. Both methods suffer from their inherent inability to handle nonlinear relationships in a data set. The limited number of input data in this study does not benefit much from the PLS’ advantage of reduced data dimension and redundancy [33].

With BPNN, proper selection of input data is perhaps one of the most crucial steps for modeling success. Typically, a larger number of input neurons provide better model performance at the cost of a more cumbersome BPNN. The larger R² and smaller RMSE (Table 3) of the corn season than the wheat season during the model calibration stage was partly associated with more sensitive wavelength bands for the corn season than for wheat season (9 vs. 5). Our BPNN accuracy was comparable to that of Camacho-Tamayo et al. [34], though our number of samples was much less. BPNN often achieves better results with a large number of data separately for model training and testing [35].

4.2 Soil characteristics

The long-term fixed fertilization scheme did not lead to significant inter-annual and seasonal variations in soil total nitrogen content. However, soil reflectance spectral patterns were different between wheat season and corn season. The spectral curves of winter wheat crop showed an increasing trend in visible and near-infrared bands, consistent with the results of Lou’s study [25]. The difference of soil spectral reflectance between summer corn and winter wheat in the same year may be partly caused by straw returned to the field. With 236 days of growth period for wheat, it is expected that continuous decomposition of straw likely released water soluble substances and crude protein. This explains why carboxyl content increased and the fatty property was weakened in soil with corn straw. The remaining substances were mainly tannin and lignin, which are stable in structure and difficult to decompose. In contrast, at the time of corn harvest, wheat straw had matured for about 90 days, and the decomposition rate of straw reached 48.88–59.95% [36]. The content of carbohydrate and sugar gradually decreased, and the soluble organic matter such as amino acid and organic acid was decomposed. Compared with corn straw, wheat straw contained relatively more cellulose and lignin which were difficult to decompose, thus affecting the physical and chemical properties of soil during the corn harvest season. This may be responsible for the different patterns at 1969 and 2377 and 2215 nm bands in this study. Some workers pointed out that the reflectivity near 1710, 2100 and 2350 nm is closely related to the concentration of hemicellulose, cellulose and lignin [37]. Lydia Serrano showed that the abundant lignin and cellulose in crop stubble resulted in obvious absorption characteristics of 2100–2300 nm [38].

Another factor attributing the difference in hyperspectral patterns between wheat and corn season is timing in the year. Corn is planted typically in October while local rainfall occurs mainly from June to September (75% of the annual rainfall) [3]. The abundant soil moisture was more conducive to the decomposition of wheat stubble and straw, so the soil color and soil reflectance in September were different from that in June. The overall distinction in seasonal soil management and hydrological pattern explained the different soil hyperspectral patterns between wheat and corn season, which in turn, provided the basis for modeling with seasonal data than with annual data (Table 3).

5. Conclusions

A quantitative tool for rapid assessment of soil total nitrogen content has now been successfully developed using characteristic hyperspectral wavelengths identified through first-order differential of squared spectra and Soil Adjusted Spectral Index (SASI) transformation, respectively, in a wheat-corn rotation system. The back propagation neural network has clear advantage over multiple linear regression and partial least square regression for its ability to fit data with non-linear relationships. The preferred number of characteristic wavelengths ranges from 5 to 9 as demonstrated in this study. While the long-term fixed fertilization scheme did not lead to significant difference in inter-annual and seasonal variations in soil total nitrogen, differences in functional groups of soil organic matter, decomposition of returned straws, and seasonal soil hydrological regimes between the wheat season and corn season resulted in different soil spectra patterns. This warranted the development of prediction models separately at the harvest of wheat and corn. These models have practical implications in providing timely fertilization recommendations in a context of precision agriculture.

Acknowledgments

The present study was supported by the National Key Research and Development Program of China (Grant # 2017YFD0200706, 2017YFD0200701), the Key Research and Development Program of Shandong Province (Grant # 2019GNC106011), the National Natural Science Foundation of China (Grant # 41977019, 41907067), the Graduate Education Teaching Reform Research Project of Shandong Agricultural University (Grant # YZD2018002). We thank undergraduates Zhenzhen Zhang, Tao Tan and Changyu Man for sample handling and analysis.

Disclosure statement

No potential conflict of interest was reported by the authors.