Prediction of total nitrogen in cropland soil at different levels of soil moisture with Vis/NIR spectroscopy

Abstract

Visible/near-infrared (Vis/NIR) spectroscopy has been proven to be an effective technique for soil total nitrogen (TN) content estimation in the laboratory conditions. However, the transferability of this technique from laboratory study to field application is complicated by soil moisture effects. This study aims to compare the performance of four spectral transformation strategies, namely, Savitzky–Golay (SG) smoothing, SG smoothing followed by first derivative (FD), orthogonal signal correction (OSC), and generalized least squares weighting (GLSW), in the removal of soil moisture effects on TN estimation. The spectral transformations were applied on 8 sets of spectral reflectance measured from 62 soil samples at 8 moisture levels. The air-dried set was used for partial least squares regression (PLSR) calibration, whereas the other seven sets with moisture gradients were used for external validations. Results show that the SG-PLSR model cannot be transferred from the air-dried samples to the samples with moisture gradients. The FD-PLSR model showed fair TN prediction performance, with five out of seven residual prediction deviations (RPD) that are greater than 1.4. Both OSC-PLSR and GLSW-PLSR had good transferability to the moist samples. More specifically, the GLSW-PLSR model (mean of , root mean square error for prediction [RMSEP] = 0.262, and RPD = 1.885) outperformed the OSC-PLSR model (mean of , RMSEP = 0.277, and RPD = 1.780). The results demonstrate the value of OSC and GLSW in eliminating the effects of moisture on TN estimation, and the GLSW-PLSR is recommended for a better Vis/NIR estimation of TN content under different soil moisture conditions.

Keywords:

• cropland soil
• total nitrogen
• visible/near-infrared spectroscopy
• orthogonal signal correction
• generalized least squares weighting
• soil moisture effects

Introduction

Nitrogen, which is one of the primary nutrients critical for the survival of all living organisms, and its cycle is significantly affected by human activities (e.g., agricultural practice) in the local and global ecosystems (Bernhard 2010). The rational use of nitrogenous fertilizers could not only improve soil fertility and promote the sustainable development of agriculture but also ensure food and nutrition safety (Yakovchenko et al. 1996; Olfs et al. 2005). Conversely, unreasonable management of nitrogenous fertilizers may lead to the decline of soil fertility, resulting in the decrease of crop yield and protein content. Moreover, this situation will cause a series of environmental problems, such as soil erosion and water eutrophication (Elmi et al. 2003; Erhart et al. 2007), which will threaten human health and survival. Soil total nitrogen (TN) is an important part of soil, and the level of its content is one of the important indicators for soil quality evaluation (Shi et al. 2013). The importance of nitrogen in terrestrial ecosystems underlies the significance of enhancing the efficiency of obtaining soil TN content to guide the precise fertilizer and the assessment of soil quality.

In recent years, visible/near-infrared (Vis/NIR) spectroscopy has been increasingly used to measure various soil properties, including the TN content (Chang & Laird 2002; Viscarra Rossel et al. 2006; Michel et al. 2009; Soriano-Disla et al. 2013). Compared with traditional biological and chemical methods for determining TN content (Keeney 1982; Felipó & Garau 1987), Vis/NIR spectroscopy has the advantages of fast, efficient, and nondestructive measurements (Demattê et al. 2004; Terhoeven‐Urselmans et al. 2006). In the laboratory, soil reflectance can be acquired under controlled conditions. However, in the field, spatial variation of “external parameters,” such as moisture and particle size, may influence soil reflectance and further degrade its ability for TN estimation. The effects of soil moisture on the Vis/NIR reflectance spectra have been extensively studied (Galvão & Vitorello 1998; Reeves III 2010). The relationship between the normalized soil reflectance and moisture is nonlinear (Idso et al. 1975). Such nonlinear relationship could explain why homogeneous moisture in soils is preferable for the Vis/NIR estimation of soil properties. The influence of soil moisture on the Vis/NIR estimation of soil properties, such as TN, has been explored in various studies. For example, Chang and Laird (2002) minimized the influence of moisture content on the spectra by using dried samples. Nevertheless, one of the most challenging issues is how to improve the transferability of Vis/NIR model among different moisture conditions. Moreover, with the development of a global reflectance spectral library from legacy soil samples (Viscarra Rossel et al. 2008), which are usually air-dried, transferring the Vis/NIR model calibrated from dry samples to the samples with moisture gradients would be desirable.

Chemo-metric strategies were used in line monitoring of petrochemical, food, and pharmaceutical field for preventing influence factors, which can occur in industrial conditions (Preys et al. 2008), such as multiplicative scatter correction (Isaksson & Næs 1988), standard normal variate transformations and de-trending methods (Barnes et al. 1989), smoothing (Savitzky & Golay 1964), orthogonal signal correction (OSC; Wold et al. 1998), external parameter orthogonalization (EPO; Roger et al. 2003), and generalized least squares weighting (GLSW, Zorzetti et al. 2011). In soil science, current efforts have been devoted to the removal of the influence of moisture on the Vis/NIR estimation of soil organic carbon (SOC) in view of some chemo-metric strategies. For example, Wu et al. (2009) noted that the first derivative (FD) of specific bands (e.g., 800–1400, 1600–1700, and 2100–2200 nm) is moisture-independent and proposed to use these spectra to estimate SOC content under field condition. Marco et al. (2013) classified the spectra according to the normalized soil moisture index proposed by Haubrock et al. (2008) and established a series of models to predict SOC content at different moisture levels. Minasny et al. (2011) used EPO to remove the effects of soil moisture from Vis/NIR spectra in the prediction of SOC content. Despite these efforts, further examination should be conducted on chemo-metric strategies that hold the potential to remove moisture effects on the Vis/NIR estimation of TN.

We investigated the influence of soil moisture on the Vis/NIR spectra and the prediction of TN. This study mainly aims to remove the soil moisture effects on the Vis/NIR estimation of soil TN. More specifically, we aim to (1) explore the transferability of partial least squares regression (PLSR) model calibrated among eight moisture gradients and to (2) compare the performance of four spectral transformation strategies, namely, Savitzky–Golay (SG) smoothing, SG smoothing followed by FD, OSC, and GLSW, in the removal of soil moisture effects on the Vis/NIR estimation of TN content.

Materials and methods

Study area

The study area is located in Honghu City (Hubei Province), Jianghan Plain, which is known for its riparian landscape as well as its dramatic land use and land cover changes since the 1950s (Fang et al. 2005). The landform of the study area is flat with an elevation not exceeding 50 m above mean sea level. Inceptisols dominate the study area according to the US Department of Agriculture Soil Taxonomy. Water body, lacustrine vegetation, floodplain, cropland, and open land are the major land cover types (Zhao et al. 2003). The field survey in December 2011 reported that artificial forest and meadow along the dams are now essential parts of land cover types in this region. Paddy field and irrigated cropland are the major land-use types comprising the cropland. The humid climate and diverse land-use types result in the spatial–temporal dynamics of soil moisture. Therefore, the Honghu region was selected as study area.

In December 2011, a total of 62 topsoil samples were collected in study area. According to the US Department of Agriculture classification system, soil texture of all samples was classified into four classifications. Descriptive statistics showing the texture, TN, and SOC content of the samples have been provided in Table 1. We can see that with the order of clay, loam, and sandy, the SOC and TN content gradually reduced. The geographical coordinates of all sampling sites were recorded using a handheld global positioning system with a positional error of less than 5 m. Figure 1 shows the spatial distribution of the sampling sites, with an Enhanced Thematic Mapper Plus (ETM+) image (panchromatic band) showing the landscape of the study area. Spatial resolution of the panchromatic band is 15 m and the spectral range is 0.52–0.90 µm. Thus, the cropland shows bright while the water body (e.g., breeding ponds) appears to be dark. It is noted that the total set of soil samples were from cropland and some were much closer to the breeding pond area. At each sampling site, about 1.0–1.5 kg of surface soil from a depth of 0–10 cm was collected in five sampling campaigns after wiping off plant material, plant residues, roots, and stones (Shi et al. 2013). That is to say, five samples were collected on the center position and four corners of a 5 × 5 m grid for each sampling site, and composed them as one soil sample. This method can reduce random error and ensure the representativeness of the sample. The collected soil samples were then kept in a sealed package for chemical analyses and spectral measurement in the laboratory.

Table 1. Basic properties and statistical descriptions of the soil samples on Honghu region, China.

			SOC (g kg^−1)			TN (g kg^−1)
Texture	Sample number	Crop	Min	Max	Mean	Min	Max	Mean
Clay	36	Winter rape, grass, idle field	15.09	26.23	20.59	1.23	2.18	1.69
Clay loam	7	Wheat, winter rape, sesame	11.71	13.74	12.36	0.87	1.6	1.24
Loam	14	Wheat, winter rape, sesame	7.45	10.25	8.60	0.61	1.03	0.78
Sandy loam	5	Peanut, winter rape	2.35	6.12	4.35	0.43	0.64	0.55

Min, minimum; Max, maximum. SOC, soil organic carbon; TN, total nitrogen.

Figure 1. Maps showing the geographical location of the study area, distribution of sampling sites, and landscape of the study area, indicated by a LANDSAT-7 ETM+ image with band 8 (panchromatic band).

Laboratory analyses and measurements

In the laboratory, the soil samples were air-dried, ground, and sieved (less than 2 mm). Each sample was thereafter placed on a petri dish with diameter and thickness of 12 and 1 cm, respectively. All samples were oven-dried at 105°C for six hours and weighted on an analytical scale (with accuracy of 0.01 g) to calculate the weight of oven-dried soil samples. A small part of each sample was analyzed for soil TN content by using the semimicro Kjeldahl method (Nelson & Sommers 1980). Subsequently, the oven-dried soil samples were rewetted to oversaturation, with the moisture content falling within the range of 350–400 g kg⁻¹. For the rewetting process, predetermined volume of distilled water was measured using a glass cylinder, and gently added to the petri dish. Each of the petri dish was covered for about two hours until all the samples were saturated. The samples were thereafter air-dried for seven days at room temperature ranging from 18 to 22 °C (Ciarlo et al. 2007). The lid of the petri dish was used to control soil evaporation so that the moisture of each sample could fall within the ranges of 300–350 g kg⁻¹, 250–300 g kg⁻¹, 200–250 g kg⁻¹, 150–200 g kg⁻¹, 100–150 g kg⁻¹, 50–100 g kg⁻¹, and 0–50 g kg⁻¹ (air-dried) for each of the seven days, respectively. The soil samples at these eight different moisture levels (including the oversaturated samples) were weighted shortly before spectral measurement. Subtracting the weights of the oven-dried soil samples facilitates the calculation of the moisture of each sample at each moisture level. With box plots, Figure 2 shows the statistical characteristics of the moisture content of the eight data-sets. It can be seen that for each of the seven moist data-sets, the moisture content of the 62 samples takes an approximately normal distribution. We assumed that such statistical distribution of moisture content could be a good representative of operational situation.

Figure 2. Box plots of soil moisture content (SMC) which are divided into eight levels: 0–50 g kg⁻¹ (air-dried), 50–100 g kg⁻¹, 100–150 g kg⁻¹, 150–200 g kg⁻¹, 200–250 g kg⁻¹, 250–300 g kg⁻¹, 300–350 g kg⁻¹, and 350–400 g kg⁻¹.

An ASD FieldSpec3 portable spectral radiometer with wavelength of 350–2500 nm was used to measure the soil spectra for each set. The sampling interval and spectral resolution were 1.4 nm and 3 nm for the 350–1000 nm range, respectively, and 2 nm and 10 nm for the 1000–2500 nm range, respectively (http://www.asdi.com). A standardized white Spectralon^® panel was used for reflectance calibration. A white light source matched with the spectroradiometer was used with a 45° incident angle. The spectra of the soil samples were measured by the spectroradiometer with a distance of 15 cm from the probe to the sample surface and a zenith angle of 90°. The entire scanning procedure was performed in a dark room at night to minimize the influence of external light (Choe et al. 2008).

Spectral preprocessing

Only the reflectance of the air-dried samples (0–50 g kg⁻¹) in the region of 400–2350 nm was used for model calibration because of the presence of noise at both edges. Four spectral preprocessing strategies were implemented and compared in terms of their performance in removing soil moisture effects on TN prediction. The reflectance was smoothed by using the SG algorithm with a window size of 15 and polynomial order of 2 (Savitzky & Golay 1964). The smoothed reflectance was thereafter preprocessed with FD (Rinnan et al. 2009), OSC, and GLSW. Thus, for each of the eight data-sets, four sets of spectra were involved: (1) the reflectance with SG smoothing, (2) the SG smoothed reflectance followed by FD, (3) the SG smoothed reflectance coupled with OSC, and (4) the SG smoothed reflectance followed by GLSW.

The OSC effectively removes all orthogonal variations in independent variables (X-block) that are unrelated or orthogonal to dependent variables (Y-block), thus simplifying the calibration model (Lin & Kuo 2013). The algorithm starts by identifying the first principal component of the X-block. Afterward, the loading is rotated to make the scores orthogonal to the Y-block. Once the rotation is complete, a partial least squares (PLS) model is estimated, which can predict these orthogonal scores from the X-block. The number of components (c) in the PLS model is adjusted to achieve a specified level of captured variance for the orthogonal scores. Finally, the weights, loadings, and predicted scores are used to remove the specified orthogonal component. This entire process can be repeated on the X-block, from which the previously identified orthogonal component was removed, for any given number of components. Each cycle results in additional PLS weights and loadings being added to the total and will be used when applied to new data (Wise et al. 2007).

The GLSW was used to down weigh the differences between replicate measurements (Zorzetti et al. 2011). The underlying principle of GLSW is a filter calculated from the differences (usually caused by external factors) between samples, which should otherwise be similar. These differences are considered interferences, which the filter attempts to down weigh. In this study, GLSW was applied to correct soil moisture effects on the Vis/NIR estimation of TN. GLSW helps down weigh the differences and make the spectra of the same sample at different soil moisture gradients appear more similar. Ten samples were selected from the air-dried set according to the spectral Euclidean distance (DISTSLCT function in PLS toolbox version 7.3.1, Eigenvector Research, Manson, WA). Then, the same samples were selected from the other seven data-sets with different moisture levels. Thereafter, a GLSW was estimated by using the SG smoothed reflectance of selected samples from each of the eight moisture levels. Finally, the filtered air-dried set was used for the PLSR calibration (Fu et al. 2012). The most commonly used algorithm for GLSW has a single adjustable parameter, α, which defines how strongly the GLSW down weight the interferences. Adjusting α toward larger values (typically above 0.001) decreases the effect of the filter. Smaller α values (typically 0.001 and below) apply more filtering.

Building the data-sets

The reflectance spectra and the TN content of all 62 soil samples were denoted as (Xi, Y), where Xi denotes the spectral matrix and Y represents the content of TN. The lower case character i represents the soil moisture level, where i = 1 means soil moisture content (SMC) = 0–50 g kg⁻¹ (the air-dried data-set), i = 2 means SMC = 50–100 g kg⁻¹, i = 3 means SMC = 100–150 g kg⁻¹, i = 4 means SMC = 150–200 g kg⁻¹, i = 5 means SMC = 200–250 g kg⁻¹, i = 6 means SMC = 250–300 g kg⁻¹, i = 7 means SMC = 300–350 g kg⁻¹, and i = 8 means SMC = 350–400 g kg⁻¹. With the four spectral transformations mentioned above, 32 data-sets, namely, (, Y), (, Y), (, Y), and (, Y), i = 1, …, 8, were built to study the influence of soil moisture on the prediction of TN content.

Model calibration and validation

PLSR modeling

PLSR coupled with leave-one-out cross validation are used for model calibration by relating the spectral data-sets Xi with TN content data-sets Y. This method was developed by Wold et al. (2001) and has become a routine modeling technique for the quantitative analysis of Vis/NIR spectra (Viscarra Rossel & Behrens 2010). The advantages of PLSR are that it handles multicollinearity. It is robust in terms of data noise and missing values (Viscarra Rossel et al. 2006). Particularly, it is useful when dealing with highly correlated predictor variables, whose number is much greater than that of samples (Wold et al. 2001; Viscarra Rossel et al. 2006). According to the four spectral pretreatment strategies, the SG-PLSR, FD-PLSR, OSC-PLSR, and GLSW-PLSR models were calibrated by using the (, Y), (, Y), (, Y), and (, Y) air-dried data-sets, respectively. To find the better c and α for the OSC-PLSR and GLSW-PLSR models, respectively, different values of c (1, 2, 3, and 4) and α (0.01, 0.001, and 0.0001) were used in the model calibrations. Leave-one-out cross validation (Gomez et al. 2008) was used to determine the optimal number of factors in the PLSR models. PLS toolbox version 7.3.1 (Eigenvector Technologies, Manson, Washington, USA) was applied for the model calibration and validation.

Evaluation of the calibration models

The performances of the calibration models were assessed by the determination coefficient for cross validation and validation ( and ), the root mean square error for prediction (RMSEP), and the residual prediction deviation (RPD; Saeys et al. 2005; Mouazen et al. 2010; Wang et al. 2014). R² indicates the variance percentage in the Y variable that can be captured by the X variables. The RPD, defined as the ratio of the standard deviation of the Y variable in the validation set to the RMSEP, demonstrates the predictive performance of the calibration models. We adopted the five level interpretations of R² given by Williams (2004) and the RPD mentioned by Viscarra Rossel et al. (2006): excellent predictions (R² ≥ 0.90, RPD ≥ 2.5), good predictions (0.81 ≤ R² < 0.90, 2.0 ≤ RPD < 2.5), approximate quantitative predictions (0.66 ≤ R² < 0.81, 1.8 ≤ RPD < 2.0), possibility to distinguish between high and low values (0.50 ≤ R² < 0.66, 1.4 ≤ RPD < 1.8), and unsuccessful predictions (R² < 0.50, RPD < 1.40).

Transferability verification

The transferability verification is two-fold. First, eight PLSR models were calibrated by using one of the data-sets (, Y) with a specific i (i = 1, …, 8). The other seven data-sets were used for validations. Such processes were designed to examine whether the PLSR models calibrated from the soil samples at a specific moisture level could be successfully applied to estimate the TN content when the moisture of the samples varied. No other spectral transformation strategies were used in this verification process. The second verification procedure involved all four spectral transformation strategies for the purpose of comparison. The (, Y), (, Y), (, Y), and (, Y) data-sets were used for model calibration. Thereafter, coupled with the other data-sets at different moisture gradients, four calibration models, namely, SG-PLSR, FD-PLSR, OSC-PLSR, and GLSW-PLSR, were used for TN prediction and transferability verification. In both verification processes, the transferability of the calibration models was assessed by using and RPD.

Flowchart of the study

Figure 3 shows a flowchart that describes the processes of data collection and preparation, model calibration, validation, and evaluation. For the comparisons of spectral transformations in the removal of soil moisture effects, only the air-dried data-sets were used in the model calibration, and the model transferability was examined with the moisture data-sets as test sets. This measure is performed not only because most previous studies have used the spectra of air-dried samples (e.g., Shi et al. 2013) but also for future use of soil spectral library in the estimation of soil properties in the field.

Figure 3. Flowchart of the processes of data collection and preparation, model calibration, validation, and evaluation.

Results and analysis

Effect of moisture content on soil spectra

Figure 4(a) shows the SG smoothed spectra. The differences in soil moisture result in the variations of spectra in the entire spectral range (400–2350 nm). The reflectance decreased with increased soil moisture. The variation is more pronounced for longer wavelengths, particularly in the water absorption bands (1450 and 1940 nm; Weidong et al. 2002). For low moisture levels, a small increase in soil moisture induces a typical decrease in soil reflectance. However, in the cases of higher soil moisture levels (i.e., SMC > 250 g kg⁻¹), the reflectance decreases slightly with increased moisture. These conclusions are consistent with those of Neema et al. (1987) and Zhu et al. (1984). Moreover, the wavelengths around 1450, 1850, and 2250 nm were identified as the important wavelengths for PLSR modeling in TN content evaluation (Shi et al. 2013). Thus, moisture may mask these features, resulting in poor Vis/NIR calibration for TN.

Figure 4. The average of soil spectra from different data-sets with four spectral transformations: (a) SG smoothing, (b) SG smoothing followed by FD, (c) SG smoothing followed by OSC, and (d) SG smoothing followed by GLSW.

Figure 4(b) shows the eight curves of the FD spectra. The water peak at 1400 and 1900 nm is exaggerated. The eight spectral curves appear more similar to one another compared with those in Figure 4(a). In Figure 4(c), the OSC applied in this study effectively reduced the influence of moisture and furthered the similarity of the spectral curves from the eight moisture levels. This finding suggests that the OSC may be used to remove the soil moisture effects on the Vis/NIR prediction of soil TN content. Figure 4(d) reveals that the eight spectral curves with GLSW transformation fluctuate significantly. However, similar trends of these curves can still be observed in the region of 400–2350 nm.

Effect of moisture content on the transferability of SG-PLSR models

The transferability of the eight SG-PLSR models was assessed by using and RPD (Figure 5). When ≥ 0.5 and RPD ≥ 1.4, models could be transferred among different moisture levels, and the larger, the better. Models calibrated from the moist samples can be transferred to samples with moisture differences of approximately 50 g kg⁻¹ (the adjacent moisture gradient). However, these models failed to predict the TN content when the differences of moisture content were greater than 100 g kg⁻¹. The model calibrated by using the air-dried samples (0–50 g kg⁻¹) exhibited poor transferability in estimating TN with the moist samples. Thus, without additional spectral transformations, the SG-PLSR models need to be calibrated at each of the moisture gradients for accurate TN estimations.

Figure 5. Transferability of SG-PLSR models for total nitrogen (TN) estimation with soil samples of eight moisture gradients as assessed by (a) determination coefficient for validation () and (b) RPD. It should be noted that the diagonals are (a) determination coefficient for calibration () and (b) standard deviation of prediction set divided by root mean square error of calibration.

Effect of spectral transformations on the correlations between soil spectra and TN

The Pearson correlations between differently pretreated spectra and soil TN at all eight moisture levels were explored. The correlation curves were line-plotted in Figure 6. The spectral wavelengths with significant correlation at the confidence interval of 0.01 were highlighted with black cross symbols. The significant wavelength variables may vary with different spectral pretreatments. No evident trend was found for all eight correlation coefficient curves from the spectra with SG smoothing [Figure 6(a)]. This result indicates that the soil moisture significantly interfered with the correlation between reflectance and TN content. The FD-based correlation curves fluctuated in the entire spectral region. The shapes of the eight curves were similar in the visible region but slightly different in the NIR region of 1400–2350 nm [Figure 6(b)]. sure 6(c) shows that the shapes of the eight correlation curves for the OSC-pretreated spectra and TN content were relatively similar. The same phenomenon was found in the GLSW-pretreated spectra [Figure 6(d)]. These results suggest that the latter two spectral preprocessing techniques could be useful in determining the common wavelengths, which are important for TN estimations at different moisture levels. Thus, the model calibrated from these common wavelengths could be transferred in the soil samples with different moisture contents.

Figure 6. Pearson’s correlation coefficient between total nitrogen (TN) content and differently pretreated spectra: (a) spectra with Savitzky–Golay (SG) smoothing, (b) spectra with SG smoothing and FD, (c) spectra with SG smoothing and OSC, and (d) spectra with SG smoothing and GLSW.

Parameters optimization for the OSC-PLSR and GLSW-PLSR models

The parameters that need to be optimized for OSC-PLSR and GLSW-PLSR are c, the number of OSC filter dimensions and α, the weights of GLSW filter intensity. Figure 7 shows the RMSE values of the TN prediction with different c and α. Results show that the lowest RMSE values were obtained when c = 3 (for the OSC-PLSR model) and α = 0.001 (for the GLSW-PLSR model). Therefore, we chose c = 3 and α = 0.001 for OSC and GLSW filters, respectively.

Figure 7. RMSEP of total nitrogen (TN) prediction models with the seven moist data-sets as validation sets: (a) OSC-PLSR models with different c values (the number of OSC filter dimensions); (b) GLSW-PLSR models with different α values (the weights of GLSW filter intensity).

Performances of the four PLSR models

Table 2 presents the results of the four PLSR models for estimating TN content. The optimal number of latent variables (N_LV) determined by leave-one-out cross validation for the SG-PLSR, FD-PLSR, OSC-PLSR, and GLSW-PLSR models were 4, 5, 4, and 3, respectively. In terms of model calibration by using the (, Y), (, Y), (, Y), and (, Y) data-sets, the determination coefficient for calibration () values were between 0.686 and 0.897, with the between 0.619 and 0.699. The results indicate successful model calibrations. Specifically, the OSC-PLSR model performs better than the SG-PLSR and the FD-PLSR models do, but not as good as the GLSW-PLSR model in terms of N_LV and .

Table 2. Prediction of soil total nitrogen (TN) content using Vis/NIR spectroscopy with different preprocessing methods.

				Mean			Standard deviation
Model	NLV				RMSEP	RPD		RMSEP	RPD
SG-PLSR	4	0.686	0.619	0.468	0.357	1.382	0.091	0.031	0.122
FD-PLSR	5	0.897	0.684	0.557	0.33	1.493	0.067	0.028	0.131
OSC-PLSR	4	0.75	0.696	0.695	0.277	1.78	0.033	0.017	0.119
GLSW-PLSR	3	0.842	0.699	0.718	0.262	1.885	0.024	0.012	0.086

The four PLSR models were used to predict the TN content in soil samples with varying moisture contents (data-sets (, Y), (, Y), (, Y), and (, Y), i = 2, …, 8). Thus, for each of the PLSR models, seven validation sets corresponding to the seven moisture gradients were employed. , RMSEP, and RPD were calculated in each of the validation processes. Their mean and standard deviation are shown in Table 2. Figure 8 details the prediction performance of the four PLSR models for each of the seven soil moisture levels.

Figure 8. Performances of the four PLSR models: SG-PLSR, FD-PLSR, OSC-PLSR, and GLSW-PLSR. The air-dried sets were used for model calibration, whereas the other seven sets with moisture gradients were used for validations.

Poor predictions were obtained for the SG-PLSR model when the model calibrated with the air-dried samples was applied to predict the TN content of moist samples. The mean values of , RMSEP, and RPD were 0.468, 0.357, and 1.382, respectively. The (0.331–0.589), RMSEP (0.315–0.402), and RPD (1.226–1.563) for the seven validation sets varied dramatically, indicating unstable prediction ability. For the FD-PLSR model, the predictions were less affected by SMC. The mean values of increased to 0.557 and the mean values of RPD increased to 1.493, whereas those of the RMSEP reduced to 0.330, which were better compared with the predictions of the SG-PLSR model. However, two out of the seven values of and RPD were below 0.5 and 1.4, respectively. This finding denotes the limitations of this model. Besides, the moisture effects on the TN estimation remained because was considerably less than .

Therefore, the OSC-PLSR and GLSW-PLSR models were calibrated to remove the soil moisture effects on TN predictions. The mean values of for these two models significantly increased and approached their . All of the seven RPDs are greater than 1.4, and the RMSEP decreased from above to below 0.29. These results show that the prediction accuracy of the soil with moisture gradients is comparable to the prediction of air-dried soils. Moreover, less fluctuation was observed in , RMSEP, and RPD curves for the OSC-PLSR and GLSW-PLSR models. This finding indicates that the predictions for all seven sets of moist samples were similar in spite of variations in moisture contents. These results proved the efficiency of the OSC-PLSR and GLSW-PLSR in the removal of soil moisture effects on the Vis/NIR estimation of TN. The GLSW-PLSR model (mean of = 0.718, mean of RMSEP = 0.262, and mean of RPD = 1.885) performed better than the OSC-PLSR model did (mean of = 0.695, mean of RMSEP = 0.277, and mean of RPD = 1.780). This finding confirms that the quantitative prediction of soil TN from the soil samples with varied SMCs is possible.

Discussion

Diffuse reflectance spectroscopy has been reported to have potential as a rapid screening method to characterize soil chemical properties (Udelhoven et al. 2003). Unfortunately, field spectroscopy performed not as well as laboratory spectroscopy probably due to strong interferences of environmental factors, such as soil moisture variations. Prediction accuracy depends upon sample preparation and spectral pretreatment selected before estimating a statistical model (Chang & Laird 2002). In this study, four modeling strategies were used to examine their ability of removing the soil moisture effects on the Vis/NIR estimation of soil TN.

Considering all the indicators used to assess the quality of model predictions, the performance of the four models can be ranked as GLSW-PLSR > OSC-PLSR > FD-PLSR > SG-PLSR. Such a result may be explained by the fact that the SG- or FD-PLSR model is just to deal with spectral data itself, and does not consider the spectral differences between the air-dried and the moist soil sample sets. Such process may miss some useful information and cannot eliminate the soil moisture effect entirely, which will further affect the quality of modeling. In addition, the FD algorithm can remove the baseline drift and background interferences (Rinnan et al. 2009), which were partially caused by soil moisture. The interferences caused by soil moisture are nonlinear. Thus, the FD cannot eliminate the influence of moisture completely. By contrast, the OSC- and GLSW-based models introduced in this study showed promising results and could effectively eliminate the effects of soil moisture on the Vis/NIR predictions of TN. However, it should be pointed out that, through too few OSC components, the OSC can only remove parts of the signals that are unrelated (orthogonal) to Y. In contrast, removing too many OSC components may result in erosion of the useful signal (Boulet et al. 2007). These defects decrease the prediction accuracy of OSC-PLSR models. Thus, appropriate number of OSC components is important for OSC-PLSR in effectively eliminating the effects of soil moisture on the Vis/NIR prediction of TN. GLSW uses the eigenvectors and eigenvalues of a covariance matrix to down weight signal identified as being from known interferences or differences between samples which should otherwise be the same (Zorzetti et al. 2011). Results indicated that GLSW-PLSR model obtained a better performance than the other models.

In addition, we compared the important wavelengths for estimating the soil TN between the models calibrated using the air-dried and moist soil sample sets (with SMC = 150–200 g kg⁻¹ as an example). Thereafter, from the perspective of important wavelengths, we further evaluated the potentials of the four spectral pretreatments in removing the interferences of soil moisture. For a specific spectral pretreatment, if the important wavelengths of the air-dried and the moist samples-based models are similar in their distribution, it may indicate that this pretreatment functions effectively in removing the soil moisture effect on the TN estimation. The variable importance in the projection (VIP) was used to identify the important wavelengths used in the PLSR calibrations (Chong and Jun 2005). If the VIP value for a specific wavelength is greater than 1, this spectral wavelength is then considered to be important (Gomez et al. 2008). The VIPs for predicting soil TN from the four preprocessed reflectance spectra are presented in Figure 9. We can see that the VIP scores of four PLSR models now display a marked difference. According to the order of Figure 9(a)–(d), with the increasing number of important wavelengths, the distribution of the bands is becoming more and more wide. This suggests that the prediction abilities of models are gradually improved. From the perspective of moisture content, the similarity of VIP curves of four models are improving step by step. The similar patterns suggest that both data-sets estimate very similar regression models to predict soil TN, which were less affected by water content. This further proved the performance of the four models illustrated as GLSW-PLSR > OSC-PLSR > FD-PLSR > SG-PLSR.

Figure 9. The VIP scores of (a) SG-PLSR, (b) FD-PLSR, (c) OSC-PLSR, and (d) GLSW-PLSR modeling for TN estimation using the air-dried and moist samples (with SMC = 150–200 g kg⁻¹ as an example), respectively.

Despite these differences, we found some common important bands of the four models. According to the VIP scores, the wavelengths near 1410, 1850, 2210, and 2350 nm were identified as the important wavelengths for TN estimation. Approximate wavelength regions around 1446, 1826, 2176, and 2366 nm were also identified by Morra et al. (1991). Similarly, the wavelengths 1450, 1850, 2250, and 2330 nm were reported by Shi et al. (2013) for TN content estimation. Specially, in our study, we found that the TN content showed high correlations with the wavelengths in the visible region. This could be explained by previous studies which reported visible wavelengths 600–700 nm (Vinogradov 1981); and 564.4 and 623.6 nm (Krishnan et al. 1980) were important for organic matter prediction. Because of the inherent link between total carbon and TN (Fystro 2002), soil TN content might be indirectly related to these wavelengths through soil total carbon (Shi et al. 2013).

Soil moisture is one of the major factors affecting soil reflectance in the Vis/NIR region. Some researchers (e.g., Hummel et al. 2001) estimated soil chemical properties by developing a calibration data-set with samples covering a wide range of moisture contents. While our study demonstrated the applications of OSC and GLSW in the removal of soil moisture effects on the Vis/NIR prediction of TN content in soil. For operational application in the field situation, the moisture of soil samples may vary spatially. In order to calibrate a TN estimation model which is less influenced by or free of soil moisture effect, the GLSW and OSC may be applied to the Vis/NIR spectra of samples with a wide range of moisture contents. Thereafter, the PLSR approach can be used for model calibrations. It should be noted that the spatial variations of other external parameters besides the moisture might weaken the predictive ability of the TN estimation model. Further investigations are needed to determine whether the effects of other factors, such as soil particle size, can be removed by algorithms, such as OSC and GLSW. In our future studies, the models calibrated in our study need to be validated in other study areas, and the formulation of “moisture-free” or “other-external-factors-free” model for the Vis/NIR estimation of soil chemical properties is encouraged to transfer this technique from laboratory study to field application. Besides, airborne and satellite hyperspectral images are useful data sources in mapping TN contents over large areas (Shi et al. 2013). Several studies have showed that the outdoor and airborne/satellite imaging spectroscopy have potentials for the estimation of soil properties (Stevens et al. 2010; Viscarra Rossel & Behrens 2010). This opens further perspectives to airborne hyperspectral sensors, such as the HyMap and AVIRIS sensors, which characterize soil reflectance in the Vis/NIR spectral region (400–2500 nm) with a relatively high spectral and spatial resolution (Udelhoven et al. 2003). Therefore, the proposed strategies for removing moisture effects on the Vis/NIR estimation of TN based on outdoor and airborne/satellite data should be explored to provide accurate information for precision agriculture.

Acknowledgments

The authors are thankful to Prof. Yanfang Liu for her support in the laboratory analyses and Prof. Guofeng Wu for his valuable comments. We would also like to thank Mr Kai Guo, Mr Pujiang Huang, and Ms JingQian for their helpful advices on the methodology.

Funding

This work is supported by the National 12th Five-Years Key Technology R&D Program of China [grant number 2011BAB01B06].

Additional information

Funding

Funding: This work is supported by the National 12th Five-Years Key Technology R&D Program of China [grant number 2011BAB01B06].