How well do fertilizer enhancers work?

ABSTRACT

Fertilizer enhancers can provide economic and environmental benefits but their efficacy needs to be assessed appropriately. This study analyzed available trial results of three different fertilizer enhancer products to determine if the results support product claims. All available trial results were considered for a meta-analysis of each product. Product effect was calculated for each experiment in a manner that could be compared with manufacturer claims. The distribution of trial results was examined and factors that may have influenced results assessed. Previously published assessments of the products were compared. N-Boost biostimulant sprayed with dissolved urea increased pasture nitrogen response by an equivalent of 18.0 kg N/ha compared to urea alone in nitrogen-responsive and label-conditions trials. For SustaiN Green urease-inhibitor-treated urea, pasture nitrogen response was 50% higher than untreated urea if low-ammonia-volatilization-risk trials were excluded. AVAIL phosphate fertilizer enhancer increased crop yields by 4.1% over untreated fertilizer in fertilizer-responsive and non-high-soil-phosphate trials.

KEYWORDS:

• efficient fertilizer use
• meta-analysis
• nutrient use efficiency

Introduction

Efficient fertilizer use (yield response per unit of nutrient applied) is crucial for economic and environmental reasons. Nutrient use efficiency (NUE) in intensive pastoral systems is often below 50% (Gourley et al. 2012).

For grazed pasture systems in New Zealand, the typical nitrogen response in kg dry matter biomass per kg nitrogen applied (kg DM/kg N) in actively growing pastures has been measured from 6 to 30 with a value of 12 kg DM/kg N being common at moderate (20 to 40 kg N/ha) fertilizer application rates (Cameron et al. 2005). For pasture with a shoot nitrogen percentage of 3%, this represents around 36% efficiency of converting fertilizer nitrogen into shoot biomass.

For cropping systems, the estimates of NUE are variable with phosphorus uptake estimated at generally being 10–30% of the annual phosphorus application (Doberman 2007) with the remainder mostly remaining in the soil and potentially available for future crops. The unutilized portion of a fertilizer may, however, be lost to the environment, potentially contributing to waterway eutrophication and, in the case of nitrogen, a contribution to greenhouse gas emissions as nitrous oxide.

One approach to improving NUE is to include a fertilizer enhancer product with nitrogen or phosphorus fertilizers. Determining the efficacy of these fertilizer enhancing products is best accomplished by analysis of several trials to overcome the variability of individual trial. Ideally a range of field conditions would be tested to enable assessment of factors affecting product efficacy.

In an individual trial, an ineffective product can show a statistically significant effect purely by chance (a Type I error). At 95% level of probability on two-tailed tests, around 2.5% of trials of an ineffective product could be expected to be statistically significantly positive. Presentation of a selected trial or group of trials with significant results could, therefore, give a false impression that the product is efficacious. The analysis of all valid multiple trials (meta-analysis) more fairly tests the statistical significance of the effect.

Individual trials of effective products can also fail to show a statistically significant effect (Type II error). This is exacerbated by natural variability in the crop as well as limitations in sample design such as small plot size and low numbers of replicates. With multiple trials, the power of the analysis is appropriately increased, which makes Type II errors less likely while leaving the chance of a Type I error the same. There are two main caveats to the meta-analysis approach.

The first caveat is that if there is any bias in the selection of the trials for a meta-analysis, then there is a risk of a product effect being over- or under-estimated. With a bias towards favorable-looking trials, the effect of even an ineffective product could be misleadingly reported as statistically significant. Thus, all appropriately designed trials should be included in an initial overall analysis.

The second caveat is that as trial numbers in a meta-analysis increase, the chance of a small mean effect being statistically significant increases. The focus should therefore not be just on the p-value but also on the size of estimated effect to assess if it is meaningful, economically worthwhile, and matches any product claims. Again individual trial estimates will be prone to imprecision and they may also reflect the conditions present in that trial. A meta-analysis will provide a more precise estimate for the range of conditions trialed and also provide an opportunity to explore the factors that may influence effect.

Weighting, REML analysis, or log transformation

Statistical analyses such as t-tests and ANOVA have an assumption that the residuals of the model will be normally distributed. If the measured effect is a percentage or a ratio then there can often be a non-normal skewed distribution of trial mean effects and residuals are likely to also show such skew. This can be addressed by using weighting, restricted maximum likelihood (REML) analysis or by log-transforming the data to improve the validity of the statistical analysis.

Cooper (2010) considered that just as statistical analysis of individual trials should be rigorous in calculating statistics correctly, so too should meta-analysis be rigorous, emphasizing that established weighting techniques should be employed where possible. The use of weighting avoids undue importance being placed on trials with high variability and, therefore, a greater chance that the mean effect in that trial has been over- or under-estimated. Generally, the highest effect values will be more variable and weighting tends to lessen their impact on the overall mean product effect. Instead of traditional weighting, the use of REML assessment in the mixed model analysis of full data (i.e. all individual plot results) reduces the impact of highly variable trials.

If individual plot information is not available to assess within-trial variability, the simpler statistical analysis provides some assessment of overall mean effect but the limitations of the analysis should be acknowledged in that there is a chance of trials with very high results having an undue influence on the overall mean estimated effect. In some cases, this can be addressed by log transformation of the trial results.

Objectives of the study

In this study, we aimed to estimate the effects of the three fertilizer-enhancing products using meta-analysis. The validity of manufacturer claims were tested and factors affecting efficacy assessed. A comparison was made with results from analyses by Edmeades and McBride (2011, 2012) of the three products to demonstrate the importance of analysis methodology.

N-Boost

N-Boost (previously named LessN) is a microbial extract product with ingredients including adenine compounds and other microbial bioactives. These compounds are considered by the product manufacturers to induce a cytokinin-like response which may improve nitrogen response by triggering energy release from chloroplasts (energy release from chloroplasts is a known response to cytokinin; Musgrave 1994). The N-Boost system involves the application of 3 L/ha N-Boost mixed with 18.4 kg N/ha sprayed dissolved urea in a total tank volume of around 200 L per hectare on pasture.

Pasture trials with N-Boost have been claimed by Donaghys Ltd (N-Boost manufacturer, Christchurch, New Zealand) to show N response increase such that mean pasture response to the N-Boost system matched the mean response to 36.8 kg N/ha urea (so implying an increase in N response equivalent to an additional 18.4 kg N/ha). The N-Boost system is recommended for use when an N response can be achieved and when soil temperature is at or above 10°C (at soil temperatures below this, pasture growth can be limited). Edmeades and McBride (2012) analyzed N-Boost pasture trials undertaken in New Zealand and reported a statistically significant yield improvement in independently conducted trials of 10% (95% confidence interval – CI – of 6 to 14%, n = 20), and in Donaghys conducted trials of 38% (95% CI 29 – 47%, n = 41) pointing to an apparent difference in the scale of effect though acknowledging that there was also a similar difference in the 36.8 kg N/ha treatment between the manufacturer and the independent trials.

SustaiN Green

SustaiN Green (Ballance Agri-Nutrients) is granulated urea coated with the urease inhibitor (AgrotaiN, N-(n-butyl) thiophosphoric triamide), which has been shown to reduce ammonia volatilization (Saggar et al. 2012). The manufacturer claim is that nitrogen response to urea is increased when there is a risk of ammonia volatilization. Edmeades and McBride (2012) sought to combine the results of all published studies on SustaiN Green on pasture and reported a mean pasture response to SustaiN Green of 4% (95% CI −3 to 11%, n = 16). The combined analysis failed to include published research trials from researchers employed by the manufacturers of SustaiN Green of Blennerhasset et al. (2006), Ramakrishnan et al. (2008) and Quin, Blennerhassett, and Zaman (2005) and by Zaman, Blennerhassett, and Quin (2006; 2008).

AVAIL

AVAIL (Specialty Fertilizer Products – SFP) is a maleic–itaconic copolymer used in conjunction with phosphate fertilizers, with the objective of improving P fertilizer efficiency. The manufacture claim is that yield of crops is increased. The product is claimed to reduce immobilization of phosphate ions by aluminium, calcium, and iron though this has been criticized by Chien et al. (2014), on the basis of the small amount of active ingredient over the volume of soil treated, and since the affinity of phosphate to maleic acid and itaconate is less than the affinity to the competitor ions (although the latter should arguably be a desirable trait for influencing phosphate availability). Greenhouse experiments conducted by Degryse et al. (2013) only found evidence of a phosphorus sparing effect (presumed to be by sequestering phosphorus-binding cations) at an unfeasibly high level of addition of the maleic–itaconic copolymer. It remains possible that there may be different mechanisms of effect and it should also be pointed out that the product is added in fertilizer granules and not distributed over the whole soil volume. Various authors (e.g. Ward 2010) have combined a discussion of calculated lack of significant active ingredient for the claimed task of phosphate protection with an apparent lack of effect of AVAIL in field trials. In the Ward (2010) experiments conducted on both wheat and corn, AVAIL treatments had generally higher mean yields than untreated fertilizer and this dataset is analyzed further in the current paper. Edmeades and McBride (2011, 2012) conducted a combined analysis of 95 AVAIL trials considered “very reliable” (defined in the paper as three or more replicates, statistical analysis available and trial design known) and reported a statistically significant 1.4% (95% CI 0.3–2.5%) effect. This combined analysis was included within the publication of Chien et al. (2014). Other publications have presented some statistically significantly positive effects of AVAIL (e.g. Gordon 2005; Randall and Vetsch 2004; Stark and Hopkins 2013; Wiatrak 2013; and Dunn and Stevens 2008 although only at a P < 0.100 significance level in the last publication).

Materials and methods

Data sources

The current study analyzed all trials that met the stated criteria of Edmeades and McBride (2012). For N-Boost, all trial results were available and analyzed. For AVAIL and SustaiN Green trials, references given by Edmeades and McBride (2011, 2012) were examined and additional searching for trials and reports was also undertaken with the internet (SFP, 2013; Google Web; Google Scholar; State University Annual Reports) and databases (CAB Abstracts and Massey University general database search) searching for keywords of AVAIL and chemistry terms for the active ingredient. Researchers were contacted, if required and possible, to obtain sufficiently detailed research reports.

Trial methodology

N-Boost pasture trials all included a standard product treatment of 3 L/ha N-Boost combined with dissolved urea at 18.4 kg N/ha in a total tank volume of around 200 L/ha termed sprayed N-Boost18.4. That was compared to either solid granular urea or sprayed urea solution at the same N rate or at 36.8 kg N/ha. The individual comparison treatments were named accordingly; solidUrea18.4, solidUrea36.8, sprayedUrea18.4, or sprayedUrea36.8. In most trials, pasture growth was determined by Grassmaster II capacitance probe (end measure minus start measure just prior to application). Capacitance probes work by creating an oscillating electric field around the base of the probe; moisture within pasture herbage surrounding the probe modifies the capacitance of this field and the probe estimates the amount of herbage, based on the change in capacitance (see Serrano et al. 2011 for further explanation). In one trial, probe data was not collected and mower clipping data was compared instead. Data for each trial was for a single grazing rotation (duration 14 to 47 days after application; mean 25 days). All trials were considered in the overall analysis and fertilizer efficiency effect was then calculated with the exclusion of non-N responsive trials and trials conducted against label recommendation soil temperature. Trials were deemed to be non-N responsive if the mean calculated N response for all urea treatments combined was less than 3 kg dry matter/kg N applied. The soil temperature criteria employed was that the 7-day mean soil temperature (9 AM, 10 cm depth) on application day was above 10°C on the farm (or at the closest meteorological station if an on-farm assessment was not carried out).

SustaiN Green pasture trial designs have been described in the original publications. Response ratios (SustaiN Green growth over check divided by urea-only growth over check) were calculated for SustaiN Green data.

AVAIL trials (conducted prior to 2011 to fit with the timeframe analyzed by Edmeades and McBride 2011) were on a wide range of crops and generally included a phosphate fertilizer (usually with N, e.g. as mono ammonium phosphate or diammonium phosphate) applied with and without a coating of AVAIL. A percentage effect (from treatment yield divided by fertilizer-only-yield) was calculated for each AVAIL trial. This study included a database of 134 researcher-conducted AVAIL trials (from 2010 or earlier) that met “very reliable” criteria of Edmeades and McBride (2011, 2012). In the current analysis, trials with less than 1% yield increase for mean treatment result over nil-check (no-fertilizer treatment) if it was present in the trial design, were termed fertilizer non-responsive but were included in the initial analysis. Soil phosphate availability was measured in some of the trials and was included in the analysis as a potential explanatory factor.

Statistical analysis

R (Version 3.2.0, R Core Team, 2014) was used for statistical analysis and figures. Since measures of trial variability were not available for all SustaiN Green and AVAIL trials, confidence intervals were constructed without weighting. SustaiN Green trial means were analyzed after natural log transformation, ln(x) to reduce the effect of a positive skew towards high response ratios and improve normality of residuals from the statistical models. For AVAIL, extremely low and high trial means were present so to improve normality of residuals in the models and also cope with negative and positive numbers, data was transformed with a symmetrical log approach (sign of the mean multiplied by the natural log of the absolute value of the mean with 1 added, sign(x).ln(abs(x+1)).

For N-Boost trials and for the AVAIL dataset of Ward (2010), meta-analysis using generalized mixed model effects (lme function in nlme package of R) procedure utilized individual plot data. Trial data are available from the authors for the analyses of N-Boost, SustaiN Green, and AVAIL, respectively. Individual plot data for Ward (2010) was accessed from the original publication.

Results

Meta-analysis of N-Boost® Trials

Over all 63 trials (including non-N-responsive trials) the mean dry matter (DM) increase for N-Boost (applied with 18.4 kg N/ha dissolved urea) above urea alone at the same N rate was estimated at 237 kg DM/ha (95% CI 216–263, REML analysis) which is equivalent to an additional 12.9 kg DM/kg N applied. The double rate of urea (36.8 kg N/ha) estimated effect was just 9 kg DM/ha greater (95% CI −18.4 to 35.6, not statistically significant P = 0.532). The nitrogen response ratio for the main comparison treatments of all trials combined is presented in Table 1.

Table 1. Meta-analysis results for the ratio of N-Boost system response (N-Boost System – Check yield) compared to urea-only at two rates and in two forms. All trials included.

N-Boost18.4^*	Urea Application Rate
Response^**	18.4 kg N/ha	36.8 kg N/ha
Compared with	(Urea18.4)	(Urea36.8)
Urea only dissolved	2.47 ± 0.29 (n = 55)	0.96 ± 0.06 (n = 51)
Urea only solid	1.77 ± 0.34 (n = 15)	1.04 ± 0.12 (n = 15)

* N-Boost 3 L/ha spray applied with 18.4 kg N/ha dissolved urea spray.

** N response is kg dry matter pasture grown above nil nitrogen check (ratios are that response divided by response to the urea only treatments ± standard error).

Of the three poorest N-Boost responses reported in independent trials, two had similarly poor Urea80 responses, and it is quite possible that there was a limitation in the ability to respond to N-Boost in these pastures simply because site-maximum yield had been reached. The third trial (Reporoa) had flooding from heavy rainfall (200 mm) two days prior to trial commencement, which may have impacted on immediate pasture response to N-Boost; this trial could potentially be considered non-representative (the response was statistically significantly different from other trials P = 0.019).

There were five independently conducted trials with low mean calculated N response (termed non-N-responsive) and a further four trials were conducted at below label soil temperature (see Tables 2 and 3) and these trials all tended to have a lower estimated N-Boost effect (soil temperature effect was statistically significant at P = 0.002). Cumulative distribution functions (CDFs) of N-Boost effect are shown by absolute effect (kg DM/ha) above sprayedUrea18.4 in Figure 1. The results for urea applied at twice the rate were similar to the N-Boost treatment at 18.4 kg N/ha sprayed urea (not statistically significantly different in any individual trial). Results for straight urea at twice the N rate are shown for each corresponding N-Boost18.4 result, providing evidence of the relationship between N-Boost18.4 and Urea36.8 results regardless of trials being independent or not. Independently conducted trials are indicated by open circles on the graph and show a similar trend of N-Boost treatments matching urea alone at twice the N rate.

Figure 1. Cumulative Distribution Function (CDF) of N-Boost effect (N-Boost with 18.4 kg N/ha yield minus 18.4 kg N/ha urea alone sprayed yield) for Donaghys-conducted and independent (indep) trials. Data for 36.8 kg N/ha urea alone (solid, sprayed or mean depending on trial) included for comparison (triangles). Horizontal error bars based on 2 standard errors each side of mean. Sigmoidal curve is a theoretical normal distribution for the N-Boost effect. Vertical dashed line is equivalent to an additional nitrogen response of 12 kg DM/kg N above the nitrogen response for 18.4 kg N/ha urea alone.

Table 2. Trials deemed non-responsive to N with mean N response for all N treatments.

Trial name	Mean N response (kg DM^*/kg N)
Hinds 2007	1.5
Northland 2011	1.3
Tirau 2011	2.1
Walton 2011	2.1

* DM: dry matter of pasture (refers to mean DM grown of all N treatments above the mean nil-nitrogen check growth).

Table 3. Soil temperature records for low soil temperature trials.

Trial name	Start date	Day of application 10 cm depth, 9 AM (°C)	7-day mean at application (°C)
Manawatu sheep	27/8/2007	7.6	9.04
Manawatu dairy	29/8/2007	8.6	9.24
Canterbury 07	05/9/2007	4.4	5.34
Rangiora 07	06/9/2007	2.6	4.69

The REML calculated N-Boost effect was equivalent to an additional 18.0 kg N/ha (95% CI 16.9–19.1) for the 39 N-responsive trials in which Urea36.8 response exceeded 1.7 times Urea18.4 response. For the nine trials where 36.8 kg N/ha urea resulted in less than 1.7 times the 18.4 kg N/ha treatments (excluding below-label-soil-temperature and one flooding trial), mean N-Boost18.4 response was 1.05 times Urea36.8 response.

Meta-analysis of sustaiN green trials

Summarized results are presented in Table 4. The distribution of response ratio results had some extreme values (common for ratio calculations) with some of these being negative and others positive. The statistical results presented were, therefore, transformed using a symmetrical log transformation to reduce the non-normal distribution of residuals.

Table 4. Mean SustaiN Green effect compared to plain urea as affected by trial selection and ammonia volatilization risk.

Data selection	Trial number	Reported volatilization risk	Response ratio	Extra kg DM/kg N
Edmeades and McBride (2012) Data	16	All included	1.10	0.9
Edmeades and McBride (2012) Data	2	Low excluded	1.56	4.7
Full Selection^*	27	All included	1.20	3.3
Full Selection^*	13	Low excluded	1.50	6.2

* Including published studies adhering to stated criteria of Edmeades and MacBride (2012) but excluded from their analysis.

The estimated mean increased response ratio (nitrogen response with SustaiN Green divided by the response with standard urea) was 1.2 (95% CI 1.11 – 1.34, n = 28, symmetrical log transformation) meaning that there were around 1.2 times the nitrogen response from SustaiN Green than from urea alone. The 13 trials conducted by Stafford, Catto, and Morton (2008), which were conducted in periods expressly intended to avoid ammonia volatilization risk, had a mean of only 1.02. Martin et al. (2008) trials resulted in response ratios of 1.25 and 1.81 for two trials, but just 1.01 for the third trial, which was reported to have limited volatilization potential due to rainfall.

The effect of reported volatilization risk was statistically significant over all combined trials (P < 0.001). When trials with a reported low risk of volatilization were excluded, the estimate for mean response ratio increased to 1.5 (95% CI 1.34 – 1.67, n = 13) which translates to a 50% increase in nitrogen response.

Mean response difference (kg DM/kg N for SustaiN Green minus kg DM/kg N for urea) over all trials was calculated as 3.3 kg DM/kg N (95% CI 1.80–4.89, n = 28). For trials with a perceived risk of ammonia volatilization, the mean response difference was 6.2 kg DM/kg N (95% CI 4.71–8.42, n = 13); this was statistically significantly different from trials defined as of low risk of ammonia volatilization (P < 0.001). Residuals from this modeling were reasonable so response differences transformation was not required.

The effect of trial selection and reported risk of ammonia volatilization on response ratio is presented in Figure 2 showing a clear difference between trials with reported low risk of volatilization compared to others. The trial results show a bimodal distribution corresponding to the low volatilization risk and the other trials as emphasized by the difference between the distribution of results and the theoretical normal distribution curve either side of the median in Figure 2.

Figure 2. CDF of SustaiN Green response ratio (kg DM/kg N for SustaiN Green urea divided by kg DM/kg N for untreated urea) identifying trials conducted in conditions with reported low risk of ammonia volatilization. The sigmoidal curve is a theoretical normal distribution for the ratios.

Meta-analysis of AVAIL trials

The skewed distribution of percentage yield effects is shown in Figure 3. Summarized results are presented in Table 5 and Figure 4 shows AVAIL effect measured for trials according to soil P test status and fertilizer responsiveness. The mean AVAIL effect was a 1.6% yield increase over the same rate of fertilizer alone (95% CI 1.04–2.41, n = 134, P < 0.001, symmetrical log transformation). Significant influences on AVAIL effect were detected for fertilizer responsiveness of trials (P < 0.001) and high soil P test results (P = 0.048). Selecting apparent fertilizer responsive trials (>1% mean response of fertilizer treatments over check) resulted in an increased estimation for AVAIL effect of 3.7% (95% CI 2.35–5.68, n = 59). Excluding known high-soil-P-test trials, the mean AVAIL effect was estimated at 1.9% (95% CI 1.25–2.82, n = 119). The combination of selecting for fertilizer responsive trials and excluding known high-soil-P-test trials resulted in a mean AVAIL effect of 4.1% (95% CI 2.56–6.22, n = 54).

Figure 3. AVAIL effect (percent increase in yield over fertilizer without AVAIL) in 134 independent crop trials defined as ‘very reliable’. The kernel density plot (probability distribution function in bold) shows smoothed proportion of data at point of the x axis, provides a visual test for normal distribution and is supplemented with a rug plot (small vertical lines at the base) to show individual trial means. A theoretical normal distribution is shown in the dotted bell curve based on the overall mean (vertical dashed line) and standard deviation of the trial means. A boxplot is added to indicate the overall median response (thick line near the center of the rectangle) and emphasize potential outlier results and skew.

Table 5. Contribution of trial selection, fertilizer responsiveness, and soil P test level to measured AVAIL effect.

Data selection	No. of trials	Fertilizer responsive	Soil P test	Percent over fertilizer only (%)
Current	134	All	All	1.6
Current	59	Exclude <1% response^*	All	3.7
Current	119	All	Not high^**	1.9
Current	54	Exclude <1% response^*	Not high^**	4.1

* Trials without nil-check (and thus with fertilizer response unknown) remained included.

** Trials without soil P test data remain included in all comparisons.

In meta-analyses of the AVAIL trials of Ward (2010), a statistically significant effect of AVAIL on percentage wheat yield was detected with a mean increase over fertilizer alone of 4.9% (standard error or SE of 1.83, P = 0.010) and an advantage in absolute yield of 79 kg/ha (SE of 33.3, P = 0.022). The effect estimated for corn was not statistically significant for mean percentage 2.5% (SE 1.82, P = 0.175) nor for mean effect on absolute yield of 224 kg/ha (SE 198.8, P = 0.262). There was no evidence of fertilizer rate influencing (i.e. interacting with) the AVAIL effect for wheat yield percentage (P = 0.352) and absolute effect (P = 0.512) nor for corn yield percentage (P = 0.405) and absolute effect (P = 0.219).

Discussion

N-Boost

The manufacturer claim that N-Boost System with 18.4 kg N/ha sprayed urea produced an equivalent pasture response to that achieved with 36.8 kg N/ha of urea alone is supported by the analysis of combined trials. In several trials, the response to the 36.8 kg N/ha were not double that of the 18.4 kg N/ha rate of urea alone. This may be due to other factors such as mineral availability or simply reaching a trial site maximum growth rate. Therefore, the N-Boost system matching the 36.8 kg N/ha rate was not always equivalent to a doubling of nitrogen response so wording of the N-Boost claim is important.

The responses could also have been measured as relative percentage increases over check growth or percentage over the 18.4 kg N/ha of urea alone. The latter calculation was used in Edmeades and McBride (2012) and they reported widely different results between Donaghys conducted and independent trials. As reported in Edmeades and McBride, however, the mean response for urea alone at 36.8 kg N/ha was the same as that for the N-Boost treatment with 18.4 kg N/ha sprayed urea in both Donaghys and independently conducted trials. Thus, the difference in percentage responses did not contradict the manufacturers claim. The percentage calculation was also confounded by a lower mean response for all treatments (including nitrogen alone) in the independent trials. Furthermore, a large proportion of the independently conducted trials were non-N-responsive and, therefore, could not be used to reliably test differences in nitrogen response effect of treatments.

The results demonstrate the importance of judging product claims with the appropriate measurement parameter (in this case comparison with the double rate of nitrogen). The calculation of absolute gain in dry matter was shown to be a useful calculation to provide an economic return context for the trial results and tests the substitution effect (the equivalent amount of urea nitrogen required to achieve the same effect). It is also crucial to acknowledge trials where there was an apparently limited response to urea over all treatments.

SustaiN green

The results were consistent with manufacturer recommendations of utilizing the product when there is a perceived risk of ammonia volatilization. While it remains possible that there are direct or indirect benefits of the product over and above avoiding volatilization losses, such mechanisms may be synergistic with ammonia volatilization avoidance (e.g. nitrogen response can sometimes be higher for a greater level of N application due to a threshold effect of overcoming soil immobilization of nitrogen) or may be coincident, e.g. related to more active pasture growth at time of application or may be related to drier soil conditions. The results again show the importance of interpreting results in terms of the conditions recommended for use and also the importance of including all known research trials in an initial assessment. Several of the individual trials had shown a statistically significant effect and the overall product effect was found to be statistically significant (in Edmeades and McBride, 2012 and in the current analysis), further highlighting the importance of paying attention to likely factors that may be influencing effect.

Of the 16 trials assessed by Edmeades and McBride (2012), 13 were from a series reported by Stafford, Catto, and Morton (2008), on sites, times, or conditions expressly chosen as being less conducive to volatilization. In the remaining three (independent) trials, reported by Martin et al. (2008), one trial had pre- and post-application rainfall considered by the original authors to have greatly reduced the potential for volatilization. The remaining two trials demonstrated statistically significant yield increases for SustaiN Green over uncoated urea. These results were consistent in response ratio effect with the manufacturer published trial results that had been missed from the Edmeades and McBride (2012) analysis. Overall results were, therefore, consistent with the way the product is marketed for use where volatilization risk is judged higher.

AVAIL

There was strong evidence of an improved crop yield with AVAIL in the meta-analysis of all trials (P < 0.001). Some trial means were highly positive and some highly negative (kurtosis of distribution). Cube root transformation or symmetrical log transformation are suitable for addressing such positive and negative extremes (Cox 2011) but cube root transformation did not sufficiently improve normality of residuals whereas symmetrical log transformation improved the residual distribution. Without transformation, the mean of all trials was 4.2% but the mean of symmetrical log transformed data (without the same influence from highly positive trials) was 1.4% (95% CI 1.04–2.41) over fertilizer alone. This was a similar mean to the 1.4% estimated by Edmeades and McBride (2011) for their untransformed analysis of “very reliable” trials.

In the current study, significant influences on the calculated mean were identified from both fertilizer responsiveness and soil P test status. This provides further support to the efficacy of AVAIL. When trials with little apparent fertilizer response or a high soil P test were excluded, the mean AVAIL percentage effect was an increase of 4.1% over fertilizer alone (95% CI 2.56–6.22).

Although the same time frame was covered by Edmeades and McBride (2011, 2012) and the current study and the same criteria was used for trial selection, there was an obvious difference in the set of trials analyzed. For instance, Edmeades and McBride (2011) reported only eight trials in total (including those considered “unreliable”) up to 2005, even though the database they cited included 18 trials fulfilling the “very reliable” criteria up to 2005 (all conducted by a university or governmental researchers). Because of the lack of clarity over which trials from some publications were actually included by Edmeades and McBride (2011, 2012), it has not been possible to provide a straight comparison.

Edmeades and McBride (2011) reported mean values of 2.1% (± 2.1, n = 55) and 0.9% (±1.2, n = 31) for fertilizer responsive (statistically significant fertilizer response detected) and those trials termed “unresponsive,” respectively. These distributions were described as almost identical and that it was “strong evidence” of AVAIL being ineffective. For those distributions, a two-tailed t-test is not statistically significant (P-value approximately 0.181). Thus, there was insufficient evidence of a difference but an important distinction in statistics is that such a finding is not strong evidence of a lack of effect particularly when the direction of the difference between responsive and unresponsive trial results was in fact in a consistent direction to what might be expected for an effective product. Furthermore, the current study showed that fertilizer responsiveness actually had significant influence on the effect of AVAIL. The results of a series of AVAIL potato trials (Stark and Hopkins 2013) also demonstrated that response to AVAIL was lower in experiments with high soil P test levels.

A meta-analysis performed in the current study on the Ward (2010) dataset found statistical significance for AVAIL effect in wheat trials (P = 0.022). The mean absolute effects were financially meaningful at US$21/ha (based on the mean advantage for each comparison with fertilizer alone of 79 kg/ha and US$260/ton wheat) for a US$7/ha mean AVAIL cost. The mean absolute effect estimated for corn was 224 kg/ha which would be equivalent to US$35/ha for corn (based on a corn price of US$157/ton) but this estimate should be treated with caution since the effect was not statistically significant (P = 0.262).

The financial calculations highlight that seemingly low percentage effects could still be financially meaningful. Contrary to Edmeades and McBride (2012) suggestion that effects below 5% were likely of little consequence to practical farmers, effects as low as 1% or 2% can, in fact, be financially meaningful. Financial effects are related to absolute effects rather than percentages. There are, furthermore, several methods of calculating percentages with the choice of method affecting magnitude. Percentages can use the nil-check (no-fertilizer treatment) as the denominator or alternatively, as in Edmeades and McBride (2011, 2012), the fertilizer without additive yield as the denominator thus yielding a smaller magnitude of percentage effect. A further option is to calculate percentage as the product result minus check divided by the fertilizer alone result minus check (providing a percentage based on the response ratio) which would usually yield a larger percentage value. Thus, the percentage calculation method needs to be clearly outlined and taken into consideration in interpretation. Discussion of financial implications should still, however, be based on measurements such as absolute yield effects that are directly related to financial effect.

Establishing statistically significant and financially meaningful effects of a product is not proof for statements of product mechanism. Chien et al. (2014) treatment of AVAIL consisted of two parts, one disputing from a soil chemistry perspective that AVAIL could effectively reduce phosphate immobilization and the second part reporting a meta-analysis of field evidence for efficacy (based largely on Edmeades and McBride 2011, 2012). Given that the current study establishes sufficient evidence of greater efficacy than that calculated by Edmeades and McBride, further investigation is warranted on the mechanisms of action and conditions affecting AVAIL efficacy.

General

As shown for N-Boost data and the AVAIL data, the percentage calculation can be a poor indicator of absolute yield effect and economic return for fertilizer efficiency products since experiments can vary widely related to the denominator value with little relationship to the absolute effect.

Fertilizer efficiency product assessment should be carried out in conditions recommended by the product marketers and reflective of standard growing conditions. To demonstrate differences in efficiency between different P fertilizers (and, therefore, P-efficiency additives), for example, it is established practice to deliberately select sites with low-medium soil P levels that are more likely to be P-responsive. Indeed, when Edmeades and McBride (2012) provided an example of a reliable product response on a CDF graph, data were taken from a variety of P-response trials all conducted on P deficient sites.

The basic CDF graph approach is open to the misleading presentation of trial result sets and does not easily provide an understanding of the distribution of data. The CDF distribution can provide enhanced information with the inclusion of individual trial error bars, identification of groupings within one CDF, and plotting of additional treatments around the CDF (e.g. Figure 1). Other graphical methods such as Q–Q (quantile–quantile) plots, boxplots, histograms, and kernel density plots facilitate understanding of the distributions of results.

Plotting two comparison distributions on a CDF graph can visually exaggerate differences making overlap of range unclear. There may also be a different range of trial conditions for one distribution. Separate curves for treatments on a CDF graph can furthermore obscure comparative results within individual trials. Fair visual comparison of trial conditions can be achieved by plotting them with different symbols on one CDF.

While there is merit in combined analysis to overcome imprecision and variable conditions of individual trials, the analysis should use appropriate statistical methods, check assumptions, and pay attention to likely causes of variability in response. The analyses by Edmeades and McBride (2011, 2012) were not suited to the data distributions, may have misrepresented results through trial selection choice tending to include less favorable results (for SustaiN Green and AVAIL), and failed to identify key factors related to variation in trial results (estimated product effects for all three products, were influenced by fertilizer responsiveness and/or environmental conditions). Trial results for the products were not independently identically distributed (due to common factors linking some trials) and had strongly skewed distributions. Given that trial results were affected by factors that would indeed be expected to influence fertilizer efficiency products, the methodology of Edmeades and McBride (2011, 2012) that was also relied on by Chien et al. (2014) requires reassessment.

Formal meta-analysis should often assign weights according to individual trial variability (Hedges and Olkin 1985). Edmeades and McBride (2012) argued against weighting stating that coefficient of variation (CV) levels will often be similar in pasture trials. The CV levels, however, can vary considerably between trials due to pasture growth and plot size and other factors, and for percentage data can be substantially greater than for absolute yields. CV would itself be unsuitable as a weighting since CV does not reflect replication level (an important aspect of trial reliability).

Log transformation is the common transformation of choice for non-binomial percentage data which often exhibit positively skewed distributions and tend to have greater variation around higher means. It is commonly known that estimation of the mean and confidence interval is highly inaccurate for heavily skewed data (see e.g. Wilcox 2003). Maindonald and Cox (1984) stated that anyone working with biological measurements should be aware of the frequent need to use data transformation to better work with such data. In some cases modern analysis methods reduce the effect of skew, e.g. analysis of the N-Boost data using the mixed model (with REML) method in Jenkins and Randhawa (2013). In the current paper, the choice of absolute yield effect of N-Boost showed no apparent skew issue.

Edmeades and McBride (2012) incorrectly described the distributions for some products as “approximately normally distributed.” This may be due to the assessment of distributions being difficult with their chosen method of CDF graphs without the plotting of a comparison normal distribution. Maindonald and Braun (2010) recommended boxplots (e.g. as in Figure 3), as they allow rapid data comprehension to the trained eye. Quantile–quantile plots are also superior to CDF for testing departures from normality. The shape of distributions and testing for multimodal distribution can also be achieved visually with histograms but kernel density plots (also presented in Figure 3) are considered superior and not prone to shape alteration due to arbitrary histogram bin width selection (Faraway 2006).

Multiple boxplots or identification of points on a CDF (e.g. Figures 1, 2, and 4 for the latter) can display categorical variables (e.g. volatilization risk, soil test levels, fertilizer responsiveness, etc.) allowing visual exploration of how those variables impact on product effect.

Figure 4. CDF of AVAIL effect (percent increase in yield over fertilizer without AVAIL), identifying trials with poor fertilizer response or high soil phosphorus test level. The sigmoidal solid line is the theoretical normal distribution.

Although combined analysis overcomes the issue of imprecision of individual trials, it makes sense to have sufficient replicates in each trial to have a reasonable chance of detecting a statistically significant effect. This provides more reliable information about performance under different specific field conditions.

Conclusions

Three fertilizer enhancer products were analyzed for their effectiveness. The results demonstrated the importance of considering all trials and taking into consideration the factors that may affect efficacy. Parameters were chosen at tested results against the manufacturer claims (thus absolute kg DM/ha effect of N-Boost, ratio of SustaiN Green urea effect over untreated urea effect and the percent increase of AVAIL-treated fertilizer over untreated fertilizer crop yield) and the importance of appropriate statistical analysis was highlighted in the case of non-normal distribution of results in the AVAIL and SustaiN Green combined trial data.

N-Boost biostimulant sprayed with dissolved urea increased pasture nitrogen response by an equivalent of 18.0 kg N/ha compared to urea alone in nitrogen-responsive and label-conditions trials. This was similar to a manufacturer claim that the product increases nitrogen response by an equivalent of 18.4 kg N/ha. For SustaiN Green urea, pasture nitrogen response was 50% higher than untreated urea if low-ammonia-volatilization-risk trials were excluded. This supported a manufacturer claim of increased nitrogen response when there is a risk of ammonia volatilization. AVAIL fertilizer increased crop yields by 4.1% over untreated fertilizer in fertilizer-responsive and non-high-soil-phosphate trials. This supported a manufacturer claim of greater yield potential for AVAIL. Identifying the significant factors affecting efficacy can assist in farmer decision making for utilising these products.

Acknowledgments

Donaghys Ltd funded the N-Boost component of this paper. Jenkins and Randhawa were involved in N-Boost research and development and Jenkins continues to be a consultant to Donaghys. The authors acknowledge the support of farmers in conducting the on-farm N-Boost trials. We express our gratitude to Bert Quin for discussion of an earlier manuscript and access to trial results.

Additional information

Funding

Donaghys Ltd, 001.