ABSTRACT
This systematic review and meta-analysis rigorously examines the relationship between glyphosate exposure and risk of lymphohematopoietic cancer (LHC) including NHL, Hodgkin lymphoma (HL), multiple myeloma (MM), and leukemia. Meta-relative risks (meta-RRs) were positive and marginally statistically significant for the association between any versus no use of glyphosate and risk of NHL (meta-RR = 1.3, 95% confidence interval (CI) = 1.0–1.6, based on six studies) and MM (meta-RR = 1.4, 95% CI = 1.0–1.9; four studies). Associations were statistically null for HL (meta-RR = 1.1, 95% CI = 0.7–1.6; two studies), leukemia (meta-RR = 1.0, 95% CI = 0.6–1.5; three studies), and NHL subtypes except B-cell lymphoma (two studies each). Bias and confounding may account for observed associations. Meta-analysis is constrained by few studies and a crude exposure metric, while the overall body of literature is methodologically limited and findings are not strong or consistent. Thus, a causal relationship has not been established between glyphosate exposure and risk of any type of LHC.
Introduction
The broad-spectrum herbicide glyphosate (N-(phosphonomethyl)glycine), as a constituent of more than 750 products for agricultural, forestry, urban, and residential applications, is the most commonly used herbicide in the world. Therefore, understanding its potential human carcinogenicity has major implications for public health and risk assessment.
In 2014, the German Federal Institute for Risk Assessment (BfR), on behalf of the European Union, reviewed all toxicological studies of glyphosate in laboratory animals, as well as over 30 epidemiological studies in humans, and concluded that “the available data do not show carcinogenic or mutagenic properties of glyphosate” and “there is no validated or significant relationship between exposure to glyphosate and an increased risk of non-Hodgkin lymphoma or other types of cancer.”Citation[1,2] This conclusion was consistent with those previously reached by the United States Environmental Protection Agency (U.S. EPA) and the Joint Meeting on Pesticide Residues (JMPR), sponsored by the Food and Agriculture Organization of the United Nations and the World Health Organization (WHO), which concluded that glyphosate was unlikely to be carcinogenic to humans.Citation[3–5]
By contrast, the International Agency for Research on Cancer (IARC) in 2015 classified glyphosate as “probably carcinogenic to humans” (Group 2A). In arriving at this classification, IARC characterized evidence of carcinogenicity in humans as “limited,” based on the data available for non-Hodgkin lymphoma (NHL).Citation[6] IARC considered the evidence of carcinogenicity in experimental animals as “sufficient.” The latter determination was based on the occurrence of renal tubule carcinoma, hemangiosarcoma, and pancreatic islet-cell adenoma in rodents, as well as mechanistic evidence.
To incorporate the IARC classification into the European Union review of glyphosate, BfR was commissioned by the German government and the European Food Safety Authority (EFSA) to review the IARC assessment.Citation[7] In its subsequent revised assessment report, BfR reached the conclusion that “no carcinogenic risk to humans is to be expected from glyphosate if it is used in the proper manner for the intended purpose.”Citation[8] This assessment was supported by all European Union member states except one (Sweden) and by EFSA.Citation[9] The WHO also has established an expert taskforce to re-evaluate the available data on glyphosate and report its findings to JMPR.Citation[10]
In summarizing the epidemiological evidence, IARC stated that “case-control studies in the USA, Canada, and Sweden reported increased risks for NHL associated with exposure to glyphosate. The increased risk persisted in the studies that adjusted for exposure to other pesticides. The [Agricultural Health Study] cohort did not show an excess of NHL. The Working Group noted that there were excesses reported for multiple myeloma in three studies; however, they did not weight this evidence as strongly as that of NHL because of the possibility that chance could not be excluded; none of the risk estimates were statistically significant nor were they adjusted for other pesticide exposures.”Citation[6] A recent meta-analysis conducted by investigators from IARCCitation[11] found a statistically significant positive association between glyphosate use and NHL risk (meta-relative risk [RR] = 1.5, 95% confidence interval [CI] = 1.1–2.0), based on six studies.Citation[12–17] The same meta-analysis also found a significant positive association between glyphosate use and risk of B-cell NHL, based on two studies.Citation[14,18]
Although Schinasi and LeonCitation[11] stated that in their meta-analysis, “[i]n an effort to use the most unbiased estimate, [they] extracted the most adjusted effect estimate,” two or arguably three of the RR estimates that they selected for inclusion were not the most highly adjusted estimates reported by the original authors.Citation[13–15] Instead, in a personal communication (11 August 2015), Dr. Schinasi indicated that other estimates were selected based on considerations of consistency of estimates across meta-analyses of other pesticides, secondary analyses, and statistical modeling approach.
Meta-analyses are not intended to identify, validate, or dispute causal relationships. Although they can be useful in providing a summary measure of association and identifying heterogeneity among research results, they can obscure important differences in methods and results among studies that can be more thoroughly evaluated in a detailed qualitative review. Schinasi and LeonCitation[11] did not assess study quality and did not specifically address the potential impact of study limitations on the findings for glyphosate, nor did they discuss whether the apparent association between glyphosate and NHL risk is likely to be causal. On the other hand, Mink et al.Citation[19] conducted a qualitative systematic review, without a meta-analysis, of epidemiologic studies of glyphosate and various cancers, including NHL. Taking into account potential sources of error, including selection bias, confounding, and especially exposure misclassification, the authors concluded that they “found no consistent pattern of positive associations indicating a causal relationship between total cancer (in adults or children) or any site-specific cancer and exposure to glyphosate.”
Given the conflicting findings surrounding this issue, we conducted this systematic review and meta-analysis to examine more rigorously the relationship between exposure to glyphosate and risk of NHL, as well as major histopathological subtypes of NHL, in human epidemiologic studies. Because NHL is often considered alongside other lymphohematopoietic cancers (LHC), whose ever-changing classification systems now characterize some leukemias and multiple myeloma (MM) as NHL subtypes,Citation[20] we also included Hodgkin lymphoma (HL), MM, and leukemia in this review. Despite the limitations of quantitative meta-analysis for observational epidemiology,Citation[21,22] we conducted a meta-analysis largely to determine the impact of using RR estimates not used in the meta-analysis by Schinasi and Leon.Citation[11] In addition, we conducted a qualitative evaluation of potential for error and bias. Thus, this article goes beyond previous work by examining all types of LHC, conducting a new meta-analysis, providing a detailed evaluation of study quality and potential for bias, and synthesizing the overall epidemiologic evidence for a causal association between glyphosate and LHC risk.
Methods
Literature search
Sources eligible for inclusion in the meta-analysis were original articles describing epidemiological studies that provided numeric point estimates of the RR (i.e., odds ratio, rate ratio, or prevalence ratio) of LHC, including NHL, HL, MM, leukemia, and any subtypes of these disease entities, associated with individual-level glyphosate exposure, along with corresponding interval estimates (e.g., 95% confidence intervals [CI]) or sufficient raw data to calculate RRs and CIs. Reviews, commentaries, letters to the editor without original data, and non-human studies were excluded, as were articles that did not report quantitative measures of association between glyphosate exposure (e.g., those assessing broadly defined categories of pesticides or herbicides) and risk of LHC (e.g., those assessing other cancers or all malignancies combined).
To identify all potentially relevant articles, we searched MEDLINE via PubMed (Supplementary methods), with additional targeted searches in Web of Science and Google Scholar, along with a review of the bibliographies of recent review articles. Based on a review of titles and abstracts to exclude articles without pertinent information, followed by a review of the full text of relevant articles, 19 articles (as well as one letter to the editorCitation[23] that contained additional results from a study described in another one of the included articles,Citation[24] and one abstractCitation[25] that preceded a full-length articleCitation[26]) were ultimately deemed eligible for inclusion (Appendix ). Two authors independently reviewed and agreed upon the list of eligible articles.
Of the 19 articles reporting on the association between glyphosate and risk of specific forms of LHC, 12 pertained to NHL or its subtypes (including hairy-cell leukemia, which is a subtype of B-cell NHL),Citation[12–18,24,27–30] 2 pertained to HL,Citation[17,31] 6 pertained to MM,Citation[12,17,26,32–34] and 3 pertained to leukemia.Citation[12,35,36]
Evaluation of study characteristics and quality
From each eligible study, we extracted the following information: first author, publication year, study location, study design, study years, source population, number of subjects, proportion of proxy respondents, exposure assessment method, outcome assessment method, confounders adjusted, number of subjects in each exposure category, and RR estimates with CIs.
In addition to summarizing study characteristics, we qualitatively evaluated the methodological quality of each study in terms of its potential for selection bias, information bias/exposure misclassification, confounding, reporting bias, and other issues affecting validity. Potential for bias was evaluated based on subject identification strategy, participation rates, investigator blinding, assessment methods for exposures, outcomes, and potential confounders, statistical approach, reporting of results, and other considerations.Citation[37–39]
Selection of data for meta-analysis
From each publication, we selected an RR point estimate for inclusion in the meta-analysis based on a set of rules specified a priori. First, if unadjusted and adjusted RRs were reported in a publication or across multiple publications from the same study population, the most fully adjusted RR was selected for inclusion. The most fully adjusted RR was defined as the RR estimate that took into consideration, by restriction or statistical adjustment, the most covariates that appeared to be confounders. The rationale for choosing the most fully adjusted RR was based on the assumption that the adjusted covariates were found by the authors to act as confounders by altering the estimate of association (either directly or by acting as a surrogate for another, unmeasured confounder); however, some authors did not explain how confounders were selected, so this assumption may not hold for all studies. If an adjusted RR was not reported, the unadjusted (crude) RR was included as reported by the authors or as calculated from available raw data. Second, if multiple eligible publications were derived from the same study population, the RR from the most recent publication was selected for inclusion unless it was based on a subset of the overall eligible study population, in which case the RR based on the most complete study population was included. Third, subject to the first two rules, the RR for dichotomous exposure with the largest number of exposed cases was selected for inclusion in the meta-analysis. In a few instances where another RR from a given study nearly met these inclusion criteria but was superseded by a more fully adjusted, more recent, or more robust RR, the alternative RR was considered in secondary analyses.
RRs for multiple categories of exposure also were extracted to enable qualitative evaluation of exposure-response trends (based on the assumption, discussed later, that studies were able to distinguish among exposure levels). However, because no two studies used the same set of three or more categories to classify glyphosate exposure, these estimates could not be combined in meta-analysis.
Statistical approach
For associations with at least two independent RR estimates from different study populations, we estimated both fixed-effects and random-effects meta-RRs with 95% CIs. We used comparison of meta-RR estimates from fixed-effects and random-effects models as one approach to the evaluation of the impact of between-study heterogeneity on the meta-RRs. As a quantitative measure of between-study heterogeneity, we calculated I2, which represents the percentage of between-study variance in RRs that is attributable to study heterogeneity (as opposed to chance).Citation[40] We also tested for statistically significant between-study heterogeneity based on Cochran's Q statistic,Citation[41] although this test has low power to detect modest heterogeneity across a limited number of studies.Citation[42]
In the absence of statistically significant heterogeneity, the presence of at least one statistically significant association, I2 < 50%, and at least four contributing studies, we evaluated evidence of publication bias (i.e., non-random selection of studies for publication, with a tendency toward submission and publication of studies that report larger, statistically significant associationsCitation[43]) by using the linear regression approach of Egger et al.,Citation[44] which measures the degree of funnel plot asymmetry. We also estimated meta-RRs corrected for publication bias by imputing results for missing studies using the trim-and-fill procedure developed by Duval and Tweedie,Citation[45] which iteratively trims asymmetric studies from the overbalanced side of a funnel plot to locate the unbiased effect, and then fills the plot by re-inserting the trimmed studies on the original side of the mean effect, along with their imputed counterparts on the opposite side. Again, we used these approaches with the understanding that they have limited power to detect publication bias based on few studies.Citation[42]
The meta-analysis was conducted using Comprehensive Meta-Analysis Software (Biostat, Inc., Englewood, NJ, USA). All calculated meta-RRs and 95% CIs were confirmed using Episheet (www.krothman.org/episheet.xls).
Sensitivity analysis
To evaluate the robustness of results to various potential sources of heterogeneity, we planned a priori to conduct a sensitivity analysis with stratification of studies by study design (case-control vs. cohort), source of controls (population-based vs. hospital-based), gender (males only vs. males and females), geographic region (North America vs. Europe), and time period of cancer diagnosis (1980s, 1990s, or 2000s, with studies contributing to a given stratum if any part of the case diagnosis period was in a given decade).
Overall evaluation
To guide a qualitative assessment of the combined epidemiologic evidence for a causal relationship between glyphosate exposure and risk of LHC, we used Sir Austin Bradford Hill's “viewpoints” as a general framework.Citation[46] Because this review is restricted to the epidemiologic literature, our consideration of the biological plausibility of the association and the coherence of the human, animal, and mechanistic evidence was limited.
Results
Study characteristics and overlap
Studies of NHL and subtypes
Twelve studies from seven independent study populations, including eleven case-control studies and one prospective cohort study, evaluated the relationship between glyphosate use and risk of NHL and/or its histopathological subtypes.Citation[12–18,24,27–30] Characteristics of these studies are summarized in . All of the studies considered glyphosate use in agricultural operations or settings, and most evaluated overall NHL as an outcome. The exceptions were Cocco et al.,Citation[18] which analyzed B-cell lymphoma and other NHL subtypes, but not overall NHL, and Nordstrom et al.,Citation[30] which included only hairy-cell leukemia. Eriksson et al.Citation[14] presented results for B-cell lymphoma and other NHL subtypes, as well as for overall NHL, while Orsi et al.Citation[17] included results for overall NHL and several specific NHL subtypes.
Table 1. Design characteristics of studies of glyphosate exposure and risk of lymphohematopoietic cancer (LHC), including non-Hodgkin lymphoma (NHL), NHL subtypes, Hodgkin lymphoma (HL), multiple myeloma (MM), and leukemia.
Table 1. Continued (additional columns).
De Roos et al.Citation[13] combined data from Cantor et al.Citation[24] with data from two other studies that did not independently report associations between glyphosate use and NHL risk;Citation[47,48] therefore, we did not further consider Cantor et al.Citation[24] as a separate study. Lee et al.Citation[29] was based on Cantor et al.Citation[24] and Hoar Zahm et al.,Citation[48] but not Hoar et al.,Citation[47] and stratified results by asthma status (with no apparent interaction between glyphosate exposure and asthma); therefore, results from De Roos et al.Citation[13] took precedence in our analysis over those from Lee et al.Citation[29] The study by Hardell et al.Citation[15] pooled data from two other studies that reported on glyphosate use and NHL risk.Citation[27,30] Consequently, the latter two studies were not considered further with respect to NHL, although Nordstrom et al.Citation[30] was evaluated separately with respect to hairy-cell leukemia. Based on the same study population as McDuffie et al.Citation[16] (except for four fewer cases excluded after pathology review), Hohenadel et al.Citation[28] reported associations with use of glyphosate with or without malathion, but not glyphosate overall; therefore, the results from McDuffie et al.Citation[16] were prioritized in our analysis.
The seven independent studies ranged markedly in size with respect to the number of NHL cases classified as exposed to glyphosate (based on reported use): Cocco et al.,Citation[18] 4 B-cell lymphoma cases exposed; Hardell et al.,Citation[15] 8 exposed; Orsi et al.,Citation[17] 12 exposed; Eriksson et al.,Citation[14] 29 exposed; De Roos et al.,Citation[13] 36 exposed; McDuffie et al.,Citation[16] 51 exposed; De Roos et al.,Citation[12] 71 exposed in the total eligible cohort. Four studies were based in EuropeCitation[14,15,17,18] and three in North AmericaCitation[12,13,16] (). Four of the case-control studies were population-based,Citation[13–16] one was hospital-based,Citation[17] and one included a mixture of population-based and hospital-based cases and controls.Citation[18] Four studies were restricted to males,Citation[13,15–17] while the rest included males and females. Two studies conducted at least some case ascertainment during the 1980s,Citation[13,15] five during the 1990s,Citation[12,14–16,18] and four during the 2000sCitation[12,14,17,18] (categories are overlapping). For reference, glyphosate entered the U.S. and European commercial markets in 1974.Citation[49]
Studies of HL
Two case-control studies estimated the OR between glyphosate use and risk of HL.Citation[17,31] Characteristics of these studies are summarized in . The study by Karunanayake et al.Citation[31] used the same methods and source population as McDuffie et al.,Citation[16] but focused on HL rather than NHL.
As described in the section on NHL studies, Orsi et al.Citation[17] was a hospital-based case-control study set in Europe (France), restricted to males, with case ascertainment in the 2000s, participation rates > 90%, and no proxy respondents. This study classified six HL cases as exposed to glyphosate. Karunanayake et al.Citation[31] was a population-based case-control study set in North America (Canada), restricted to males, with case ascertainment in the 1990s, participation rates of 68% for cases and 48% for controls, and an unspecified proportion of proxy respondents. In this study, 38 HL cases were classified as glyphosate-exposed.
Studies of MM
Six studies from four independent study populations, including four case-control studies and two prospective cohort studies, evaluated the association between glyphosate use and risk of MM.Citation[12,17,26,32–34] These studies are described in . A cross-sectional analysis within a subset of the Agricultural Health Study Cohort examined the association between glyphosate use and risk of monoclonal gammopathy of unknown significance (MGUS), an MM precursor;Citation[50] this study was not included in the present review.
The studies by De Roos et al.Citation[12] and SorahanCitation[26] were based on virtually identical datasets from the Agricultural Health Study cohort (except that the dataset used by Sorahan was stripped of data on race, state of residence, and applicator type due to privacy concerns; these differences should not have affected the results substantively). Because the SorahanCitation[26] study included all eligible cohort members, whereas the De Roos et al.Citation[12] study was based on a restricted subset of the cohort with complete data,Citation[51] the SorahanCitation[26] results were prioritized in our analysis of MM. Brown et al.Citation[32] employed the same methods and source population as Cantor et al.,Citation[24] which was included in the pooled analysis of NHL by De Roos et al.Citation[13] Pahwa et al.Citation[34] and Kachuri et al.Citation[33] conducted overlapping analyses in the same Canadian source population as McDuffie et al.,Citation[16] Hohenadel et al.,Citation[28] and Karunanayake et al.Citation[31] Pahwa et al.Citation[34] included more controls in their analysis, but these controls were excluded from Kachuri et al.Citation[33] because they were younger than any enrolled MM cases (≤29 years) and thus did not contribute meaningfully to the analysis. Kachuri et al.Citation[33] also controlled for more confounders, and therefore was prioritized in our analysis.
With respect to glyphosate use, the four independent studies of MM included, respectively, 5 exposed cases,Citation[17] 11 exposed cases,Citation[32] 24 exposed cases,Citation[26] and 32 exposed cases.Citation[33] All but one study, which was based in France,Citation[17] were conducted in North America, and all except oneCitation[26] were restricted to males. One of the two case-control studies was population-basedCitation[32] and the other was hospital-based.Citation[17] Case ascertainment took place during the early 1980s in one study,Citation[32] at least partly during the 1990s in two studies,Citation[26,33] and at least partly during the 2000s in two studies.Citation[17,26]
Studies of leukemia
Two case-control studies and one prospective cohort study investigated the relationship between glyphosate use and risk of leukemia.Citation[12,35,36] Key characteristics of these studies are provided in . The study by Brown et al.Citation[35] used the same methods and source population as Brown et al.,Citation[32] which was described in the section on MM, and Cantor et al.,Citation[24] which was included as part of De Roos et al.Citation[13] in a pooled analysis of NHL.
As described earlier, De Roos et al.,Citation[12] the only prospective cohort study included, was based in North America (Iowa and North Carolina), enrolled both males and females, ascertained cancer incidence in the 1990s and 2000s, and had a 99.5% follow-up rate through 2001. In the total eligible cohort, 43 leukemia cases occurred among glyphosate users. Brown et al.Citation[35] was a population-based case-control study set in North America (Iowa and Minnesota), restricted to white males, with cases identified in 1980–1983, participation rates of 86% for cases and 77–79% for controls, and proxy respondent rates of 41% for cases and 34% for controls. Fifteen leukemia cases in this study were classified as having used glyphosate. The other case-control study of leukemia, by Kaufman et al.,Citation[36] was a hospital-based study set in Asia (Thailand), with males and females, case ascertainment in the 1990s and 2000s, participation rates of 100%, and no proxy respondents for cases or controls.
Meta-analysis
NHL
All relevant RRs and 95% CIs for the association between reported glyphosate use and risk of overall NHL, including those not used in the meta-analysis, such as estimates within subgroups, minimally adjusted estimates, and estimates of exposure-response patterns, are provided in . The estimates selected from each independent study population for inclusion in the meta-analysis, according to the rules specified in the methods section, are provided in .
Table 2. Estimated associations between glyphosate exposure and risk of lymphohematopoietic cancer (LHC), including non-Hodgkin lymphoma (NHL), NHL subtypes, Hodgkin lymphoma (HL), multiple myeloma (MM), and leukemia.
Table 3. Selected estimates included in meta-analyses and calculated meta-analysis relative risks (meta-RRs) of the association between glyphosate exposure and risk of (LHC), including non-Hodgkin lymphoma (NHL), NHL subtypes, Hodgkin lymphoma (HL), multiple myeloma (MM), and leukemia.
As shown in and , the combined meta-RR for overall NHL in association with any use of glyphosate, based on six studies,Citation[12–17] was 1.3 (95% CI = 1.0–1.6). The results were identical in the random-effects and fixed-effects models, suggesting limited between-study heterogeneity in the association. Little heterogeneity also was indicated by the I2 value of 0.0% and the highly non-significant P-value of 0.84 for Cochran's Q. Given the lack of heterogeneity and at least one statistically significant association, we tested for publication bias using Egger's linear regression approach to evaluating funnel plot asymmetry, and found no significant asymmetry (one-tailed P-value = 0.20). Using Duval and Tweedie's trim-and-fill approach to adjust for publication bias, the imputed meta-RR for both the random-effects and fixed-effects models was 1.2 (95% CI = 1.0–1.6).
Figure 1. Forest plots of relative risk (RR) estimates and 95% confidence intervals (CIs) for the association between glyphosate exposure and risk of non-Hodgkin lymphoma. Meta-RRs were identical in random-effects and fixed-effects models.
In secondary analyses, we replaced the RR estimated by De Roos et al.Citation[13] using a hierarchical (i.e., multistage) regression model with the RR estimated using a more traditional logistic regression model (). (The hierarchical regression RR was selected for the primary analysis because, as stated by the authors, hierarchical regression models can yield “increased precision and accuracy for the ensemble of estimates” when modeling multiple pesticides simultaneously, and the more conservative prior assumptions specified in these models “seemed appropriate in a largely exploratory analysis of multiple exposures for which there is little prior knowledge about how pesticide exposures interact in relation to the risk of NHL.”) Using the logistic regression RR did not appreciably affect the results of the meta-analysis (meta-RR = 1.3, 95% CI = 1.0–1.6; identical for random-effects and fixed-effects models).
In another secondary analysis, we replaced the RR reported by McDuffie et al.Citation[16] with the results reported by Hohenadel et al.Citation[28] in the same study population (minus four previously misclassified NHL cases) (). Because Hohenadel et al.Citation[28] reported two estimates for glyphosate use—one in the absence of malathion use and one in the presence of malathion use—we combined these two estimates into a single estimate (RR = 1.40, 95% CI = 0.62–3.15) using random-effects meta-analysis. Using this alternative estimate also did not appreciably affect the meta-RR (1.3, 95% CI = 1.0–1.7; identical for random-effects and fixed-effects models). Finally, using both the logistic regression RR instead of the hierarchical regression RR from De Roos et al.Citation[13] and the combined RR from Hohenadel et al.Citation[28] instead of the RR from McDuffie et al.Citation[16] slightly but non-significantly increased the meta-RR to 1.4 (95% CI = 1.0–1.8; identical for random-effects and fixed-effects models) ().
As noted earlier, in their meta-analysis of the association between glyphosate use and NHL risk, Schinasi and LeonCitation[11] included RR estimates from Eriksson et al.Citation[14] and Hardell et al.Citation[15] that were not the most highly adjusted estimates reported by the authors (shown in as univariate odds ratios). They also used the logistic regression estimate from De Roos et al.Citation[13] that arguably was not as highly adjusted as the hierarchical regression estimate. When we included these estimates in the meta-analysis, along with the same estimates from De Roos et al.,Citation[13] McDuffie et al.,Citation[16] and Orsi et al.Citation[17] as included in our main meta-analysis, we obtained the same results as reported by Schinasi and Leon:Citation[11] random-effects meta-RR = 1.5, 95% CI = 1.1–2.0 (I2 = 32.7%, pheterogeneity = 0.19). The fixed-effects meta-RR based on these estimates (not reported by Schinasi and LeonCitation[11]) was 1.4 (95% CI = 1.1–1.8).
NHL subtypes
All reported RRs and 95% CIs for the association between glyphosate use and risk of various NHL subtypes are shown in . The estimates included in meta-analyses, which were conducted for B-cell lymphoma, diffuse large B-cell lymphoma, chronic lymphocytic leukemia/small lymphocytic lymphoma, follicular lymphoma, and hairy-cell leukemia (i.e., all NHL subtypes for which at least two estimates from independent studies were available), are shown in . Too few studies of any given NHL subtype were conducted to justify testing for publication bias.
The meta-RR for the association between any use of glyphosate and risk of B-cell lymphoma, based on two studies,Citation[14,18] was 2.0 (95% CI = 1.1–3.6) according to both the random-effects and the fixed-effects model (I2 = 0.0%, pheterogeneity = 0.58) (). These results are the same as reported by Schinasi and Leon.Citation[11] The four B-cell lymphoma cases who were classified by Cocco et al.Citation[18] as having used glyphosate consisted of one patient with diffuse large B-cell lymphoma, one with chronic lymphocytic leukemia, one with unspecified B-cell lymphoma, and one with MM. Eriksson et al.Citation[14] did not report the number of exposed cases, but overall the B-cell lymphomas in their study comprised 29% diffuse large B-cell lymphoma, 24% chronic lymphocytic leukemia/small lymphocytic lymphoma, 20% follicular lymphoma grades I–III, 16% other specified B-cell lymphoma, and 11% unspecified B-cell lymphoma; MM cases were not included.
The meta-RR for the association between any use of glyphosate and risk of diffuse large B-cell lymphoma, based on two studies,Citation[14,17] was 1.1 (95% CI = 0.5–2.3) using both the random-effects and the fixed-effects models (I2 = 0.0%, pheterogeneity = 0.79) ().
Based on the same two studies,Citation[14,17] the meta-RR for the association between any use of glyphosate and risk of chronic lymphocytic leukemia/small lymphocytic lymphoma was 1.3 (95% CI = 0.2–10.0) according to the random-effects model and 1.9 (95% CI = 0.9–4.0) according to the fixed-effects model, with significant heterogeneity between the two included estimates (I2 = 83.7%, pheterogeneity = 0.01) ().
Results for follicular lymphoma from these two studies,Citation[14,17] by contrast, were not significantly heterogeneous (I2 = 0.0%, pheterogeneity = 0.73), with a meta-RR of 1.7 (95% CI = 0.7–3.9) in both the random-effects and the fixed-effects models ().
Finally, the two studies that reported associations between any glyphosate use and risk of hairy-cell leukemiaCitation[17,30] yielded a meta-RR of 2.5 (95% CI = 0.9–7.3) in the random-effects and fixed-effects models (I2 = 0.0%, pheterogeneity = 0.63) ().
HL
Both of the published, fully adjusted RRs and 95% CIs for the association between any glyphosate use and HL risk () were included in the meta-analysis (). Based on two studies,Citation[17,31] the meta-RR was 1.1 (95% CI = 0.7–1.6) in both the random-effects and the fixed-effects models, with I2 = 0.0% and pheterogeneity = 0.36 (). Publication bias was not evaluated due to the availability of only two studies of HL.
MM
All relevant RRs and 95% CIs for the association between glyphosate use and risk of MM, including estimates that did not contribute to the meta-analysis, are shown in . The independent estimates selected for inclusion in the meta-analysis are shown in .
The combined meta-RR for the association between any glyphosate use and risk of MM, based on four studies,Citation[17,26,32,33] was 1.4 (95% CI = 1.0–1.9) according to the random-effects and fixed-effects models (, ). On the basis of the I2 value of 0.0% and the P-value of 0.63 for Cochran's Q statistic, between-study heterogeneity was not evident. Egger's linear regression approach yielded no significant evidence of publication bias (one-tailed P-value for asymmetry = 0.10), while the imputed meta-RR using the trim-and-fill procedure to adjust for publication bias was 1.3 (95% CI = 0.9–1.8).
Figure 2. Forest plots of relative risk (RR) estimates and 95% confidence intervals (CIs) for the association between glyphosate exposure and risk of multiple myeloma. Meta-RRs were identical in random-effects and fixed-effects models.
Several secondary analyses were conducted for MM by replacing RRs in the primary meta-analysis with alternative estimates (). When the RR reported by De Roos et al.,Citation[12] who excluded cohort members with missing data from their analysis, was substituted for the one reported by Sorahan,Citation[26] who included such subjects by creating a separate category for missing or unknown data, the meta-RR was slightly increased to 1.5 (95% CI = 1.0–2.1) and was the same for random-effects and fixed-effects models. When the main RR from Kachuri et al.Citation[33] was replaced with the RR from the same study after exclusion of data reported by proxy respondents, the meta-RR was not appreciably different from the original estimate (alternative meta-RR = 1.4, 95% CI = 0.9–1.9 in random-effects and fixed-effects models). Another secondary analysis included the RR reported by Pahwa et al.,Citation[34] who adjusted for a slightly different (and smaller) set of confounders than Kachuri et al.Citation[33] and also retained controls who were too young to have any age-matched MM cases in this Canadian study. This change had minimal impact on the meta-RR (1.4, 95% CI = 1.0–2.0; same for random-effects and fixed-effects models). When both the De Roos et al.Citation[12] and the Pahwa et al.Citation[34] substitutions were made, the resultant meta-RR was the same as that when only De Roos et al.Citation[12] was used (meta-RR = 1.5, 95% CI = 1.0–1.2 in random-effects and fixed-effects models).
Leukemia
Of the four published RRs and 95% CIs for the association between any use of glyphosate and risk of leukemia (), three (excluding one age-adjusted RR in favor of a more fully adjusted RR from De Roos et al.Citation[12]) were included in the meta-analysis (). The meta-RR based on three studiesCitation[12,35,36] was 1.0 (95% CI = 0.6–1.5) using the random-effects model and the fixed-effects model (I2 = 0.0%, pheterogeneity = 0.92) (). Publication bias was not assessed because only three studies of leukemia were available.
Sensitivity analysis
A sensitivity analysis was conducted for overall NHL only (), because other outcomes had an insufficient number of studies for stratification. In all strata, the random-effects and fixed-effects meta-RRs were identical and I2 was 0.0%. Results did not differ substantially from the main meta-RR (1.3, 95% CI = 1.0–1.6) when the analysis was restricted to case-control studies (meta-RR = 1.3, 95% CI = 1.0–1.7) or those with population-based controls (meta-RR = 1.4, 95% CI = 1.0–1.8). Meta-analysis could not be conducted for cohort studies or studies with hospital-based controls because only one of each of these study types was available. No major differences were detected between studies restricted to males (meta-RR = 1.3, 95% CI = 1.0–1.7) and those that included males and females (meta-RR = 1.2, 95% CI = 0.8–1.8) or between those conducted in North America (meta-RR = 1.2, 95% CI = 1.0–1.6) and those conducted in Europe (meta-RR = 1.3, 95% CI = 0.8–2.1). Prompted by Schinasi and Leon,Citation[11] we also conducted a stratified meta-analysis of the two studies conducted in SwedenCitation[14,15] and found a stronger, albeit statistically non-significant, association in these particular studies (meta-RR = 1.6, 95% CI = 0.9–2.8). The estimated meta-RR declined somewhat from studies that ascertained cases in the 1980s (meta-RR = 1.6, 95% CI = 1.0–2.7) to those conducted in the 1990s (meta-RR = 1.2, 95% CI = 1.0–1.6) to those conducted in the 2000s (meta-RR = 1.2, 95% CI = 0.8–1.7).
Table 4. Sensitivity analysis of the association between glyphosate exposure and risk of non-Hodgkin lymphoma (NHL).
Exposure-response trends
NHL and subtypes
Three studies evaluated exposure-response trends between glyphosate use and NHL risk, with exposure classified as cumulative lifetimeCitation[12,14] or annualCitation[16] days of glyphosate use (). Two studies detected some evidence of a positive exposure-response trend (statistical significance not reported),Citation[14,16] whereas the other did not.Citation[12] All of these studies relied wholly or in part on evaluating days of glyphosate use in an attempt to quantify exposure; however, this metric has been shown to be a poor indicator of actual glyphosate dose received.Citation[52]
In a model adjusted for age, sex, and year of diagnosis or enrollment, Eriksson et al.Citation[14] found that the RR of NHL was higher with > 10 days of lifetime glyphosate use (RR = 2.36, 95% CI = 1.04–5.37) than with ≤ 10 days (RR = 1.69, 95% CI = 0.70–4.07), compared with no pesticide use. Also, the RR of NHL was higher after more than 10 years since first use of glyphosate (RR = 2.26, 95% CI = 1.16–4.40) than after 1–10 years (RR = 1.11, 95% CI = 0.24–5.08). Statistical tests for trend were not performed, and exposure-response analyses adjusted for other potential confounders (i.e., 2-methyl-4-chlorophenoxyacetic acid (MCPA), 2,4,5-trichlorophenoxyacetic acid (2,4,5-T) and/or 2,4-dichlorophenoxyacetic acid (2,4-D), mercurial seed dressing, arsenic, creosote, and tar) were not presented, even though adjustment for these characteristics attenuated the RR for overall glyphosate use from 2.02 to 1.51.
McDuffie et al.Citation[16] reported that the RR for more than two days of glyphosate use per year (RR = 2.12, 95% CI = 1.20–3.73) was higher than that for up to two days per year (RR = 1.00, 95% CI = 0.63–1.57), compared with never use, adjusting for age and province of residence. Tests for a significant exposure-response trend were not performed, and results were not reported after adjustment for other potential confounders (i.e., personal medical history and family history of cancer; adjustment for these characteristics attenuated the RR for overall glyphosate use from 1.26 to 1.20) or significantly associated pesticides (i.e., aldrin, dicamba, and mecoprop) in this study population.
The most detailed analysis of glyphosate-NHL exposure-response trends was performed by De Roos et al.,Citation[12] who examined tertiles of cumulative lifetime days of glyphosate use (1–20, 21–56, or 57–2,678 days) and tertiles of intensity-weighted cumulative days of use (i.e., years of use × days per year × intensity level, where intensity was defined as (mixing status + application method + equipment repair status) × personal protective equipment use). In analyses adjusted for age, education, smoking, alcohol, family history of cancer, and state of residence, no significant trend was detected for NHL risk in association with increasing cumulative days of glyphosate use (RRs for tertiles 1, 2, and 3, respectively = 1.0 (referent), 0.7 (95% CI = 0.4–1.4), and 0.9 (95% CI = 0.5–1.6); ptrend = 0.73) or intensity-weighted cumulative exposure days (RRs = 1.0 (referent), 0.6 (95% CI = 0.3–1.1), and 0.8 (95% CI = 0.5–1.4); ptrend = 0.99).
Exposure-response trends between glyphosate use and risk of specific NHL subtypes were not evaluated in any of the included studies.
HL
No studies assessed exposure-response trends between glyphosate use and risk of HL.
MM
Three studies reported exposure-response trends between glyphosate use and MM risk, including the two analyses based on the same Agricultural Health Study cohort datasetCitation[12,26] and the Canadian case-control studyCitation[33] (). The case-control study found mixed evidence of a positive trend (statistical significance not reported), while a positive trend was detected in one analysis of the cohort dataCitation[12] but not the other.Citation[25]
The Canadian case-control study found a lower risk of MM among those who used glyphosate for up to two days per year than those who had never used glyphosate (RR = 0.72, 95% CI = 0.39–1.32).Citation[33] However, risk was higher in those with more than two days of glyphosate use per year (RR = 2.04, 95% CI = 0.98–4.23), adjusting for age, province of residence, proxy status, smoking, personal medical history, and family history of cancer. Results were similar after exclusion of data reported by proxy subjects. The authors did not conduct statistical tests for exposure-response trends.
Based on the 55% of Agricultural Health Study cohort members who had available exposure and covariate data, De Roos et al.Citation[12] reported a positive, albeit statistically non-significant, trend between MM risk and increasing tertiles of cumulative days of glyphosate use (RRs for tertiles 1, 2, and 3, respectively = 1.0 (referent), 1.1 (95% CI = 0.4–3.5), and 1.9 (95% CI = 0.6–6.3); ptrend = 0.27) or intensity-weighted cumulative days of use (RRs = 1.0 (referent), 1.2 (95% CI = 0.4–3.8), and 2.1 (95% CI = 0.6–7.0); ptrend = 0.17). These estimates were adjusted for age, education, smoking, alcohol, family history of cancer, state of residence, the five pesticides for which cumulative-use variables were most highly associated with glyphosate cumulative use days (i.e., 2,4-D, alachlor, atrazine, metolachlor, and trifluralin), and the five pesticides that were most highly associated with ever use of glyphosate (i.e., benomyl, maneb, paraquat, carbaryl, and diazinon). When intensity alone was analyzed in association with MM risk, the RR for the highest versus the lowest tertile was 0.6 (95% CI = 0.2–1.8), indicating that the suggested trend was due only to total days of use. When subjects who never used glyphosate were set as the reference group, the RRs for tertiles 1, 2, and 3 of cumulative days of use were 2.3 (95% CI = 0.6–8.9), 2.6 (95% CI = 0.6–11.5), and 4.4 (95% CI = 1.0–20.2); ptrend = 0.09. When cumulative use was categorized into quartiles, the RR for the highest quartile versus never use was 6.6 (95% CI = 1.4–30.6); ptrend = 0.01.
In contrast to De Roos et al.,Citation[12] SorahanCitation[26] included more than 53,000 eligible cohort members in the analysis (excluding only those with a history of cancer before enrollment, loss to follow-up, missing data on age at enrollment, or missing data on glyphosate use) by creating separate categories for missing or unknown exposure and covariate data. Adjusting for age, sex, education, smoking, alcohol, family history of cancer, and the same 10 pesticides as De Roos et al.,Citation[12] the RRs for each tertile of cumulative days of glyphosate use, compared with never use, were 1.14 (95% CI = 0.43–3.03), 1.52 (95% CI = 0.54–4.34), and 1.38 (95% CI = 0.42–4.45); ptrend = 0.48 using category scores of 1–4, ptrend > 0.50 using mean exposures within categories. RRs for increasing tertiles of intensity-weighted days of use versus never use were 1.00 (95% CI = 0.33–3.00), 1.27 (95% CI = 0.45–3.56), and 1.87 (95% CI = 0.67–5.27); ptrend = 0.22 using scores, ptrend = 0.18 using means. When SorahanCitation[26] expanded the eligible cohort to 55,934 subjects to include those with unknown use of glyphosate, he again detected no significant exposure-response trends with respect to either cumulative days of use (for tertiles 1, 2, and 3 and unknown use versus never use, respectively, RRs = 1.11 (95% CI = 0.44–2.83), 1.45 (95% CI = 0.54–3.88), 1.17 (95% CI = 0.40–3.41), and 1.19 (95% CI = 0.25–5.65); ptrend > 0.50 across categories of known use using scores or means, excluding unknown) or intensity-weighted cumulative days of use (RRs = 0.95 (95% CI = 0.33–2.75), 1.19 (95% CI = 0.44–3.19), 1.58 (95% CI = 0.62–4.05), and 1.04 (95% CI = 0.22–4.92); ptrend = 0.30 using scores, ptrend = 0.26 using means, excluding unknown).
Leukemia
The De Roos et al.Citation[12] study based on the Agricultural Health Study cohort was the only study that reported exposure-response trends between glyphosate use and risk of leukemia (). No significant trend was observed between increasing tertiles of cumulative days of glyphosate use (RRs = 1.0 (referent), 1.9 (95% CI = 0.8–4.5), and 1.0 (95% CI = 0.4–2.9) for tertiles 1, 2, and 3, respectively; ptrend = 0.61) or intensity-weighted cumulative days of use (RRs = 1.0 (referent), 1.9 (95% CI = 0.8–4.7), and 0.7 (95% CI = 0.2–2.1); ptrend = 0.11), adjusting for demographic and lifestyle factors as well as other pesticides.
Evaluation of bias
Selection bias
All studies of the association between glyphosate exposure and risk of LHC were case-control studies except for the Agricultural Health Study, the prospective cohort study that served as the basis for the studies by De Roos et al.Citation[12] and Sorahan.Citation[26] In case-control studies, differences in participation patterns between cases and controls can result in selection bias if participation is related to the exposure of interest. In cohort studies, selection bias can occur if loss to follow-up is related to the exposure and outcome of interest or, less commonly, if baseline participation differs by exposure status and risk of developing the outcome of interest in the future (e.g., based on having a positive family history of an outcome with a genetic susceptibility component). Selection bias in any study also can occur if inclusion in the data analysis, e.g., predicated on data completeness, differs by exposure and outcome status. In general, lower participation, follow-up, or data completeness and large differences in participation between groups increase the potential magnitude of selection bias.
shows the reported participation and follow-up proportions in all reviewed studies. Most studies did not report data completeness. The substantial differences in participation between cases and controls in the European multi-center study,Citation[18] the most recent Swedish study,Citation[14] and the Canadian study, which also had relatively low absolute participation proportions of <70% for cases and <50% for controls,Citation[16,28,31,33,34] are of particular concern. However, the smaller discrepancies between case and control participation in other studies also could have produced selection bias. Moreover, even identical participation by cases and controls can obscure differences in reasons for study participation that could result in bias.
Given that several case-control studies were originally designed to evaluate associations between pesticides and risk of LHC,Citation[13–16,28,31–35] it is plausible that cases with a history of agricultural pesticide use were more likely than controls to participate, thereby biasing results toward a positive association for glyphosate as well as other pesticides. It is also possible that certain sources of controls in some of these studies (e.g., residential telephone calls and voter lists) were more likely to identify individuals who were not farmers, again biasing results toward a positive association. Investigators from the Canadian studyCitation[16,28,31,33,34] reported that an analysis of postal codes showed that respondents and non-respondents did not differ significantly in terms of rural versus urban residence, but they could not examine differences in occupation or pesticide use.
Although the initial follow-up completion of >99% in the Agricultural Health Study was high,Citation[12,25] the sizeable proportions of subjects with missing data raise concerns about selection bias. Specifically, 88% of the eligible cohort (excluding those who were diagnosed with cancer before enrollment or were lost to follow-up) provided usable data on ever use of glyphosate and key demographic and lifestyle covariates, 73% additionally provided data on use of other pesticides, 65–66% contributed to analyses of cumulative days of glyphosate use (with or without intensity weighting), and 55% contributed to analyses of cumulative use additionally adjusted for other pesticides. Questionnaire completion could conceivably have varied by demographic and lifestyle factors that are associated with LHC risk, thereby producing bias. Neither analysis accounted for missing data using methods such as multiple imputation or inverse probability weighting.
Differential data completeness by disease status is more likely to occur in case-control studies, such as the pooled Midwestern U.S. study conducted by De Roos et al.Citation[13] In this study, the analysis of multiple pesticides excluded 25% of cases and 25% of controls who lacked complete data. Although the overall frequency of missing data was the same between cases and controls, this exclusion could have led to selection bias if subjects' reasons for providing complete data, and thus being included in the analysis, differed by disease status and were related to glyphosate exposure status. The authors also excluded subjects who had lived or worked on a farm before age 18 years. If glyphosate use was more common in such subjects, then RR estimates would have been biased upward if a childhood farm environment was inversely associated with NHL riskCitation[53] and biased downward if the association was positive.Citation[54]
Exposure misclassification
All of the included studies assessed use of glyphosate and other pesticides based on self-reported information (), which is prone to various types of error, such as better recall by cases than controls and by subjects than proxies, inaccurate recall of specific pesticides and amounts used, and a lack of the best measure of biological dose received.Citation[55] Thus, probable exposure misclassification is a key limitation of all of these studies. The degree of misclassification may vary by mode of data collection, for example, by written questionnaire, telephone interview, or in-person interview.Citation[56] The extent of misclassification also may depend on questionnaire structure, for example, whether subjects were asked in an open-ended manner to report use of any pesticides or whether they were prompted to report use of specific pesticides based on a prepared list.Citation[57] Some authors did not clearly describe the structure of their study's questions on pesticide use.
Of the eight independent study populations included in this review (seven studies of NHL with or without other types of LHC and one study of leukemia), three provided information on validation of their exposure assessment methods: the Canadian case-control study,Citation[16,28,31,33,34] the Agricultural Health Study,Citation[12,26] and the Kansas case-control studyCitation[47] that contributed to the pooled Midwestern U.S. study by De Roos et al.Citation[13] Overall, these studies do not establish the validity of self-reported information on glyphosate use; rather, the limited results suggest considerable error and inconsistency in such data.
Specifically, in the Canadian study, Dosman et al.Citation[58] reported on the results of a validation pilot study of 21 volunteer farmers whose self-reported pesticide use was compared with written records of pesticide purchases through their local agrochemical supplier. Of the 21 farmers, 17 (81%) had a supplier who had retained written records; the remaining four transactions were conducted with cash. Based on the written records, 146 (65%) of 226 chemicals reported by farmers were verified; 50 of the unverified reports were potentially explained by aerial applications, home and garden use, use more than five years in the past (i.e., during 1958–1984), or use outside of Canada. In 32 instances (for 25 chemicals) the suppliers' records indicated a purchase of chemicals that was unreported by the farmer; 2 of these were for glyphosate. Detailed self-reported exposure (e.g., frequency, intensity, and duration of use of specific pesticides) could not be validated in this pilot study.
Likewise, Hoar et al.Citation[47] reported that suppliers for 110 subjects in the Kansas study (out of 130 sought) were located and provided information on the subjects' crops and herbicide and insecticide purchases as “corroborative evidence” of self-reported pesticide use. The authors observed that suppliers usually reported less pesticide use than subjects; that agreement on specific years of use was better for insecticide use than herbicide use; that the differences between agreement for cases and controls were not consistent; and that agreement between suppliers and subjects was better for pesticide use within the last 10 years than for earlier use. Quantitative results on concordance were not provided by Hoar et al.,Citation[47] but in a summary of this study shared with Dosman et al.Citation[58] the authors stated that reports on herbicide use agreed 59% of the time, with little variation by crop type, and that reports on insecticide use also agreed 59% of the time, but differed by crop type.
In the Agricultural Health Study, the reliability of the question on ever having mixed or applied glyphosate was evaluated by comparing responses to two questionnaires completed one year apart by 3,763 pesticide applicators.Citation[59] Agreement on a positive response to the question was 82%, and the kappa statistic value for inter-rater agreement was moderate (0.54, 95% CI = 0.52–0.58). For more detailed questions about glyphosate use, including years mixed or applied, days per year mixed or applied, and decade first applied, the percentage with exact agreement ranged from 52% to 62% and kappa ranged from 0.37 to 0.71. These metrics evaluated only the reliability (i.e., reproducibility) of self-reported glyphosate use, not its accuracy.
Subsequent exposure validation studies for other pesticides in the Agricultural Health Study, based on comparisons between exposure intensity estimated from an expert-derived algorithm using self-reported or directly observed exposure data and pesticide biomarker levels measured in urine, yielded Spearman correlation coefficients between 0.4 and 0.8, depending on the type of pesticide.Citation[60,61] Correlations with urinary biomarker levels were poorer for self-reported determinants of pesticide exposure such as kilograms of active ingredient, hours spent mixing and applying, and number of acres treated, with correlation coefficients of −0.4 to 0.2, but application method and use of personal protective equipment were found to be important determinants of exposure intensity. However, the latter factors were evaluated in the study questionnaire only for pesticides or pesticide classes in general, not for glyphosate or other individual pesticides;Citation[62] thus, limitations remain in the assessment of specific pesticide exposures.
Several studies included a sizeable proportion of surveys that were completed by proxy respondents for deceased or otherwise unavailable cases and controls (). The use of exposure data reported by surrogates most likely resulted in even poorer accuracy of exposure information in these studies. Although some exposure misclassification may have been non-differential by disease status, such error does not inevitably result in underestimated exposure-disease associations unless additional strict conditions are met, such as independence from other classification errors.Citation[63,64]
Furthermore, differential exposure misclassification in case-control studies can readily result in overestimated associations. Reasonable scenarios include more accurate and/or detailed recollection of past exposures by cases, who are more motivated than controls to try to understand the potential causes of their disease; false recollection by cases, who are more aware of scientific hypotheses or media reports that a certain exposure has been linked to their disease; and unconscious influence by study investigators who are aware of causal hypotheses and subjects' case-control status. Only the authors of the Swedish studies,Citation[14,15] the French study,Citation[17] and the Nebraska component of the pooled Midwestern U.S. studyCitation[48] specifically stated that investigators were blinded to case-control status. In reality, such blinding is often difficult to achieve in studies that collect interview data.
Others have discussed in detail the problems of estimating individual subjects' exposure to glyphosate from responses to interviews and questionnaires asking about days of use, mixing and application procedures, use of personal protective equipment, and other work practices.Citation[19,52] Acquavella et al.Citation[52] reported that any given day of pesticide use can entail highly variable amounts of pesticides used and numbers of mixing operations, and that urine concentrations of glyphosate were poorly correlated with lifetime average exposure intensity scores derived from data self-reported by farmers using this agent. Although recall bias between cases and controls generally might be anticipated to affect all specific pesticides (including glyphosate) equally, variation in the degree of misclassification due to these and other factors affecting usage and exposure could result in different pesticide-specific associations.
Most of the case-control studies did not use procedures to exclude glyphosate exposure that might have occurred after disease onset. The Swedish studies omitted glyphosate use within one year prior to diagnosis or the index date in controls,Citation[15,30] or within the same calendar year or the year before.Citation[14] In some cases, however, these restrictions may not have been sufficient to exclude exposure that occurred during the latency period between disease onset and diagnosis. Inclusion of any such post-disease exposure would have led to misclassification.
Finally, exposure misclassification resulting from the crude dichotomization of glyphosate use as ever versus never is an important limitation of most of the included studies. This classification conflates individuals with considerably different frequencies, intensities, and durations of glyphosate use, and precludes potentially informative analyses of any gradient in LHC risk with increasing glyphosate exposure. As described earlier in the section on exposure-response trends, only three independent studies reported on glyphosate use in more than two (ever vs. never) categories, and only the Agricultural Health Study evaluated more than three exposure categories.
Confounding
As shown in , the degree of control for confounding varied widely among the reviewed studies. Although several studies considered potential confounding by other pesticides or pesticide families, only a minorityCitation[12–15,26,28] reported RR estimates for the association between glyphosate use and LHC risk adjusted for use of other pesticides. Given that Schinasi and LeonCitation[11] found significant associations between NHL risk and several other types of pesticides, including carbamate insecticides, organophosphorus insecticides, lindane, and MCPA, and numerous other associations of specific pesticides with LHC risk have been reported in the literature (e.g.,Citation[65,66])—and because most people who use pesticides occupationally are exposed to multiple pesticides—it is important to control for confounding, whether direct or indirect (if pesticides are surrogates for other risk factors), by these agents.
None of the studies controlled for potential confounding by agricultural exposures other than pesticides, such as other agricultural chemicals, farm animals, allergens, and infectious agents. These exposures have been hypothesized, and in some studies shown, to be associated with risk of NHL, HL, MM, or leukemia,Citation[67–73] and they are probably correlated with glyphosate use, making them potential confounders of associations between glyphosate and LHC risk. Medical history, certain infections, diet, alcohol consumption, and obesity also may be associated with risk of these malignanciesCitation[74–77] and could vary by glyphosate use, again making them possible confounders. Even in studies where numerous confounders were included in multivariable regression models, crude categorization or other misclassification of confounders could have enabled residual confounding of observed associations. The direction and magnitude of confounding depend on the relationships of each factor with glyphosate use and LHC risk, and are therefore difficult to predict.
Other issues
Additional issues related to the design, conduct, and reporting of the included studies also could have affected study results and their interpretation. For instance, Hardell et al.Citation[15] enrolled some prevalent rather than incident cases, since eligible NHL cases were diagnosed in 1987–1990 but interviewed in 1993–1995.Citation[27] The relatively long time interval between diagnosis and interview may have hampered recollection of past exposures, thereby undermining the accuracy of self-reported exposure data in this study. The delay between diagnosis and interview also almost certainly increased the proportion of cases and matched controls who were deceased (43%) and had proxy interviews, leading to further exposure misclassification.
In the studies by De Roos et al.Citation[13] and Brown et al.,Citation[32,35] LHC cases were diagnosed in 1979–1986, 1980–1983, and 1980–1984, respectively. With glyphosate having come to market in 1974, the cases in these studies would have had a relatively short potential induction time since first use of glyphosate. However, few studies to date have considered the issue of induction time. The Agricultural Health Study collected information on decade of first use of glyphosate in the baseline questionnaire for private pesticide applicators,Citation[62] but did not use this information in the published analysis.Citation[12] If glyphosate is a cause of LHC, the actual induction time is unknown because the mechanism of carcinogenesis is not established.
Orsi et al.,Citation[17] Kaufman et al.,Citation[36] and four of the six study centers included in Cocco et al.Citation[18] enrolled hospital-based rather than population-based cases and controls. Given that farmers have lower hospitalization rates than non-farmers,Citation[78] hospital-based controls may be less likely than population-based controls to report agricultural occupational exposures, including pesticides, thereby resulting in overestimated RRs for pesticide use. On the other hand, occupational injuries are more common in agriculture than in general private industry,Citation[79] possibly leading to oversampling of farmers from hospital trauma/emergency and orthopedics departments, which might result in underestimated RRs. We did not observe any meaningful change in the meta-RR after restriction to population-based case-control studies.
As noted in , many possible analyses were not conducted or not reported by authors. De Roos et al.Citation[13] specifically acknowledged that they did not report results for pesticide combinations that were analyzed but yielded statistically null associations for joint effects, and Hohenadel et al.Citation[28] likewise did not show results for pesticide combinations without evidence of joint effects. Most other authors did not explicitly state when null results were not reported, but the Methods sections of several papers suggested that certain analyses were performed, yet not shown. Given the widespread predilection for emphasizing statistically significant associations in published research articles,Citation[80] unreported results probably are usually statistically null. The omission of null results is a form of reporting bias that favors positive associations.
Other evidence suggests that statistically null associations between glyphosate and LHC risk have been underreported in the epidemiologic literature. For example, two of the studies that contributed to the pooled analysis conducted by De Roos et al.Citation[13] apparently collected information on glyphosate use, yet associations between glyphosate and NHL risk were not reported in the original publications.Citation[47,48] In an analysis of interactions between pesticide use and asthma, allergies, or hay fever diagnosis in relation to NHL risk in the Canadian case-control study,Citation[81] results were reported for several specific pesticides, but not glyphosate, even though information was available for glyphosate use. The most probable scenario in each of these cases is that no significant association was detected between glyphosate use and NHL risk. The omission of such results from the published literature represents a distortion of the body of epidemiologic evidence.
The largest number of studies included in any of the meta-analyses described here was six (in the analysis of NHL), and the majority of meta-analyses (of HL, B-cell lymphoma, diffuse large B-cell lymphoma, chronic lymphocytic leukemia/small lymphocytic lymphoma, follicular lymphoma, and hairy-cell leukemia) included only two studies. The small number of available studies limits the robustness of the estimated meta-RRs, as well as the ability to perform informative sensitivity analysis and evaluation of heterogeneity and publication bias. Even with 10 contributing studies (which we lacked), the statistical power to detect modest heterogeneity using Cochran's Q statistic is “low.”Citation[42] The small number of studies also provides little opportunity to qualitatively investigate possible sources of heterogeneity by subject characteristics or study design. Thus, the results of the meta-analyses and related statistical tests reported here should be interpreted cautiously in light of the sparse and possibly selectively published literature, as well as the high potential for bias and confounding in most of the available studies.
Overall evaluation
The validity of the meta-RRs for glyphosate use and LHC risk reported here and by othersCitation[11] is uncertain because systematic error due to bias and confounding cannot reasonably be ruled out as explanations for the observed associations (including both positive and null associations). In addition, an evaluation of the association between glyphosate exposure and risk of LHC based on the Bradford Hill viewpointsCitation[46] does not favor a causal relationship with NHL, any NHL subtype, HL, MM, or leukemia. These nine viewpoints are strength, consistency, specificity, temporality, biological gradient, plausibility, coherence, experiment, and analogy.
To evaluate the strength of the association between glyphosate use and risk of each type of LHC, we considered the magnitude of study-specific RRs and the corresponding meta-RRs. In individual studies, estimates of the association between glyphosate use and risk of NHL ranged between 1.0 and 2.1, and estimates of the association with NHL subtypes ranged between 0.4 and 3.35 (). For HL, the two estimates of association were 0.99 and 1.7. For MM, RRs ranged between 1.0 and 2.4, and those for leukemia ranged between 0.9 and 1.40. Most study-specific estimates were between 1.0 and 1.5. The estimated meta-RRs for all LHC outcomes, including those calculated in secondary and sensitivity analyses, ranged between 1.0 (for leukemia) and 2.5 (for hairy-cell leukemia). The meta-RRs calculated based on at least four studies ranged between 1.3 and 1.4. These associations are not of sufficient magnitude to exclude modest bias or confounding as reasonable explanations for the observed results.
Results were not consistent between case-control studies of NHL and the one prospective cohort study of NHL, which reported no association.Citation[12] Even among the six studies that contributed to the meta-analysis of NHL, RR point estimates varied by more than two-fold, only one statistically significant positive association was observed, and results from some studies were internally inconsistent (). Another, arguably more appropriately adjusted RR (from a hierarchical regression model) that was 24% lower and statistically non-significant was reported in the same study that found a significant association.Citation[13] The lack of statistically significant heterogeneity among studies of NHL, based on an underpowered statistical test, does not indicate consistency of results. For NHL subtypes, RR estimates also were variable, except for diffuse large B-cell lymphoma, for which both estimates were close to 1.0. Only one statistically significant positive association was detected (for chronic lymphocytic leukemia/small lymphocytic lymphoma),Citation[14] and this result was contradicted by a non-significant inverse association in the other study of this outcome.Citation[17] No significant associations with ever use of glyphosate were detected for HL, MM, or leukemia, and for MM the RR point estimates varied by more than two-fold. Results for MM in the Agricultural Health Study were internally inconsistent;Citation[12,26] and the positive association with cumulative glyphosate exposure probably was due largely to selection bias.
Numerous associations have been hypothesized between glyphosate exposure and diverse health outcomes, and between various exposures and risk of NHL, NHL subtypes, HL, MM, or leukemia. Thus, the putative associations are not specific to either the exposure or any of the outcomes. As noted by Bradford Hill,Citation[46] “diseases may have more than one cause” and “one-to-one relationships are not frequent”; therefore, a lack of specificity does not detract from a causal hypothesis.
In case-control studies, where exposure assessment was retrospective, a temporal sequence was not definitively established with glyphosate use preceding the time of disease onset. Although some studies attempted to exclude use close to the time of case diagnosis (or enrollment, for controls),Citation[14,15,30] in practice individuals may not accurately recall the timing of use. Only the prospective Agricultural Health StudyCitation[12,26] was designed to collect information on glyphosate use prior to cancer ascertainment. However, the authors did not exclude malignancies diagnosed close to (e.g., within one year of) study enrollment, nor did they report the distribution of diagnoses with respect to time since first use of glyphosate. Thus, some preclinical cancers may have existed prior to study entry and, possibly, prior to at least some reported glyphosate use.
As discussed in detail earlier, in the three studies of NHL with information on frequency, intensity, and/or duration of glyphosate use,Citation[12,14,16] a positive biological gradient was not consistently demonstrated and was notably lacking in the Agricultural Health Study,Citation[12] which had the most detailed exposure information (). One case-control studyCitation[33] and one prospective cohort studyCitation[12] of MM reported results suggesting a positive biological gradient with glyphosate use, but the alternative analysis of the Agricultural Health Study dataCitation[26] did not demonstrate such a trend. No data were available to evaluate exposure-response trends between glyphosate and risk of NHL subtypes or HL, and the single study with such data for leukemia found no apparent trend.Citation[12]
Inhalation exposure to glyphosate from agricultural or residential uses is likely to be slight due to glyphosate's extremely low vapor pressure.Citation[82] Although dermal contact can be considerable, the very low skin penetrability of glyphosateCitation[83] should result in minimal, if any, biologically absorbed dose. A study of farm families with a lower limit of detection of 0.001 µg/mL (1 ppb) found that 40% of glyphosate applicators had undetectable urinary glyphosate, which reflects all routes of exposure (dermal, inhalation, and oral).Citation[84] Among those with detectable urinary glyphosate, the distribution of concentrations was right skewed, with a peak geometric mean concentration of 0.0032 µg/mL (3.2 ppb) on the day of application and declining thereafter. A review of seven human biomonitoring studies of glyphosate (includingCitation[84]) yielded the conclusion that “no health concern was revealed because the resulting exposure estimates were by magnitudes lower” than the science-based acceptable daily intake and the acceptable operator exposure level proposed by EFSA.Citation[85] Glyphosate is usually applied in agricultural operations only a few days per year. Given the low biological dose of glyphosate that is expected to be sustained, along with the lack of information on the mechanism of carcinogenesis that may exist in humans, the biological plausibility of LHC development due to typical glyphosate exposure has not been established.
IARC recently determined based on their process that there is “sufficient” evidence of carcinogenicity of glyphosate in experimental animals and mechanistic evidence of genotoxicity and oxidative stress.Citation[6] By contrast, U.S. EPA,Citation[86] JMPR,Citation[3] BfR,Citation[1] EFSA,Citation[9] and othersCitation[87,88] concluded that glyphosate does not have genotoxic, mutagenic, or carcinogenic effects in in vivo animal and in vitro studies, and that the negative findings constitute evidence against carcinogenicity. Given these widely divergent opinions, one cannot unambiguously conclude whether the scientific evidence is coherent with the hypothesis that glyphosate causes any or all LHC.
No true experimental evidence exists regarding the association between glyphosate exposure and risk of LHC in humans. However, positive associations between farming and risk of LHC were detected prior to 1974, when glyphosate was first commercially marketed.Citation[89,90] Thus, if the apparent associations between farming and risk of LHC are due to causal agricultural exposures, they cannot be explained only by glyphosate exposure. Likewise, the recent worldwide increase (followed by a plateau or decline) in NHL incidence began before the 1970sCitation[91,92]—although any impact of glyphosate on NHL incidence trends might be obscured by stronger risk factors. No marked increase in the incidence of HL, MM, or leukemia has been observed in parallel with the introduction and expansion of glyphosate use.Citation[93–96]
Finally, numerous analogies exist to support or oppose the hypothesis of a causal link between glyphosate exposure and risk of LHC. On balance, such analogies do not strengthen or weaken a conclusion of causality.
In summary, although none of the Bradford Hill viewpoints can establish or disprove causality, we did not find compelling evidence in support of causality based on any of the nine viewpoints. Thus, on balance, the existing epidemiologic evidence does not favor a causal effect of glyphosate on NHL, HL, MM, leukemia, or any subtype of these malignancies.
Discussion
Our meta-analysis yielded borderline significant RRs of 1.3 and 1.4 between glyphosate use and risk of NHL and MM, respectively, and no significant association with risk of HL or leukemia. Based on more fully adjusted RRs, our NHL meta-RR of 1.3 (95% CI = 1.0–1.6) was weaker than that reported by Schinasi and LeonCitation[11] (RR = 1.5, 95% CI = 1.1–2.0). The largest meta-RR of 2.5 (for hairy-cell leukemia) and the only meta-RR with a lower 95% confidence limit that excluded 1.0 (for B-cell lymphoma) were based on only two studies each, and the maximum number of studies contributing to any meta-analysis was six. The few studies with available data did not consistently detect positive exposure-response trends between quantitative measures of glyphosate use and risk of any LHC.
Consideration of the available epidemiologic evidence in light of the Bradford Hill viewpoints does not substantiate a causal relationship between glyphosate exposure and risk of any type of LHC. A conclusion in favor of causality also is undermined by the studies' methodological limitations, which could reasonably account for at least part of the observed associations. These limitations include exposure misclassification (which may differ by outcome status especially in case-control studies, which constitute nearly all available studies), selection bias (due to differential enrollment, follow-up, or data completeness), poor adjustment for confounding (by other agricultural exposures, for instance), small numbers (which lead to low statistical power as well as a higher probability that a statistically significant finding is falseCitation[97]), and potential reporting and publication bias. Although underpowered statistical tests did not formally detect publication bias, we identified several examples of studies with available data that did not report associations between glyphosate use and LHC risk, and these unreported associations were most likely null.
Underpowered statistical tests also generally did not detect heterogeneity of results among studies, except for chronic lymphocytic leukemia/small lymphocytic lymphoma and MM. Nevertheless, our sensitivity analysis revealed some evidence of stronger associations with NHL risk in studies based in Sweden and those that ascertained cases in the 1980s, whereas the meta-RRs for studies that ascertained cases in the 2000s were close to the null and statistically non-significant The stronger association with NHL diagnosed in the 1980s raises questions about whether glyphosate, an agent first introduced in 1974 in the United States and Europe, could plausibly cause lymphoma less than a decade later. However, deliberation on the potential induction time requires an understanding of the presumed mechanism of carcinogenesis, which is unknown for glyphosate. The classification system for lymphoid tumors underwent major changes in 1994 and 2001,Citation[20] such that the definition of NHL as a disease entity is not entirely comparable between recent studies and those conducted in the 1980s. Study quality also may have improved over time, for example, due to refinements in survey design, interviewing techniques, data management, and other methods to augment data integrity.
The stronger association in Swedish studies probably is not explained by geographical differences in glyphosate use or effect modifiers related to NHL risk. One possible explanation is that of the six NHL studies, only the two Swedish studiesCitation[14,15] compared subjects who used glyphosate with those who did not use any pesticides as the reference group, whereas the other studies defined the reference group as those who did not use glyphosate in particular. Comparisons with subjects who do not use any pesticides are more likely to be confounded by other pesticides and agricultural exposures.
Meta-analysis can be problematic when applied to observational epidemiology.Citation[21,22] Meta-analysis increases statistical precision by combining results from studies that may differ substantially in terms of source population, exposure and outcome assessment and classification, control for confounding, and other key characteristics. In the presence of such heterogeneity, even if not detectable using formal statistical tests, a single summary estimate may not be scientifically meaningful. Additionally, even when studies are statistically homogeneous, meta-analysis may not yield valid results, since this technique cannot overcome problems in the design and conduct of the underlying studies. Instead, given that bias can seldom be ruled out and unmeasured and uncontrolled confounding can never be eliminated from observational epidemiologic studies, modest meta-RRs detected across multiple studies may simply be due to shared biases, rather than a true association.Citation[21] As stated earlier, the purpose of meta-analysis is not to evaluate whether associations are causal. We conducted a meta-analysis primarily for comparison with published findings.
Considering the shortcomings of the existing literature, what can be done to shed further light on whether glyphosate causes LHC in humans? Perhaps the foremost need is better exposure assessment. Self-reported information on use of specific pesticides, unless validated by comparison with sales records (which most likely would need to be collected prospectively, and might not be closely correlated with pesticide use) or other objective documentation, is not sufficiently accurate and reliable to yield credible estimates of association, especially exposure-response trends. Urinary glyphosate levels would provide more accurate and quantitatively detailed information on biological dose of glyphosate received, but would probably have to be measured repeatedly to reflect long-term exposure.
Information about temporal aspects of glyphosate exposure, such as the putative induction time since first use of glyphosate, duration of use, and time since last use, could help to shed light on the exposure-outcome relationship. Results from additional prospective cohort studies are necessary to alleviate concerns about selection and reporting bias in case-control studies.
More specific outcome classification also is needed. Only two studiesCitation[14,17] examined associations between glyphosate use and more than one histological subtype of NHL, despite growing evidence of important etiologic heterogeneity among NHL subtypes.Citation[74] Information on NHL subtypes also is available in the Agricultural Health Study,Citation[66] and publication of risk associations with glyphosate is anticipated. Risk factors for HL and leukemia also are known to differ by subtype,Citation[76,77] yet no studies estimated associations with glyphosate separately for subtypes of these tumors. (Chronic lymphocytic leukemia and hairy-cell leukemia, which were analyzed as distinct outcomes, are classified as NHL subtypes.Citation[20]) Large, probably pooled studies with histopathological data can determine whether associations with specific tumor subtypes might be obscured by analyzing overall NHL, HL, MM, or leukemia as a single disease entity.
Conclusion
In conclusion, we found marginally significant positive meta-RRs for the association between glyphosate use and risk of NHL and MM, and statistically null associations with HL and leukemia. A statistically significant positive meta-RR for B-cell lymphoma, but not other NHL subtypes, was calculated based on only two studies. Combining these results with recognition of the methodological weaknesses of the small number of existing studies and an overall body of literature that is not strong, consistent, temporally unambiguous, or indicative of a positive biological gradient, we determined that no causal relationship has been established between glyphosate exposure and risk of NHL, HL, MM, leukemia, or any subtype of LHC.
Acknowledgments
The authors thank John Acquavella and Thomas Sorahan for their thoughtful comments on earlier drafts of this manuscript, and Bernard Beckerman for his technical review of the tables.
Disclosure statement
The sponsors were provided the opportunity to review the manuscript prior to journal submission, but inclusion of their suggestions was left to the discretion of the authors, who retained sole control of the manuscript content and the findings. Statements in this paper are those of the authors and not those of the authors' employer or the sponsors. The authors are employed by Exponent, a scientific research and consulting firm that provides services for private and governmental clients, including on projects concerning glyphosate and other pesticides. In the past five years, Ellen Chang has provided consulting services through Exponent on behalf of Monsanto Company on other issues, and she also has provided consulting services on other pesticides and lymphohematopoietic cancers for other clients.
Funding
This work was supported by Monsanto Company, the original producer and marketer of glyphosate formulations.
References
- BfR. The BfR has finalised its draft report for the re-evaluation of glyphosate. Press release. Bundesinstitut für Risikobewertung (BfR): Berlin, 2014. Available at http://www.bfr.bund.de/en/the_bfr_has_finalised_its_draft_report_for_the_re_evaluation_of_glyphosate-188632.html (accessed Nov 2015).
- BfR. Does glyphosate cause cancer? BfR Communication No 007/2015, 23 March 2015. Bundesinstitut für Risikobewertung (BfR): Berlin, 2015. Available at http://www.bfr.bund.de/cm/349/does-glyphosate-cause-cancer.pdf (accessed Nov 2015).
- JMPR. Pesticide residues in food - 2004. Joint FAO/WHO Meeting on Pesticide Residues. Evaluations 2004. Part II - Toxicological. World Health Organization: Geneva, 2006.
- U.S. EPA. Reregistration Eligibility Decision (RED): Glyphosate. United States Environmental Protection Agency (U.S. EPA), Office of Prevention, Pesticides and Toxic Substances: Washington, DC, 1993.
- U.S. EPA. Glyphosate; Pesticide Tolerances. 40 CFR Part 180 [EPA-HQ-OPP-2012-0132; FRL-9384-3]. Final Rule. Federal Register. 2013, 78, 25396–25401.
- IARC. Glyphosate. In IARC Monographs on the Evaluation of Carcinogenic Risks to Humans. Volume 112. Some Organophosphate Insecticides and Herbicides: Diazinon, Glyphosate, Malathion, Parathion, and Tetrachlorvinphos; International Agency for Research on Cancer (IARC): Lyon, France, 2015.
- BfR. Frequently asked questions on the procedure for the re-assessment of glyphosate within the framework of the EU active substance review. Available at http://www.bfr.bund.de/en/frequently_asked_questions_on_the_procedure_for_the_re_assessment_of_glyphosate_within_the_framework_of_the_eu_active_substance_review-195637.html (accessed Nov 2015). Bundesinstitut für Risikobewertung (BfR): Berlin, 2015.
- BfR. Frequently asked questions on the assessment of the health risk of glyphosate. Available at http://www.bfr.bund.de/en/frequently_asked_questions_on_the_assessment_of_the_health_risk_of_glyphosate-127871.html#topic_195669 (accessed Nov 2015). Bundesinstitut für Risikobewertung (BfR): Berlin, 2015.
- EFSA. Conclusion on the peer review of the pesticide risk assessment of the active substance glyphosate. EFSA (European Food Safety Authority) Journal. 2015, 13, 4302.
- JMPR. World Health Organization. Food safety. Joint FAO/WHO Meeting on Pesticide Residues (JMPR). Expert Taskforce on Diazinon, Glyphosate and Malathion. Available at http://www.who.int/foodsafety/areas_work/chemical-risks/jmpr/en/ (accessed Nov 2015.).
- Schinasi, L.; Leon, M.E. Non-Hodgkin lymphoma and occupational exposure to agricultural pesticide chemical groups and active ingredients: a systematic review and meta-analysis. Int. J. Environ. Res. Public Health. 2014, 11, 4449–4527.
- De Roos, A.J.; Blair, A.; Rusiecki, J.A.; Hoppin, J.A.; Svec, M.; Dosemeci, M.; Sandler, D.P.; Alavanja, M.C. Cancer incidence among glyphosate-exposed pesticide applicators in the Agricultural Health Study. Environ. Health Perspect. 2005, 113, 49–54.
- De Roos, A.J.; Zahm, S.H.; Cantor, K.P.; Weisenburger, D.D.; Holmes, F.F.; Burmeister, L.F.; Blair, A. Integrative assessment of multiple pesticides as risk factors for non-Hodgkin's lymphoma among men. Occup. Environ. Med. 2003, 60, E11.
- Eriksson, M.; Hardell, L.; Carlberg, M.; Akerman, M. Pesticide exposure as risk factor for non-Hodgkin lymphoma including histopathological subgroup analysis. Int. J. Cancer. 2008, 123, 1657–1663.
- Hardell, L.; Eriksson, M.; Nordstrom, M. Exposure to pesticides as risk factor for non-Hodgkin's lymphoma and hairy cell leukemia: pooled analysis of two Swedish case-control studies. Leuk. Lymphoma. 2002, 43, 1043–1049.
- McDuffie, H.H.; Pahwa, P.; McLaughlin, J.R.; Spinelli, J.J.; Fincham, S.; Dosman, J.A.; Robson, D.; Skinnider, L.F.; Choi, N.W. Non-Hodgkin's lymphoma and specific pesticide exposures in men: cross-Canada study of pesticides and health. Cancer Epidemiol. Biomarkers Prev. 2001, 10, 1155–1163.
- Orsi, L.; Delabre, L.; Monnereau, A.; Delval, P.; Berthou, C.; Fenaux, P.; Marit, G.; Soubeyran, P.; Huguet, F.; Milpied, N.; Leporrier, M.; Hemon, D.; Troussard, X.; Clavel, J. Occupational exposure to pesticides and lymphoid neoplasms among men: results of a French case-control study. Occup. Environ. Med. 2009, 66, 291–298.
- Cocco, P.; Satta, G.; Dubois, S.; Pili, C.; Pilleri, M.; Zucca, M.; t Mannetje, A.M.; Becker, N.; Benavente, Y.; de Sanjose, S.; Foretova, L.; Staines, A.; Maynadie, M.; Nieters, A.; Brennan, P.; Miligi, L.; Ennas, M.G.; Boffetta, P. Lymphoma risk and occupational exposure to pesticides: results of the Epilymph study. Occup. Environ. Med. 2013, 70, 91–98.
- Mink, P.J.; Mandel, J.S.; Sceurman, B.K.; Lundin, J.I. Epidemiologic studies of glyphosate and cancer: a review. Regul. Toxicol. Pharmacol. 2012, 63, 440–452.
- Swerdlow, S.H.; Campo, E.H., N.L.; Jaffe, E.S.; Pileri, S.A.; Stein, H.; Thiele, J.; Vardiman, J.W. WHO Classification of Tumours of Haematopoietic and Lymphoid Tissues, 4th Ed. World Health Organization (WHO): Geneva, 2008.
- Shapiro, S. Meta-analysis/Shmeta-analysis. Am. J. Epidemiol. 1994, 140, 771–778.
- Weed, D.L. Meta-analysis and causal inference: a case study of benzene and non-Hodgkin lymphoma. Ann. Epidemiol. 2010, 20, 347–355.
- Cantor, K.P.; Blair, A.; Brown, L.M.; Burmeister, L.F.; Everett, G. Pesticides and other agricultural risk factors for non-Hodgkin's lymphoma among men in Iowa and Minnesota. Cancer Res. 1992, 52, 2447–2455.
- Cantor, K.P.; Blair, A.; Everett, G.; Gibson, R.; Burmeister, L.F.; Brown, L.M.; Schuman, L.; Dick, F.R. Pesticides and other agricultural risk factors for non-Hodgkin's lymphoma among men in Iowa and Minnesota. Cancer Res. 1992, 52, 2447–2455.
- Sorahan, T. Multiple myeloma and glyphosate use: a re-analysis of US Agricultural Health Study data. Toxicol. Lett. 2012, 211, S169–S169.
- Sorahan, T. Multiple myeloma and glyphosate use: a re-analysis of US Agricultural Health Study (AHS) data. Int. J. Environ. Res. Public Health. 2015, 12, 1548–1559.
- Hardell, L.; Eriksson, M. A case-control study of non-Hodgkin lymphoma and exposure to pesticides. Cancer. 1999, 85, 1353–1360.
- Hohenadel, K.; Harris, S.A.; McLaughlin, J.R.; Spinelli, J.J.; Pahwa, P.; Dosman, J.A.; Demers, P.A.; Blair, A. Exposure to multiple pesticides and risk of non-Hodgkin lymphoma in men from six Canadian provinces. Int. J. Environ. Res. Public Health. 2011, 8, 2320–2330.
- Lee, W.J.; Cantor, K.P.; Berzofsky, J.A.; Zahm, S.H.; Blair, A. Non-Hodgkin's lymphoma among asthmatics exposed to pesticides. Int. J. Cancer. 2004, 111, 298–302.
- Nordström, M.; Hardell, L.; Magnuson, A.; Hagberg, H.; Rask-Andersen, A. Occupational exposures, animal exposure and smoking as risk factors for hairy cell leukaemia evaluated in a case-control study. Br. J. Cancer. 1998, 77, 2048–2052.
- Karunanayake, C.P.; Spinelli, J.J.; McLaughlin, J.R.; Dosman, J.A.; Pahwa, P.; McDuffie, H.H. Hodgkin lymphoma and pesticides exposure in men: a Canadian case-control study. J. Agromedicine. 2012, 17, 30–39.
- Brown, L.M.; Burmeister, L.F.; Everett, G.D.; Blair, A. Pesticide exposures and multiple myeloma in Iowa men. Cancer Causes Control. 1993, 4, 153–156.
- Kachuri, L.; Demers, P.A.; Blair, A.; Spinelli, J.J.; Pahwa, M.; McLaughlin, J.R.; Pahwa, P.; Dosman, J.A.; Harris, S.A. Multiple pesticide exposures and the risk of multiple myeloma in Canadian men. Int. J. Cancer. 2013, 133, 1846–1858.
- Pahwa, P.; Karunanayake, C.P.; Dosman, J.A.; Spinelli, J.J.; McDuffie, H.H.; McLaughlin, J.R. Multiple myeloma and exposure to pesticides: a Canadian case-control study. J Agromedicine. 2012, 17, 40–50.
- Brown, L.M.; Blair, A.; Gibson, R.; Everett, G.D.; Cantor, K.P.; Schuman, L.M.; Burmeister, L.F.; Van Lier, S.F.; Dick, F. Pesticide exposures and other agricultural risk factors for leukemia among men in Iowa and Minnesota. Cancer Res. 1990, 50, 6585–6591.
- Kaufman, D.W.; Anderson, T.E.; Issaragrisil, S. Risk factors for leukemia in Thailand. Ann. Hematol. 2009, 88, 1079–1088.
- Woodruff, T.J.; Sutton, P. The Navigation Guide systematic review methodology: a rigorous and transparent method for translating environmental health science into better health outcomes. Environ. Health Perspect. 2014, 122, 1007–1014.
- Higgins, J.P.T.; Altman, D.G.; Sterne, J.A.C. Chapter 8: Assessing risk of bias in included studies. In Cochrane Handbook for Systematic Reviews of Interventions; Higgins, J.P.T.; Green, S. Eds.; The Cochrane Collaboration, 2011. Version 5.1.0 [updated March 2011]. Available at http://handbook.cochrane.org/ (accessed Nov 2015).
- Guyatt, G.H.; Oxman, A.D.; Vist, G.; Kunz, R.; Brozek, J.; Alonso-Coello, P.; Montori, V.; Akl, E.A.; Djulbegovic, B.; Falck-Ytter, Y.; Norris, S.L.; Williams, J.W., Jr.; Atkins, D.; Meerpohl, J.; Schunemann, H.J. GRADE guidelines: 4. Rating the quality of evidence–study limitations (risk of bias). J. Clin. Epidemiol. 2011, 64, 407–415.
- Higgins, J.P.; Thompson, S.G.; Deeks, J.J.; Altman, D.G. Measuring inconsistency in meta-analyses. BMJ. 2003, 327, 557–560.
- Cochran, W.G. The combination of estimates from different experiments. Biometrics. 1954, 10, 101–129.
- Ioannidis, J.P. Interpretation of tests of heterogeneity and bias in meta-analysis. J. Eval. Clin. Pract. 2008, 14, 951–957.
- Dickersin, K.; Min, Y.I. Publication bias: the problem that won't go away. Ann. N. Y. Acad. Sci. 1993, 703, 135–146; discussion 146–138.
- Egger, M.; Davey Smith, G.; Schneider, M.; Minder, C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997, 315, 629–634.
- Duval, S.; Tweedie, R. Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000, 56, 455–463.
- Hill, A.B. The Environment and Disease: Association or Causation? Proc. R. Soc. Med. 1965, 58, 295–300.
- Hoar, S.K.; Blair, A.; Holmes, F.F.; Boysen, C.D.; Robel, R.J.; Hoover, R.; Fraumeni, J.F., Jr. Agricultural herbicide use and risk of lymphoma and soft-tissue sarcoma. JAMA. 1986, 256, 1141–1147.
- Hoar Zahm, S.; Weisenburger, D.D.; Babbitt, P.A.; Saal, R.C.; Vaught, J.B.; Cantor, K.P.; Blair, A. A case-control study of non-Hodgkin's lymphoma and the herbicide 2,4-dichlorophenoxyacetic acid (2,4-D) in eastern Nebraska. Epidemiol. 1990, 1, 349–356.
- Glyphosate Task Force. History of glyphosate. 2013. Available at http://www.glyphosate.eu/glyphosate-basics/history-glyphosate. (accessed Nov 2015).
- Landgren, O.; Kyle, R.A.; Hoppin, J.A.; Beane Freeman, L.E.; Cerhan, J.R.; Katzmann, J.A.; Rajkumar, S.V.; Alavanja, M.C. Pesticide exposure and risk of monoclonal gammopathy of undetermined significance in the Agricultural Health Study. Blood. 2009, 113, 6386–6391.
- Lash, T.L. Bias analysis applied to Agricultural Health Study publications to estimate non-random sources of uncertainty. J. Occup. Med. Toxicol. 2007, 2, 15.
- Acquavella, J.F.; Alexander, B.H.; Mandel, J.S.; Burns, C.J.; Gustin, C. Exposure misclassification in studies of agricultural pesticides: insights from biomonitoring. Epidemiol. 2006, 17, 69–74.
- Rudant, J.; Orsi, L.; Monnereau, A.; Patte, C.; Pacquement, H.; Landman-Parker, J.; Bergeron, C.; Robert, A.; Michel, G.; Lambilliotte, A.; Aladjidi, N.; Gandemer, V.; Lutz, P.; Margueritte, G.; Plantaz, D.; Mechinaud, F.; Hemon, D.; Clavel, J. Childhood Hodgkin's lymphoma, non-Hodgkin's lymphoma and factors related to the immune system: the Escale Study (SFCE). Int. J. Cancer. 2011, 129, 2236–2247.
- Hofmann, J.N.; Hoppin, J.A.; Lynch, C.F.; Poole, J.A.; Purdue, M.P.; Blair, A.; Alavanja, M.C.; Beane Freeman, L.E. Farm characteristics, allergy symptoms, and risk of non-Hodgkin lymphoid neoplasms in the agricultural health study. Cancer Epidemiol. Biomarkers Prev. 2015, 24, 587–594.
- Blair, A.; Zahm, S.H. Methodologic issues in exposure assessment for case-control studies of cancer and herbicides. Am. J. Ind. Med. 1990, 18, 285–293.
- Bowling, A. Mode of questionnaire administration can have serious effects on data quality. J. Public Health (Oxf). 2005, 27, 281–291.
- Griffith, L.E.; Cook, D.J.; Guyatt, G.H.; Charles, C.A. Comparison of open and closed questionnaire formats in obtaining demographic information from Canadian general internists. J. Clin. Epidemiol. 1999, 52, 997–1005.
- Dosman, J.A.; McDuffie, H.H.; Pahwa, P.; Fincham, S.; McLaughlin, J.R.; Robson, D.; Theriault, G. Pesticides, Soft Tissue Sarcoma, Lymphoma, and Multiple Myeloma. A Case Control Study in Three Regions of Canada. Report to Health and Welfare Canada on Project 6008–1223; University of Saskatchewan: Saskatoon, Canada, 1990.
- Blair, A.; Tarone, R.; Sandler, D.; Lynch, C.F.; Rowland, A.; Wintersteen, W.; Steen, W.C.; Samanic, C.; Dosemeci, M.; Alavanja, M.C. Reliability of reporting on life-style and agricultural factors by a sample of participants in the Agricultural Health Study from Iowa. Epidemiol. 2002, 13, 94–99.
- Blair, A.; Thomas, K.; Coble, J.; Sandler, D.P.; Hines, C.J.; Lynch, C.F.; Knott, C.; Purdue, M.P.; Zahm, S.H.; Alavanja, M.C.; Dosemeci, M.; Kamel, F.; Hoppin, J.A.; Freeman, L.B.; Lubin, J.H. Impact of pesticide exposure misclassification on estimates of relative risks in the Agricultural Health Study. Occup. Environ. Med. 2011, 68, 537–541.
- Coble, J.; Thomas, K.W.; Hines, C.J.; Hoppin, J.A.; Dosemeci, M.; Curwin, B.; Lubin, J.H.; Beane Freeman, L.E.; Blair, A.; Sandler, D.P.; Alavanja, M.C. An updated algorithm for estimation of pesticide exposure intensity in the agricultural health study. Int. J. Environ. Res. Public Health. 2011, 8, 4608–4622.
- Agricultural Health Study. Agricultural Health Study Questionnaires & Study Data. Available at http://aghealth.nih.gov/collaboration/questionnaires.html (accessed 28 November 2015). 1996.
- Jurek, A.M.; Greenland, S.; Maldonado, G. How far from non-differential does exposure or disease misclassification have to be to bias measures of association away from the null? Int. J. Epidemiol. 2008, 37, 382–385.
- Jurek, A.M.; Greenland, S.; Maldonado, G.; Church, T.R. Proper interpretation of non-differential misclassification effects: expectations vs observations. Int. J. Epidemiol. 2005, 34, 680–687.
- Weichenthal, S.; Moase, C.; Chan, P. A review of pesticide exposure and cancer incidence in the Agricultural Health Study cohort. Environ. Health Perspect. 2010, 118, 1117–1125.
- Alavanja, M.C.; Hofmann, J.N.; Lynch, C.F.; Hines, C.J.; Barry, K.H.; Barker, J.; Buckman, D.W.; Thomas, K.; Sandler, D.P.; Hoppin, J.A.; Koutros, S.; Andreotti, G.; Lubin, J.H.; Blair, A.; Beane Freeman, L.E. Non-Hodgkin lymphoma risk and insecticide, fungicide and fumigant use in the Agricultural Health Study. PLoS One. 2014, 9, e109332.
- Pearce, N.; McLean, D. Agricultural exposures and non-Hodgkin's lymphoma. Scand. J. Work Environ. Health. 2005, 31(Suppl 1), 18–25; discussion 15–17.
- McDuffie, H.H.; Pahwa, P.; Spinelli, J.J.; McLaughlin, J.R.; Fincham, S.; Robson, D.; Dosman, J.A.; Hu, J. Canadian male farm residents, pesticide safety handling practices, exposure to animals and non-Hodgkin's lymphoma (NHL). Am. J. Ind. Med. 2002, Suppl 2, 54–61.
- Fritschi, L.; Johnson, K.C.; Kliewer, E.V.; Fry, R.; Canadian Cancer Registries Epidemiology Research, G. Animal-related occupations and the risk of leukemia, myeloma, and non-Hodgkin's lymphoma in Canada. Cancer Causes Control. 2002, 13, 563–571.
- Khuder, S.A.; Mutgi, A.B.; Schaub, E.A.; Tano, B.D. Meta-analysis of Hodgkin's disease among farmers. Scand. J. Work Environ. Health. 1999, 25, 436–441.
- Pahwa, P.; McDuffie, H.H.; Dosman, J.A.; Robson, D.; McLaughlin, J.R.; Spinelli, J.J.; Fincham, S. Exposure to animals and selected risk factors among Canadian farm residents with Hodgkin's disease, multiple myeloma, or soft tissue sarcoma. J. Occup. Environ. Med. 2003, 45, 857–868.
- Perrotta, C.; Staines, A.; Cocco, P. Multiple myeloma and farming. A systematic review of 30 years of research. Where next? J. Occup. Med. Toxicol. 2008, 3, 27.
- Keller-Byrne, J.E.; Khuder, S.A.; Schaub, E.A. Meta-analysis of leukemia and farming. Environ. Res. 1995, 71, 1–10.
- Morton, L.M.; Slager, S.L.; Cerhan, J.R.; Wang, S.S.; Vajdic, C.M.; Skibola, C.F.; Bracci, P.M.; de Sanjose, S.; Smedby, K.E.; Chiu, B.C.; Zhang, Y.; Mbulaiteye, S.M.; Monnereau, A.; Turner, J.J.; Clavel, J.; Adami, H.O.; Chang, E.T.; Glimelius, B.; Hjalgrim, H.; Melbye, M.; Crosignani, P.; di Lollo, S.; Miligi, L.; Nanni, O.; Ramazzotti, V.; Rodella, S.; Costantini, A.S.; Stagnaro, E.; Tumino, R.; Vindigni, C.; Vineis, P.; Becker, N.; Benavente, Y.; Boffetta, P.; Brennan, P.; Cocco, P.; Foretova, L.; Maynadie, M.; Nieters, A.; Staines, A.; Colt, J.S.; Cozen, W.; Davis, S.; de Roos, A.J.; Hartge, P.; Rothman, N.; Severson, R.K.; Holly, E.A.; Call, T.G.; Feldman, A.L.; Habermann, T.M.; Liebow, M.; Blair, A.; Cantor, K.P.; Kane, E.V.; Lightfoot, T.; Roman, E.; Smith, A.; Brooks-Wilson, A.; Connors, J.M.; Gascoyne, R.D.; Spinelli, J.J.; Armstrong, B.K.; Kricker, A.; Holford, T.R.; Lan, Q.; Zheng, T.; Orsi, L.; Dal Maso, L.; Franceschi, S.; La Vecchia, C.; Negri, E.; Serraino, D.; Bernstein, L.; Levine, A.; Friedberg, J.W.; Kelly, J.L.; Berndt, S.I.; Birmann, B.M.; Clarke, C.A.; Flowers, C.R.; Foran, J.M.; Kadin, M.E.; Paltiel, O.; Weisenburger, D.D.; Linet, M.S.; Sampson, J.N. Etiologic heterogeneity among non-Hodgkin lymphoma subtypes: the InterLymph Non-Hodgkin Lymphoma Subtypes Project. J. Natl. Cancer Inst. Monogr. 2014, 2014, 130–144.
- Becker, N. Epidemiology of multiple myeloma. Recent Results Cancer Res. 2011, 183, 25–35.
- Glaser, S.L.; Chang, E.T.; Clarke, C.A.K., T.H. Chapter 1. Epidemiology. In Hodgkin Lymphoma. A Comprehensive Overview. 2nd ed.; Engert, A.; Younes, A. Eds.; Springer: New York, 2015; 3–26.
- Linet, M.S.; Devesa, S.S.; Morgan, G.J. Chapter 44. The Leukemias. In Cancer Epidemiology and Prevention, 3rd Ed.; Schottenfeld, D.; Fraumeni, J.F., Jr. Eds.; Oxford University Press: New York, 2006; 841–871.
- Stiernstrom, E.L.; Holmberg, S.; Thelin, A.; Svardsudd, K. A prospective study of morbidity and mortality rates among farmers and rural and urban nonfarmers. J. Clin. Epidemiol. 2001, 54, 121–126.
- McCurdy, S.A.; Carroll, D.J. Agricultural injury. Am. J. Ind. Med. 2000, 38, 463–480.
- Kavvoura, F.K.; Liberopoulos, G.; Ioannidis, J.P. Selection in reported epidemiological risks: an empirical assessment. PLoS Med. 2007, 4, e79.
- Pahwa, M.; Harris, S.A.; Hohenadel, K.; McLaughlin, J.R.; Spinelli, J.J.; Pahwa, P.; Dosman, J.A.; Blair, A. Pesticide use, immunologic conditions, and risk of non-Hodgkin lymphoma in Canadian men in six provinces. Int. J. Cancer. 2012, 131, 2650–2659.
- Acquavella, J.; Farmer, D.; Cullen, M.R. A case-control study of non-Hodgkin lymphoma and exposure to pesticides. Cancer. 1999, 86, 729–731.
- Wester, R.C.; Melendres, J.; Sarason, R.; McMaster, J.; Maibach, H.I. Glyphosate skin binding, absorption, residual tissue distribution, and skin decontamination. Fundam. Appl. Toxicol. 1991, 16, 725–732.
- Acquavella, J.F.; Alexander, B.H.; Mandel, J.S.; Gustin, C.; Baker, B.; Chapman, P.; Bleeke, M. Glyphosate biomonitoring for farmers and their families: results from the Farm Family Exposure Study. Environ. Health Perspect. 2004, 112, 321–326.
- Niemann, L.; Sieke, C.; Pfeil, R.; Solecki, R. A critical review of glyphosate findings in human urine samples and comparison with the exposure of operators and consumers. J. Verbr. Lebensm. 2015, 10, 3–12.
- U.S. EPA. Glyphosate (CASRN 1071-83-6). Integrated Risk Information System. Carcinogenicity Assessment for Lifetime Exposure, 1993. Available at http://www.epa.gov/iris/subst/0057.htm. (accessed Nov 2015).
- Kier, L.D.; Kirkland, D.J. Review of genotoxicity studies of glyphosate and glyphosate-based formulations. Crit. Rev. Toxicol. 2013, 43, 283–315.
- Greim, H.; Saltmiras, D.; Mostert, V.; Strupp, C. Evaluation of carcinogenic potential of the herbicide glyphosate, drawing on tumor incidence data from fourteen chronic/carcinogenicity rodent studies. Crit. Rev. Toxicol. 2015, 45, 185–208.
- Fasal, E.; Jackson, E.W.; Klauber, M.R. Leukemia and lymphoma mortality and farm residence. Am. J. Epidemiol. 1968, 87, 267–274.
- Milham, S., Jr. Leukemia and multiple myeloma in farmers. Am. J. Epidemiol. 1971, 94, 507–510.
- Sandin, S.; Hjalgrim, H.; Glimelius, B.; Rostgaard, K.; Pukkala, E.; Askling, J. Incidence of non-Hodgkin's lymphoma in Sweden, Denmark, and Finland from 1960 through 2003: an epidemic that was. Cancer Epidemiol. Biomarkers Prev. 2006, 15, 1295–1300.
- Holford, T.R.; Zheng, T.; Mayne, S.T.; McKay, L.A. Time trends of non-Hodgkin's lymphoma: are they real? What do they mean? Cancer Res. 1992, 52, 5443s–5446s.
- Thygesen, L.C.; Nielsen, O.J.; Johansen, C. Trends in adult leukemia incidence and survival in Denmark, 1943–2003. Cancer Causes Control. 2009, 20, 1671–1680.
- Hjalgrim, H. On the aetiology of Hodgkin lymphoma. Dan. Med. J. 2012, 59, B4485.
- Hirabayashi, Y.; Katanoda, K. Comparison of time trends in multiple myeloma incidence (1973–1997) in East Asia, Europe and United States, from Cancer Incidence in Five Continents, Vols IV-VIII. Jpn. J. Clin. Oncol. 2008, 38, 720–721.
- Morton, L.M.; Wang, S.S.; Devesa, S.S.; Hartge, P.; Weisenburger, D.D.; Linet, M.S. Lymphoma incidence patterns by WHO subtype in the United States, 1992–2001. Blood. 2006, 107, 265–276.
- Button, K.S.; Ioannidis, J.P.; Mokrysz, C.; Nosek, B.A.; Flint, J.; Robinson, E.S.; Munafo, M.R. Power failure: why small sample'- size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 2013, 14, 365–376.
Appendix
Literature search methods
The authors conducted a search of MEDLINE via PubMed using the following search string, which includes Chemical Abstracts Service (CAS) Registry Numbers for glyphosate and its salts:
(glyphosat* OR glifosat* OR glyfosat* OR gliphosat* OR Roundup OR Round-up OR 1071-83-6 OR 38641-94-0 OR 70901-12-1 OR 39600-42-5 OR 69200-57-3 OR 34494-04-7 OR 114370-14-8 OR 40465-66-5 OR 69254-40-6 OR (aminomethyl w phosphonic*) OR 1066-51-9 OR pesticid* OR herbicid* OR organophosphorus compounds [MeSH] OR pesticides [MeSH] OR herbicides [MeSH]) AND (leukemi* OR leukaemi* OR lymphoma* OR NHL OR lymphopoietic OR hemato* OR hematopoie* or hematolog* OR lymphoid OR myeloid OR myeloma OR leukemia [MeSH] OR lymphoma [MeSH] OR multiple myeloma [MeSH]) AND (cases OR controls OR case-control OR cohort).
As of June 23, 2015, this search string identified a total of 11,755 articles in PubMed. We conducted additional targeted searches in PubMed, Web of Science, and Google Scholar using simpler keyword combinations such as (glyphosate AND lymphoma), (pesticides AND lymphoma), and (herbicides AND lymphoma). References also were identified from the bibliographies of recent review articles.
Altogether, a total of 12,709 articles were identified from these combined sources (). Based on a review of titles and abstracts, 321 articles were identified as potentially containing estimates of the association between glyphosate exposure and LHC risk, and were obtained for further evaluation. Forty-seven of these articles contained the word “glyphosate” or “Roundup” (or alternative spellings of these terms) in the text; as specified earlier, articles that did not mention glyphosate were ineligible for inclusion. Following a review of the full text of each of the 47 articles mentioning glyphosate, 19 articles were ultimately deemed eligible for inclusion.
Figure A1. Flow chart of literature identification and selection process.
