Abstract
Increasing the transparency of the evidence base behind health interventions such as pharmaceuticals, biologics, and medical devices, has become a major point of critique, conflict, and policy focus in recent years. Yet the lack of publicly available information regarding the immunogenicity assays upon which many important, widely used vaccines are based has received no attention to date. In this paper we draw attention to this critical public health problem by reporting on our efforts to secure vaccine assay information in respect of 10 vaccines through Canada's access to information law. We argue, under Canadian law, that the public health interest in having access to the methods for these laboratory procedures should override claims by vaccine manufacturers and regulators that this information is proprietary; and, we call upon several actors to take steps to ensure greater transparency with respect to vaccine assays, including regulators, private firms, researchers, research institutions, research funders, and journal editors.
Why Transparency of Vaccine Assays is Important
Today, members of the scientific community, publicly funded research institutions, industry, and regulatory agencies face higher expectations for transparency and accountability.Citation1-4 Registration of clinical trials is now legally required in several jurisdictions.Citation5-6 Due to poor compliance,Citation7-11 many are pushing for disclosure of full “clinical study reports” and anonymized patient-level data.Citation12-14 Whether government regulators such as the European Medicines Agency (EMA) or the United States’ Food and Drug Administration (FDA) will follow suit remains to be seen.Citation15 In the meantime, we draw attention to an issue that has yet to enter the policy conversation: open access to the scientific methods used to determine the immunogenicity of vaccines.
Vaccine efficacy and effectiveness can be quantified directly by evaluating protection from infection or clinical illness, or assessed indirectly by measuring immune responses (immunogenicity) to a given vaccine. Depending on the particular pathogen, vaccine, and other factors, immune responses can correlate, more or less accurately, with clinical protection. Although direct measurement of clinical protection is preferred, so-called “correlates of protection” are substituted when an efficacy trial would be inordinately large, expensive, not feasible because of the rarity of outcomes, or not ethically permissible when comparable vaccines are already available.
Quantitative assessment of serum antibodies before and after vaccination is the most commonly used correlate of protection, and often the only outcome accepted by regulatory authorities. Serologic measures are diverse, and the evidence in support of their validity as indirect measures of efficacy varies.Citation16-20 Accurate and consistent findings in repeated studies across laboratories using such assays are thus critical to the successful application of correlates of protection.
However, efforts to standardize methods across settings cannot occur without access to the specific testing methodology, and vaccine manufacturers frequently treat the methods used to support the regulatory approval of vaccines as “proprietary.” In addition they may use reagents that are developed in house, are not available commercially, and are not described in sufficient detail to permit others to replicate the assay or findings.
This proprietary practice precipitates several problems. First, the lack of access to the reagents or standard operating procedures (SOPs) precludes independent validation of reported results. Second, the lack of availability to assays that are, in some circumstances, considered reference assays, precludes the ability of diagnostic laboratories to determine the performance characteristics of commercially available assays to measure antibody levels. Third, if summaries of the validation data that were generated when the assays were approved are not publicly available, there is no way to ensure that methods used in diagnostic laboratories are comparable. Fourth, treating the assays as proprietary precludes application of the assay to new populations that may not have been evaluated in the original studies (e.g. the immune compromised, aboriginal peoples).
This proprietary practice is not universally observed. A few manufacturers have taken an intermediate step of enabling technological transfer of assay methodology for their proprietary vaccines to select groups (e.g., Novartis’ meningococcal B vaccine)Citation21-22 in order to ensure quality assurance by training specific individuals to perform the assay. However, details about the assays tied to several widely licensed vaccines (see ) remain entirely secret. We argue that the public interest in having the laboratory methodology used to determine the correlates of protection and summaries of validation data open to independent scrutiny is a threat to public health and clearly outweighs manufacturers’ competing business interests.
Table 1. Ten vaccines licensed by Health Canada encompassed in our access to information request
Challenging Proprietary Practices in Vaccine Research and Regulation
We sought to challenge the status quo using Canada's Access to Information Act (ATI Act).Citation23 In a letter dated March 12, 2012, we requested from Health Canada information that we believe to be contained in the SOP accompanying 10 vaccines that have been licensed for sale and use in Canada. Specifically, we requested 2 types of information from each vaccine's SOP: (a) the detailed assay protocols (i.e. methods) for the serological tests that were used to measure the immunogenicity of the vaccine in support of its market authorization; and, (b) the assay validation data that was submitted to Health Canada by the manufacturer. (See )
Table 2. Vaccine assay information encompassed by the access to information request.
To facilitate processing of our request, we indicated that assay protocols and validation data connected to several vaccines were likely to cross-reference one another and we did not require multiple copies of the same information. Second, we excluded information about the manufacturing processes as they have been characterized as a potential “trade secret” in previous court decisions.Citation24 Third, we anticipated that Health Canada would likely regard the information as “third party information” (i.e., a) trade secrets, b) financial, commercial, scientific or technical information, c) information that could result in financial or competitive harm, or d) information that could interfere with contractual negotiations by a third party) and thus exempt from disclosure under the ATI Act. Therefore, we invoked a provision in the ATI Act that allows the government institution to disclose third party information (save for trade secrets) if it is deemed clearly in the public interest.
After considerable delay and complaints to the Information Commissioner of Canada about Health Canada's non-response to our request, on August 19, 2013 we finally received a response to our ATI request. The letter from Health Canada, which enclosed a 272-page PDF document, noted that “some records, or portions of records, are withheld from disclosure pursuant to” sections 19(1) (personal information), 20(1)(b) (financial, commercial, scientific or technical information), and 20(1)(c) (information that could result in financial loss or gain, or prejudice the competition position of a third party) of the ATI Act.
Our review revealed that the disclosure provides no information regarding 8 of the 10 vaccines encompassed in our ATI request. Two HPV vaccines included in our request, Cervarix (manufacturer: GSK) and Gardasil (manufacturer: Merck) were included; however, most of the 272 pages are blank apart from a notation indicating that the information in question was withheld on the basis of the above exemptions. (See Supplementary Information for access to the complete disclosure) Other pages in the disclosure contain table of contents, names of figures and tables, and reference to validation data but all of the substantive information is redacted from those pages. At most, these pages suggest that one of the manufacturers (GSK) did in fact submit appropriate validation data to Health Canada during the regulatory review process. But it is not possible to evaluate or independently replicate those findings without further information about the procedures and correlates of protection that were used.
Health Canada's response was not surprising. Since it began publishing access to information data in 1991, the regulator has relied upon third party information exemptions more often than any other type of exemption under the ATI Act to deny, in whole or in part, access to information (See ). Nevertheless, we contend that treating vaccine assays as proprietary is clearly contrary to the public interest.
Figure 1. Number of access to information requests processed and partially or fully denied by Health Canada, and number of exemptions relied upon to partially or fully deny such requests, 1991-2012. *Figure 1 was compiled using information contained in Health Canada's annual reports regarding access to information requests, which are publicly available. Data for fiscal years 1992-1993, 1996-1997, 1997-1998, and 1998-1999 is not publicly available, and only some data is available for fiscal years 2007-2008, 2009-2010, and 2011-2012).
A Legal Argument for Greater Openness
It is important to note at the outset that any personal information captured by our request was inadvertent. We do not contest Health Canada's decision to withhold the names of the individuals who carried out and reviewed the assay protocols and validation data in question. Our argument concerns Health Canada's decision to exempt all substantive information regarding the assay protocols and validation data from disclosure because it believed disclosure could harm a manufacturer's competitive position or deemed the information to fall within the scope of confidential information.
The business harm exemption should not apply to the information we requested. Conceivably, a manufacturer may assert that disclosure of the information may preclude its ability to commercialize an assay; thus disclosure of the assay protocol and validation data could result in financial loss, competitive harm, or compromise business negotiations. However, courts have repeatedly demanded more than mere assertions of harm (to competitive position) by companies, Citation25 and there are no signs that the manufacturers implicated by our request are attempting to commercialize the assays accompanying their vaccines. As noted below, the assay associated with Merck's zoster virus vaccine provides a telling example to the contrary.
Conversely, the statutory exemption pertaining to confidential information appears to be the most cogent reason for denying disclosure of assay protocols and validation data. Health Canada referenced this exemption throughout the (largely blank) 272-page PDF it provided. Under Canadian law, to qualify for this exemption the information must be “(i) financial, commercial, scientific or technical information; (ii) confidential and consistently treated in a confidential manner by the third party; and (iii) supplied to a government institution by a third party.”Citation25 Each of these criteria appears to be met in this case.
This is why, in our original request to Health Canada, we invoked the “public interest override” contained in the ATI Act. The override allows for the disclosure of information notwithstanding its confidential nature if it “would be in the public interest as it relates to public health” and it “clearly outweighs” any financial loss or gain to a third party, or impact on its competitive position or any contractual or other negotiations.
Although Canadian courts have had little occasion to offer guidance around the scope of this kind of public interest overrideCitation25-27 there are several reasons why it should apply in the case of vaccine assays.
First, the 10 vaccines encompassed in our request have tremendous public health importance. Hundreds of thousands of Canadians are vaccinated on the assumption that the studies submitted to the regulator accurately estimate the degree of clinical protection against the various pathogens targeted by the vaccines in our request. If those assays are somehow suboptimal, the measurements of antibody response may lead to inaccurate interpretations about vaccine efficacy, and thus inaccurate estimates of protection against serious infectious diseases. Open access to vaccine assays can, over time, help ensure decisions by health care payers are properly informed of a given vaccine's strengths and limitations as well as enable health care providers to have a better understanding of whether individuals and populations are actually clinically protected against a pathogen.
Second, regulators do not replicate the assay protocols or verify the validation data submitted by vaccine manufacturers. Rather, regulators simply review the protocols and data as submitted. Given the inherent challenges of utilizing serological measures as correlates of protection and demonstrated problems of vaccine assay standardization, opening up assays to independent scrutiny is essential to ensuring that the chosen correlates of protection do, in fact, provide adequate measures of vaccine safety and efficacy. Independent analysis will become increasingly important in the future as more vaccines are licensed on the strength of correlates of protection.
Third, on a related note, manufacturers have been known to manipulate the evidence base behind other health products such as pharmaceuticals in order to secure regulatory approval. GlaxoSmithKline's selective data reporting of “study 329,” which masked risks of increased suicidal ideation among adolescents taking a course of paroxetine (Paxil/seroxat), is perhaps the most infamous example.Citation28-29 There are limits to regulators’ ability to detect this thus at least one major regulator, the EMA, is contemplating open access to anonymized patient-level clinical trial data, in part, to improve regulatory decision-making.Citation30 There is a clear public health interest in improving access to clinical trial data. Precisely the same logic should apply to vaccine assays.
Fourth, disclosure of these diagnostic assays also carries benefits beyond enhancing decision-making of health care payers, providers, patients, and regulators. Specifically, it has benefits for research and research participants. At present, specific sub-populations (e.g. aboriginal populations, immune-compromised individuals) that were not included in the trials that supported licensure of these vaccines will be denied the benefit of these vaccines (assuming the benefits are confirmed) unless and until access to the correlates of protection is provided. That is, researchers planning to study the safety and efficacy of these vaccines in those sub-populations cannot do so absent access to the assays. Denying access to the assay information undermines equitable access to important public health interventions for particular populations. It also precludes efforts to assess the duration of protection over time.
Fifth, and finally, although it is possible that disclosure of the assay information may preclude manufacturers’ ability to commercialize an assay, there are no signs that the manufacturers implicated by our request are attempting to do so. We do not think the speculative possibility of commercializing a vaccine assay—a task that manufacturers have not undertaken despite years of assay use—should preclude disclosure, especially given the public health importance of having access to this information.
An informative example of this final point is the assay associated with Merck's varicella-zoster vaccine. Measuring immunity against varicella zoster virus (VZV) is challenging. While the fluorescent-antibody-to-membrane-antigen (FAMA) assay has been considered the gold standard for measuring immunity to VZV, this assay is cumbersome and is not amenable to automation or high-throughput testing. Merck developed a gpELISA, an enzyme-linked immunosorbent assay based on highly purified glycoproteins (gp) from VZV, which is more sensitive than FAMA. This gpELISA has been the standard method Merck has used to measure immunity in all of its subsequent VZV vaccine trials.Citation31-33 While the methods were reasonably detailed in scientific publications, the validation of the assay was limited. In 2006, Merck published a paper on a second-generation assay which optimized the assay further and improved its performance characteristics.Citation33 In this paper, Merck detailed the extensive validation undertaken; although the methods described were standard ELISA methodology, the antigen used and the standard operating procedures for completing the assay were considered proprietary, i.e. not publicly available. While Merck has granted limited access to the reagents and its standard operating procedures to reference laboratories in the past, Canada no longer has access to this method. As a result of Merck's decision to treat the gpELISA as proprietary, the majority of commercially available assays have not been validated against the gpELISACitation34-35 and their performance in determining VZV immunity is quite variable. Meanwhile, the Canadian Immunization Guide has recently recommended serologic testing for all health care workers who do not have previous VZV infection documented by a health care provider. As the cohort that has received VZV vaccine as part of the primary vaccination series enters the health care workforce, the shortcomings of the commercially available enzyme immunoassays in terms of detecting vaccine-induced immunity will translate into many more vaccinations that would be necessary because diagnostic laboratories do not have access to the gpELISA or its antigen to use for screening or as a reference method for the validation of other enzyme immunoassays. Nevertheless, despite the regulator's endorsement of Merck's gpELISA and a ready-made market of health care workers required to be protected against VZV, in a 10-year period, Merck has neither sought to commercialize its assay nor shown any signs of doing so.
From Argument to Action
The issue of transparency in the development and regulation of health products such as drugs and vaccines presently commands a great deal of attention. Although currently focused on clinical trial registration requirements and access to clinical trial data, we believe that access to vaccine assay protocols and validation data is critical from public health and ethical perspectives. Without access to that information, it is not possible to independently replicate and evaluate the assays used to establish the safety and efficacy of widely utilized vaccines.
We assert that Canadian law should not bar access to vaccine assay protocols and validation data despite manufacturers’ decision to treat this information as proprietary. On the contrary, the public interest override contained in Canada's access to information law should allow the regulator to disclose the information. To the extent they are available, similar flexibilities in other jurisdictions’ laws should be invoked to support disclosure by other regulators.
The inevitable delays in the access to information process and the public health importance of the vaccines included in our request and the fact that future vaccines are increasingly likely to be granted regulatory approval on the strength of serological correlates of protection requires more upfront transparency. Researchers, research institutions, research funders, vaccine manufacturers, and national regulators alike should proactively disclose assay protocols and validation data. As a start, we think that entities such as the International Committee of Medical Journal Editors and research funding organizations could integrate a requirement to make assay protocols and summary validation data into their policies regarding authorship and open access.Citation36-37 With the support of their institutions, researchers involved in vaccine trials could stipulate a commitment to make protocols and validation data open to independent scrutiny, and regulators should be legally empowered to enforce such transparency as part of the regulatory process. Transparency may come in different forms. Ultimately, to ensure the putative public health benefits of a vaccine are achieved, meaningful independent scrutiny of vaccine assay protocols and validation data must occur.
Disclosure of Potential Conflicts of Interest
The lead author (MH) is a member of the Health Policy Translation Group of the Canadian Centres for Vaccinology (CCfV), which has carried out a number of clinical trials in partnership with vaccine manufacturers. MH has not been involved, in any way, with the conduct of these trials. The remaining authors have, however, received contract funding from numerous vaccine manufacturers to undertake vaccine clinical trials. Specifically, TH is currently a co-investigator on an investigator-driven collaborative agreement with GlaxoSmithKline and has had similar agreements with Immunovaccine and Microbix in the past 5-10 years. SH is an investigator in multiple clinical trials related to multiple vaccines funded by vaccine manufacturers including Sanofi Pasteur, GlaxoSmithKline, Merck, Pfizer, Novartis, PREVENT, and Folia. JL's institution has received funding for the conduct of clinical trials from Sanofi Pasteur, Merck, GlaxoSmithKline, Pfizer, Novartis, Immunovaccine, the Canadian Institutes of Health Research and for advisory boards (Sanofi Pasteur, Novartis, Merck, GlaxoSmithKline). JL serves on committees advisory to provincial and federal governments in Canada for vaccine policy. JL currently holds the Canadian Institutes of Health – GlaxoSmithKline Chair in Pediatric Vaccinology (2013–2018). All funding for this Chair is paid to Dalhousie University. As well, SH has served on ad hoc scientific advisory boards for GlaxoSmithKline, Novartis, Sanofi Pasteur, Pfizer and Merck and is a member of the Scientific Advisory Board for ImmunoVaccine. He has also served on advisory committees and working groups of the Advisory Committee on Immunization Practice of the US Centers for Disease Control, the Canadian National Advisory Committee on Immunization, the Nova Scotia Department of Health and Wellness, and the World Health Organization SAGE. He has served on the Data Monitoring Committee for Medicago.
References
- Asamoah AK, Sharfstein JM. Transparency at the food and drug administration. N Engl J Med 2010; 362(25):2341-3; PMID:20484392
- Drazen JM. Transparency for clinical trials–the tEST act. N Engl J Med 2012; 367(9):863-4; PMID:22873430
- The Lancet. European medicines agency–more transparency needed. Lancet 2010; 375(9728):1753; PMID:20494710
- Psaty BM, Furberg CD. Rosiglitazone and cardiovascular risk. N Engl J Med 2007; 356(24):2522-4; PMID:17517854
- United States. Food and drug administration amendments act of 2007, Pub. L. No. 110-85, § 801, 121 Stat. 823, 904-22 (2007)
- European Commission, Communication from the commission regarding the guideline on the data fields contained in the clinical trials database provided for in article 11 of directive 2001/20/EC to be included in the database on medicinal products provided for in article 57 of regulation (EC) No 726/2004, 2008 OFFICIAL J. EUR. UNION C 168/3, C 168/3 (2008), available at: (last visited: 22, 2013).
- Viergever RF, Ghersi D. The quality of registration of clinical trials. PLoS ONE 2011; 6(2):e14701; PMID:21383991
- Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on clinical trials.gov: cross sectional study. BMJ 2012; 344(jan03 1):d7373-d7373; PMID:22214756
- Mathieu S BI. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA 2009; 302(9):977-84; PMID:19724045
- Sekeres M, Gold JL, Chan A-W, Lexchin J, Moher D, Van Laethem MLP, Maskalyk J, Ferris L, Taback N, Rochon PA. Poor reporting of scientific leadership information in clinical trial registers. PLoS ONE 2008; 3(2):e1610; PMID:18286168
- Law MR, Kawasumi Y, Morgan SG. Despite law, fewer than one in eight completed studies of drugs and biologics are reported on time on clinical trials.gov. Health Aff 2011; 30(12):2338-2345; PMID:22147862
- Wieseler B, Wolfram N, McGauran N, Kerekes MF, Vervölgyi V, Kohlepp P, Kamphuis M, Grouven U. Completeness of reporting of patient-relevant clinical trial outcomes: comparison of unpublished clinical study reports with publicly available ata. PLoS Med 2013; 10(10):e1001526; PMID:24115912
- Gopal RK, Yamashita TE, Prochazka AV. Research without results: inadequate public reporting of clinical trial results. Contemp Clin Trials 2012; 33(3):486-91; PMID:22342449
- Doshi P, Jefferson T, Del Mar C. The Imperative to share clinical study reports: recommendations from the tamiflu experience. PLoS Med 2012; 9(4):e1001201; PMID:22505850
- Herder M. Government regulators must steward drug transparency. Nature Med 2014; 20(8):806; PMID:25100518
- Plotkin SA. Correlates of protection induced by vaccination. Clin Vaccine Immunol 2010; 17(7):1055-65; PMID:20463105; http://dx.doi.org/10.1128/CVI.00131-10
- Plotkin SA, Gilbert PB. Nomenclature for immune correlates of protection after vaccination. Clin Infect Dis 2012; 54(11):1615-7; PMID:22437237; http://dx.doi.org/10.1093/cid/cis238
- Plotkin SA. Complex correlates of protection after vaccination. Clin Infect Dis 2013; 56(10):1458-65; PMID:23386629; http://dx.doi.org/10.1093/cid/cit048
- Plotkin SA. Vaccines: correlates of vaccine-induced immunity. Clin Infect Dis 2008; 47(3):401-9; PMID:18558875; http://dx.doi.org/10.1086/589862
- Plotkin SA. Immunologic correlates of protection induced by vaccination. Pediatr Infect Dis J 2001; 20(1):63-75; PMID:11176570
- Donnelly J, Medini D, Boccadifuoco G, Biolchi A, Ward J, Frasch C, Moxon ER, Stella M, Comanducci M, Bambini S, et al. Qualitative and quantitative assessment of meningococcal antigens to evaluate the potential strain coverage of protein-based vaccines. PNAS 2010; 20; 201013758; PMID:20962280
- Plikaytis BD, Stella M, Boccadifuoco G, DeTora LM, Agnusdei M, Santini L, Brunelli B, Orlandi L, Simmini I, Giuliani M, et al. Interlaboratory standardization of the sandwich enzyme-linked immunosorbent assay designed for MATS, a rapid, reproducible method for estimating the strain coverage of investigational vaccines. Clin Vaccine Immunol 2012; 19(10):1609-17; PMID:22875603
- Canada. Access to Information Act, R.S.C., 1985, c. A-1
- AstraZeneca Canada Inc. v. Canada (Minister of Health), 2005 FC 189 at para. 65 (supplemental reasons 2005 FC 648, aff’d 2006 FCA 241).
- Merck Frosst Canada Ltd. v. Canada (Health), [2012]1 S.C.R. 23.
- Order PO-2557; Ontario (Ministry of the Environment), [2007]O.I.P.C. No. 61.
- Canada Packers Inc. v. Canada (Minister of Agriculture), [1989]1 F.C. 47 (F.C.A.)
- Jureidini JN, McHenry LB, Mansfield PR. Clinical trials and drug promotion: selective reporting of study 329. Int J Risk Saf Med 2008; 20(1):73-81
- McGoey L, Jackson E. Seroxat and the suppression of clinical trial data: regulatory failure and the uses of legal ambiguity. J Med Ethics 2009; 35(2):107-12; PMID:19181884
- European medicines agency, Publication and access to clinical-trial data, Policy/0070: Draft for public consultation, Jun. 24, 2013, available at: http://www.ema.europa.eu/docs/en_GB/document_library/Other/2013/06/WC500144730.pdf ( last visited: Aug. 22, 2013)
- Wasmuth EH, Miller WJ. Sensitive enzyme-linked immunosorbent assay for antibody to varicella-zoster virus using purified VZV glycoprotein antigen. J Med Virol 1990; 32(3):189-93; PMID:2177782
- Provost PJ, Krah DL, Kuter BJ, Morton DH, Schofield TL, Wasmuth EH, White CJ, Miller WJ, Ellis RW. Antibody assays suitable for assessing immune responses to live varicella vaccine. Vaccine 1991; 9(2):111-6; PMID:1647574
- Hammond O, Wang Y, Green T, Antonello J, Kuhn R, Motley C, Stump P, Rich B, Chirmule N, Marchese RD. The optimization and validation of the glycoprotein ELISA assay for quantitative varicella-zoster virus (VZV) antibody detection. J Med Virol 2006; 78(12):1679-87; PMID:17063506
- Maple CPA, Gunn A, Sellwood J, Brown DW, Gray JJ. Comparison of fifteen commercial assays for detecting varicella zoster virus IgG with reference to a time resolved fluorescence immunoassay (TRFIA) and the performance of two commercial assays for screening sera from immunocompromised individuals. J Virol Methods 2009; 155(2):143-9; PMID:18996415
- Sauerbrei A, Schäfler A, Hofmann J, Schacke M, Gruhn B, Wutzler P. Evaluation of three commercial varicella-zoster virus IgG enzyme-linked immunosorbent assays in comparison to the fluorescent-antibody-to-membrane-antigen test. Clin Vaccine Immunol 2012; 19(8):1261-8; PMID:22718131; http://dx.doi.org/10.1128/CVI.00183-12
- International Committee of Medical Journal Editors, Uniform Requirements for manuscripts submitted to biomedical journals: ethical considerations in the conduct and reporting of research: Authorship and Contributorship, 2013, available at: http://www.icmje.org/ethical_1author.html (last visited: Nov. 14, 2013).
- Canadian institutes of health research, CIHR Open Access Policy, Jan. 1, 2013, available at: http://www.cihr-irsc.gc.ca/e/32005.html (last visited: Nov. 14, 2013).