Recommendations on ELISpot Assay Validation by the GCC

Abstract

Gene therapy, cell therapy and vaccine research have led to an increased need to perform cellular immunity testing in a regulated environment to ensure the safety and efficacy of these treatments. The most common method for the measurement of cellular immunity has been Enzyme-Linked Immunospot assays. However, there is a lack of regulatory guidance available discussing the recommendations for developing and validating these types of assays. Hence, the Global CRO Council has issued this white paper to provide a consensus on the different validation parameters required to support Enzyme-Linked Immunospot assays and a harmonized and consistent approach to Enzyme-Linked Immunospot validation among contract research organizations.

Keywords:

• cell therapy
• cellular immunity
• ELISpot
• Enzyme-Linked Immunospot
• GCC
• gene therapy
• global CRO council in bioanalysis
• regulated bioanalysis
• vaccine
• validation

Background

The Global CRO Council in Bioanalysis (GCC) was created in 2010 as an independent global consortium bringing together many contract research organization (CRO) leaders to discuss various topics and challenges in scientific and regulatory issues related to bioanalysis [1]. Since its formation, the GCC has held regular meetings and published conference reports to share discussions and opinions [2–10]. White papers on specific topics of widespread interest in bioanalysis have also been published to provide unified GCC recommendations helpful to the global bioanalytical community [11–19].

Introduction

Gene therapy, cell therapy and vaccine research have led to an increased need to perform cellular immunity testing in a regulated environment to ensure the safety and efficacy of these treatments. Cellular immunity assays are more complex than traditional immunoassays due to the fact that they include cell culture and not traditional immuno-sandwich. This can result in assays that are less reproducible. Furthermore, cellular immunity assays must be sensitive enough to reliably detect potentially low levels of T-cell populations [20]. It is also known that the reliability of the results can be dependent on the experience of the operator, especially in the handling of primary blood cells [21]. Finally, the lack of appropriate reference standards and positive control samples, particularly those that mimic test samples, can be a challenge.

The most common method for the measurement of cellular immunity has been Enzyme-Linked Immunospot (ELISpot) assays; however, there is a lack of regulatory guidance available discussing the recommendations for developing and validating these types of assays. The available literature can provide examples of cellular immunity testing assays [22–24], but the Clinical and Laboratory Standards Institute (CLSI) published a request for clear guidance for validating these assays as long ago as 2004 [25]. Historically, bioanalysts have attempted to adapt bioanalytical method validation guidance documents [26,27] into a fit-for-purpose approach to method validation, but these documents do not consider ELISpot assays in scope and many parameters are not applicable. In an effort to provide specific recommendations for improving assay performance, white papers have been published [20,28–30], which, when considered together, can help bioanalysts who are validating ELISpot assays.

A survey was provided to representatives in the GCC in order to determine if any of the existing white paper recommendations are being applied in industry, or if other approaches are being used. This survey received 52 responses, and 35 respondents confirmed that they perform ELISpot assays at their organization. This white paper provides a summary of the results of the survey containing questions and answers on the different approaches to ELISpot validation (refer to Table 1), as well as a consensus on the different validation parameters required to support these assays and a harmonized, consistent approach to ELISpot validation among CROs.

Discussion

ELISpot assays are no longer used simply for research or exploratory purposes; survey results indicate that 76% of ELISpot methods are used for pre-clinical or clinical regulated bioanalysis. In fact, the majority of laboratories (>50%) that run regulated ELISpot assays follow good laboratory practice (GLP) or good clinical practice (GCP) regulations. Less than 25% of Clinical Laboratory Improvement Amendments (CLIA)/College of American Pathologists (CAP) laboratories use regulated ELISpot assays and no International Organization for Standardization (ISO)/good manufacturing practice (GMP) laboratories use these assays.

When queried on which reagents are considered “critical,” overwhelming consensus was reached that these include peripheral blood mononuclear cell (PBMC) (89% of respondents), detection antibodies (96% of respondents) and positive controls (89% of respondents). Other reagents could be considered critical depending on the assay (e.g., Streptavidin-AP, polyvinylidene fluoride [PVDF]–backed 96-well microplates) and should be indicated as such in the validation documentation. Consensus was also reached that wash buffers and dilution buffers are not considered critical. Furthermore, respondents overwhelmingly agreed that lot-to-lot bridging must be performed for ELISpot critical reagents.

Respondents were asked to indicate which parameters are being assessed during ELISpot assay validation. The survey results unanimously demonstrate that positive controls must be included in each run. Although the survey does not delineate the type of positive control, it is important to note that these controls can include mitogens such as calcium ionomycin, phorbol 12-myristate 13-acetate (PMA) or phytohemagglutinin (PHA) to determine PBMC functionality; peptide controls such as CEF or CEFT to determine presentation dependent activation; or superantigens such as Staphylococcus aureus enterotoxins. In addition, a responding PBMC donor or cell line can be used as a positive control to verify analytical test peptide responses [33]. Furthermore, consensus was reached that precision, sensitivity (LOD), specificity, ruggedness and robustness are required during validation. Additional parameters suggested, but without overwhelming agreement, included dilutional linearity, reportable range and sample stability. It was also interesting to note that only just over half of respondents use patient samples for ELISpot validation. Since patient samples may be limited or unavailable during early-stage development, alternative approaches may need to be taken to extrapolate and assess the utility of the test for clinical samples.

In order to determine the existing harmonization of criteria among those who perform ELISpot validations, respondents were asked to outline what criteria are applied to the evaluations. Three respondents ensure that the positive control is greater than a pre-defined threshold such as the negative control. One respondent required that the response is ≥30 SFU/well, allowing the reporting of both standard deviation (SD) and % CV. For wells with fewer than 30 spots, only SD should be reported. Precision for samples with a mean spot count of greater than 100 will be <25%. For samples with a mean spot count of >30 spots/well up to 100 spots/well, the % CV should be <50%. The remaining proposals for intra- and inter-assay precision varied between 20 and 30% without any mention of dependency on the number of spots per well, and one respondent used a criterion of ≤25% RSD. Finally, two respondents reported stability criteria of either ≤20% bias between each run or 30% CV between time points. Almost half of the 28 respondents who answered this question (46%) use existing recommendations by Maecker et al. [20], Janetzki et al. [28], Piccoli et al. [29] and/or Corsaro et al. [30]. Table 2 summarizes these recommendations.

The last question discussed the criteria for sample analysis. Several specified that positive and negative controls should be assayed on each plate and used for acceptance. Most respondents confirm that the same criteria as assay validation should be used.

Recommendations

Following the survey results, the GCC supports prior recommendations for ELISpot assay validation presented in Maecker et al. [20], Janetzki et al. [28], Piccoli et al. [29] and Corsaro et al. [30]. Table 2 summarizes these recommendations. Table 3 contains a summary of the additional GCC recommendations following this survey.

Conclusion

In an attempt to harmonize ELISpot validation, the GCC highly recommends the industry adopt the parameters and acceptance criteria provided in Table 3.

Future perspective

The GCC as a global organization will continue to provide recommendations on hot topics of global interest in bioanalysis. Please contact the GCC [34] for the exact date and time of future meetings, and for all membership information.

Table 1. GCC survey on ELISpot.

Question	Response
What is the intended use of your ELISpot assays?	Research exploratory: 59.5% Pre-clinical: 70.3% Clinical: 81.2%
Do you perform regulated ELISpot assays?	Yes: 75.7% No: 24.3% It should be noted that those who answered no to this question were forwarded directly to the question regarding critical reagents.
What are the percentages of ELISpot assays you perform in a regulated environment?	GLP or GCP labs: ∼50% CLIA/CAP labs: ∼25% ISO/GMP labs: 0%
What do you consider a “critical reagent” for ELISpot assays?	PBMC: 89.3% PVDF-backed 96-well microplate: 28.6% Detection antibody: 96.4% Streptavidin-AP: 32.1% BCIP/NBT: 14.3% Positive control: 89.3% Dilution buffers: 0% Wash buffer: 0% Other: 17.9% Other reagents included the diluent (in some instances), assay specific stimuli (e.g., peptide pools), novel therapeutics, peptides, antigens.
Do you use lot-to-lot bridging protocols for ELISpot critical reagents?	Yes: 89.3% No: 10.7%
Do you use positive controls for each run in ELISpot assays?	Yes: 100.0% No: 0.0%
What validation parameters do you use for ELISpot assays?	Precision: 96.4% Sensitivity (LOD): 85.7% Specificity: 78.6% Dilutional linearity: 46.4% Reportable range: 35.7% Ruggedness and robustness: 71.4% Other: 32.1% Other parameters included PBMC/splenocytes F/T stability, matrix stability, whole blood/frozen PBMC stability, plate imaging and sample stability, selectivity, parameters outlined in Corsaro et al. [30]
Do you use patient samples for ELISpot validation?	Yes 57.1% No 42.9%
What assay acceptance criteria do you use for ELISpot validation?	The following criteria were used by multiple organizations: • Criteria outlined in Piccoli et al. [29] and/or Corsaro et al. [30] (nine respondents) • Criteria outlined in Maecker et al. [20] (two respondents) • Criteria outlined in Janetzki et al. [28] (two respondents) • Replicate CV: 20% (one respondent) or 30% (three respondents) • Intra-assay precision: 30% (two respondents), 25% (two respondents), 20% (two respondents) • Inter-assay precision: 30% (three respondents), 25% (one respondent), 20% (two respondents) • Stability of whole blood or PBMCs: 30% (one respondent) or 20% (one respondent) • Positive control greater than negative control (two respondents)
What assay acceptance criteria do you use for ELISpot sample analysis?	The following criteria were used by multiple organizations: • Criteria outlined in Piccoli et al. [29] and/or Corsaro et al. [30] (seven respondents) • Replicate CV: 30% (three respondents), 20% (one respondent) • Positive stimulation greater than pre-defined threshold (ten respondents)

CAP: College of American Pathologists; CLIA: Clinical Laboratory Improvement Amendments; ELISpot: Enzyme-Linked Immunospot; GCP: Good clinical practice; GLP: Good laboratory practice; GMP: Good manufacturing practice; ISO: International Organization for Standardization; PBMC: Peripheral blood mononuclear cell; PVDF: Polyvinylidene fluoride.

Table 2. Summary of prior recommendations presented in Maecker et al. [20], Janetzki et al. [28], Piccoli et al. [29] and Corsaro et al. [30].

	Janetzki et al. [28] Piccoli et al. [29]	Maecker et al. [20]	Corsaro et al. [30]
SOP	• Counting method for apoptotic cells • Overnight rest of cells prior to plating and incubation • Human auditing during reading process • Adequate adjustments for technical artifacts • Training requirements	Does not discuss	Definition of a counting template can be useful
Serum	Pre-tested and optimized for low background:high signal ratio	Does not discuss	• Pre-screened to ensure lack of reactivity • One to two positive samples
Training	Only trained personnel to conduct assays	Does not discuss	Recommends Janetski et al. [28]
Replicates per sample	Six	Three to six	• Replicate variability criteria <twofold • Well acceptance criteria should be established
Sensitivity/LOD	• Antigen-specific spot counts per 2 × 10^5 PBMCs >10 • At least 3x as high as the background reactivity [24,31]	Two SDs above the mean of replicate negative control samples	• One sample to assess ULOD; ULOD ≤450 SFC/well • Media-only wells used to assess LOD • Statistical approach, such as with dual criteria or mean + two SDs of pre-existing immunity/background • Positivity criteria for a sample established based on meaningful level of reactivity that is above the background reactivity
Precision	Does not discuss	• Intra-assay: six replicates per assay • Inter-assay: eight assays on different days • CV 4–133% for medium and high responders • Use SD for low responders • Depends on counting statistics – 2 × 10^5 PBMCs/well gave highest counting efficiency	• Required during validation • Use ≤10 samples • Intermediate % CV ≤40% for ≥80% of samples having SFU/10^6 PBMCs greater than LOQ • Six samples tested at three cell inputs
Ruggedness	Does not discuss	Three different operators on the same day	• Required during validation • Maximum fold difference between assays <twofold • Different parameters that can vary over time during routine operation should be tested
Linearity	Does not discuss	• Serial dilute PBMC from a high responder (triplicate samples) into PBMC from a non-responder • R^2 values >0.97	• Required during validation • Six samples at three cell inputs • Sample reactivity expected to decrease as cell input decreases
LOQ	Does not discuss	Does not discuss	• Required during validation • Lowest value that can be quantified with acceptable precision (intermediate precision % CV ≤40%) • LLOQ ≥ LOD
Specificity	Does not discuss	Does not discuss	• Required during validation • Determined with a negative cutoff determination for vector and transgene peptide pools using a correction factor specific to each peptide pool
Normalization of results	Does not discuss	Does not discuss	Cells can be normalized, that is, PBMCs analyzed by flow cytometry and then PBMCs adjusted to the fixed/pre-defined number of T-cells (executed at the site of the ELISpot analysis)
Critical reagents	Does not discuss	Does not discuss	• Do not to use cells with a viability <80% • High-quality frozen PBMC preparations • Capture and detection antibody pair • Antigen source can be overlapping synthetic peptides or whole proteins • Filter plates may be PVDF or other variety • Side-by-side assessment of new reagent lots against qualified lots is necessary for reagent bridging and trending

ELISpot: Enzyme-Linked Immunospot; PBMC: Peripheral blood mononuclear cell; PVDF: Polyvinylidene fluoride; SD: Standard deviation; SFC: Spot-forming cell; SFU: Spot-forming unit; SOP: Standard operating procedure; ULOD: Upper limit of detection.

Data taken from [20,28–30].

Table 3. Additional GCC recommendations on ELISpot assay validation.

Parameter	Recommendation
SOP	Should include PBMC isolation, counting method, cell handling, plate reading method, training requirements and equipment specific procedures It is suggested that the type of collection tube (heparin), sample collection and handling times, shipping times determined during method development and validation also be included
Normalization of results	• Intra- and inter-subject, as needed • Multiple baselines possible • Normalization against negative control
Critical reagents	• PBMC (refer to Corsaro et al. [30] for recommendation for using high-quality PBMC; alternative criteria may be utilized when evaluated during method development and confirmed to support the context of use); also, it is possible to consider the option of CPT tubes for collection as an alternative to removal of granulocytes at point of analysis • Detection antibodies • Positive controls • Lot-to-lot bridging should be performed • In the absence of vendor-provided stability data, stability experiments must be performed to demonstrate critical reagent stability as per usage in the assay
Sample type	• Patient samples reflecting study population should be used during method development and validation, if available • All runs should include positive controls and negative controls • Run all controls and samples in triplicate (three wells per result) during validation and sample analysis
Quality controls	• At least two levels of positive control and one negative control (media only) • Acceptable range should be established during validation • Reference sample/trending control for each day ELISpot is run
Validation parameters based on COU	• Precision • Sensitivity (LLOQ and LOD) • ULOQ/reportable range • Specificity • Ruggedness and robustness • Linearity • Critical reagent stability • Whole blood or PBMC stability
Precision	• Minimum ten donors • Inter-assay: should include a minimum of six runs with three replicates each by two analysts over multiple days • Intra-assay: minimum one run and six replicates • CV ≤30% • Total error <40% (LLOQ <50%)
Sensitivity (LLOQ and LOD)	• Determined based on precision data; the intermediate precision as LLOQ should be based on the acceptable intermediate precision of 40% • LOD is determined based on two SDs above the mean of replicate negative control samples • Due to the mathematical considerations of a high % CV at low spot numbers per well, statistical testing [32] is recommended for samples that are below 30 spots per well and above the LOD • LLOQ ≥ LLOD
ULOQ	Defined as the maximum number of individual spots per well the ELISpot plate reader software can discriminate; this can be achieved by counting spots using a series of cell dilutions treated with mitogen, or peptide for a donor with a very strong peptide response
Specificity	Positive control greater than negative control; should also be tested with non-specific peptides such as beta-actin; in the case of non-specific peptides, the response must be less than LLOQ; in addition, specificity should examine the full extent to which an assay responds to all subsets of an analyte [30]; often this may also include an assessment of specificity for the target cell type, especially for assays aiming to measure this component
Ruggedness and robustness	• Maximum fold difference between assay ruggedness factor levels is expected to be less than twofold for tenfold dilution of cells; in the case of analysts and instruments – should meet the % CV criteria • Inter-laboratory comparison studies may be performed to demonstrate assay ruggedness
Linearity	• Serial dilute PBMC from a high responder into PBMC from a non-responder • Use at least six donors (high responders) and at least three dilutions • Sample reactivity expected to decrease as cell input decreases • R^2 values >0.97
Selectivity	• Ten different lots/donors of PBMC (refer to Corsaro et al. [30] for recommendation for using high-quality PBMC) • Decide on and establish level of response needed for LLOQ from the PBMCs in MD, to determine the level of reactivity needed for selectivity determination of the assay in validation • Controls for selectivity are based on media and PBMC positivity criteria for a selectivity sample established based on meaningful level of reactivity of that which is below the background reactivity [30]; the difference between selectivity samples is the media controls with PBMC responses ≤ LLOQ; +/- 40% • ≥70% of the lots should pass this criterium

COU: Context of use; ELISpot: Enzyme-Linked Immunospot; LLOD: Lower limit of detection; MD: Method development; PBMC: Peripheral blood mononuclear cell; SD: Standard deviation; SOP: Standard operating procedure.

Financial & competing interests disclosure

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending or royalties.

No writing assistance was utilized in the production of this manuscript.