Performance metrics for evaluating system suitability in liquid chromatography—Mass spectrometry peptide mass mapping of protein therapeutics and monoclonal antibodies

Figures & data

Figure 1. Representative total ion chromatogram of the BSA digest spiked with synthetic peptides used in this study for evaluation of system suitability. The intra-scan peptides (highlighted in red) were spiked in pairs consisting of one light peptide and one heavy isotope labeled peptide (schematic mass spectrum representation shown in insert). The light peptides were present at signal normalized concentrations corresponding to 0.1% to 100% of the heavy peptide concentrations in each pair. The inter-scan peptides (highlighted in blue) are closely related synthetic peptides eluting at similar retention times around 30 min. The other peaks in the chromatogram correspond to BSA peptides. The signal from the inter-scan peptides present at signal normalized concentrations of 1% and 0.1% are too low to be directly observed on the total ion chromatogram.

Table 1. Experimental method settings with systematic variation of several parameters. “+” sign indicates high source voltage; long MS¹/MS² scan time/accumulation; or high MS² precursor selection threshold. “−” sign corresponds to lower settings. M0 is a MS¹ only method, thus the MS² parameters are listed as “NA” (not applicable)

Method ID	Source Voltage	MS^1 Scan Time	MS^2 Scan Time	Selection Threshold
M0	−	+	NA	NA
M1	−	−	+	−
M2	+	−	+	−
M3	−	+	+	−
M4	−	−	−	−
M5	−	−	+	+
M6	−	+	−	−
M7	−	+	+	+
M8	−	−	−	+
M9	−	+	−	+

Figure 2. Influence of method settings on protein sequence coverage (black bars) for data from (a) Q-Exactive and (b) QTOF instrument as reported by Proteome Discoverer. Gray bars indicate the average percentage of MS² spectra matched to peptides for each method (number of spectra associated with high-confidence peptides divided by the total number of spectra input into the search algorithm). The number of MS² spectra inputs into the search algorithm is also included as line plots following the y-axis on the right.

Figure 3. MS² scores for inter-scan peptide standards at signal normalized concentrations of 10% for Q-Exactive and QTOF systems. Higher scores were associated with higher confidence in identification. Diamonds indicate the confidence interval, with the center line indicating the mean and the vertical span representing the 95% confidence interval.

Figure 4. Average absolute mass error associated with each acquisition method on either MS platform. ANOVA analysis identifies significant differences among the Q-Exactive methods (P< 0.0001), but not for the QTOF methods (p = 0.2097). A pairwise t-test, however, suggests that significant differences in mass error between some of the methods on QTOF exist.

Figure 5. Average absolute ppm errors of high-confidence BSA peptides identified for data acquired on the Q-Exactive and QTOF sorted by differences in MS¹ scan time. The (+) corresponds to a higher MS¹ AGC target (Q-Exactive) or a longer MS¹ scan (QTOF) than the (−). Significant differences were observed between the different MS¹ settings on both systems (p < 0.0001 for Q-Exactive and p = 0.0083 for QTOF).

Figure 6. Summary of accuracy and precision results for intra-scan (left column) and inter-scan (right column) peptide pairs at different signal normalized concentration ratios on the Q-Exactive (top row) and QTOF (bottom row). Percent accuracy is defined as the experimental result divided by the expected value. The precision is represented by the standard deviation as shown with the error bars. Missing points on broken lines indicate the peptide was not detected at least twice in the triplicate runs using the associated method. Data with CV > 20% are highlighted with yellow edges on the points.

Figure 7. Variation in quantitation results relative to the MS¹ only method (M0) due to variation in peak sampling for 2 inter-scan peptides on both the QTOF and Q-Exactive. The sampling factor featured good correlation with the QTOF data, but not with the Q-Exactive data as manifested by the correlation coefficients (R) noted in the plots. The ranges where the normalized results are 0.8∼1.2 (relative quantitation result within ± 20% of that by method M0) are highlighted in blue in each plot.

Table 2. Summary of the proposed metrics for system suitability test, with proposed acceptance criteria and possible troubleshooting targets. The thresholds of 15% for CV and +/− 20% for relative quantitation are adapted from current FDA recommendations³⁴

Metric	Purpose	Definition	Recommended Threshold	Possible Cause for Failure
CV of Peak Area	Check signal stability	%CV of peak area of EIC at 100%	<15%	Instable signal, likely an issue at the electrospray
Protein Sequence Coverage	Check method settings for identification capability	Determined from search algorithm based on the confidence level chosen	Near complete is ideal	Acquisition setting, intrinsic sensitivity, LC separation
Average Mass Error	Report overall mass accuracy	Average of absolute ppm error of all identified peptides with high confidence (or known peptides)	From system specification <5 ppm or lower (High resolution)	Temperature change, invalid calibration, incorrect method settings (e.g., lock mass, AGC)
Mass Resolution	Check instrument resolution	Full width at half maximum mass peak resolution at 100% concentration	Within instrument specification (instrument and operation mode dependent)	Invalid instrument tuning/qualification, instrument malfunctioning
Ion Source Settings	Minimize non-native species generated in source	Monitor the relative abundance of a fragile peptide in the standard	Maintain sensitivity while minimize excess activation of fragile species	Source tuning, source acceleration voltage too high
S/N of EIC	Defining LOD	S/N of target peptide EIC	> 3	Bad instrument tuning, bad EIC peak shape, bad spraying condition, intrinsic sensitivity limit of system
MS^2 Identification Score	Check MS^2 quality at different concentrations	Target peptide score/cutoff threshold for intra and inter-scan	≥1	Acquisition setting, intrinsic sensitivity, LC separation
Mass Accuracy/Resolution Change	Evaluate confidence of mass measurement at low concentrations	Compare mass error/resolution at low and 100% concentration	Ideally minimal change at low concentrations	Instrument specific and can be normal to see mass error/resolution change at different signal intensities. Need to adjust data analysis procedure accordingly.
Accuracy of Relative Quantitation	Check accuracy and dynamic range of quantitation	Percent accuracy of the experimental result relative to the expected result	80% –120%	Bad EIC extraction method, co-eluting species, instrument intrinsic limit
CV of Relative Quantitation Result	Check precision of quantitation	%CV of peak area ratios of a target peptide pair	<15% unless only intended for semi-quantitation	Low sensitivity/bad peak shape at the concentration, incorrect data processing, interfering co-eluting species
Differential Peak Sampling	Check potential error in quantitation due to stochastic sampling across EIC peak	Ratio of relative quantitation results between the proposed method and the MS^1 only method	0.8∼1.2 Use the MS^1 only method for quantitation if exceeds the range	Improper MS^2 settings such as the selection threshold, MS^1 and MS^2 scan rate, etc.

Figure 8. Proposed workflow for a system suitability test.

U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER) and Center for Veterinary Medicine (CVM). Guidance for industry: Bioanalytical method validation. Silver Spring, MD: FDA; 2001

 Google Scholar