Evaluation of vaccine seroresponse rates and adverse event rates through Bayesian and frequentist methods

Abstract

In the evaluation of vaccine seroresponse rates and adverse reaction rates, extreme test results often occur, with substantial adverse event rates of 0% and/or seroresponse rates of 100%, which has produced several data challenges. Few studies have used both the Bayesian and frequentist methods on the same sets of data that contain extreme test cases to evaluate vaccine safety and immunogenicity. In this study, Bayesian methods were introduced, and the comparison with frequentist methods was made based on practical cases from randomized controlled vaccine trials and a simulation experiment to examine the rationality of the Bayesian methods. The results demonstrated that the Bayesian non-informative method obtained lower limits (for extreme cases of 100%) and upper limits (for extreme cases of zero), which were similar to the limits that were identified with the frequentist method. The frequentist rate estimates and corresponding confidence intervals (CIs) for extreme cases of 0 or 100% always equaled and included 0 or 100%, respectively, whereas the Bayesian estimations varied depending on the sample size, with none equaling zero or 100%. The Bayesian method obtained more reasonable interval estimates of the rates with extreme data compared with the frequentist method, whereas the frequentist method objectively expressed the outcomes of clinical vaccine trials. The two types of statistical results are complementary, and it is proposed that the Bayesian and frequentist methods should be combined to more comprehensively evaluate clinical vaccine trials.

Keywords:

• adverse event rates
• Bayesian method
• extreme data
• frequentist method
• seroresponse rates
• vaccine

Abbreviations:

• BCIs, Bayesian credible intervals
• C, control group
• CIs, confidence intervals
• CL_l, lower confidence limit
• CLu, upper confidence limit
• FDA/CBER, Food and Drug Administration/ Center for Biologics Evaluation and Research
• HAV, hepatitis A virus
• Hib conjugate, haemophilus influenza type b conjugate vaccine
• HPV 6/11/16/18, human papillomavirus 6/11/16/18 quadrivalent vaccine
• HPV 16/18 AS04-adjuvanted, human papillomavirus vaccine -16/18 AS04-adjuvanted vaccine
• MMRV, measles, mumps, rubella and varicella vaccine
• T, test group

Introduction

The seroresponse rates (seroprotection rates and seroconversion rates) and adverse event rates are often used as the primary endpoints to evaluate vaccine immunogenicity and safety. The superiority, equivalence or non-inferiority of 2 vaccines, with a rate used as an endpoint, are typically demonstrated via the confidence interval (CI) approach, with the risk difference representing the parameter of interest.¹ In the estimation of vaccine seroresponse rates and adverse reaction rates, extreme test results often occur, with substantial adverse event rates of 0% and/or seroprotection rates of 100%,^2-10 which are more likely to occur when the sample size is small.¹

The evaluation of vaccine immunogenicity and safety with extreme data (i.e., data for which the point estimate of the seroresponse and adverse event rates reach 100 or 0%, respectively) has produced several data challenges. The challenges originate from both clinical practice and statistical methods. Regarding clinical vaccine practice, for adverse event rates of 0% and seroprotection rates of 100%, the inherent problems of false negatives and false positives are inevitably associated with clinical vaccine trials because none of the currently available human vaccines can achieve 100% protection and/or zero risk.⁵ For the statistical methods, the calculation of the CI for a rate is 'chaotic' when the proportion under consideration is very low (e.g., 0 or 0.01) or very high (e.g.,, 0.99 or 1);¹¹ this is because in the case of CIs for proportions, the relevant distribution is a binomial, which is discontinuous.¹² This discontinuity precludes any 95% CI from including the true population proportion exactly 95% of the time in repeated experiments.^12-19 For example, for frequentist statistics, the Wald method of point estimations minus/plus a multiplier of the standard error, which is the most widely taught method for the calculation of binomial CIs, fails to produce effective statistical inferences when the point that is estimated is 0 or 100%, and the standard error is 0.¹⁶ Other methods that are frequently used in clinical vaccine trials, such as the Walson (Score) method and the Clopper-Person method, produce statistical inferences with a lower confidence limit (CL_l) of 0% or an upper confidence limit (CLu) of 100%, regardless of the sample size, when the sample rate is 0 or 100%, respectively^20,21.

Various procedures have been proposed to determine the CI of a binomial proportion.^{13,16,17,21,22} Most results that evaluate the seroresponse rates and adverse reaction rates are derived from frequentist statistics.^2,3,5,8 However, the less well-known but rapidly developing Bayesian approach^23,24 is seldom involved. The Bayesian approach may provide effective solutions to problems of this nature via the incorporation of prior information in conjunction with current experimental results.²³ Many independently published studies have applied Bayesian statistics to the evaluation of the performance of medical devices and diagnostic tests.^25-28 However, few studies have used both the Bayesian and frequentist methods on the same sets of data that contain extreme test cases to evaluate vaccine safety and immunogenicity. In this study, Bayesian methods with non-informative and informative priors were introduced for the estimation of seroresponse rates and adverse reaction rates. A comparison between the Bayesian and frequentist methods was performed based on several practical cases from randomized, controlled vaccine trials for the human papillomavirus (HPV),^2,4,6,8 the hepatitis A virus (HAV),³ haemophilus influenza type b (Hib),⁵ varicella¹⁰ and the influenza virus.⁹ To examine the rationality of the Bayesian approaches, simulations were performed that were based on practical scenarios of seroresponse and adverse events.

Results

Practical cases

The adverse event rates or incidence, the seroprotection rates or seroconversion rates, the rate difference between the test and control groups, the corresponding 95% CIs and 95% Bayesian credible intervals (BCIs) from 8 randomized, controlled trials that were designed to evaluate vaccine safety and immunogenicity are summarized in Table 1.

Table 1. Comparison of Bayesian and frequentist methods for the estimation of the seroresponse rates and adverse event rates from clinical vaccine randomized control trials

				Rate and 95% CI (%)	Rate and 95% BCI (%)	Rate difference, 95% CI and BCI (%)
Case	Source	Vaccine	Event/Total	Frequentist	Bayesian	Frequentist	Bayesian
Adverse events rates or incidence
1	Garland et al.^8 (2007)	HPV 6/11/16/18	T: 1/2673	0.04(0.01, 0.21)	0.06(0.01, 0.21)	0.04(−0.11, 0.21)	0.03(−0.08, 0.18)
			C: 0/2672	0.00(0.00, 0.14)	0.03(0.00*, 0.14)
2	Romanowski et al.^2 (2014)	HPV 16/18 AS04-adjuvanted	T: 0/240	0.00(0.00, 1.58)	0.28(0.01, 1.54)	0.00(−1.58, 1.58)	−0.01(−1.27, 1.22)
			C: 0/239	0.00(0.00, 1.58)	0.30(0.01, 1.52)
3	Cui et al.^3 (2014)	Hepatitis A virus	T: 0/84412	0.00(0.00, 0.00)	0.00* (0.00*, 0.00*)	−0.03(−0.04, −0.02)	−0.03(−0.04, −0.02)
			C: 22/79254	0.03(0.02, 0.04)	0.03(0.02, 0.04)
4	Nauta^1 (2011)	MMRV	T: 2/148	1.35(0.37, 4.79)	1.76(0.42, 4.79)	1.35(−1.50, 4.80)	1.14(−1.29, 4.24)
			C: 0/132	0.00(0.00, 2.83)	0.52(0.02, 2.68)
5	LU et al.^10 (2008)	Frozen-dried live attenuated varicella	T: 2/320	0.63(0.17, 2.25)	0.84(0.19, 2.21)	0.63(−1.73, 2.25)	0.35(−1.51, 1.91)
			C: 0/160	0.00(0.00, 2.34)	0.43(0.02, 2.29)
Seroprotection rates or seroconversion rates
6	ZHANG et al.^9 (2005)	Influenza Virus	T: 62/62	100.00(94.17, 100.00)	98.92(94.30, 99.96)	0.00(−5.83, 7.41)	0.24(−4.29, 6.02)
			C: 48/48	100.00(92.59, 100.00)	98.59(92.69, 99.95)
7	LU et al.^10 (2008)	Frozen-dried live attenuated varicella	T: 82/82	100.00(95.52, 100.00)	99.15(95.74, 99.97)	0.00(−4.48, 11.70)	1.21(−2.80, 10.30)
			C: 29/29	100.00(88.30, 100.00)	97.75(88.28, 99.92)
8	Romanowski et al.^2 (2014)	HPV-16/18 AS04-adjuvanted	T: 139/139	100.00(97.31, 100.00)	99.51(97.37, 99.98)	0.00(−2.69, 2.89)	0.02(−2.07, 2.27)
			C: 129/129	100.00(97.11, 100.00)	99.46(97.21, 99.98)
9	Wang^4 (2013)	HPV 16/18	T: 318/318	100.00(98.81, 100.00)	99.79(98.85, 99.99)	99.37(97.73, 99.83)	98.85(97.28, 99.65)
			C: 2/318	0.63(0.17, 2.26)	0.83(0.20, 2.27)
10	Low et l.^5 (2013)	Hib conjugate	T: 232/232	100.00(98.37, 100.00)	99.69(98.44, 99.99)	0.00(−1.63, 1.51)	−0.01(−1.28, 1.16)
			C: 250/250	100.00(98.49, 100.00)	99.73(98.57, 99.99)

BCI, Bayesian credible interval; C, control group; CI, confidence interval; HPV 6/11/16/18, human papillomavirus 6/11/16/18 quadrivalent vaccine; HPV 16/18 AS04-adjuvanted, hum an papillomavirus vaccine -16/18 AS04-adjuvanted vaccine; MMRV, (measles, mumps, rubella, and varicella) vaccine; Hib conjugate, haemophilus influenza type b conjugate vaccine; T, test group. *"0.00" occured because th decimal digits rounded to 0.00%.

For the adverse event rates and incidence, as shown in Table 1, the upper limits from the Bayesian and frequentist methods were similar. However, for the seroprotection rates or seroconversion rates, the low limits from both methods were similar. Moreover, for the rate difference, the 2 methods presented the same statistical inference. For example, for cases 3 and 9 (Table 1), their 95% CIs and BCIs of the rate differences did not cover zero, which indicates that the test and control groups were statistically different. However, it is worth noting that in the cases where the numerator was zero or the cases that equaled 100%, the point estimators and the 95% lower limits or upper limits from the frequentist methods were all zero or 100%, respectively. The Bayesian estimation varied depending on the sample size, with none of the lower or upper limits equal to zero or 100% ("0.00" occurred in case 1 and case 3 because the decimal digits rounded to 0.00%).

Simulation study

To investigate the performance of Bayesian and frequentist methods in the conditions of different sample sizes and prior information, a simulation experiment was designed. Table 2 shows that for different sample sizes, the Bayesian estimate of the population rate and the credible limits did not contain a value of 100% or zero in both the non-informative and informative priors, even if the rate in the sample was equal to 100% or zero. Moreover, it is clear that the Bayesian non-informative method obtained lower limits (for extreme cases of 100%) or upper limits (for extreme cases of zero) which were similar to the limits that were obtained by the frequentist method. Table 2 shows that for the case where x (number of event) equals 1 or n-1 (number of total minus 1), the same characteristics were observed compared with when x was equal to zero or n. It is clear that for the cases of non-extreme data (x=n *80% or x=n *20%), the lower limits and upper limits form the Bayesian non-informative method were similar to from the frequentist method. However, for the Bayesian informative method, substantial differences were especially present when the sample size was small, and the Bayesian informative method provided narrower credible intervals.

Table 2. 95% confidence interval and 95% credible intervals (as percent) estimated by frequentist and Bayesian methods for different sample sizes

Simulation scenarios for seroprotection and serpconversion rates					Simulation scenarios for adverse event rates and incidences
			Bayesian					Bayesian
Test	x/n	Frequentist	Non-informative	Informative*	Test	x/n	Frequentist	Non-informative	Informative^♦
x=n					x=0
1	20/20	100.00(83.89, 100.00)	96.81(83.88, 99.88)	98.64(93.28, 99.93)	6	0/100	0.00(0.00, 3.70)	0.68(0.03, 3.57)	0.58(0.03, 2.85)
2	40/40	100.00(91.24, 100.00)	98.29(91.61, 99.94)	98.97(94.92, 99.95)	7	0/200	0.00(0.00, 1.88)	0.35(0.01, 1.79)	0.33(0.02, 1.65)
3	60/60	100.00(93.98, 100.00)	98.88(94.32, 99.96)	99.18(96.08, 99.96)	8	0/500	0.00(0.00, 0.76)	0.14(0.01, 0.72)	0.15(0.01, 0.72)
4	80/80	100.00(95.42, 100.00)	99.14(95.65, 99.97)	99.32(96.72, 99.97)	9	0/1000	0.00(0.00, 0.38)	0.07(0.00, 0.37)	0.08(0.00, 0.37)
5	100/100	100.00(96.30, 100.00)	99.32(96.46, 99.98)	99.43(97.17, 99.97)	10	0/10000	0.00(0.00, 0.04)	0.01(0.00, 0.04)	0.01(0.00, 0.04)
x=n −1					x=1
11	19/20	95.00(76.39, 99.11)	92.15(76.25, 98.83)	96.90(90.48, 99.51)	16	1/100	1.00(0.18, 5.45)	1.65(0.24, 5.41)	1.29(0.22, 4.17)
12	39/40	97.50(87.12, 99.56)	95.96(87.18, 99.43)	97.69(92.69, 99.64)	17	1/200	0.50(0.09, 2.78)	0.84(0.12, 2.72)	0.76(0.11, 2.37)
13	59/60	98.33(91.14, 99.71)	97.23(91.20, 99.61)	98.19(94.27, 99.71)	18	1/500	0.20(0.04, 1.12)	0.33(0.05, 1.09)	0.32(0.05, 1.04)
14	79/80	98.75(93.25, 99.78)	97.96(93.34, 99.68)	98.49(95.19, 99.78)	19	1/1000	0.10(0.02, 0.56)	0.16(0.02, 0.56)	0.17(0.03, 0.55)
15	99/100	99.00(94.55,99.82)	98.37(94.61, 99.75)	98.72(95.90, 99.80)	20	1/10000	0.01(0.00, 0.06)	0.02(0.00, 0.06)	0.02(0.00, 0.06)
x=n *80%					x=n *20%
21	16/20	80.00(58.40,91.93)	78.09(58.29, 91.69)	81.08(71.48, 88.64)	26	20/100	20.00(13.34,28.88)	20.39(13.39, 28.98)	19.53(13.76, 26.12)
22	32/40	80.00(65.24,89.50)	79.01(65.12, 89.32)	80.82(72.19, 87.84)	27	40/200	20.00(15.05,26.09)	20.19(15.00, 25.99)	19.66(15.12,24.80)
23	48/60	80.00(68.22,88.17)	79.36(68.27, 88.13)	80.73(73.06, 87.17)	28	100/500	20.00(16.73,23.73)	20.07(16.74, 23.72)	19.85(16.68, 23.28)
24	64/80	80.00(69.95,87.30)	79.52(69.91, 87.14)	80.57(73.48, 86.56)	29	200/1000	20.00(17.64,22.59)	20.05(17.64, 22.63)	19.93(17.60, 22.42)
25	80/100	80.00(71.12,86.66)	79.58(71.09, 86.56)	80.55(73.73, 86.28)	30	2000/10000	20.00(19.23,20.80)	20.00(19.23, 20.80)	19.99(19.23, 20.78)

*95% credible interval of prior rates for x=n and x=n -1: (90, 99.9%) with Beta(36.9, 1.1), for x=n *80%: (70, 90%) with Beta(46.3,10.8); ^♦95% credible interval of prior rate for x=0 and x=1 : (0.1, 10%) with Beta(1.1, 36.9), for x=n *20%: (10, 30%) with Beta(10.8,46.3); x, number of event; n, number of total.

Discussion

Through the use of practical cases and the simulation experiment, this study demonstrated that the Bayesian method is more reasonable to use in the evaluation of seroresponse rates and adverse event rates in cases that contain extreme data compared with the frequentist method. In this section, the reasons for the outcomes are presented from both clinical practice and statistical methods. Because of the features of clinical vaccine trials, classical frequentist statistical methods have certain difficulties in the evaluation of vaccine safety and immunogenicity. For example, for cases of extreme data, the classical point estimates of the adverse event rates or the seroresponse rates and their corresponding lower or upper 95% CLs always reached zero or 100%, respectively. Clearly, this is not reasonable for the evaluation of clinical vaccine trials because none of current available human vaccine can achieve 100% protection and/or zero risk.⁵ However, for Bayesian statistics, this dilemma can be solved via the incorporation of prior information. The Bayesian estimate of the population rate and the corresponding CLs are not able to have a value of 100 or 0% when prior information is provided (non-informative or informative). This result is consistent with the experience of medical experts and the understanding of individuals regarding current clinical vaccine trials.⁵ For the statistical methods, it is well known that the 95% confidence or credible intervals are used to measure the reliability of the statistical estimates.^11,23 In the case of extreme data, Bayesian credible interval was indeed superior to its frequentist companion. According to tests 1 and 5 in Table 2, the point estimate of the seroprotection rate in test 5 was more reliable, based on common sense. However, both tests produced a point estimate value of 100% no matter of the sample size when the frequentist method was used. The explanation for this apparent contradiction is that the sample size of test 5 was 5 times larger compared with test 1 (Table 2). This difference in reliability could be expressed by a Bayesian posterior median and a 95% BCI.

When the sample size is small, inferences that are based on the Bayesian method are very suitable in the analysis of extreme data because a priori information has a substantive influence on the Bayesian posterior distribution. For test 1 in Table 2, if π was unknown prior parameter to the experiment, one of the most natural approaches would be to choose a uniform prior in the range of 0–100% (i.e., Beta(1, 1)). After 20 tests, there would be 20 positive results. With the assigned prior updated via Bayes' theorem, the posterior median estimate and the 95% BCI of π would be 96.81% and (83.88, 99.88%), respectively. Nevertheless, if one researcher was 95% credible regarding π having a lower limit of 90% and an upper limit of 99.9% (i.e., π ∼ Beta(36.9, 1.1)),²⁹ the updated results would be π = 98.64% and a 95% BCI of (93.28, 99.93%) (Table 2). The Bayesian posterior distribution was narrowed toward the prior information.²³ By applying informative prior information, Bayesian statistics has the potential to improve the quality of analyses that include extreme data when the sample size is small. However, the influence of the prior distribution on Bayesian inferences becomes weak and even diffuse when sample sizes are large.

There are many theoretical differences between the BCI and the frequentist CI, and only the distinctions between their interpretations were discussed in this paper. In the viewpoint of frequentist statistics, the 95% refers to the fact that if the same study were repeated many times and the CI similarly calculated for each case, 95% of such intervals would include the population proportion. The explanation, obviously, is not easy to understand by clinicians. Advocates of the CI do acknowledge this strict definition; however, many clinicians do not share this insight.³⁰ Instead of finding an interval that covers the true population proportion on 95% of the time for any population proportion, the Bayesian interval covers the true population proportion 95% of the time given the observed proportion.²³ In other words, the Bayesian interval covers 95% of the population proportions that could lead to the experimentally determined proportion being observed. This property is appealing because it enables the researcher to propose a direct probability statement regarding the parameters. Many individuals find this concept to be a more natural approach to understand probability intervals and easier to explain to non-statisticians.³⁰

The following discussion addresses the similarity of the results that were obtained through the use of the Bayesian and frequentist methods. This study demonstrates that the upper limits (for extreme rates of zero) or lower limits (for extreme rates of 100%) from the frequentist and Bayesian non-informative methods were very similar, even under the condition of small sample sizes (see practical cases in Table 1 and simulation tests in Table 2). The same characteristics were observed for the cases of non-extreme data because the posterior is simply proportional to the product of the prior and likelihood. This result has important significance for clinical vaccine trials because the upper limit of 95% CIs is often a primary endpoint for the evaluation of safety,² and the lower limit of 95% CIs is typically the primary endpoint for the evaluation of immunogenicity.^31,32 For example, the FDA/CBER (Food and Drug Administration/Center for Biologics Evaluation and Research) published guidance documents regarding the licensure of influenza vaccines for an adult population, and the lower limit of the 2-sided 95% CI for the seroprotection rate was required to meet or exceed 0.7.^31,32 For the evaluation of safety, the focus will typically be on the upper limit because it provides the upper boundary of the rate with which the reaction is expected to occur in subjects who receive the vaccine.¹ The boundary is often translated into a less-than- 1-in rate.¹ If the upper confidence limit for the rate of a specific reaction is CL_U, then the expected rate of the reaction is <1 in 1/CL_U vaccinated subjects, with 1/CL_U often rounded down to the nearest multiplier of 100. For example, Garland et al. reported⁸ that in a phase III trial that evaluated the efficacy of a prophylactic, quadrivalent vaccine that prevents anogenital diseases associated with HPV 6/11/16/18, when the serious event (vaccine-related) in the vaccine group was 1/2673, both of the upper limits from the frequentist and Bayesian non-informative methods were 0.21% (see case 1 in Table 1). Thus, the expected rate of the vaccine-related serious event was <1 in 476 (i.e., <1 in 450) vaccinated subjects. For the same set of data, when the Bayesian non-informative and frequentist methods produced very similar results, this increased the reliability of the statistical results. For the discussion regarding the similarity of both methods, it must be emphasized that this condition is limited to the Bayesian non-informative method. Once an informative prior is available, such as a meta-analysis, published articles, previous similar studies or expert opinions, which are often the source of informative priors, the Bayesian method potentially provides more informative results, which demonstrates the significant advantage of Bayesian compared with frequentist methods, especially regarding phase I and phase II clinical vaccine trials in which the sample sizes are typically small.^1,9,10

In recent decades, Bayesian statistics have progressed more substantially in theory and practice compared with classic frequentist statistics because the development of numerical algorithms and computer technology has removed the constraint of high-dimensional integration.²⁴ Therefore, it is useful to introduce a Bayesian method in the evaluation of clinical vaccine trials because the Bayesian approach may address the weaknesses of the frequentist method, thereby enhancing the ability to assess very high seroresponse rates and extremely low adverse event rates. The frequentist estimates of adverse event rates and seroresponse rates objectively express the outcomes of clinical vaccine trials, such as by the indication of a 0% adverse event rate in the reported study, where the adverse event truly did not occur; thus, the 2 types of statistical methods are complementary in the evaluation of clinical vaccine trials. It is proposed that the Bayesian and frequentist methods should be combined to enable the more comprehensive evaluation of clinical vaccine trials. Especially, in the evaluation of vaccine seroresponse rates and adverse reaction rates with extreme test results, when Bayesian point estimates and credible intervals are presented, the point estimates from frequentist method should be reported simultaneously. This proposal agrees with Efron³³ and Wijeysundera et al.,³⁰ in that the Bayesian statistical inference enhances the interpretation of contemporary biological and epidemiological data.

Materials and Methods

Frequentist method

To determinate the CI of a rate, the frequentist methods can only use population information and sample information. Population information reflects the assumption that the outcomes (positive or negative) of the evaluation follow a binomial distribution,^16,18 whereas sample information consists of the experimental data. Given these 2 types of information, frequentist methods typically estimate the rate based on the maximum likelihood estimation and calculate the associated CIs. Regarding the several methods that can be used to construct frequentist CIs, the Wilson method, also known as the Score method, is specifically recommended.^20,21 In this study, the Wilson-type CLs were calculated using the statistical software SAS 9.3 (SAS Institute, Inc., Cary, NC, USA).

Bayesian method

The Bayesian approach is not only based on population and sample information but also on the information from a prior distribution.²⁷ To proceed further, a few notations must be introduced. Let x_t, x_c denote the number of observations of the test and control groups, respectively, regarding the characteristic being studied (e.g., positive number), and let n_t, n_c denote the sample size of the test and control groups, respectively. Let p_t, p_c denote the estimator of the unknown proportion π_t, π_c of the test and control groups, respectively. Moreover, supposing that n_t, n_c subjects are sampled randomly with the proportions π_t, π_c, respectively, x_t, x_c have the following binomial sampling distributions:

${\displaystyle x_t\sim Binomial({\pi }_t,n_t),x_c\sim Binomial({\pi }_c,n_c).}$

A Bayesian approach to this problem requires the specification of a prior distribution for parameters π_t and π_c. A convenient and reasonable prior that is typically used in Bayesian analyses of binomial processes is the family of Beta distributions. If there is a Beta(a, b) prior and x occurrences of the event are observed in n trials, the posterior distribution is Beta(a+x, b+n-x). The information regarding p that is represented by the Beta(a, b) distribution can be interpreted as equivalent to having seen a patients who had a serious reaction out of a + b total patients who were exposed to the new treatment.²⁷ Two types of prior distributions were adopted in this study, as described below.

1. Non-informative prior distribution: judging by common sense, the range of values used to represent a rate is [0, 1]. Therefore, the uniform distribution in [0, 1] (i.e., x ∼ Uniform(0, 1) or x ∼ Beta(1, 1)) may be reasonably applied to the analysis of a rate as a non-informative prior distribution in the absence of relevant knowledge. With this choice of a prior distribution, the conclusions that result from the Bayesian and frequentist approaches are often consistent if the sample size is sufficiently large.^23,24,27 In the present study, Beta(1, 1) was adopted to perform a rational comparison of Bayesian statistics with frequentist statistics.

2. Informative prior distribution: A fuller usage of the Bayesian approach would include the use of informative priors, which increase the precision of the interval estimates. There are a variety of approaches to assess prior distributions.²³ It is easy to assess certain prior quantitles (for example, the 2.5th and 97.5th quantitles for a 95% credible interval) when using a small freeware of Parameter Solver (Version 2.3),²⁹ which can be downloaded from the http://biostatistics.mdanderson.org/SoftwareDownload/ web site. The goal of the software is to calculate the parameters that describe a standard probability distribution when certain input quantiles or a certain input mean and variance are provided. The prior parameters, such as quantiles, mean and variance, can be obtained through published articles, previous studies or expert opinions. In this study, the prior parameters of Bayesian informative method were assigned by Dr. Zhu (co-author, an epidemiologist in the field of clinical vaccine trials).

The posteriors of the parameter distributions were calculated using Markov chain Monte Carlo (MCMC) techniques²⁴ in the freeware WinBUGS 1.4 (available at http://www2.mrc-bsu.cam.ac.uk/bugs/). In this study the inferences were based on 25,000 iterations after an initial burn-in of 5,000 iterations was discarded, with convergence assessed by running multiple chains from various starting values.³⁴ The WinBUGS code that was used in this paper is provided in the Notes and can be easily altered and used with different data.

Simulation study

The simulation study was design into 2 scenarios that one is for seroprotection and seproconversion rates and another is for adverse event rates and incidence. The range of sample sizes (n) of the simulation study varied from 20 to 10000 (Table 2). For seroprotection rates and seroconversion rates, small sample sizes were simulated; in contrast, for adverse event rates and incidences, large sample sizes were simulated. The limits of the interval for the binomial distribution are symmetrical around the proportion 0.5;²⁰ for example, the limits of the interval for 0 out of 100 (0.00, 3.70) is identical to 100% minus the limits of the interval for 100 out of 100 (96.30, 100.00) (see tests 5 and 6 in Table 2). Therefore, the sample size that is listed in Table 2 covers a large range to estimate CIs and BCIs in clinical vaccine trials. For the simulation study of Bayesian informative method, the informative priors were assigned based on potential situations of clinical vaccine trials. For the scenarios for adverse event rates and incidences, the 95% prior rates were assumed to be (0.01, 10%) with Beta(1.1, 36.9) and (10, 30%) with Beta(10.8,46.3); for the scenarios for seroprotection rates and seroconversion rates, the 95% prior rates were assumed to be (90, 99.9%) with Beta(36.9, 1.1) and (70, 90%) with Beta(46.3, 10.8). The Beta distributions were calculated using the freeware of Parameter Solver.²⁹

Disclosure of Potential Conflicts of Interest

No potential conflicts of interest were disclosed.

Funding

This work was supported by the National Natural Science Foundation of China [grant number 81302512 and 81273184].