Performance of Bayesian Priors in Validation of Correlate of Protection for High Efficacy Vaccine Trials

Igwebuike Enweonye^*, Edith Uzoma Umeh

Department of Statistics, Nnamdi Azikiwe University, Awka, Nigeria.

DOI: 10.4236/oalib.1107978                

Abstract

Although the use of intermediate clinical endpoint or surrogate (correlate) of protection (CoP) has increased over the years, the validation of CoP for high efficacy vaccine trials has remained a challenge due to sparse data and conventional statistical methods which are not adequate. Be it in the frequentist or the Bayesian world, the meta-analytic approach is a well accepted method of validation. However, the full joint bivariate models suffer computational issues. And there is a push for the use of individual level instead of aggregate data in validation process. In this quest, the Bayesian approach is emerging as the future as regards the validation of CoP but one recurring criticism about this method is its application of prior distributions. To elucidate which makes better sense, the non-informative (NIP) and weakly informative prior (WIP) distributions are compared in a meta-analytic approach using simulated data. It was found that, 1) there are no convergence issues when either of the models is used, 2) WIP models take about 20% longer time than NIP models to converge, and 3) the NIP models consistently perform better than the WIP models.

Keywords

Validation, Correlate of Protection, Clinical Endpoint, Bayesian Hierarchical Modelling, Prior Distribution

1. Introduction

1.1. Background

The protective threshold of a vaccine is desirable in identifying the level of an immune marker above which vaccinees have a defined probability of being protected and make a statement over the vaccine efficacy. Such quantity defines the vaccine response threshold, used to calculate the response rate [1]. Sadly, during clinical development, vaccine correlate of protection is generally unknown [2]. Vaccines are mostly given as prophylactics, of which the true clinical endpoint is difficult to measure, therefore, approval relies largely on correlate of protection achievable by immunogenicity data.

Correlate of protection (CoP) may be used in lieu [3] when vaccine clinical endpoints of primary interest are hard to measure, or unethical. CoP is useful when they can be measured earlier, more conveniently, or more frequently than the true endpoints [4]. Its use in clinical studies has increased, necessitating the development of sound statistical methods in the validation process [5]. Health authorities around the world are opening doors to CoP, for example, between 2010 and 2012, the United States Food and Drug Administration (US FDA) approved 45 percent of new drugs applications based on various surrogate endpoints. A beneficial surrogate (correlate) of protection generally allows for more efficient drug development programs.

The Bayesian statistics provide a flexible tool for complex applications including the validation of correlate of protection. The beauty of Bayesian inference lies in the prior distribution which is its backbone although it has caused controversies among the scientific community; with some arguing that prior distribution introduces extra data. Distinction should be made between non-informative (also known as reference or objective) and informative priors. Lunn, Jackson, Best, Thomas & Spiegelhalter (2012) [6] state that non-informative priors are intended for use in situations where scientific objectivity is at premium, for example, when presenting results to a regulator or in scientific journal, illustrating that the Bayesian tool is a convenient way of dealing with complex multi-dimensional models. They did not support the use of the term non-informative prior but suggested to replace it with either vague or diffuse prior. In this paper the non-informative (NIP) and weakly informative (WIP) priors are compared as tools for validation of CoP using Gibbs sampler.

Gibbs Sampling is MCMC method which involves successive sampling from the complete conditional densities. For the working of MCMC algorithm, we refer to [7]-[10]. Samples may be drawn from standard densities or non-standard densities [10]. If the full conditionals are non-standard but of a certain mathematical form, then adaptive rejection sampling [11] may be used within the Gibbs sampling for those parameters. In other cases, alternative schemes based on the Metropolis-Hastings algorithm, may be used to sample from non-standard densities [12]. In JAGS there is no flexibility of specifying any one sampling method rather it runs as a black box and chooses the most efficient sampling method among those available. Enweonye & Umeh (2021) [13] considered the validation of correlate of protection in the context of high vaccine efficacy trials [2] in the Bayesian perspective using individual level data. This current work is an extension of [13] where the non-informative (NIP) and weakly informative prior (WIP) distributions are compared in a meta-analytic approach using simulated clinical data. Both papers are similar in methods, software, simulation of data and modelling.

1.2. Purpose of the Study

Although the idea of statistical validation of surrogate was developed in the context of single trial, the meta analytic validation has become a well accepted method. The conventional statistical methods are not adequate for the high vaccine efficacy trials due to sparse data. Joint modeling of correlate of protection and true clinical endpoints in the frequentist world poses computational issues. The two-stage meta analysis leads to loss of information causing a push toward the use of individual level instead of aggregate data in validation process.

In this quest, the Bayesian approach is emerging as the future as regards the validation of CoP but one recurring criticism about this method is its application of prior distributions. To elucidate which makes better sense, the non-informative (NIP) and weakly informative prior (WIP) distributions are compared in a meta-analytic approach using simulated data.

1.3. Meta-Analytic Validation

Several authors have discussed the validation of surrogate using data from multiple randomized trials [14]. Daniels & Hughes (1997) [15] gave a first formal Bayesian approach. These ideas were extended to the theory of linear mixed-effects models [16]. The model of Daniels & Hughes (1997) considered cases that individual data are not available. Later, [14] adopted Bayesian approach for continuous endpoints. Renard & Geys (2005) [17] discussed meta-analytic validation with binary outcomes. They adopted a latent variable approach, with the assumption that the observed binary variables result from dichotomizing an unobserved continuous variable based on certain threshold. Qin, Gilbert, Corey, McElrath & Self (2007) [18] proposed a hierarchical framework for assessing immunological correlates of protection in vaccine trials, while [19] provided the relationship between the causal-inference and meta-analytic approach.

Bujkiewicz, Thompson, Riley & Abrams (2016) [20] utilized Bayesian meta-analysis to incorporate multiple surrogate endpoints in drug development process. They extended Bayesian multivariate models to include multiple surrogate endpoints with the potential benefit of reducing the uncertainty when making predictions [20]. Callegaro & Tibaldi (2019) [2] used aggregated data to develop solutions for assessing CoP in the context of high vaccine efficacy. None of the authors has investigated a binary endpoint and normally distributed surrogate endpoint with individual level data. It turns out that the Bayesian approach to validate correlate of protection in the context of high vaccine efficacy with a true binary endpoint and a normal surrogate using individual data remains novel.

2. Study Design and Methods

We simulated data sets including both true clinical and surrogate endpoints. Each data set consists of 50 centers (used as trials) characterised by a 1:1 randomization and sample size of 100 participants per trial and 5000 participants for each data set. Bayesian hierarchical model using Markov Chain Monte Carlo (MCMC) was applied to each of the simulated data sets. The model combined non-informative prior (NIP) and then weakly informative prior (WIP) distributions with simulated data to obtain posterior information for inferences. Each MCMC simulation has 3 parallel chains, adapting every 1000 simulation steps, 1000 draws were discarded as burn-in samples, and another 10000 draws were used for inference. Standard inference calls R to run the model through Just Another Gibbs Sampler (JAGS) and extract predicted values for the monitored parameters, variance-covariance matrix between random treatment effects of the endpoints, $D$ and coefficient of determination, $R^2$ .

We consider the same models as in [13]. Let $S_{ij}$ and $T_{ij}$ represent the continuous and binary underlying values of the surrogate and the true endpoints, respectively, for subject $j$ in trial $i$ and $Z_{ij}$ an indicator for treatment effect. And further consider the meta-analytic framework in the single trial setting, in which the units are randomized subgroups of centers. At the first level of the hierarchical Bayesian meta-analytic approach, a bivariate model is specified as follows:

$S_{ij}={\mu }_S+m_{Si}+\left(\alpha +a_i\right)Z_{ij}+{\epsilon }_{Sij}$ (1)

$logit\left(T_{ij}=1\right)={\mu }_T+m_{Ti}+\left(\beta +b_i\right)Z_{ij}$ (2)

where ${\mu }_S$ and ${\mu }_T$ are fixed intercepts, $m_{Si}$ and $m_{Ti}$ are trial specific random intercepts, $\alpha$ and $\beta$ are fixed effects of treatment $Z$ on the endpoints in trial $i$ , $a_i$ and $b_i$ are the trial specific random effects of treatment on the endpoints. That a subject $j$ in trial $i$ has the disease is depicted with $T_{ij}=1$ . The error structure ${\epsilon }_{Sij}$ are surrogate associated normally distributed random error terms with mean zero and variance ${\sigma }_{{\mu }_S}^2$ . The random effects ($m_{Si}$ , $m_{Ti}$ , $a_i$ , $b_i$ ) are assumed to be mean-zero normally distributed with variance-covariance matrix,

$D=\left(\begin{array}{llll}{d}_{SS}\hfill & d_{ST}\hfill & d_{Sa}\hfill & d_{Sb}\hfill \\ \hfill & d_{TT}\hfill & d_{Ta}\hfill & d_{Tb}\hfill \\ \hfill & \hfill & d_{aa}\hfill & d_{ab}\hfill \\ \hfill & \hfill & \hfill & d_{bb}\hfill \end{array}\right)$ (3)

The surrogate endpoint validation is captured by means of the quantity, the trial-level $R^2$ . Provided (3) is positive definite, we have,

$R_{trial}^2=R_{b_i|m_{Si},a_i}^2=\frac{{\left(\begin{array}{l}{d}_{Sb}\hfill \\ d_{ab}\hfill \end{array}\right)}^{\text{T}}{\left(\begin{array}{ll}{d}_{SS}\hfill & d_{Sa}\hfill \\ d_{Sa}\hfill & d_{aa}\hfill \end{array}\right)}^{-1}\left(\begin{array}{l}{d}_{sb}\hfill \\ d_{ab}\hfill \end{array}\right)}{d_{bb}}$ (4)

Reduction of the above model is achieved by removing the trial-specific intercept and the error term in (1) and the trial-specific intercept in (2), assuming full mediation, leading to,

$S_{ij}={\alpha }_0+\left({\alpha }_1+a_i\right)Z_{ij}+{\epsilon }_{Sij}$ (5)

$logit\left(T_{ij}=1\right)={\beta }_0+\left({\beta }_1+b_i\right)Z_{ij}$ (6)

where,

$\left(\begin{array}{l}{S}_{ij}\hfill \\ T_{ij}\hfill \end{array}\right)~N\left[\left(\begin{array}{l}{\alpha }_0+\left({\alpha }_1\right)Z_{ij}\hfill \\ {\beta }_0+\left({\beta }_1\right)Z_{ij}\hfill \end{array}\right),\Sigma \right]$ (7)

And,

$\left(\begin{array}{l}{a}_i\hfill \\ b_i\hfill \end{array}\right)~N\left[\left(\begin{array}{l}0\hfill \\ 0\hfill \end{array}\right),D\right],D=\left(\begin{array}{ll}{d}_{aa}\hfill & d_{ab}\hfill \\ d_{ab}\hfill & d_{bb}\hfill \end{array}\right)$ (8)

The $R^2$ for the reduced models becomes,

$R_{trial\left(r\right)}^2=R_{b_i|a_i}^2=\frac{d_{ab}^2}{d_{aa}{d}_{bb}}$ (9)

The CoP $~N\left({\alpha }_0+{\alpha }_1\cdot Z_{ij},{\theta }_{{\mu }_S}^2\right)$ , The true clinical endpoint, $T~Bern\left(n_T,p_T\right)$ , where $n_T$ is the number of subjects and $p_T$ the probability of being protected by vaccination. The fixed treatment effects ${\alpha }_0~N\left(0,{\tau }_{\alpha 0}^2\right)$ , ${\alpha }_1~N(0,{\tau }_{\alpha 1}^2$ , ${\beta }_0~N\left(0,{\tau }_{\beta 0}^2\right)$ and ${\beta }_1~N\left(0,{\tau }_{\beta 1}^2\right)$ . At the second level of the hierarchical model, the priors for the fixed effects. For NIP models the following hyper-priors are specified:

$\begin{array}{l}{\theta }_T~Bern\left[p_T\right],\\ {\mu }_S~N\left(0,{\theta }_{\mu s}^2\right),\\ {\alpha }_0~N\left(0,{\tau }_{\alpha 0}^2\right),\\ {\alpha }_1~N\left(0,{\tau }_{\alpha 1}^2\right),\\ {\beta }_0~N\left(0,{\tau }_{\beta 0}^2\right),\\ {\beta }_1~N\left(0,{\tau }_{\beta 1}^2\right),\\ {\theta }_{\mu s}^{-2}~U\left(0,100\right),\\ {\tau }_{\alpha 0}^{-2}~Gamma\left({10}^{-4},{10}^{-4}\right),\\ {\tau }_{\alpha 1}^{-2}~Gamma\left({10}^{-4},{10}^{-4}\right),\\ {\tau }_{\beta 0}^{-2}~Gamma\left({10}^{-4},{10}^{-4}\right),\\ {\tau }_{\beta 1}^{-2}~Gamma\left({10}^{-4},{10}^{-4}\right),\end{array}$ (10)

Next, specify a prior distribution for the association between the treatment effects of the two endpoints and the random effects. As the hyper-prior distribution for the variance-covariance matrices, a Wishart distribution is assumed:

$\begin{array}{l}{D}^{-1}~Whishart\left(R_D\right)\\ {\Sigma }^{-1}~Whishart\left(R_{\Sigma }\right)\end{array}$ (11)

For WIP model the default prior as a function of the variances ($d_{aa}$ and $d_{bb}$ ) and the correlation between the two varying random effects $a_i$ and $b_i$ is given by:

$p\left(D\right)\propto {\left|D\right|}^{\frac{1}{2}}=d_{aa}{d}_{bb}\sqrt{1-{\rho }^2}$ (12)

The NIP and the WIP models differ in the assignment of priors for variance-covariance matrices of the random effects. Chung, Gelman, Rabe-Hesketh, Liu & Dorie (2015) [21] suggested these weakly informative priors for the variance-covariance of the random effects. All other priors remain same as for NIP model. This is a huge difference since the coefficients of determination R² depends on it as seen earlier.

$\begin{array}{l}{d}_{aa}^{-1}~Gamma\left(1.5,{10}^{-2}\right),\\ d_{bb}^{-1}~Gamma\left(1.5,{10}^{-2}\right),\\ D^{-1}~Beta\left(1.5,1.5\right)\end{array}$ (13)

The trial-level surrogacy is assessed using the posterior means for the coefficients of determination, Equation (9). A sufficiently large R² is an indicator of a good surrogate. Beside statistics, clinical and epidemiological judgments, as deemed fit by the experts, are taken into account before a surrogate is finally adopted.

Gibbs Sampling is MCMC method which involves successive sampling from the complete conditional densities. Markov chain is an integer-time process, $\left\{X_n\ge 0\right\}$ , for which the sample values for each random variable $X_n$ , $n\ge 1$ , lie in a countable set $S$ and depend on the past only through the most recent random variable $X_{n-1}$ [9]. Samples may be drawn from standard densities or non-standard densities [10]. The adaptive rejection sampling (Gilks and Wild, 1992) may be used within the Gibbs sampling for those parameters if the full conditionals are non-standard but of a certain mathematical form. In other cases, alternative schemes based on the Metropolis-Hastings algorithm, may be used to sample from non-standard densities (Morgan, 2000). For further details working of MCMC algorithms refer to [10] and [8]. Modeling was performed using Just Another Gibbs Sampler in R (RJAGS) as an interface to JAGS (JAGS 4.3.0 release July 18 2017). In JAGS there is no flexibility of specifying any one sampling method rather it runs as a black box and chooses the most efficient sampling method among those available [13].

3. Results and Discussion

3.1. Results

With a range of vaccine efficacy (VE = 0.30, 0.70, 0.75, 0.82, 0.95, 0.96, 0.97, 0.98 and 1), a total of 70 scenarios were simulated in R. The simulated data contain both a true binary outcome and a continuous immunogenicity values as correlate op protection, using the reduced models in Equations (5) and (6) without random intercepts. Each scenario has a sample size N = 5000 subjects. Randomisation was performed within 50 trials in a 1:1 ratio to treated or untreated groups of 100 subjects in each trial.

The following parameters were used in the Simulation: ${\mu }_S=4.609$ ; ${\mu }_T=\left(-2.0,-3.5,-4.0,4.5,-5.0,-5.6,-7\right)$ ; $\alpha =5.458$ ; $\beta =\left(-1.43,-1.45,-1.7591,-3\right)$ ; $Var\left(a_i\right)=10$ ; $Var\left(b_i\right)=4$ . The correlation between the treatment random effects is $\rho =Cor\left(a_i,b_i\right)=\sqrt{0.9}$ , with $R^2=0.9$ .

Let $P\left(T=1|Z=1\right)$ be the probabilities of disease among vaccinated individuals and $P\left(T=1|Z=0\right)$ be the probabilities of disease among unvaccinated individuals, respectively.

The expression for vaccine efficacy is:

$VE=\left(1-\frac{P\left(T=1|Z=1\right)}{P\left(T=1|Z=0\right)}\right)100\text{\%}$ (14)

Reduced models without random intercepts (5) and (6) were applied. The simulated data were loaded and prior values specified for MCMC. The modelling steps were performed for both NIP and WIP alike. Each MCMC simulation used 1000 samples as burn in, while 10,000 iterations were used for inference. The sampler adapts its behaviour to maximize its efficiency after every 1000 iterations. Trace plots reveal the stability and proper mixing of the monitored parameters R² and variance-covariance matrix across the 3 parallel chains.

The small subgroups were used as trials for the meta-analysis. For VE = 30%, 70%, 80% and 95%, the results of NIP and WIP are compared in Table 1, Table 2, Table 3 and Table 4 respectively. Both models converge without problems. However, the NIP model consistently outperforms the WIP model in the sense of mean R² and lower precision.

Table 1. Comparison of results VE = 30%.

Param	NIP				WIP
	Mean	SD	Naive SE	Time-series SE	Mean	SD	Naive SE	Time-series SE
Dmat[1,1]	4.0533	0.92149	0.005320	0.014254	3.9449	0.89939	0.005193	0.013943
Dmat[2,1]	6.5132	1.42206	0.008210	0.014736	6.1999	1.35424	0.007819	0.014354
Dmat[1,2]	6.5132	1.42206	0.008210	0.014736	6.1999	1.35424	0.007819	0.014354
Dmat[2,2]	11.5323	2.44915	0.014140	0.021057	11.2986	2.36500	0.013654	0.022884
R2	0.9087	0.03288	0.000190	0.000623	0.8639	0.04793	0.000277	0.001509
Dmat[1,1]	5.9584	1.33923	0.007732	0.017447	5.8872	1.29869	0.007498	0.018611
Dmat[2,1]	9.0683	1.97891	0.011425	0.016122	8.7076	1.87628	0.010833	0.019701
Dmat[1,2]	9.0683	1.97891	0.011425	0.016122	8.7076	1.87628	0.010833	0.019701
Dmat[2,2]	15.1804	3.22186	0.018601	0.024470	14.8890	3.09118	0.017847	0.032181
R2	0.9098	0.03115	0.000180	0.000521	0.8657	0.04572	0.000264	0.001411
Dmat[1,1]	4.7558	1.07016	0.006179	0.015427	4.6378	1.04233	0.006018	0.01675
Dmat[2,1]	6.8041	1.48787	0.008590	0.016261	6.4635	1.41823	0.008188	0.01803
Dmat[1,2]	6.8041	1.48787	0.008590	0.016261	6.4635	1.41823	0.008188	0.01803
Dmat[2,2]	10.5903	2.27877	0.013157	0.025884	10.3270	2.19202	0.012656	0.02647
R2	0.9203	0.03052	0.000176	0.000718	0.8733	0.04697	0.000271	0.00179
Dmat[1,1]	2.8118	0.64071	0.003699	0.009013	2.7921	0.62779	0.003625	0.008954
Dmat[2,1]	4.3419	0.96447	0.005568	0.009116	4.1523	0.92241	0.005326	0.009421
Dmat[1,2]	4.3419	0.96447	0.005568	0.009116	4.1523	0.92241	0.005326	0.009421
Dmat[2,2]	7.7676	1.67559	0.009674	0.015418	7.6992	1.63248	0.009425	0.015319
R2	0.8643	0.04683	0.000270	0.000927	0.8036	0.06496	0.000375	0.001885
Dmat[1,1]	4.7940	1.09171	0.006303	0.014241	4.6427	1.04932	0.006058	0.016024
Dmat[2,1]	6.8001	1.49962	0.008658	0.012974	6.4538	1.42252	0.008213	0.016375
Dmat[1,2]	6.8001	1.49962	0.008658	0.012974	6.4538	1.42252	0.008213	0.016375
Dmat[2,2]	10.7203	2.29062	0.013225	0.020582	10.6217	2.22064	0.012821	0.021601
R2	0.9005	0.03531	0.000204	0.000686	0.8454	0.05433	0.000314	0.001692
Dmat[1,1]	3.7355	0.84472	0.004877	0.011363	3.6527	0.81543	0.004708	0.011284
Dmat[2,1]	5.8277	1.28343	0.007410	0.012311	5.5241	1.21619	0.007022	0.013316
Dmat[1,2]	5.8277	1.28343	0.007410	0.012311	5.5241	1.21619	0.007022	0.013316
Dmat[2,2]	10.1792	2.17406	0.012552	0.020188	9.9498	2.09989	0.012124	0.021414
R2	0.8938	0.03753	0.000217	0.000699	0.8403	0.05427	0.000313	0.001635

Table 2. Comparison of results VE = 70%.

Param	NIP				WIP
	Mean	SD	Naive SE	Time-series SE	Mean	SD	Naive SE	Time-series SE
Dmat[1,1]	5.7900	1.60136	0.009246	0.080407	5.8065	1.67774	0.009687	0.094235
Dmat[2,1]	7.5704	1.78781	0.010322	0.061251	7.1525	1.71157	0.009882	0.058613
Dmat[1,2]	7.5704	1.78781	0.010322	0.061251	7.1525	1.71157	0.009882	0.058613
Dmat[2,2]	10.6336	2.31954	0.013392	0.038409	10.2679	2.18146	0.012595	0.031980
R2	0.9365	0.02997	0.000173	0.001007	0.8658	0.06055	0.000350	0.003383
Dmat[1,1]	3.2587	0.81166	0.004686	0.023335	3.2397	0.80682	0.004658	0.023183
Dmat[2,1]	4.8893	1.10962	0.006406	0.022208	4.6432	1.04631	0.006041	0.015379
Dmat[1,2]	4.8893	1.10962	0.006406	0.022208	4.6432	1.04631	0.006041	0.015379
Dmat[2,2]	8.0961	1.75978	0.010160	0.022593	7.9282	1.67500	0.009671	0.017433
R2	0.9096	0.03658	0.000211	0.000985	0.8437	0.06246	0.000361	0.002558
Dmat[1,1]	5.1164	1.26863	0.007325	0.044367	5.0483	1.25725	0.007259	0.048321
Dmat[2,1]	7.0250	1.59357	0.009201	0.040029	6.7147	1.51666	0.008756	0.035424
Dmat[1,2]	7.0250	1.59357	0.009201	0.040029	6.7147	1.51666	0.008756	0.035424
Dmat[2,2]	10.2632	2.23856	0.012924	0.042112	9.9694	2.11865	0.012232	0.028725
R2	0.9427	0.02557	0.000148	0.000743	0.8991	0.04429	0.000256	0.002104
Dmat[1,1]	4.3508	1.09760	0.006337	0.03569	4.5347	1.14335	0.006601	0.040648
Dmat[2,1]	5.8134	1.34244	0.007751	0.02562	5.6567	1.31957	0.007619	0.030489
Dmat[1,2]	5.8134	1.34244	0.007751	0.02562	5.6567	1.31957	0.007619	0.030489
Dmat[2,2]	9.1136	1.97332	0.011393	0.02381	9.1855	1.96122	0.011323	0.027995
R2	0.8553	0.05192	0.000300	0.00128	0.7718	0.07445	0.000430	0.002449
Dmat[1,1]	3.5487	0.8804	0.005083	0.026541	3.614	0.89989	0.0051955	0.028021
Dmat[2,1]	5.2356	1.2127	0.007002	0.022425	4.996	1.16611	0.0067325	0.020663
Dmat[1,2]	5.2356	1.2127	0.007002	0.022425	4.996	1.16611	0.0067325	0.020663
Dmat[2,2]	9.1817	1.9830	0.011449	0.022378	9.142	1.93127	0.0111502	0.017196
R2	0.8318	0.05771	0.000333	0.001280	0.759	0.07882	0.0004551	0.002473
Dmat[1,1]	3.5222	0.88773	0.005125	0.028679	3.5165	0.9004	0.0051987	0.029457
Dmat[2,1]	4.7017	1.07601	0.006212	0.019884	4.4540	1.0267	0.0059275	0.017449
Dmat[1,2]	4.7017	1.07601	0.006212	0.019884	4.4540	1.0267	0.0059275	0.017449
Dmat[2,2]	7.0281	1.54080	0.008896	0.017866	6.9352	1.4835	0.0085648	0.017234
R2	0.8969	0.04313	0.000249	0.001311	0.8185	0.0714	0.000412	0.002841

Table 3. Comparison of results VE = 80%.

Param	NIP				WIP
	Mean	SD	Naive SE	Time-series SE	Mean	SD	Naive SE	Time-series SE
Dmat[1,1]	3.9215	0.98386	0.005680	0.031238	3.7936	0.93076	0.005374	0.025544
Dmat[2,1]	5.0557	1.15058	0.006643	0.020997	4.7433	1.07112	0.006184	0.017635
Dmat[1,2]	5.0557	1.15058	0.006643	0.020997	4.7433	1.07112	0.006184	0.017635
Dmat[2,2]	7.1312	1.55360	0.008970	0.019162	6.9447	1.47657	0.008525	0.015217
R2	0.9175	0.03465	0.000200	0.000981	0.8574	0.05984	0.000346	0.002708
Dmat[1,1]	2.8296	0.70304	0.004059	0.017729	2.8292	0.6876	0.003970	0.015377
Dmat[2,1]	3.9520	0.91457	0.005280	0.012849	3.9429	0.9082	0.005244	0.012665
Dmat[1,2]	3.9520	0.91457	0.005280	0.012849	3.9429	0.9082	0.005244	0.012665
Dmat[2,2]	6.4745	1.41441	0.008166	0.013470	6.4703	1.4197	0.008197	0.013851
R2	0.8553	0.05577	0.000322	0.001503	0.8517	0.0569	0.000329	0.001474
Dmat[1,1]	3.2473	0.79477	0.0045886	0.019687	3.1911	0.77408	0.004469	0.019433
Dmat[2,1]	4.8114	1.08332	0.0062545	0.014528	4.5528	1.01763	0.005875	0.014647
Dmat[1,2]	4.8114	1.08332	0.0062545	0.014528	4.5528	1.01763	0.005875	0.014647
Dmat[2,2]	7.9002	1.70454	0.0098412	0.014984	7.7583	1.64555	0.009501	0.015100
R2	0.9053	0.03882	0.0002241	0.001058	0.8411	0.06277	0.000362	0.002624
Dmat[1,1]	4.4536	1.12449	0.006492	0.031982	4.5638	1.13772	0.006569	0.027283
Dmat[2,1]	6.4673	1.51733	0.008760	0.038792	6.1527	1.42107	0.008205	0.019979
Dmat[1,2]	6.4673	1.51733	0.008760	0.038792	6.1527	1.42107	0.008205	0.019979
Dmat[2,2]	11.0120	2.42494	0.014000	0.051043	10.8586	2.28621	0.013199	0.021878
R2	0.8552	0.05148	0.000297	0.001234	0.7674	0.07426	0.000429	0.002065
Dmat[1,1]	3.6452	0.88019	0.005082	0.01936	3.7243	0.90668	0.005235	0.02269
Dmat[2,1]	5.0568	1.16755	0.006741	0.01772	4.8895	1.14526	0.006612	0.02171
Dmat[1,2]	5.0568	1.16755	0.006741	0.01772	4.8895	1.14526	0.006612	0.02171
Dmat[2,2]	8.4496	1.83979	0.010622	0.02659	8.5219	1.82092	0.010513	0.02721
R2	0.8318	0.05771	0.000333	0.00128	0.7558	0.07851	0.000453	0.00222
Dmat[1,1]	4.1261	1.0094	0.005828	0.027354	4.0037	0.97946	0.005655	0.024051
Dmat[2,1]	6.1411	1.3828	0.007984	0.023597	5.7731	1.30059	0.007509	0.018684
Dmat[1,2]	6.1411	1.3828	0.007984	0.023597	5.7731	1.30059	0.007509	0.018684
Dmat[2,2]	10.0831	2.1704	0.012531	0.025903	9.9406	2.10173	0.012134	0.022174
R2	0.9092	0.0364	0.000210	0.000952	0.8408	0.06053	0.000350	0.002298

Table 4. Comparison of results 95%.

Param	NIP				WIP
	Mean	SD	Naive SE	Time-series SE	Mean	SD	Naive SE	Time-series SE
Dmat[1,1]	1.0901	0.6997	0.004040	0.06956	1.1261	0.8512	0.004915	0.08368
Dmat[2,1]	2.3933	1.1262	0.006502	0.09791	1.3831	0.9997	0.005772	0.06248
Dmat[1,2]	2.3933	1.1262	0.006502	0.09791	1.3831	0.9997	0.005772	0.06248
Dmat[2,2]	8.8972	1.8953	0.010942	0.01606	9.5796	2.1271	0.012281	0.02597
R2	0.6344	0.2219	0.001281	0.01568	0.2529	0.1930	0.001114	0.01062
Dmat[1,1]	1.9628	1.2498	0.0072160	0.15305	1.6457	1.1713	0.006763	0.11282
Dmat[2,1]	3.5079	1.4537	0.0083927	0.14525	2.1113	1.1799	0.006812	0.08162
Dmat[1,2]	3.5079	1.4537	0.0083927	0.14525	2.1113	1.1799	0.006812	0.08162
Dmat[2,2]	8.4924	1.8466	0.0106611	0.02173	8.9194	1.9596	0.011314	0.02292
R2	0.7868	0.1548	0.0008935	0.01148	0.3749	0.2163	0.001249	0.01250
Dmat[1,1]	1.7748	1.0017	0.0057833	0.104102	1.160	0.8601	0.004966	0.07584
Dmat[2,1]	4.0094	1.5388	0.0088845	0.140845	2.078	1.3820	0.007979	0.12022
Dmat[1,2]	4.0094	1.5388	0.0088845	0.140845	2.078	1.3820	0.007979	0.12022
Dmat[2,2]	11.837	2.5109	0.0144967	0.021245	12.557	2.7445	0.015845	0.03471
R2	0.8022	0.1345	0.0007763	0.008629	0.357	0.2213	0.001277	0.01490
Dmat[1,1]	1.1874	0.7746	0.004472	0.07767	1.1094	1.2385	0.007151	0.16124
Dmat[2,1]	2.8639	1.3421	0.007748	0.12163	1.7543	1.3439	0.007759	0.13620
Dmat[1,2]	2.8639	1.3421	0.007748	0.12163	1.7543	1.3439	0.007759	0.13620
Dmat[2,2]	10.480	2.2441	0.012956	0.01847	11.0808	2.4535	0.014165	0.04168
R2	0.6989	0.2030	0.001172	0.01621	0.3277	0.2165	0.001250	0.01338
Dmat[1,1]	2.0189	1.2892	0.007443	0.146744	1.8263	1.4999	0.008660	0.15756
Dmat[2,1]	3.6996	1.4835	0.008565	0.134978	2.4119	1.5062	0.008696	0.11473
Dmat[1,2]	3.6996	1.4835	0.008565	0.134978	2.4119	1.5062	0.008696	0.11473
Dmat[2,2]	9.0266	1.9749	0.011402	0.025145	9.5311	2.0957	0.012099	0.02728
R2	0.7988	0.1285	0.000742	0.007326	0.3972	0.2152	0.001242	0.01243
Dmat[1,1]	2.526	1.26800	0.0073208	0.123232	2.5511	1.5449	0.008920	0.167465
Dmat[2,1]	4.801	1.61778	0.0093402	0.140299	3.7371	1.5216	0.008785	0.100247
Dmat[1,2]	4.801	1.61778	0.0093402	0.140299	3.7371	1.5216	0.008785	0.100247
Dmat[2,2]	11.164	2.40545	0.0138879	0.022957	11.4913	2.4507	0.014149	0.031537
R2	0.8510	0.09706	0.0005604	0.005979	0.5298	0.1783	0.001029	0.009937

3.2. Discussion

Molenbergs et al. (2004) investigated computational issues of random-effects. They concluded that when the between-trial variability gets smaller, convergence problems do arise and worsen as the number of trials decreases. Renard et al. (2002) simulation study show that both R² (trial and individual) tend to be biased in small samples but, bias in R² (individual) can be eliminated by increasing overall sample size (i.e., trial size and/or number of trials), whereas bias in R² (trial) full models (1) and (2) can be reduced by increasing replication at the trial level.

With a sample size of N = 5000 for each scenario and 10,000 MCMC iterations huge database is created which edges the any frequentist method. The success of the large database meant that the between-trial variability is reduced, also, the MCMC tool does not allow convergence problems to arise. For all vaccine efficacy VE = 30%, 70%, 80% and 95%, Tables 1-4, the NIP model consistently showed better coefficient of determination R².

4. Conclusion

We applied a reduced bivariate model with trial specific random treatment effects on the endpoints with no correlated residuals. Comparison of coefficient of determination showed R² consistently better for NIP compared to WIP model for all vaccine efficacy VE = 30% - 95%. The standard errors are also smaller for NIP than WIP. In both models, a significant improvement of R² was observed, as expected. From the study, these conclusions were made: 1) Although the full joint bivariate models can suffer computational issues, no convergence problems were experienced for any of the models used; 2) WIP models take about 20% longer time than NIP models to converge; and 3) the NIP models consistently perform better (higher mean R² and lower precision) than the WIP models.

Conflicts of Interest

The authors declare no conflicts of interest.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Voysey, M., Sadarangani, M., Pollard, A.J. and Fanshawe, T.R. (2018) Computing Threshold Antibody Levels of Protection in Vaccine Clinical Trials: An Assessment of Methodological Bias. PLOS ONE, 13, e0202517. https://doi.org/10.1371/journal.pone.0202517
[2]	Callegaro, A. and Tibaldi, F. (2019) Assessing Correlates of Protection in Vaccine Trials: Statistical Solutions in the Context of High Vaccine Efficacy. BMC Medical Research Methodology, 19, Article No. 47. https://doi.org/10.1186/s12874-019-0687-y
[3]	Buyse, M. and Molenberghs, G. (1998) Criteria for the Validation of Surrogate Endpoints in Randomized Experiments. Biometrics, 54, 1014-1029. https://doi.org/10.2307/2533853
[4]	Ellenberg, S.S. and Hamilton, J.M. (1989) Surrogate Endpoints in Clinical Trials: Cancer. Statistics in Medicine, 8, 405-413. https://doi.org/10.1002/sim.4780080404
[5]	Burzykowski, T., Molenberghs, G. and Buyse, M. (2006) The Evaluation of Surrogate Endpoints. Springer Science & Business Media.
[6]	Lunn, D., Jackson, C., et al. (2012) The BUGS Book: A Practical Introduction to Bayesian Analysis. CRC Press.
[7]	Gelman, A. and Shalizi, C.R. (2012) Philosophy and the Practice of Bayesian Statistics. British Journal of Mathematical and Statistical Psychology, 66, 8-38. https://doi.org/10.1111/j.2044-8317.2011.02037.x
[8]	Gelman, A., Carlin, J.B., et al. (2013) Bayesian Data Analysis. CRC Press.
[9]	Gallager, R.G. (2013) Stochastic Processes. Cambridge University Press. https://doi.org/10.1017/cbo9781139626514
[10]	Congdon, P. (2014) Applied Bayesian Modelling. Wiley. https://doi.org/10.1002/9781118895047
[11]	Gilks, W.R. and Wild, P. (1992) Adaptive Rejection Sampling for Gibbs Sampling. Applied Statistics, 41, 337-348. https://doi.org/10.2307/2347565
[12]	Brooks, S.P., Catchpole, E.A. and Morgan, B.J.T. (2000) Bayesian Animal Survival Estimation. Statistical Science, 15, 357-376. https://doi.org/10.1214/ss/1009213003
[13]	Enweonye, I. and Umeh, E.U. (2022) Bayesian Meta-Analysis to Validate Correlate of Protection for High Vaccine Efficacy Clinical Trials. European Journal of Information Technologies and Computer Science, 2, 19-25. https://doi.org/10.24018/compute.2022.2.1.34
[14]	Shkedy, Z. and Barbosa, F.T. (2005) Bayesian Evaluation of Surrogate Endpoints. In: Statistics for Biology and Health, Springer, 253-270. https://doi.org/10.1007/0-387-27080-9_15
[15]	Daniels, M.J. and Hughes, M.D. (1997) Meta-Analysis for the Evaluation of Potential Surrogate Markers. Statistics in Medicine, 16, 1965-1982. https://doi.org/10.1002/(sici)1097-0258(19970915)16:17<1965::aid-sim630>3.0.co;2-m
[16]	Buyse, M., Molenberghs, G., Burzykowski, T., Renard, D. and Geys, H. (2000) The Validation of Surrogate Endpoints in Meta-Analyses of Randomized Experiments. Biostatistics, 1, 49-67. https://doi.org/10.1093/biostatistics/1.1.49
[17]	Renard, D. and Geys, H. (2005) Meta-Analytic Validation with Binary Outcomes. In: Statistics for Biology and Health, Springer, 153-162. https://doi.org/10.1007/0-387-27080-9_10
[18]	Qin, L., Gilbert, P.B., Corey, L., McElrath, M.J. and Self, S.G. (2007) A Framework for Assessing Immunological Correlates of Protection in Vaccine Trials. The Journal of Infectious Diseases, 196, 1304-1312. https://doi.org/10.1086/522428
[19]	Alonso, A., Van der Elst, W., Molenberghs, G., Buyse, M. and Burzykowski, T. (2014) On the Relationship between the Causal-Inference and Meta-Analytic Paradigms for the Validation of Surrogate Endpoints. Biometrics, 71, 15-24. https://doi.org/10.1111/biom.12245
[20]	Bujkiewicz, S., Thompson, J.R., Riley, R.D. and Abrams, K.R. (2015) Bayesian Meta-Analytical Methods to Incorporate Multiple Surrogate Endpoints in Drug Development Process. Statistics in Medicine, 35, 1063-1089. https://doi.org/10.1002/sim.6776
[21]	Chung, Y., Gelman, A., Rabe-Hesketh, S., Liu, J. and Dorie, V. (2015) Weakly Informative Prior for Point Estimation of Covariance Matrices in Hierarchical Models. Journal of Educational and Behavioral Statistics, 40, 136-157. https://doi.org/10.3102/1076998615570945