A trivariate Bernoulli regression model

Abstract

A trivariate Bernoulli regression model is proposed in this paper. There is extensive need for analysing repeated binary outcomes where correlated binary outcomes are obtained from repeated measures or longitudinal data. The proposed model is based on marginal and conditional probabilities as functions of covariates. The estimation and test procedures are shown. The tests include testing of hypotheses for first- and second-order associations among outcome variables. It is noteworthy that the test procedures for dependence in outcome variables can be demonstrated in terms of vectors of regression parameters for models on outcome variables. The proposed model can be extended for more than three correlated outcome variables conveniently.

Keywords:

• trivariate Bernoulli
• regression model
• repeated measures
• marginal probability
• conditional probability
• joint probability
• correlated outcomes

PUBLIC INTEREST STATEMENT

This research work provides an important development in modelling correlated outcomes data and shows the test for different types of associations. This model is developed for three binary outcomes. In various fields, the presence of correlation in outcome variables poses formidable difficulty to model the relationship between potential explanatory variables and outcome variables. However, if the dependence in outcome variables is not considered in modelling, the relationships between explanatory and outcome variables may be affected to a large extent resulting in misleading results. This paper provides a trivariate Bernoulli regression model and tests for the potential relationships of first, second and third orders are proposed. The applications of these procedures will make the analysis of correlated binary outcomes possible with appropriate interpretation and underlying mechanism of relationships among outcome variables and between outcome and explanatory variables.

1. Introduction

The use of Bernoulli regression model is well established. The logistic regression model has been one of the most extensively used techniques in various applications which is based on univariate Bernoulli distribution. Since the development of generalized linear models, it has been presented more explicitly using the logit link function from exponential family of distributions for binary data. There have been attempts to develop regression models for bivariate and multivariate Bernoulli regression models. Some noteworthy works were presented by McCullagh and Nelder (1989), Glonek and McCullagh (1995), Yee and Dirnbock (2009) and Dai, Ding, and Wahba (2013). McCullagh and Nelder (1989) considered proportional odds model as a starting point for constructing proportional odds model for three ordinal categories. Glonek and McCullagh generalized the model for several categorical responses. Alternative conditional models based on Markovian assumptions were proposed by Islam and Chowdhury (2006, 2008, 2017), Islam, Chowdhury, and Huda (2009) and Islam et al. (2012a). On the other hand, Dai et al. (2013) showed a multivariate Bernoulli model to estimate structure of graphs with binary nodes. Islam, Alzaid, Chowdhury, and Sultan (2013) proposed an alternative procedure using marginal-conditional approach to construct a bivariate Bernoulli model and provided tests for dependence in outcome variables. In this paper, a trivariate Bernoulli regression model is shown using marginal-conditional approach. Tests for dependence in trivariate models are also shown. The proposed model can be used very extensively in longitudinal studies with trivariate binary outcomes in various fields.

2. Trivariate Bernoulli distribution

Marshall and Olkin (1985) showed the bivariate Bernoulli form of $Y_1$ and $Y_2$with Bernoulli marginal. Let us consider three binary variables $Y_1$, $Y_2$ and $Y_3$, which, in longitudinal studies, can be considered as status of outcome variables at time points $T_1$, $T_2$ and $T_3$, respectively. The probability distribution is displayed below:

$\begin{array}{cccccc}{Y}_1& Y_2& \hspace{0.17em}& Y_3& \hspace{0.17em}& \mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}\\ 0& 0& 0& \hspace{0.17em}& p_{001}& p_{00}\\ 0& 1& p_{000}& \hspace{0.17em}& p_{011}& p_{01}\\ 1& 0& p_{010}& \hspace{0.17em}& p_{101}& p_{10}\\ 1& 1& p_{100}& \hspace{0.17em}& p_{111}& p_{11}\\ \mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}& \hspace{0.17em}& p_{110}& \hspace{0.17em}& {p_{..}}_1& 1\end{array}$

and the trivariate Bernoulli probability can be represented as follows

(1) $\begin{array}{c}P(Y_1=y_1,Y_2=y_2,Y_3=y_3)=p_{000}^{(1-y_1)(1-y_2)(1-y_3)}{p}_{001}^{(1-y_1)(1-y_2)y_3}\\ p_{010}^{(1-y_1)y_2(1-y_3)}{p}_{100}^{y_1(1-y_2)(1-y_3)}{p}_{011}^{(1-y_1)y_2{y}_3}{p}_{101}^{y_1(1-y_2)y_3}{p}_{110}^{y_1{y}_2(1-y_3)}{p}_{111}^{y_1{y}_2{y}_3}.\end{array}$(1)

We can easily find the expression for the above trivariate Bernoulli in exponential family form and after taking log the log-likelihood for n = 1 is

(2) $\begin{array}{rl}& l=y_{1}ln\left(\frac{p_{100}}{p_{000}}\right)+y_{2}ln\left(\frac{p_{010}}{p_{000}}\right)+y_{3}ln\left(\frac{p_{001}}{p_{000}}\right)+y_1{y}_{2}ln\left(\frac{p_{110}{p}_{000}}{p_{100}{p}_{010}}\right)+\\ & y_1{y}_{3}ln\left(\frac{p_{101}{p}_{000}}{p_{100}{p}_{001}}\right)+y_2{y}_{3}ln\left(\frac{p_{011}{p}_{000}}{p_{010}{p}_{001}}\right)+y_1{y}_2{y}_{3}ln\left(\frac{p_{100}{p}_{010}{p}_{001}{p}_{111}}{p_{110}{p}_{011}{p}_{101}{p}_{000}}\right)+ln{p}_{000}.\end{array}$(2)

The natural link functions are:

(3) $\begin{array}{rl}& {\theta }_1=ln\left(\frac{p_{100}}{p_{000}}\right),{\theta }_2=ln\left(\frac{p_{010}}{p_{000}}\right),{\theta }_3=ln\left(\frac{p_{001}}{p_{000}}\right),\\ & {\theta }_{12}=ln\left(\frac{p_{110}{p}_{000}}{p_{100}{p}_{010}}\right),{\theta }_{13}=ln\left(\frac{p_{101}{p}_{000}}{p_{100}{p}_{001}}\right),{\theta }_{23}=ln\left(\frac{p_{011}{p}_{000}}{p_{010}{p}_{001}}\right),\\ & {\theta }_{123}=ln\left(\frac{p_{100}{p}_{010}{p}_{001}{p}_{111}}{p_{110}{p}_{011}{p}_{101}{p}_{000}}\right),{\theta }_0=ln{p}_{000}.\end{array}$(3)

Islam, Alzaid, Chowdhury, and Sultan (2013) showed for bivariate Bernoulli regression model that underlying relationships can be explored more conveniently if a marginal-conditional approach is employed. We can use the underlying conditional and marginal models for expressing the association parameters in the proposed model. The bivariate Bernoulli model is comprised of 3 ($2^2-1$) models, two conditional and one marginal. Extension to trivariate Bernoulli shows that there are 7 (2³–1) models. It appears from the above link functions that there are underlying relationships between link functions of three first, three second and one third orders that emerge from the natural link functions. The first-order link functions are simply odds between respective cell probability for an outcome variable and baseline probability of non-occurrence of any of the outcomes at three time points.

3. The marginal-conditional models for trivariate Bernoulli

We can express the joint probability of three outcome variables Y₁, Y₂ and Y₃ for given X as follows:

(4) $P(Y_1=y_1,Y_2=y_2,Y_3=y_3|X)=P(Y_1=y_1|X)P(Y_2=y_2|y_1,X)P(Y_3=y_3|y_1,y_2,X)$(4)

Here is $P(Y_1=y_1|X)={\pi }_{y_1}(X)$ marginal probability for Y₁, $P(Y_2=y_2|y_1,X)={\pi }_{y_1{y}_2}(X)$ is conditional probability for Y₂ given Y₁, $P(Y_3=y_3|y_1,y_2,X)={\pi }_{y_1{y}_2{y}_3}(X)$ is conditional probability for Y₃ given Y₁ and Y₂, and $X=(1,X_1,...,X_p)$. Using the relationship shown in Equation (4), the joint probabilities are

(5) $\begin{array}{rl}& p_{000}(X)=\left[1-{\pi }_1(X)\right]\left[1-{\pi }_{01}(X)\right]\left[1-{\pi }_{001}(X)\right]\\ & p_{100}(X)={\pi }_1(X)\left[1-{\pi }_{11}(X)\right]\left[1-{\pi }_{101}(X)\right]\\ & p_{010}(X)=\left[1-{\pi }_1(X)\right]{\pi }_{01}(X)\left[1-{\pi }_{011}(X)\right]\\ & p_{001}(X)=\left[1-{\pi }_1(X)\right]\left[1-{\pi }_{01}(X)\right]{\pi }_{001}(X)\\ & p_{110}(X)={\pi }_1(X){\pi }_{11}(X)\left[1-{\pi }_{111}(X)\right]\\ & p_{011}(X)=\left[1-{\pi }_1(X)\right]{\pi }_{01}(X){\pi }_{011}(X)\\ & p_{101}(X)={\pi }_1(X)\left[1-{\pi }_{11}(X)\right]{\pi }_{101}(X)\\ & p_{111}(X)={\pi }_1(X){\pi }_{11}(X){\pi }_{111}(X)\end{array}$(5)

The marginal model ${\pi }_1(X)$ is

(6) ${\pi }_1(X)=\frac{e^{X{\beta }_1}}{1+e^{X{\beta }_1}}$(6)

where ${\beta }_1=({\beta }_{10},{\beta }_{11},...,{\beta }_{1p})$ and ${\pi }_0(X)=\frac{1}{1+e^{X{\beta }_1}}$ as ${\pi }_0(X)+{\pi }_1(X)=1$.

The conditional probabilities can be obtained from the first- and second-order Markov models with covariate dependence (Islam et al., 2012b, 2012a; Islam et al., 2013; Islam & Chowdhury, 2017). The conditional models for first-order Markov chain transition probabilities in the relationships (5) are displayed in Table 1 shown below for outcome variables Y₁ and Y₂:

Table 1. Transition probabilities for outcome variables Y₁ and Y₂

Y1	Y2
	0	1
0	${\pi }_{00}(X)$	${\pi }_{01}(X)$
1	${\pi }_{10}(X)$	${\pi }_{11}(X)$

The transition probabilities are functions of covariates as displayed below

(7) ${\pi }_{01}(X)=\frac{e^{X{\beta }_{01}}}{1+e^{X{\beta }_{01}}},$(7)

(8) ${\pi }_{11}(X)=\frac{e^{X{\beta }_{11}}}{1+e^{X{\beta }_{11}}}$(8)

where ${\beta }_{01}=({\beta }_{010},{\beta }_{011},...,{\beta }_{01p}),{\beta }_{11}=({\beta }_{110},{\beta }_{111},...,{\beta }_{11p})$,

$\begin{array}{c}{\pi }_{00}(X)=\frac{1}{1+e^{X{\beta }_{01}}},{\pi }_{10}(X)=\frac{1}{1+e^{X{\beta }_{11}}},\mathrm{a}\mathrm{n}\mathrm{d}{\pi }_{00}(X)+{\pi }_{01}(X)=1\\ \mathrm{a}\mathrm{n}\mathrm{d}{\pi }_{10}(X)+{\pi }_{11}(X)=1\end{array}$

Four second-order conditional models ${\pi }_{001}(X),{\pi }_{011}(X),{\pi }_{101}(X),\mathrm{a}\mathrm{n}\mathrm{d}{\pi }_{111}(X)$ are needed for the trivariate Bernoulli model. The conditional probabilities for second-order Markov models satisfy ${\pi }_{000}(X)+{\pi }_{001}(X)=1$,

${\pi }_{010}(X)+{\pi }_{011}(X)=1$, ${\pi }_{100}(X)+{\pi }_{101}(X)=1$ , ${\pi }_{110}(X)+{\pi }_{111}(X)=1$

The second-order transition probabilities are shown in Table 2:

Table 2. Transition probabilities for outcome variables Y₁, Y₂ and Y₃

Y1	Y2	Y3
		0	1
0	0	${\pi }_{000}(X)$	${\pi }_{001}(X)$
0	1	${\pi }_{010}(X)$	${\pi }_{011}(X)$
1	0	${\pi }_{100}(X)$	${\pi }_{101}(X)$
1	1	${\pi }_{110}(X)$	${\pi }_{111}(X)$

The covariate-dependent second-order models are

(9) ${\pi }_{001}(X)=\frac{e^{X{\beta }_{001}}}{1+e^{X{\beta }_{001}}},{\pi }_{000}(X)=\frac{1}{1+e^{X{\beta }_{001}}},$(9)

(10) ${\pi }_{011}(X)=\frac{e^{X{\beta }_{011}}}{1+e^{X{\beta }_{011}}},{\pi }_{010}(X)=\frac{1}{1+e^{X{\beta }_{011}}}$(10)

(11) ${\pi }_{101}(X)=\frac{e^{X{\beta }_{101}}}{1+e^{X{\beta }_{101}}},{\pi }_{100}(X)=\frac{1}{1+e^{X{\beta }_{101}}}$(11)

and

(12) ${\pi }_{111}(X)=\frac{e^{X{\beta }_{111}}}{1+e^{X{\beta }_{111}}},{\pi }_{110}(X)=\frac{1}{1+e^{X{\beta }_{111}}}.$(12)

Here, ${\beta }_{001}=({\beta }_{0010},{\beta }_{0011},...,{\beta }_{001p}),{\beta }_{011}=({\beta }_{0110},{\beta }_{0111},...,{\beta }_{011p})$

${\beta }_{101}=({\beta }_{1010},{\beta }_{1011},\dots ,{\beta }_{101p}),{\beta }_{111}=({\beta }_{1110},{\beta }_{1111},\dots ,{\beta }_{111p})$

4. Link functions and estimating equations for trivariate Bernoulli model

The link functions for trivariate Bernoulli regression models are displayed in (3). In Section 3, we have shown the relationship between joint probabilities and marginal-conditional probabilities. The conditional probabilities are obtained from the first- and second-order transition probabilities. We have introduced seven logistic regression models, one marginal model in Equation (6) and six conditional models in Equations 7–12. Using the relationships between joint and marginal-conditional probabilities, we can redefine the link functions which are summarized below:

(13) ${\theta }_0=ln{p}_{000}=-ln(1+e^{X{\beta }_1})-ln(1+e^{X{\beta }_{01}})-ln(1+e^{X{\beta }_{001}}),$(13)

(14) ${\theta }_1=ln\left(\frac{p_{100}}{p_{000}}\right)=X{\beta }_1-\left\{ln(1+e^{X{\beta }_{11}})-ln(1+e^{X{\beta }_{01}})\right\}-\left\{ln(1+e^{X{\beta }_{101}})-ln(1+e^{X{\beta }_{001}})\right\}$(14)

(15) ${\theta }_2=ln\left(\frac{p_{010}}{p_{000}}\right)=X{\beta }_{01}-\left\{ln(1+e^{X{\beta }_{011}})-ln(1+e^{X{\beta }_{001}})\right\},$(15)

(16) ${\theta }_3=ln\left(\frac{p_{001}}{p_{000}}\right)=X{\beta }_{001},$(16)

(17) $\begin{array}{c}{\theta }_{12}=ln\left(\frac{p_{110}{p}_{000}}{p_{100}{p}_{010}}\right)=X{\beta }_{11}-X{\beta }_{01}-ln(1+e^{X{\beta }_{111}})+ln(1+e^{X{\beta }_{011}})\\ +ln(1+e^{X{\beta }_{101}})-ln(1+e^{X{\beta }_{001}})\end{array}$(17)

(18) ${\theta }_{13}=ln\left(\frac{p_{101}{p}_{000}}{p_{100}{p}_{001}}\right)=X{\beta }_{101}-X{\beta }_{001},$(18)

(19) ${\theta }_{23}=ln\left(\frac{p_{011}{p}_{000}}{p_{010}{p}_{001}}\right)=X{\beta }_{011}-X{\beta }_{001},$(19)

(20) ${\theta }_{123}=ln\left(\frac{p_{111}{p}_{100}{p}_{010}{p}_{001}}{p_{110}{p}_{011}{p}_{101}{p}_{000}}\right)=X{\beta }_{111}+X{\beta }_{001}-X{\beta }_{101}-X{\beta }_{011}.$(20)

The estimating equations are obtained by differentiating the log-likelihood function (2) with respect to regression parameters of seven marginal and conditional models. It may be noted here that for simplicity, the summation signs for i = 1,…,n is ignored here and the equations are shown for n = 1. The estimating equations are:

(21) ${\left[\frac{\mathrm{\partial }l}{\mathrm{\partial }{\beta }_{sk}}\right]}^{\mathrm{\prime }}=\left[\begin{array}{c}\frac{\mathrm{\partial }l}{\mathrm{\partial }{\beta }_{1k}}\\ \frac{\mathrm{\partial }l}{\mathrm{\partial }{\beta }_{01k}}\\ \frac{\mathrm{\partial }l}{\mathrm{\partial }{\beta }_{11k}}\\ \frac{\mathrm{\partial }l}{\mathrm{\partial }{\beta }_{001k}}\\ \frac{\mathrm{\partial }l}{\mathrm{\partial }{\beta }_{011k}}\\ \frac{\mathrm{\partial }l}{\mathrm{\partial }{\beta }_{101k}}\\ \frac{\mathrm{\partial }l}{\mathrm{\partial }{\beta }_{111k}}\end{array}\right]=\left[\begin{array}{c}{X}_k\left(y_1-{\pi }_1(X)\right)\\ X_k\left[(1-y_1)\left(y_2-{\pi }_{01}(X)\right)\right]\\ y_1{X}_k\left(y_2-{\pi }_{11}(X)\right)\\ X_k\left[(1-y_1-y_2+y_1{y}_2)\left(y_3-{\pi }_{001}(X)\right)\right]\\ y_2{X}_k\left[(1-y_1)\left(y_3-{\pi }_{011}(X)\right)\right]\\ y_1{X}_k\left[(1-y_2)\left(y_3-{\pi }_{101}(X)\right)\right]\\ y_1{y}_2{X}_k\left(y_3-{\pi }_{111}(X)\right)\end{array}\right]=\left[\begin{array}{c}0\\ 0\\ 0\\ 0\\ 0\\ 0\\ 0\end{array}\right],k=0,1,\dots ,p.$(21)

where $X_k,k=1,...,p$ is the kth explanatory variable corresponding to the coefficient ${\beta }_{sk}$ , s denotes marginal and conditional models represented by 1, 01,11,001, 011, 101 and 111. The information matrix is comprised of information matrices for seven sets of parameters for marginal and conditional models. The $(k,k^{\prime })$ th element of the information matrix for model s is $-\frac{{\mathrm{\partial }}^{2}l}{\mathrm{\partial }{\beta }_{sk}\mathrm{\partial }{\beta }_{s{k}^{\prime }}}$. Let us denote the $7(p+1)\times 7(p+1)$ matrix containing 7 diagonal $(p+1)\times (p+1)$ matrices as shown below:

(22) $\begin{array}{rl}& I=\left[\begin{array}{ccccccc}{I}_1& 0& 0& 0& 0& 0& 0\\ 0& I_2& 0& 0& 0& 0& 0\\ 0& 0& I_3& 0& 0& 0& 0\\ 0& 0& 0& I_4& 0& 0& 0\\ 0& 0& 0& 0& I_5& 0& 0\\ 0& 0& 0& 0& 0& I_6& 0\\ 0& 0& 0& 0& 0& 0& I_7\end{array}\right]\\ & \end{array}$(22)

where

$I_1={\left[X_k{X_k}^{\mathrm{\prime }}{\pi }_1(X)(1-{\pi }_1(X)),k,k^{\prime }=0,1,\dots ,p\right]}_{(p+1)\times (p+1)},$

$I_2={\left[X_k{X_k}^{\mathrm{\prime }}(1-y_1){\pi }_{01}(X)(1-{\pi }_{01}(X)),k,k^{\prime }=0,1,\dots ,p\right]}_{(p+1)\times (p+1)},$

$I_3={\left[X_k{X_k}^{\mathrm{\prime }}{y}_1{\pi }_{11}(X)(1-{\pi }_{11}(X)),k,k^{\prime }=0,1,\dots ,p\right]}_{(p+1)\times (p+1)},$

$I_4={\left[X_k{X_k}^{\mathrm{\prime }}(1-y_1-y_2+y_1{y}_2){\pi }_{001}(X)(1-{\pi }_{001}(X)),k,k^{\prime }=0,1,\dots ,p\right]}_{(p+1)\times (p+1)},$

$I_5={\left[X_k{X_k}^{\mathrm{\prime }}(1-y_1)y_2{\pi }_{011}(X)(1-{\pi }_{011}(X)),k,k^{\prime }=0,1,\dots ,p\right]}_{(p+1)\times (p+1)},$

$I_6={\left[X_k{X_k}^{\mathrm{\prime }}{y}_1(1-y_2){\pi }_{101}(X)(1-{\pi }_{101}(X)),k,k^{\prime }=0,1,\dots ,p\right]}_{(p+1)\times (p+1)},$

$I_7={\left[X_k{X_k}^{\mathrm{\prime }}{y}_1{y}_2{\pi }_{111}(X)(1-{\pi }_{111}(X)),k,k^{\prime }=0,1,\dots ,p\right]}_{(p+1)\times (p+1)}.$

5. Tests for models and dependence of outcome variables

For trivariate Bernoulli regression models, we have seven sets of parameters, one set for marginal for Y₁ (Model 1for s = 1), two sets for conditional models from first-order Markov chains for transition from Y₁ to Y₂ (Models 1 and 2 for s = 01, 11, respectively) and four sets from second-order Markov chain for outcomes Y₁, Y₂ and Y₃ (s = 001, 011, 101, 111, respectively) . The likelihood ratio test for overall joint model is

$-2\left[lnL\left({\beta }_0\right)-lnL\left(\beta \right)\right]\sim {{\chi }^2}_{7p}$

where ${\beta }_0=\left({\beta }_{s0},s=1,01,11,001,011,101,111\right)$ and ${\beta }_{\hspace{0.17em}}=\left({\beta }_s,s=1,01,11,001,011,101,111\right)$ . Let $\beta \ast =\left({\beta }_s\ast ,s=1,01,11,001,011,101,111\right)$ where ${\beta }_s\ast =\left({\beta }_{s1},...,{\beta }_{sp}\right)$. The null hypothesis of the above test is $H_0:\beta \ast =0$.

Tests for the marginal and conditional models can be performed separately as well. In that case, the null hypothesis for each model is $H_0:{\beta }_s\ast =0$ , s = 1, 01, 11, 001, 011, 101, 111. The likelihood ratio test for each model can be shown as

$-2\left[lnL\left({\beta }_{s0}\right)-lnL\left({\beta }_s\right)\right]\sim {{\chi }^2}_p$

The Wald test will be applied to test for significance of a parameter for each model.

The tests for association parameters, ${\theta }_{12},{\theta }_{13},{\theta }_{23},{\theta }_{123}$, can be performed based on the parameters of regression models (17)—(20). The null hypotheses and test statistics are displayed here:

1. Test for independence of Y₂ and Y₃

The null hypothesis is

$H_{01}:{\beta }_{001}={\beta }_{011}$

The test statistic for equality of parameters

$A_1={\left({\hat{\beta }}_{001}-{\hat{\beta }}_{011}\right)}^{\mathrm{\prime }}{\left[V({\hat{\beta }}_{001}-{\hat{\beta }}_{011})\right]}^{-1}\left({\hat{\beta }}_{001}-{\hat{\beta }}_{011}\right)$

which is asymptotically chi-square with p degrees of freedom. It may be noted here that ${\left[V({\hat{\beta }}_{001}-{\hat{\beta }}_{011})\right]}^{-1}\simeq I_4+I_5$.

• (ii) Test for independence of Y₁ and Y₃

The null hypothesis is

$H_{01}:{\beta }_{001}={\beta }_{101}$

The test statistic for equality of parameters

$A_2={\left({\hat{\beta }}_{001}-{\hat{\beta }}_{101}\right)}^{\mathrm{\prime }}{\left[V({\hat{\beta }}_{001}-{\hat{\beta }}_{101})\right]}^{-1}\left({\hat{\beta }}_{001}-{\hat{\beta }}_{101}\right)$

which is asymptotically chi-square with p degrees of freedom. It may be noted here that ${\left[V({\hat{\beta }}_{001}-{\hat{\beta }}_{101})\right]}^{-1}\simeq I_4+I_6$.

• (iii) Test for independence of Y₁ and Y₂.

It appears from Equation (17) that independence of Y₁ and Y₂ depends not only on equality of regression parameters from Models 7 and 8 but also on equality of parameters of Models 9 and 11 and also Models 10 and 12.

The null hypotheses are

$\begin{array}{c}{H}_{01}:{\beta }_{01}={\beta }_{11}\\ H_{02}:{\beta }_{001}={\beta }_{101}\\ H_{03}:{\beta }_{011}={\beta }_{111}.\end{array}$

The test statistic for equality of regression parameters from Models 7 and 8 ($H_{01}$) can be performed using the following statistic

$A_3={\left({\hat{\beta }}_{01}-{\hat{\beta }}_{11}\right)}^{\mathrm{\prime }}{\left[V({\hat{\beta }}_{01}-{\hat{\beta }}_{11})\right]}^{-1}\left({\hat{\beta }}_{01}-{\hat{\beta }}_{11}\right)$

The denominator is obtained approximately from

${\left[V({\hat{\beta }}_{01}-{\hat{\beta }}_{11})\right]}^{-1}\simeq I_2+I_3$

Test for $H_{02}$ is shown in (ii). Similarly, the test for $H_{03}$ is based on Models 10 and 12 and the test statistic is

$A_4={\left({\hat{\beta }}_{011}-{\hat{\beta }}_{111}\right)}^{\mathrm{\prime }}{\left[V({\hat{\beta }}_{011}-{\hat{\beta }}_{111})\right]}^{-1}\left({\hat{\beta }}_{011}-{\hat{\beta }}_{111}\right)$

where ${\left[V({\hat{\beta }}_{011}-{\hat{\beta }}_{111})\right]}^{-1}\simeq I_5+I_7$.

• (iv) Test for independence of Y₁, Y₂ and Y₃

It appears from Equation (20) that independence of Y₁, Y₂ and Y₃ depends on equality of regression parameters from Models 9 and 11 but also on equality of parameters of Models 10 and 12.

The null hypotheses are

$\begin{array}{c}{H}_{01}:{\beta }_{001}={\beta }_{101}\\ H_{02}:{\beta }_{011}={\beta }_{111}\end{array}$

The first null hypothesis is shown in (ii) which is the test for independence of Y₁ and Y₃(A₂) and the second null hypothesis is discussed in (iii) for partial test for independence of Y₁ and Y₂ (A₄). In other words, independence of Y₁, Y₂ and Y₃ depends on both (a) conditional independence of Y₁ and Y₂ (A₄) and (b) independence of Y₁ and Y₃ (A₂). Independence of three outcome variables depends on these pairwise conditional or unconditional independence of outcome variables that are shown in previous tests.

An alternative null hypothesis is quite straightforward from the model shown in Equation (20). Let us define ${\beta }_{...}={\beta }_{111}+{\beta }_{001}-{\beta }_{101}-{\beta }_{011}$ then the independence of Y₁, Y₂ and Y₃ can be tested alternatively for null hypothesis $H_{01}:{\beta }_{...}=0$. We can test the null hypothesis using

$A_5={{\hat{\beta }}_{\dots }}^{\mathrm{\prime }}{\left[V({\hat{\beta }}_{\dots })\right]}^{-1}{\hat{\beta }}_{\dots }$

where ${\left[V({\hat{\beta }}_{...})\right]}^{-1}\simeq I_4+I_5+I_6+I_7$.

6. Concluding remarks

We need to analyse binary repeated measures data in many instances where the outcome variables are correlated. The modelling of correlated outcome variables have been of interest in many fields due to recent emergence of need for analysing repeated measures data in the presence of correlation among outcome variables in addition to models for identifying explanatory variables associated with outcome variables. Several attempts have been made in the past to model such data but due to inbuilt complexity in modelling multivariate data with correlated outcomes it remained a challenge for a long time. This study shows an alternative approach based on marginal-conditional formulation to describe a joint model and provides a set of marginal and conditional models that can provide joint probabilities for a trivariate binary case. This procedure can be extended for more than three correlated outcomes easily using the same approach which is not shown in this paper to keep the exposition simple. Several tests are displayed in this paper for testing the overall model for trivariate Bernoulli as well as for examining dependence in outcome variables.

Acknowledgements

This work was supported by the HEQEP sub-project CP 3293 sponsored by UGC, Bangladesh and World Bank. I would like to express my gratitude to the anonymous reviewers for their helpful suggestions.

Additional information

Funding

This work was supported by the HEQEP sub-project [Grant Number CP 3293] sponsored by UGC, Bangladesh and World Bank.

Notes on contributors

M. Ataharul Islam

M. Ataharul Islam is currently the QM Husain Professor, ISRT, University of Dhaka. He was a former professor of statistics at the University Sains Malaysia, King Saud University, University of Dhaka and the East West university. He was a visiting faculty at the University of Hawaii and University of Pennsylvania. He is recipient of the Pauline Stitt Award, the WNAR Biometric Society Award for content and writing, University Grants Commission Award for book and research, Ibrahim Gold Medal for research, etc. He published more than 100 research papers in international journals on various topics, extensively on longitudinal and repeated measures data, including multistate and multistage hazards models, statistical models for repeated measures data, Markov models with covariate dependence, generalized linear models, conditional and joint models for correlated outcomes, etc. He authored several books either published or being published.