Comparing BSEM and EUPD Estimates for Two-Group SB-MTMM Experiments

Abstract

In 2018, two different approaches have been suggested to solve the estimation problems that have been detected during the analysis of the data of the two-group split ballot multi-trait multi-method (SB-MTMM experiments performed in many countries between 2002 and 2010 in the European Social Survey). One group suggested using the Bayesian estimation. The other group suggested a new estimation procedure (EUPD) that makes use of the pooled data across the different countries. In this note, we present a comparison of the results of the two approaches on the same generated data, which are comparable with the data that created problems in the ESS.

Keywords:

• Two-group SB-MTMM
• Bayesian analysis
• EUPD estimator
• Pooled data analysis
• invariance
• Monte Carlo experiment

Notes

¹ This design was chosen because all respondents are asked the same questions in the main questionnaire, and only the supplementary questionnaire at the end is different between the two groups.

² The population covariance matrix implied by the parameter values in Table 1 was also used in the simulation study of Helm et al. (2018). There are two parameterizations that generate the same covariance matrix. Helm et al. used the parameterization fixing the variances of the traits to 1. In this study, we fix the first loading for each trait to 1. The latter parameterization is necessary for the EUPD approach to avoid arbitrary differences between the loadings due to differences in the variances of the variables.

³ In the other tables, the loadings are denoted by ij, where i is the number of variables and j is the number of factors, traits or methods. The factor variances will be identified by jj, the covariances will be represented by jj´where j ≠ j´, and the error variances will be denoted by ii.

⁴ The results for the other 10 datasets can be obtained from the first author of this paper.

⁵ For the specification of BSEM method, we used the same specification that was used by Helm et al. (2018).