A comparison of parameter covariance estimation methods for item response models in an expectation-maximization framework

Figures & data

Table 1. Data generating parameters for Model cyh1. Group 2 did not contain items 13-16

Item	a1	a2	a3	a4	a5	c
1	1.00	0.80				1.00
2	1.40	1.50				0.25
3	1.70	1.20
4	2.00	1.00
5	1.40		1.00			1.00
6	1.70		0.80			0.25
7	2.00		1.50
8	1.00		1.20
9	1.70			1.20		1.00
10	2.00			1.00		0.25
11	1.00			0.80
12	1.40			1.50
13	2.00				1.50	1.00
14	1.00				1.20	0.25
15	1.40				1.00
16	1.70				0.80

Note: Nonzero parameters were estimated.

Table 2. Descriptive summary of the Monte Carlo simulation studies

		Unidentified
m2pl5	10	13	16.1	0.665	0.84	35
m3pl15	31	6	8.5	0.306	0.55	90
grm20	60	0	16.1	0.111	0.22	369
cyh1	56	1	8.5	0.055	0.14	281

Notes: The first column is the number of free parameters in the model. Where the Unidentified column is 0, all trials were included. Trials were considered unidentified if the iteration limit was reached or the log condition number using the covariance of the gradients was greater than log(CondNum). V is the Monte Carlo parameter covariance matrix.

Table 3. Percentage of trials that failed to converge by model and algorithm

	RE	Oakes	Agile	Tian	MR SEM
m2pl5	2.6	2.6	3.6	3.8	4.8
m3pl15	1.0	1.0	1.0	1.0	1.2
grm20	0.0	0.0	0.4	0.0	95.4
cyh1	0.0	0.0	0.0	20.0	70.2

Notes: Failure was due to either iteration limit or a non-positive definite covariance matrix. Since some trials were genuinely unidentified, these trials failed to converge for all algorithms. Compare with the unidentified column in Table 2.

Table 4. Mean elapsed time and accuracy of parameter covariance matrix estimators

	RE	Oakes	Agile	Tian	MR SEM	M-step
m2pl5
Seconds	0.015	0.015	0.019	0.035	0.08	0.012
	3.225	3.225	3.232	4.537	4.364	4.532
	1.428	1.402	1.369	1.758	1.685	1.751
m3pl15
Seconds	0.275	0.056	0.111	0.094	0.177	0.048
	11.893	11.893	11.893	12.014	11.944	12.014
	5.273	3.408	5.271	4.772	5.07	4.771
grm20
Seconds	8.039	0.143	0.478	0.515	0.247	0.026
	0.862	0.862	0.899	1.326	2.159	2.15
	0.675	0.675	0.687	0.859	1.55	1.532
cyh1
Seconds	46.358	0.829	2.953	9.657	12.18	0.072
	1.406	1.395	1.395	1.482	1.626	4.919
	1.286	1.243	1.26	2.028	4.216	3.772

Notes: RE is central difference with Richardson extrapolation and Mstep is the completed data information matrix (Equation (1)). Since unconveraged trials were excluded, the performance of MR-SEM and Tian are shown in a most positive light. The scales of and are model specific and should not be compared between models.

Figure 1. Scatterplot of Oakes vs. Richardson extrapolation derived standard errors.