A comparison of two estimation methods for common principal components

Figures & data

Table 1 Decomposition of the log-likelihood ratio statistic into partial log-likelihood ratio statistics.

Higher model	Lower model	Degrees of freedom
Equality	Proportionality	G – 1
Proportionality	CPC	$(p-1)(G-1)$
CPC	CPC(q)	$\frac{1}{2}(G-1)(p-q)(p-q-1)$
CPC(q)	Unrelated	$\frac{1}{2}(G-1)(2pq-q^2-q)$

Table 2 Simulation results for normally distributed data.

									$\left|\right|{\widehat{\mathbf{\Lambda }}}_g-\text{diag}$	$\left|\right|{\mathbf{D}}_g-\text{diag}$
	$\Vert {\widehat{\mathit{\pi }}}_1-{\mathit{\pi }}_1\Vert$	$\Vert {\mathbf{p}}_1-{\mathit{\pi }}_1\Vert$	$\Vert {\widehat{\mathit{\pi }}}_p-{\mathit{\pi }}_p\Vert$	$\Vert {\mathbf{p}}_p-{\mathit{\pi }}_p\Vert$	${\Vert \widehat{\mathbf{\Pi }}-\mathbf{\Pi }\Vert }_{F_p}$	${\Vert \mathbf{P}-\mathbf{\Pi }\Vert }_{F_p}$	${\Vert {\widehat{\mathbf{\Lambda }}}_g^{*}-{\mathbf{\Lambda }}_g^{*}\Vert }^{\text{AVG}}$	${\Vert {\mathbf{D}}_g^{*}-{\mathbf{\Lambda }}_g^{*}\Vert }^{\text{AVG}}$	$({\widehat{\mathbf{\Lambda }}}_g)|{|}_{F_p}^{\text{AVG}}$	$({\mathbf{D}}_g)|{|}_{F_p}^{\text{AVG}}$
G = 2	0.7954	0.6447	0.5939	0.4182	0.3263	0.2622	0.5407	0.5124	0.0628	0.0735
G = 4	0.7444	0.4928	0.5293	0.2698	0.3197	0.1983	0.4983	0.4688	0.0825	0.0898
G = 10	0.6799	0.3070	0.4183	0.1460	0.3069	0.1139	0.4576	0.4400	0.0937	0.0972
G = 50	0.5672	0.1230	0.3611	0.0609	0.2961	0.0422	0.4306	0.4263	0.0996	0.1003
N = 50	0.8952	0.6383	0.7241	0.3893	0.3531	0.2536	0.7229	0.6644	0.1139	0.1264
N = 100	0.7444	0.4928	0.5293	0.2698	0.3197	0.1983	0.4983	0.4688	0.0825	0.0898
N = 500	0.4599	0.2403	0.2390	0.1157	0.2265	0.0940	0.2204	0.2156	0.0385	0.0407
$N=1,000$	0.3498	0.1713	0.1714	0.0819	0.1848	0.0654	0.1576	0.1557	0.0274	0.0288
$N=10,000$	0.1176	0.0512	0.0606	0.0250	0.0774	0.0196	0.0510	0.0509	0.0088	0.0091
p = 5	0.4560	0.2522	0.3187	0.1610	0.2930	0.1353	0.3354	0.3300	0.0763	0.0831
p = 10	0.7444	0.4928	0.5293	0.2698	0.3197	0.1983	0.4983	0.4688	0.0825	0.0898
p = 20	1.0261	0.7825	0.8313	0.4655	0.2738	0.2194	0.7644	0.6654	0.0839	0.0921
p = 50	1.2560	1.1161	1.2385	0.8507	0.1910	0.1808	1.6694	1.3378	0.0853	0.0931
$\varphi =-0.9$	0.7366	0.7102	0.5406	0.4717	0.3070	0.2814	0.5559	0.5641	0.0529	0.0559
$\varphi =0$	0.7444	0.4928	0.5293	0.2698	0.3197	0.1983	0.4983	0.4688	0.0825	0.0898
$\varphi =0.9$	0.7301	0.7023	0.5413	0.4613	0.3081	0.2818	0.5516	0.5599	0.0528	0.0557

The benchmark parameter values are G = 4, N = 100, p = 10, $\varphi =0$, and the eigenvectors of the MLE are ordered by the eigenvalues of the first group.

${\widehat{\mathbf{\Lambda }}}_g^{*}=({\widehat{\lambda }}_{g1},\dots ,{\widehat{\lambda }}_{\mathit{gp}})$ and ${\mathbf{D}}_g^{*}=(d_{g1},\dots ,d_{\mathit{gp}})$, where ${\widehat{\lambda }}_{g1}$ and d_gi are the ith estimated eigenvalue for group g using MLE and Krzanowski’s method, respectively.

Table 3 Simulation results for chi-square distributed data with 10 degrees of freedom.

									$\Vert {\widehat{\mathbf{\Lambda }}}_g-\text{diag}$	${\Vert {\mathbf{D}}_g-\text{diag}({\mathbf{D}}_g)\Vert }_{F_p}^{\text{AVG}}$
	$\Vert {\widehat{\mathit{\pi }}}_1-{\mathit{\pi }}_1\Vert$	$\Vert {\mathbf{p}}_1-{\mathit{\pi }}_1\Vert$	$\Vert {\widehat{\mathit{\pi }}}_p-{\mathit{\pi }}_p\Vert$	$\Vert {\mathbf{p}}_p-{\mathit{\pi }}_p\Vert$	${\Vert \widehat{\mathbf{\Pi }}-\mathbf{\Pi }\Vert }_{F_p}$	${\Vert \mathbf{P}-\mathbf{\Pi }\Vert }_{F_p}$	${\Vert {\widehat{\mathbf{\Lambda }}}_g^{*}-{\mathbf{\Lambda }}_g^{*}\Vert }^{\text{AVG}}$	${\Vert {\mathbf{D}}_g^{*}-{\mathbf{\Lambda }}_g^{*}\Vert }^{\text{AVG}}$	$({\widehat{\mathbf{\Lambda }}}_g){\Vert }_{F_p}^{\text{AVG}}$	$({\mathbf{D}}_g){\Vert }_{F_p}^{\text{AVG}}$
G = 2	0.8034	0.6562	0.6082	0.4330	0.3309	0.2670	0.5864	0.5570	0.0655	0.0766
G = 4	0.7745	0.5151	0.5479	0.2806	0.3261	0.2042	0.5345	0.5021	0.0859	0.0940
G = 10	0.7100	0.3211	0.4341	0.1537	0.3137	0.1206	0.4936	0.4722	0.0979	0.1019
G = 50	0.6185	0.1289	0.3590	0.0636	0.3025	0.0445	0.4630	0.4576	0.1042	0.1050
N = 50	0.9055	0.6548	0.7445	0.4022	0.3575	0.2585	0.7737	0.7107	0.1180	0.1315
N = 100	0.7745	0.5151	0.5479	0.2806	0.3261	0.2042	0.5345	0.5021	0.0859	0.0940
N = 500	0.4664	0.2528	0.2534	0.1221	0.2313	0.0985	0.2364	0.2311	0.0402	0.0425
$N=1,000$	0.3527	0.1760	0.1799	0.0847	0.1884	0.0680	0.1684	0.1662	0.0287	0.0302
$N=10,000$	0.1235	0.0550	0.0561	0.0265	0.0784	0.0206	0.0544	0.0543	0.0092	0.0096
p = 5	0.4840	0.2712	0.3391	0.1717	0.3073	0.1474	0.3755	0.3683	0.0819	0.0896
p = 10	0.7745	0.5151	0.5479	0.2806	0.3261	0.2042	0.5345	0.5021	0.0859	0.0940
p = 20	1.0308	0.7991	0.8382	0.4727	0.2748	0.2219	0.8038	0.7010	0.0859	0.0945
p = 50	1.2630	1.1215	1.2349	0.8619	0.1912	0.1811	1.7166	1.3741	0.0862	0.0942
$\varphi =-0.9$	0.7657	0.7000	0.5551	0.4350	0.3150	0.2733	0.5783	0.5787	0.0649	0.0691
$\varphi =0$	0.7745	0.5151	0.5479	0.2806	0.3261	0.2042	0.5345	0.5021	0.0859	0.0940
$\varphi =0.9$	0.7643	0.6894	0.5461	0.4447	0.3143	0.2736	0.5806	0.5813	0.0647	0.0689

The benchmark parameter values are G = 4, N = 100, p = 10, $\varphi =0$, and the eigenvectors of the MLE are ordered by the eigenvalues of the first group. ${\widehat{\mathbf{\Lambda }}}_g^{*}=({\widehat{\lambda }}_{g1},\dots ,{\widehat{\lambda }}_{\mathit{gp}})$ and ${\mathbf{D}}_g^{*}=(d_{g1},\dots ,d_{\mathit{gp}})$, where ${\widehat{\lambda }}_{g1}$ and d_gi are the ith estimated eigenvalue for group g using MLE and Krzanowski’s method, respectively.

Table 4 Simulation results for chi-square distributed data with 2 degrees of freedom.

									$\Vert {\widehat{\mathbf{\Lambda }}}_g-\text{diag}$	$\Vert {\mathbf{D}}_g-\text{diag}$
	$\Vert {\widehat{\mathit{\pi }}}_1-{\mathit{\pi }}_1\Vert$	$\Vert {\mathbf{p}}_1-{\mathit{\pi }}_1\Vert$	$\Vert {\widehat{\mathit{\pi }}}_p-{\mathit{\pi }}_p\Vert$	$\Vert {\mathbf{p}}_p-{\mathit{\pi }}_p\Vert$	${\Vert \widehat{\mathbf{\Pi }}-\mathbf{\Pi }\Vert }_{F_p}$	${\Vert \mathbf{P}-\mathbf{\Pi }\Vert }_{F_p}$	${\Vert {\widehat{\mathbf{\Lambda }}}_g^{*}-{\mathbf{\Lambda }}_g^{*}\Vert }^{\text{AVG}}$	${\Vert {\mathbf{D}}_g^{*}-{\mathbf{\Lambda }}_g^{*}\Vert }^{\text{AVG}}$	$({\widehat{\mathbf{\Lambda }}}_g){\Vert }_{F_p}^{\text{AVG}}$	$({\mathbf{D}}_g){\Vert }_{F_p}^{\text{AVG}}$
G = 2	0.8700	0.7139	0.6732	0.4851	0.3437	0.2860	0.7298	0.6923	0.0729	0.0871
G = 4	0.8493	0.5719	0.6093	0.3210	0.3416	0.2261	0.6745	0.6286	0.0971	0.1078
G = 10	0.8010	0.3797	0.5126	0.1789	0.3336	0.1416	0.6224	0.5853	0.1116	0.1176
G = 50	0.7160	0.1506	0.4160	0.0743	0.3260	0.0521	0.5836	0.5627	0.1195	0.1214
N = 50	0.9686	0.7111	0.8014	0.4511	0.3675	0.2796	0.9841	0.9030	0.1318	0.1490
N = 100	0.8493	0.5719	0.6093	0.3210	0.3416	0.2261	0.6745	0.6286	0.0971	0.1078
N = 500	0.5230	0.2924	0.2920	0.1422	0.2525	0.1145	0.2927	0.2845	0.0465	0.0495
$N=1,000$	0.4051	0.2065	0.2164	0.1000	0.2082	0.0807	0.2081	0.2048	0.0333	0.0352
$N=10,000$	0.1383	0.0638	0.0710	0.0305	0.0887	0.0241	0.0667	0.0665	0.0107	0.0111
p = 5	0.5680	0.3367	0.4146	0.2157	0.3503	0.1807	0.5072	0.4930	0.0978	0.1092
p = 10	0.8493	0.5719	0.6093	0.3210	0.3416	0.2261	0.6745	0.6286	0.0971	0.1078
p = 20	1.0752	0.8323	0.8692	0.5158	0.2784	0.2307	0.9505	0.8239	0.0926	0.1029
p = 50	1.2751	1.1319	1.2425	0.8856	0.1914	0.1820	1.8893	1.4998	0.0892	0.0980
$\varphi =-0.9$	0.8207	0.7669	0.6068	0.4881	0.3284	0.2938	0.7276	0.7242	0.0716	0.0762
$\varphi =0$	0.8493	0.5719	0.6093	0.3210	0.3416	0.2261	0.6745	0.6286	0.0971	0.1078
$\varphi =0.9$	0.8167	0.7590	0.6049	0.4881	0.3280	0.2931	0.7297	0.7267	0.0714	0.0760

The benchmark parameter values are G = 4, N = 100, p = 10, $\varphi =0$, and the eigenvectors of the MLE are ordered by the eigenvalues of the first group.

${\widehat{\mathbf{\Lambda }}}_g^{*}=({\widehat{\lambda }}_{g1},\dots ,{\widehat{\lambda }}_{\mathit{gp}})$ and ${\mathbf{D}}_g^{*}=(d_{g1},\dots ,d_{\mathit{gp}})$, where ${\widehat{\lambda }}_{g1}$ and d_gi are the ith estimated eigenvalue for group g using MLE and Krzanowski’s method, respectively.

Table 5 List of the innovation related variables included in the analysis.

	Variables
Human capital	Average age of the population
	Number of people with compulsory education <9 years
	Number of people with compulsory education 9–10 years
	Number of people with high school education <2 years
	Number of people with high school education 3 years
Tax levels	State property tax (thousands of SEK)
Externalities	Diversity^a
	Related variety^b
	Herfindahl-Hirschman index^c
	Firm density^d
Firm activities	Mean establishment size of firms^e
	Export (SEK)
Firm performance	Average social security contributions (thousands of SEK)
	Average wage costs (thousands of SEK)
	Average net sales (thousands of SEK)
	Average gross investment (thousands of SEK)
Accessibility	Number of commuting people
Regional performance	Gross regional product (millions of SEK)

The following descriptions of the variables are taken from Holgersson and Kekezi (2017):

a Diversity is measured by the total number of different firms on a five-level NACE industry code.

b Related variety is calculated the same way as in Frenken, Van Oort, and Verburg (2007). It is the weighted sum of five-digit levels within each two-digit group which measures variety within sectors that comes from regional knowledge spillovers. The underlying idea is that innovation comes from the recombination of existing knowledge. Frenken, Van Oort, and Verburg (2007) argue that this is the best estimate of Jacobs externalities.

c The Herfindahl-Hirschman index is a measure of concentration. It ranges from many small firms (value close to zero) to a single monopolistic producer (value of one). A high value means that employment is concentrated in only a few sectors while a lower one shows an even distribution of employees between sectors and hence more diversification (Essletzbichler 2007).

d Firm density is calculated as the number of firms divided with the total number of people living in the municipality.

e Mean establishment size of firm is a measure of the proportion between the number of employees in the municipality and the number of firms.

Table 6 Model selection for the innovation data $(G=11,p=18)$.

Model		${\chi }^2$	df	$\frac{{\chi }^2}{\text{df}}$	AIC${}^{\mathrm{a}}$	BIC${}^{\mathrm{a}}$
Higher	Lower
MLE CPC
Equality	Proportionality	100.22	10	10.02	3,395.36	3,395.36
Proportionality	CPC	522.44	170	3.07	3,315.14	3,375.77*
CPC	CPC(16)	12.12	10	1.21*	3,132.70	4,224.10
CPC(16)	CPC(15)	27.35	20	1.37	3,140.57	4,292.61
CPC(15)	CPC(14)	90.34	30	3.01	3,153.22	4,426.53
CPC(14)	Unrelated	2,642.88	1,470	1.80	3,122.88*	4,578.09
Unrelated	–				3,420	27,576.75
Equality	Unrelated	3,395.36	1,710
Krzanowski’s CPC approximation
Equality	Proportionality	100.22	10	10.02	3,395.36	3,395.36
Proportionality	CPC	468.28	170	2.75	3,315.14	3,375.77*
CPC	CPC(16)	11.76	10	1.18*	3,186.86	4,278.27
CPC(16)	CPC(15)	27.46	20	1.37	3,195.10	4,347.14
CPC(15)	CPC(14)	94.98	30	3.17	3,207.64	4,480.95
CPC(14)	Unrelated	2,692.66	1,470	1.83	3,172.66*	4,627.87
Unrelated	–				3,420	27,576.75
Equality	Unrelated	3,395.36	1,710

The group-specific portions of the CPC(q) models were selected based on correlations (described in Sec. A.2 in the Appendix) between the CPCs (see Table A.1 in the Appendix). CPC(16) omitted components 15 and 17, CPC(15) omitted components 9, 15, and 17, and, CPC(14) omitted components 9, 13, 15, and 17.

a AIC and BIC values for the higher model.

Table 7 Model selection for the Iris data $(G=3,p=4)$.

Model		${\chi }^2$	df	$\frac{{\chi }^2}{\text{df}}$	AIC^a	BIC^a
Higher	Lower
MLE CPC
Equality	Proportionality	34.34	2	17.17	146.66	146.66
Proportionality	CPC	48.41	6	8.07	116.32	122.34
CPC	CPC(2)	39.53	2	19.77	79.91	104.00
CPC(2)	Unrelated	24.38	10	2.44*	44.38	74.48*
Unrelated	–				40.00*	200.43
Equality	Unrelated	146.66	20
Krzanowski’s CPC approximation
Equality	Proportionality	34.34	2	17.17	146.66	146.66
Proportionality	CPC	25.71	6	4.29*	116.32	122.34
CPC	CPC(2)	39.58	2	19.79	102.61	126.69
CPC(2)	Unrelated	47.03	10	4.70	67.03	97.14*
Unrelated	–				40.00*	200.43
Equality	Unrelated	146.66	20

a AIC and BIC values for the higher model.

Table 8 The effectiveness of the simultaneous diagonalization for the two datasets measured using the Frobenius norm.

Dataset	${\Vert {\widehat{\mathbf{\Lambda }}}_g-\text{diag}({\widehat{\mathbf{\Lambda }}}_g)\Vert }_{F_p}^{\text{AVG}}$	${\Vert {\mathbf{D}}_g-\text{diag}({\mathbf{D}}_g)\Vert }_{F_p}^{\text{AVG}}$
Municipality data	0.0416	0.0409
Iris data	0.6236	0.7321

Table A.1 Correlations with absolute values higher than 0.3 for the innovation data using the ML estimated CPC model.

Year	Components	Correlation
2000	(1,15)	0.32
2001	(1,15)	0.35
2001	(1,9)	–0.32
2001	(4,13)	–0.32
2009	(1,17)	0.41
2010	(1,17)	0.45

Table A.2 Correlations with absolute values higher than 0.3 for the innovation data using the Krzanowski’s estimated CPC model.

Year	Components	Correlation
2000	(1,15)	0.33
2001	(1,15)	0.37
2001	(1,9)	–0.32
2001	(4,13)	–0.31
2009	(1,17)	0.30
2009	(1,15)	–0.31
2010	(1,17)	0.34
2010	(1,15)	–0.35

Table A.3 Correlation matrix for the Iris flower data using the ML estimated CPC model.

Component	Flower type	Correlation
1	Versicolor	1
2		0.18	1
3		–0.07	–0.06	1
4		0.10	0.07	–0.14	1
1	Virginica	1
2		–0.08	1
3		0.11	–0.27	1
4		–0.15	0.39	–0.35	1
1	Setosa	1
2		–0.09	1
3		–0.74	0.01	1
4		–0.05	–0.25	0.12	1

Table A.4 Correlation matrix for the Iris flower data using the Krzanowski’s estimation CPC model.

Component	Flower type	Correlation
1	Versicolor	1
2		0.2	1
3		0.07	–0.08	1
4		0.16	–0.29	0.19	1
1	Virginica	1
2		–0.01	1
3		0.3	–0.25	1
4		–0.02	0.23	–0.29	1
1	Setosa	1
2		–0.47	1
3		–0.71	0.43	1
4		–0.18	–0.15	0.36	1

Table A.5 The first nine ML-estimated CPC coefficients for the innovation data, consisting of 18 variables and 11 groups (years 2006–2010).

	1	2	3	4	5	6	7	8	9
1. Average age	0.576	0.732	–0.270	0.000	0.235	0.014	0.025	–0.012	0.033
	(0)	(0)	(1)	(0)	(0)	(0)	(0)	(1)	(–1)
2. Compulsory education <9 years	–0.184	0.194	–0.167	0.105	–0.175	0.036	–0.398	–0.405	–0.397
	(0)	(0)	(0)	(0)	(0)	(–5)	(2)	(–3)	(0)
3. Compulsory education 9–10 years	–0.241	0.108	–0.190	0.098	–0.036	0.007	–0.190	–0.159	–0.073
	(0)	(0)	(0)	(0)	(0)	(–3)	(1)	(–1)	(0)
4. High school education <2 years	–0.216	0.137	–0.216	0.112	–0.134	0.058	–0.260	–0.083	0.041
	(0)	(0)	(0)	(0)	(0)	(–3)	(–1)	(–2)	(1)
5. High school education 3 years	–0.264	0.120	–0.220	0.125	–0.033	0.027	–0.121	0.081	0.077
	(0)	(0)	(1)	(0)	(0)	(–1)	(–1)	(0)	(0)
6. Property tax	–0.297	0.063	–0.291	0.051	0.241	0.047	0.386	0.421	–0.590
	(0)	(0)	(1)	(1)	(0)	(5)	(–3)	(3)	(–3)
7. Diversity	–0.105	0.070	–0.120	0.041	–0.041	0.008	–0.010	0.094	–0.007
	(0)	(0)	(0)	(0)	(0)	(0)	(–1)	(–1)	(2)
8. Related variety	–0.099	0.040	–0.205	0.015	–0.100	–0.001	0.027	0.220	0.108
	(0)	(0)	(0)	(0)	(0)	(1)	(–2)	(–2)	(7)
9. Herfindahl–Hirschman index	0.094	–0.095	0.135	–0.081	0.313	0.381	–0.657	0.515	–0.050
	(0)	(0)	(0)	(–1)	(1)	(–9)	(–9)	(–4)	(0)
10. Firm density	0.023	–0.001	–0.063	–0.035	–0.038	0.033	0.172	0.115	0.031
	(0)	(0)	(0)	(0)	(0)	(3)	(–1)	(0)	(3)
11. Mean establishment size of firms	–0.042	0.085	0.119	0.127	–0.081	–0.088	–0.158	0.045	0.153
	(0)	(0)	(0)	(0)	(0)	(–2)	(1)	(0)	(–3)
12. Export	–0.399	0.458	0.358	–0.705	–0.045	0.020	0.036	0.016	0.023
	(0)	(–1)	(–1)	(–1)	(0)	(0)	(–1)	(0)	(0)
13. Avg. social security contributions	–0.087	0.156	0.353	0.290	0.150	–0.187	0.002	–0.018	–0.118
	(0)	(0)	(1)	(–1)	(–1)	(0)	(2)	(0)	(5)
14. Average wage costs	–0.070	0.143	0.311	0.288	0.127	–0.253	–0.037	0.070	–0.059
	(0)	(0)	(1)	(0)	(–1)	(0)	(3)	(–1)	(0)
15. Average net sales	–0.080	0.151	0.332	0.306	0.170	–0.265	–0.032	0.066	–0.080
	(0)	(0)	(1)	(0)	(–1)	(0)	(3)	(–1)	(–1)
16. Average gross investment	–0.051	0.144	0.291	0.313	–0.081	0.815	0.278	–0.171	0.003
	(0)	(0)	(1)	(–2)	(3)	(5)	(–9)	(1)	(0)
17. Commuting	–0.283	–0.105	–0.168	–0.038	0.783	0.058	0.042	–0.366	0.313
	(0)	(0)	(0)	(0)	(1)	(–2)	(1)	(1)	(0)
18. Gross regional product	–0.280	0.215	–0.135	0.255	–0.168	–0.018	0.068	0.331	0.565
	(0)	(0)	(1)	(0)	(0)	(2)	(–1)	(2)	(–3)

The difference in coefficient estimation ($\times 100$) between ML and Krzanowski’s estimation is displayed in parentheses.

Table A.6 The last six ML-estimated CPC coefficients for the innovation data, consisting of 18 variables and 11 groups (years 2000–2010).

	10	11	12	13	14	15	16	17	18
1. Average age	–0.017	0.031	–0.016	0.008	–0.022	–0.016	–0.003	–0.002	0.004
	(0)	(0)	(0)	(0)	(0)	(0)	(1)	(0)	(0)
2. Compulsory education <9 years	0.027	–0.339	0.188	–0.391	0.243	–0.012	–0.106	–0.053	0.004
	(4)	(–1)	(–2)	(0)	(–2)	(–1)	(–7)	(0)	(0)
3. Compulsory education 9–10 years	0.014	–0.050	–0.007	0.285	–0.830	–0.162	–0.063	–0.101	0.048
	(0)	(0)	(0)	(0)	(–5)	(–2)	(21)	(4)	(–1)
4. High school education <2 years	–0.013	0.259	–0.210	0.378	0.172	0.562	–0.011	0.410	–0.102
	(–2)	(0)	(1)	(0)	(1)	(11)	(–4)	(–12)	(2)
5. High school education 3 years	–0.021	0.215	–0.098	0.298	0.392	–0.714	0.107	–0.056	–0.003
	(–1)	(0)	(1)	(–1)	(3)	(–4)	(–11)	(17)	(0)
6. Property tax	–0.225	0.058	–0.028	–0.128	–0.033	0.072	–0.034	0.087	–0.010
	(7)	(0)	(0)	(0)	(–1)	(2)	(1)	(–1)	(0)
7. Diversity	0.215	–0.059	0.088	0.025	0.016	0.245	0.826	–0.382	0.106
	(1)	(0)	(–1)	(1)	(22)	(–9)	(2)	(–9)	(–1)
8. Related variety	0.767	0.329	0.165	–0.295	–0.067	0.001	–0.243	0.026	–0.027
	(1)	(0)	(–4)	(0)	(–7)	(1)	(1)	(1)	(0)
9. Herfindahl–Hirschman index	0.017	–0.114	–0.010	0.000	–0.024	0.007	–0.033	–0.015	0.008
	(2)	(0)	(0)	(0)	(–1)	(0)	(0)	(0)	(0)
10. Firm density	0.287	–0.629	0.149	0.214	–0.006	–0.156	0.170	0.565	–0.170
	(1)	(0)	(–2)	(0)	(4)	(14)	(2)	(6)	(2)
11. Mean establishment size of firms	–0.276	0.289	0.084	–0.503	–0.231	–0.169	0.382	0.471	–0.164
	(–1)	(–1)	(2)	(–1)	(9)	(12)	(8)	(6)	(2)
12. Export	0.012	0.018	0.016	0.016	0.004	0.002	–0.001	0.001	0.001
	(0)	(0)	(0)	(0)	(0)	(0)	(0)	(0)	(0)
13. Avg. social security contributions	0.289	–0.144	–0.748	–0.141	–0.020	–0.043	0.039	0.008	–0.012
	(–3)	(1)	(–2)	(1)	(1)	(0)	(1)	(1)	(0)
14. Average wage costs	0.036	0.047	0.354	0.150	0.056	0.029	–0.048	0.200	0.713
	(3)	(0)	(0)	(0)	(–2)	(5)	(–1)	(–2)	(1)
15. Average net sales	0.021	0.049	0.387	0.205	0.039	0.070	–0.069	–0.202	–0.639
	(3)	(0)	(0)	(–1)	(–1)	(–5)	(–2)	(0)	(–1)
16. Average gross investment	0.038	0.087	0.077	0.008	–0.018	–0.014	0.011	0.000	0.004
	(0)	(0)	(–1)	(0)	(0)	(0)	(1)	(0)	(0)
17. Commuting	0.050	–0.025	0.065	–0.117	0.046	0.041	0.030	0.060	–0.002
	(–4)	(0)	(–1)	(0)	(1)	(2)	(–1)	(–1)	(0)
18. Gross regional product	–0.254	–0.357	–0.032	–0.180	0.019	0.143	–0.214	–0.190	0.049
	(–5)	(0)	(3)	(0)	(–5)	(–4)	(–1)	(–4)	(0)

The difference in coefficient estimation ($\times 100$) between ML and Krzanowski’s estimation is displayed in parentheses.

Table A.7 Estimated eigenvalue difference for the innovation data using the CPC model.

	Eigenvalue difference ($\times 100$)
Eigenvalue	2000	2001	2002	2003	2004	2005	2006	2007	2008	2009	2010
1	0	0	0	0	0	0	0	0	0	0	0
2	0	1	0	0	0	0	0	0	0	0	0
3	0	1	0	0	0	0	0	1	1	0	0
4	0	–2	0	0	0	0	0	0	0	–1	0
5	0	0	0	0	0	0	0	0	0	0	0
6	0	0	0	0	0	0	0	0	0	1	0
7	0	0	0	0	0	0	0	0	0	–1	0
8	0	0	0	0	0	0	0	0	0	0	1
9	0	0	0	0	0	0	0	0	0	0	0
10	0	0	0	0	0	0	0	0	0	0	0
11	0	0	0	0	0	0	0	0	0	0	0
12	0	0	0	0	0	0	0	0	0	0	0
13	0	0	0	0	0	0	0	0	0	0	0
14	0	0	0	0	0	0	0	0	0	0	0
15	0	0	0	0	0	0	0	0	0	0	0
16	0	0	0	0	0	0	0	0	0	0	0
17	0	0	0	0	0	0	0	0	0	0	0
18	0	0	0	0	0	0	0	0	0	0	0

Table A.8 ML estimated CPC for the Iris data.

	Coefficients of common principal components
	1	2	3	4
Sepal length	0.737	–0.647	–0.164	0.108
	(0)	(–1)	(–22)	(–12)
Sepal width	0.247	0.466	–0.835	–0.161
	(–7)	(28)	(4)	(16)
Petal length	0.605	0.500	0.522	–0.334
	(3)	(–8)	(6)	(2)
Petal width	0.175	0.338	0.063	0.922
	(2)	(–14)	(22)	(7)

Table A.9 Estimated eigenvalue difference for the Iris flower data using the CPC model.

	Coefficients ($\times 100$)
Eigenvalue	Versicolor	Virginica	Setosa
1	5	77	–156
2	38	55	–61
3	–26	–254	253
4	–17	122	–36

Table A.10 Simulation results for normally distributed data.

									$\Vert {\widehat{\mathbf{\Lambda }}}_g-\text{diag}$	$\Vert {\mathbf{D}}_g-\text{diag}$
	$\Vert {\widehat{\mathit{\pi }}}_1-{\mathit{\pi }}_1\Vert$	$\Vert {\mathbf{p}}_1-{\mathit{\pi }}_1\Vert$	$\Vert {\widehat{\mathit{\pi }}}_p-{\mathit{\pi }}_p\Vert$	$\Vert {\mathbf{p}}_p-{\mathit{\pi }}_p\Vert$	${\Vert \widehat{\mathbf{\Pi }}-\mathbf{\Pi }\Vert }_{F_p}$	${\Vert \mathbf{P}-\mathbf{\Pi }\Vert }_{F_p}$	${\Vert {\widehat{\mathbf{\Lambda }}}_g^{*}-{\mathbf{\Lambda }}_g^{*}\Vert }^{\text{AVG}}$	${\Vert {\mathbf{D}}_g^{*}-{\mathbf{\Lambda }}_g^{*}\Vert }^{\text{AVG}}$	$({\widehat{\mathbf{\Lambda }}}_g){\Vert }_{F_p}^{\text{AVG}}$	$({\mathbf{D}}_g){\Vert }_{F_p}^{\text{AVG}}$
G = 2	0.6695	0.6447	0.4371	0.4182	0.2747	0.2622	0.5407	0.5124	0.0628	0.0735
G = 4	0.5289	0.4928	0.2827	0.2698	0.2188	0.1983	0.4983	0.4688	0.0825	0.0898
G = 10	0.3365	0.3070	0.1445	0.1460	0.1288	0.1139	0.4576	0.4400	0.0937	0.0972
G = 50	0.1287	0.1230	0.0561	0.0609	0.0418	0.0422	0.4306	0.4263	0.0996	0.1003
N = 50	0.6894	0.6383	0.4317	0.3893	0.2775	0.2536	0.7229	0.6644	0.1139	0.1264
N = 100	0.5289	0.4928	0.2827	0.2698	0.2188	0.1983	0.4983	0.4688	0.0825	0.0898
N = 500	0.2368	0.2403	0.1072	0.1157	0.0957	0.0940	0.2204	0.2156	0.0385	0.0407
$N=1,000$	0.1622	0.1713	0.0729	0.0819	0.0622	0.0654	0.1576	0.1557	0.0274	0.0288
$N=10,000$	0.0427	0.0512	0.0215	0.0250	0.0162	0.0196	0.0510	0.0509	0.0088	0.0091
p = 5	0.2474	0.2522	0.1502	0.1610	0.1354	0.1353	0.3354	0.3300	0.0763	0.0831
p = 10	0.5289	0.4928	0.2827	0.2698	0.2188	0.1983	0.4983	0.4688	0.0825	0.0898
p = 20	0.8512	0.7825	0.5263	0.4655	0.2366	0.2194	0.7644	0.6654	0.0839	0.0921
p = 50	1.1728	1.1161	0.9453	0.8507	0.1854	0.1808	1.6694	1.3378	0.0853	0.0931
$\varphi =-0.9$	0.6854	0.7102	0.4425	0.4717	0.2792	0.2814	0.5559	0.5641	0.0529	0.0559
$\varphi =0$	0.5289	0.4928	0.2827	0.2698	0.2188	0.1983	0.4983	0.4688	0.0825	0.0898
$\varphi =0.9$	0.6789	0.7023	0.4360	0.4613	0.2802	0.2818	0.5516	0.5599	0.0528	0.0557

The benchmark parameter values are G = 4, N = 100, p = 10, $\varphi =0$, and the eigenvectors of the MLE are ordered by the eigenvalues of $\overline{\mathbf{S}}$.

${\widehat{\mathbf{\Lambda }}}_g^{*}=({\widehat{\lambda }}_{g1},\dots ,{\widehat{\lambda }}_{\mathit{gp}})$ and ${\mathbf{D}}_g^{*}=(d_{g1},\dots ,d_{\mathit{gp}})$, where ${\widehat{\lambda }}_{g1}$ and d_gi are the ith estimated eigenvalue for group g using MLE and Krzanowski’s method, respectively.

Table A.11 Simulation results for chi-square distributed data with 10 degrees of freedom.

									$\Vert {\widehat{\mathbf{\Lambda }}}_g-\text{diag}$	$\Vert {\mathbf{D}}_g-\text{diag}$
	$\Vert {\widehat{\mathit{\pi }}}_1-{\mathit{\pi }}_1\Vert$	$\Vert {\mathbf{p}}_1-{\mathit{\pi }}_1\Vert$	$\Vert {\widehat{\mathit{\pi }}}_p-{\mathit{\pi }}_p\Vert$	$\Vert {\mathbf{p}}_p-{\mathit{\pi }}_p\Vert$	${\Vert \widehat{\mathbf{\Pi }}-\mathbf{\Pi }\Vert }_{F_p}$	${\Vert \mathbf{P}-\mathbf{\Pi }\Vert }_{F_p}$	${\Vert {\widehat{\mathbf{\Lambda }}}_g^{*}-{\mathbf{\Lambda }}_g^{*}\Vert }^{\text{AVG}}$	${\Vert {\mathbf{D}}_g^{*}-{\mathbf{\Lambda }}_g^{*}\Vert }^{\text{AVG}}$	$({\widehat{\mathbf{\Lambda }}}_g){\Vert }_{F_p}^{\text{AVG}}$	$({\mathbf{D}}_g){\Vert }_{F_p}^{\text{AVG}}$
G = 2	0.6843	0.6562	0.4569	0.4330	0.2806	0.2670	0.5864	0.5570	0.0655	0.0766
G = 4	0.5535	0.5151	0.2981	0.2806	0.2254	0.2042	0.5345	0.5021	0.0859	0.0940
G = 10	0.3652	0.3211	0.1550	0.1537	0.1394	0.1206	0.4936	0.4722	0.0979	0.1019
G = 50	0.1412	0.1289	0.0582	0.0636	0.0462	0.0445	0.4630	0.4576	0.1042	0.1050
N = 50	0.7086	0.6548	0.4481	0.4022	0.2840	0.2585	0.7737	0.7107	0.1180	0.1315
N = 100	0.5535	0.5151	0.2981	0.2806	0.2254	0.2042	0.5345	0.5021	0.0859	0.0940
N = 500	0.2507	0.2528	0.1122	0.1221	0.1010	0.0985	0.2364	0.2311	0.0402	0.0425
$N=1,000$	0.1684	0.1760	0.0765	0.0847	0.0660	0.0680	0.1684	0.1662	0.0287	0.0302
$N=10,000$	0.0452	0.0550	0.0226	0.0265	0.0168	0.0206	0.0544	0.0543	0.0092	0.0096
p = 5	0.2713	0.2712	0.1667	0.1717	0.1522	0.1474	0.3755	0.3683	0.0819	0.0896
p = 10	0.5535	0.5151	0.2981	0.2806	0.2254	0.2042	0.5345	0.5021	0.0859	0.0940
p = 20	0.8633	0.7991	0.5323	0.4727	0.2388	0.2219	0.8038	0.7010	0.0859	0.0945
p = 50	1.1768	1.1215	0.9585	0.8619	0.1857	0.1811	1.7166	1.3741	0.0862	0.0942
$\varphi =-0.9$	0.6831	0.7000	0.4130	0.4350	0.2744	0.2733	0.5783	0.5787	0.0649	0.0691
$\varphi =0$	0.5535	0.5151	0.2981	0.2806	0.2254	0.2042	0.5345	0.5021	0.0859	0.0940
$\varphi =0.9$	0.6784	0.6894	0.4254	0.4447	0.2746	0.2736	0.5806	0.5813	0.0647	0.0689

The benchmark parameter values are G = 4, N = 100, p = 10, $\varphi =0$, and the eigenvectors of the MLE are ordered by the eigenvalues of $\overline{\mathbf{S}}$.

${\widehat{\mathbf{\Lambda }}}_g^{*}=({\widehat{\lambda }}_{g1},\dots ,{\widehat{\lambda }}_{\mathit{gp}})$ and ${\mathbf{D}}_g^{*}=(d_{g1},\dots ,d_{\mathit{gp}})$, where ${\widehat{\lambda }}_{g1}$ and d_gi are the ith estimated eigenvalue for group g using MLE and Krzanowski’s method, respectively.

Table A.12 Simulation results for chi-square distributed data with 2 degrees of freedom.

									$\Vert {\widehat{\mathbf{\Lambda }}}_g-\text{diag}$	$\Vert {\mathbf{D}}_g-\text{diag}$
	$\Vert {\widehat{\mathit{\pi }}}_1-{\mathit{\pi }}_1\Vert$	$\Vert {\mathbf{p}}_1-{\mathit{\pi }}_1\Vert$	$\Vert {\widehat{\mathit{\pi }}}_p-{\mathit{\pi }}_p\Vert$	$\Vert {\mathbf{p}}_p-{\mathit{\pi }}_p\Vert$	${\Vert \widehat{\mathbf{\Pi }}-\mathbf{\Pi }\Vert }_{F_p}$	${\Vert \mathbf{P}-\mathbf{\Pi }\Vert }_{F_p}$	${\Vert {\widehat{\mathbf{\Lambda }}}_g^{*}-{\mathbf{\Lambda }}_g^{*}\Vert }^{\text{AVG}}$	${\Vert {\mathbf{D}}_g^{*}-{\mathbf{\Lambda }}_g^{*}\Vert }^{\text{AVG}}$	$({\widehat{\mathbf{\Lambda }}}_g){\Vert }_{F_p}^{\text{AVG}}$	$({\mathbf{D}}_g){\Vert }_{F_p}^{\text{AVG}}$
G = 2	0.7509	0.7139	0.5144	0.4851	0.3008	0.2860	0.7298	0.6923	0.0729	0.0871
G = 4	0.6364	0.5719	0.3529	0.3210	0.2536	0.2261	0.6745	0.6286	0.0971	0.1078
G = 10	0.4534	0.3797	0.1893	0.1789	0.1756	0.1416	0.6224	0.5853	0.1116	0.1176
G = 50	0.2250	0.1506	0.0776	0.0743	0.0773	0.0521	0.5836	0.5627	0.1195	0.1214
N = 50	0.7796	0.7111	0.5193	0.4511	0.3085	0.2796	0.9841	0.9030	0.1318	0.1490
N = 100	0.6364	0.5719	0.3529	0.3210	0.2536	0.2261	0.6745	0.6286	0.0971	0.1078
N = 500	0.3005	0.2924	0.1335	0.1422	0.1214	0.1145	0.2927	0.2845	0.0465	0.0495
$N=1,000$	0.2028	0.2065	0.0904	0.1000	0.0814	0.0807	0.2081	0.2048	0.0333	0.0352
$N=10,000$	0.0540	0.0638	0.0268	0.0305	0.0201	0.0241	0.0667	0.0665	0.0107	0.0111
p = 5	0.3573	0.3367	0.2165	0.2157	0.2004	0.1807	0.5072	0.4930	0.0978	0.1092
p = 10	0.6364	0.5719	0.3529	0.3210	0.2536	0.2261	0.6745	0.6286	0.0971	0.1078
p = 20	0.9110	0.8323	0.5758	0.5158	0.2474	0.2307	0.9505	0.8239	0.0926	0.1029
p = 50	1.1878	1.1319	0.9613	0.8856	0.1862	0.1820	1.8893	1.4998	0.0892	0.0980
$\varphi =-0.9$	0.7662	0.7669	0.4765	0.4881	0.2973	0.2938	0.7276	0.7242	0.0716	0.0762
$\varphi =0$	0.6364	0.5719	0.3529	0.3210	0.2536	0.2261	0.6745	0.6286	0.0971	0.1078
$\varphi =0.9$	0.7532	0.7590	0.4751	0.4881	0.2972	0.2931	0.7297	0.7267	0.0714	0.0760

The benchmark parameter values are G = 4, N = 100, p = 10, $\varphi =0$, and the eigenvectors of the MLE are ordered by the eigenvalues of $\overline{\mathbf{S}}$.

${\widehat{\mathbf{\Lambda }}}_g^{*}=({\widehat{\lambda }}_{g1},\dots ,{\widehat{\lambda }}_{\mathit{gp}})$ and ${\mathbf{D}}_g^{*}=(d_{g1},\dots ,d_{\mathit{gp}})$, where ${\widehat{\lambda }}_{g1}$ and d_gi are the ith estimated eigenvalue for group g using MLE and Krzanowski’s method, respectively.

Holgersson, T., and O. Kekezi. 2017. “Towards a Multivariate Innovation Index.” Economics of Innovation and New Technology 27 (3):254–72. doi: 10.1080/10438599.2017.1331788.

 Web of Science ®Google Scholar

Frenken, K., F. Van Oort, and T. Verburg. 2007. “Related Variety, Unrelated Variety and Regional Economic Growth.” Regional Studies 41 (5):685–97. doi: 10.1080/00343400601120296.

 Web of Science ®Google Scholar

Essletzbichler, J. 2007. “Diversity, Stability and Regional Growth in the United States, 1975–2002.” In Applied Evolutionary Economics and Economic Geography, edited by Koen Frenken, 203. Cheltenham: Edward Elgar.

 Google Scholar