The Immigration-Skill Nexus: Similarities and Differences among the Nordic Countries

ABSTRACT

We examine if the OECD survey of adult skills, PIAAC, can provide information about the relative quality of education and training in Denmark, Finland, Norway and Sweden. We find that the average population scores are inappropriate while the mean scores of natives well approximate the skills of individuals that had the chance to attend the countries’ education and training systems in full. Native mean scores are invariably higher than population scores in all four countries, for all skills: literacy, numeracy, and ICT problem-solving. Thus, viewed as quality measures, average population scores are biased downwards. The native-population differences are small – at most 3.5% of population scores – but statistically significant. We also consider how the ranks of these Nordic countries in the international skill distribution are changed when mean native scores are substituted for average population scores. Except for an improvement in Sweden’s numeracy rank, all changes are statistically insignificant.

KEYWORDS:

• PIAAC
• natives vs full populations
• Nordic
• scores and ranks

Introduction

During the years 2011 and 2012 the OECD adult skill survey, PIAAC¹, was conducted in 24 different countries, including the Nordic countries Denmark, Finland, Norway, and Sweden, cf. OECD (2013a). It might be tempting to interpret the average results of the participating countries as (relative) measures of the quality of their education and training systems. While, in the context of PIAAC’s predecessor, IALS², it has been demonstrated that, in general, differences with respect to cultures, languages, and institutions invalidate such interpretations, cf., e.g., Blum, Goldstein, and Guérin-Pace (2001) and Bonnet (2002), this objection should be less applicable to comparisons within the Nordic region than in other regions.

However, in this article we consider another issue which makes it questionable to treat the average PIAAC results as relative indicators of education and training quality even among the Nordic countries. The issue is immigration and the problem it poses is that many of a country’s immigrants are partly, or entirely, educated abroad, which means that the average PIAAC results of the country’s (full) population may not be an appropriate measure of the skills generated by its national education and training systems.

We know that, in general, there are substantial differences in measured skills of immigrants and natives, not only in PIAAC but also in youth surveys, like PISA³ and TIMSS.⁴ Several studies have shown that much of the variance in these skill gaps can be accounted for by factors that can be attributed to one of the following four aspects: cultures, languages, institutions, and policies – immigration and integration policies; see, e.g., Levels, Dronkers, and Jencks (2017), Rindermann and Thompson (2016), Driessen and Merry (2011), and Marks (2005).

As implied above, the Nordic countries are quite comparable with respect to culture, languages⁵, and institutions. Jakobsen, Korpi, and Lorentzen (2018) contend that until the beginning of the 2000s the Scandinavian countries, i.e., Denmark, Norway and Sweden, were quite similar with respect to immigration and integration policies, too. In 2002 Denmark started to deviate, however, especially from Sweden, through more restrictive immigration and integration policies. The subsequent developments have been described by Brochmann and Hagelund (2010) as Denmark emphasizing the stick, Sweden the carrot, and Norway a combination of the two.

Meanwhile, Finland has distinguished itself from the other Nordic countries by very limited immigration. As a result, Finland’s shares of immigrant participants in international skill surveys has consistently been far below the corresponding shares in the Scandinavian countries. In PISA 2000, only 1.2% of the Finnish participants had a foreign background while the corresponding share for Sweden was 10.5%, cf. Entorf and Minoiu (2005). While slightly reduced, the difference continues to be very large; as shown by Table 5 below, the Finnish and Swedish shares of foreign born participants in PIAAC were 5.7% and 17.5%, respectively.

Table 5. Immigrant population shares in the Nordic countries and in an average of all the countries that participated PIAAC 2011–2012.

Country	Foreign born, %	Thereof: immigrated at age 6+, %	Thereof: immigrated before age 6, %
Denmark	11.8	9.9	1.9
Finland	5.7	4.8	0.9
Norway	13.4	12.2	1.2
Sweden	17.5	15.0	2.5
PIAAC average^a	11.1	9.6	1.5

Source: OECD (2013a) and own computations.

^aAustralia, Cyprus and the Russian Federation are not included in the PIAAC average. Access to data for Australia requires permission from the Australian Bureau of Statistics. The Cypriote data have been withdrawn by the OECD. The data for the Russian Federation are preliminary and do not include the Moscow area. The PIAAC average with respect to PS-TRE skills further does not include Italy, France, and Spain, as these countries (like Cyprus) opted out of the test of PS-TRE skills. Finally, the partition of the PIAAC averages into those that immigrated before and after age 6 does not include Germany, as Germany does not report any information about immigrants’ ages.

In this paper, our objective is to see if the Nordic cross-country differences in immigration and integration policies matter for the countries’ average PIAAC results and, hence, for the interpretation of these results as indicators of the relative quality of their education and training systems. To this end, we test in two ways (detailed below) if the results of the countries’ immigrants impact on the countries’ population results.

The two tests we conduct are based on an absolute and a relative measure, respectively. The absolute measure is the difference in average results in PIAAC among the country’s native-born individuals, on the one hand, and the average results of its entire population, on the other hand. The relative measure involves the corresponding comparison with respect to the country’s rank in the international PIAAC skill distribution. In both tests, we check if the differences between including and excluding the results of the immigrants are statistically significant.

It should be noted that while the test employing the absolute measure is determined only by the features of the country in question, the test involving the relative measure is also influenced by the conditions in all the other countries that participated in PIAAC. In consequence, the results of the two tests might well point in different directions. For example, in spite of an absolute measure yielding the conclusion that the country’s average result in PIAAC has been negatively influenced by immigration, the country’s position in the international skill distribution, i.e., its rank, may be higher when the skill distribution encompasses both immigrants and natives, i.e., the official PIAAC rank, than when skill distribution for natives only is considered.

The absolute measure is relevant for the country’s education and training policies and, therefore, clearly of interest. The relative measure, being largely dependent on factors beyond the individual country’s control, would seem to be of less interest. Nevertheless, it is the country rankings – the country “league tables” – that invariably attract the most attention when international skills surveys are made public. This circumstance, by itself, motivates our relative measure.

Apart from the contributions just described, our analysis adds to the literature by considering, in addition to literacy and numeracy skills, how skills in problem solving in technology-rich environments (PS-TRE) impact on the immigration-skill nexus. Moreover, our empirics do not merely involve intra-Nordic comparisons; we also compare the Nordic countries to an average of all countries participating in PIAAC.⁶

Research Questions

The first of our research questions concerns how to, for our purposes, appropriately make operational the concept of an immigrant. Specifically, we pose the question:

R.1 If we define immigrants as individuals born outside the host country – foreign born individuals – is this group likely to provide a good approximation to respondents that could not have had their entire education in the host country?

The reason for this research question is simplicity, both in terms of definition and with respect to empirical analysis. Given the research question R.4 below, the group that we are interested in is, strictly speaking, not immigrants in general but immigrants who have partly, or entirely been educated abroad. Making the latter group empirically operational requires a somewhat intricate immigrant definition, however, raising the question if a simple and intuitive definition can do the job well enough.

The following three research questions concern the differences in average results in PIAAC of the Nordic countries’ natives and their (full) populations:

R.2 Are the average results of the Nordic countries’ native populations statistically different from the average results of their full populations, with respect to skills in literacy, numeracy, or skills in problem solving in technology-rich environments (PS-TRE)?

R.3 Do the answers to R.2 differ by country and/or type of skill?

R.4 Can the average, official, PIAAC results be viewed as relative indicators of the quality of the Nordic countries’ education and training systems?

The reason for grouping together R.2, R.3, and R.4 is that they are dependent. For example, if the answers to both R.2 and R.3 should happen to be in the negative, then we would claim that, indeed, for the Nordic countries the average PIAAC results can be viewed as relative indicators of the average quality of their respective education and training systems. If, instead, the answers to either R.2 or R.3 should be in the affirmative, then the answer to R.4 would either be no, or, possibly, yes with some qualifications.

Our final research question concerns the relation between our two measures of how immigration might affect a (Nordic) country’s outcome in PIAAC:

R.5 Do the two analyses of how the immigrants influence a (Nordic) country’s performance in PIAAC, in terms of average results and ranking, respectively, yield different conclusions? If so, are these different conclusions consistent across types of skills and/or countries?

General Information about (the First Wave of) PIAAC

PIAAC was initiated by the OECD and conducted for the first time 2011–2012. Altogether, 166 000 individuals in 24 countries participated. The sample frames spanned 16–65 year old (non-institutionalized) residents, irrespective of nationality, citizenship and language. The 24 countries were: Australia, Austria, Belgium (Flanders), Canada, Cyprus, the Czech Republic, Denmark, Estonia, Finland, France, Germany, Ireland, Italy, Japan, Korea, the Netherlands, Norway, Poland, the Russian Federation, the Slovak Republic, Spain, Sweden, the UK (England and Northern Ireland), and the USA.

Implementation, Content, and Sample Sizes

PIAAC was conducted in the respondent’s home, according to a two-step procedure. In the first step, a Background Questionnaire was used to collect information on, i.a., demographics, attained and ongoing education and training, employment and unemployment experiences, and the respondent’s use of various skills, at work and at home. The second step was made up of the cognitive assessments, i.e., the skill tests, which concerned (OECD, 2013a):

• Literacy: The ability to understand, evaluate, use and engage with written texts to participate in society, to achieve one’s goals and to develop one’s potential.

• Numeracy: The ability to access, use, interpret and communicate mathematical information and ideas in order to engage in and manage the mathematical demands of a range of situations in adult life.

• Problem solving in technology-rich environments (PS-TRE): The ability to use digital technology, communication tools and networks to acquire and evaluate information, communicate with others and perform practical tasks.

Assessment of PS-TRE was optional and four countries – Cyprus, France, Italy, and Spain – opted out.

Like the PISA and TIMSS surveys, PIAAC has been designed to provide accurate estimates of proficiency across the population and major subgroups of it, rather than for individuals. This makes it possible to limit the response burden of individual respondents to subsets of the test items; no individual undertakes tests in all of the three skill domains assessed, neither does (s)he face all test items within a given skill domain.

By means of the respondent’s answers and the answers of other respondents with similar background, a statistical distribution is estimated, showing the range to which the correspondent’s “true” skills belong and the probabilities associated with the different parts of this range. The individual’s skills were represented by 10 independent draws from the distribution. This approach, based on Item Response Theory (IRT)⁷, implies that the PIAAC results will involve two types of statistical uncertainty: the IRT-induced uncertainty associated with the individual-level results, and the sampling error.

The sampling error is, of course, partly determined by sample size. Minimum sample size requirements were formulated by the OECD. For countries conducting tests of all of the skills considered in PIAAC, the minimum sample size was set to 5 000. This number was adjusted upward if the country implemented PIAAC in more than one language.

Table 1 contains information about the implementation of PIAAC in the Nordic countries, as well as for an average of all the countries that participated in PIAAC.

Table 1. Information about the implementation of PIAAC in the Nordic countries and for an average of the countries that participated in PIAAC 2011–2012.

Country	# of tested skills^b	Languages^c	Minimum sample size^d	# of respondents^e	Response rate, %^f
Denmark	3	1: Danish	5 000	7 328	50
Finland	3	2: Finnish, Swedish	5 276	5 464	66
Norway	3	1: Norwegian	5 000	5 128	62
Sweden	3	1: Swedish	5 000	4 469	45
PIAAC average^a	2.8	1.3	5 341	7 082	61

Sources: OECD (2013b, Table 3.7), OECD (2013c, Table 14-19, Table 16-7).

^aThe Russian Federation is not included in the PIAAC average, due to the Russian data being preliminary and not including the Moscow area.

^bLiteracy and numeracy tests were conducted in all of the participating countries. Four countries choose to opt out of the PS-TRE test: Cyprus, France, Italy and Spain.

^cOutside the Nordic region, PIAAC was implemented in multiple languages in Canada (English, French), Estonia (Estonian, Russia), Slovak Republic (Slovak, Hungarian) and Spain (Castilian, Basque, Catalan, Galician, and Valencian).

^dThe minimum sample size depends on the number of skills tested and the number of languages in which PIAAC was implemented.

^eFive countries expanded their samples with respect to various sub-populations, namely, Australia: certain states and provinces; Denmark: 55–65 year-olds and newly arrived immigrants; Canada: 16–25 year-olds, certain provinces, language minorities, aboriginals and newly arrived immigrants; Poland: 19–26 year-olds; Czech Republic: 16–29 year-olds.

^fComputed as the ratio of the number of respondents and the size of the sample frame.

According to Table 1, the implementation of PIAAC in the Nordic countries was not very different from the implementation in the hypothetical average PIAAC-participating country. However, the number of respondents was lower in Finland, Norway, and, in particular, in Sweden. Furthermore, the Danish and, especially, the Swedish response rates were well below the PIAAC average. Indeed, Sweden was the only country that did not comply with the required minimum sample size. Still, the OECD decided that, due to extensive efforts of the Swedish National Project Management team to reach respondents in all strata of the population, the sample could nevertheless be deemed representative.

The response rates in Table 1 include all respondents that, at least, have provided answers to some of the questions in the Background Questionnaire (BQ). Accordingly, they have not necessarily taken the skill tests. This matters for our analysis, for two reasons: immigrants are over-represented among the respondents that have not taken the skill tests and, secondly, there are differences across countries with respect to the handling of respondents that did not take the skill tests due to language difficulties. These differences, in turn, depend on the extent to which measures were taken to alleviate language difficulties encountered by respondents filling out the BQ.

Unlike when taking the skill tests, the respondents were allowed to make use of help from others to resolve language difficulties when they filled out the BQ. In some countries, considerable efforts were taken to provide professional help in the form of interpreters. In other countries the respondents had to rely on whatever help they could muster themselves. In the former countries, including, Finland and Sweden, most, or all, respondents filled out the BQ in full. By means of their BQ information, coupled with test results of other reasonably similar respondents, they could be assigned imputed test results. Generally, the imputed test results were very low; cf. OECD (2013c).

In countries that did not provide translation services, some respondents with language difficulties could not answer sufficiently many of the BQ questions to enable imputation of test scores. These respondents were categorized as literacy-related non-respondents and were not included in the computation of the country’s average skill scores. In effect, this is equivalent to assigning literacy-related non-respondents the average scores of the respondents that did, in fact, take the tests. In consequence, countries supplying extra resources to help respondents with language difficulties came out worse with respect to average skills than countries which abstained from doing so.

Table 2 shows that in Finland and Sweden all respondents that could not take the skill tests due to language difficulties were assigned imputed scores⁸, whereas that was not the case in Denmark and, in particular, not in Norway. Still, it can be inferred from the table that imputation was used to a greater extent in each of the Nordic countries than in the average of all of the countries participating in PIAAC.

Table 2. Proportions^a that did not take the skill assessment tests, due to language difficulties; partitioned according to whether or not they were assigned imputed scores^b.

Country	Respondents that did not take the skill tests, %	Whereof: with imputed scores %	Whereof: without imputed scores (non-response) %
Denmark	5.4	5.0	0.4
Finland	6.1	6.1	0.0
Norway	6.8	4.6	2.2
Sweden	5.9	5.9	0.0
PIAAC average^c	4.8	2.8	2.0

Source: OECD (2013b, Table 3.10).

^aWeighted so as to reflect population shares.

^bThe imputed scores concern literacy and numeracy skills, no scores were imputed with respect to PS-TRE skills.

^cThe Russian Federation is not included in the PIAAC average as the data from the Russian Federation are preliminary and do not include the population in the Moscow area.

Average Scores for the Countries’ Full Populations

Table 3 reports mean outcomes by type of skill, for the Nordic countries and for the PIAAC average. In PIAAC, the results are reported both in terms of average scores, on a scale between 0 and 500, and in terms of skill levels defined by score intervals. For literacy and numeracy, six skills levels have been defined, whereas there are four levels with respect to PS-TRE skills. Except for the highest and lowest skill levels, all the levels correspond to intervals of 50 score points, see further notes b and c to Table 3.

Table 3. Mean scores for the Nordic countries and for an average of all countries that participated in PIAAC 2011–2012 (standard errors in parentheses).

Country	Literacy^b	Numeracy^b	PS-TRE skills^c
Denmark	270.8 (0.62)	278.3 (0.73)	283.1 (0.68)
Finland	287.5 (0.67)	282.2 (0.70)	289.4 (0.83)
Norway	278.4 (0.61)	278.3 (0.79)	286.5 (0.57)
Sweden	279.2 (0.68)	279.1 (0.82)	287.8 (0.65)
PIAAC average^a	272.4 (0.76)	268.8 (0.82)	282.8 (0.89)

Source: OECD (2013a) and own computations.

^aAustralia, Cyprus and the Russian Federation are not included in the PIAAC averages with respect literacy and numeracy. Access to data for Australia requires permission from the Australian Bureau of Statistics. The Cypriote data have been withdrawn by the OECD. The data for the Russian Federation are preliminary and do not include the Moscow area. The PIAAC average with respect to PS-TRE skills further does not include Italy, France, and Spain, as these countries (like Cyprus) opted out of the test of PS-TRE skills.

^bThe scores are set on a scale between 0 and 500. They correspond to six skill levels: Below level 1 < 176, 176 ≤ Level 1 < 226, 226 ≤ Level 2 < 276, 276 ≤ Level 3 < 326, 326 ≤ Level 4 < 376, 376 ≤ Level 5.

^cThe scores are set on a scale between 0 and 500. They correspond to four skill levels: Below level 1 < 241, 241 ≤ Level 1 < 291, 291 ≤ Level 2 < 341, 341 ≤ Level 3.

According to Table 3, each of the mean skill scores in all the Nordic countries are higher than the PIAAC average, with the exception of literacy in Denmark. Beside this exception, and PS-TRE skills in Denmark, all the differences between the Nordic countries and the PIAAC average are positively statistically significant.^{9, 10}

In terms of skill levels, Table 3 shows that, for literacy and numeracy, the Nordic countries belong to level 3, except for the Danish literacy, which corresponds to level 2. The PIAAC average belongs to skill level 2 with respect to both literacy and numeracy. In contrast, for skills in PS-TRE the Nordic countries and the PIAAC average all belong to the same skill level, namely level 1.

Results among Immigrants, Compared to Natives

In the introductory section it was pointed out that if one wants to interpret the results in international skill surveys as indicators of the quality of the participating countries’ education and training systems, one should (only) consider the results of individuals that have attended the countries’ educational systems in full. While it is reasonable to assume that this condition applies to (most of) a country’s natives, it may also be true for some of the immigrants, namely those that have immigrated before school starting age. In addition to differences between natives and immigrants it may thus be relevant to consider immigrants that immigrated before vs. after school starting age. Table 4 provides the information required for these comparisons.

Table 4. Mean scores for immigrants and natives in the Nordic countries and an average of all the countries that participated in PIAAC 2011–2012 (standard errors in parentheses).

Skill Country	Foreign born	Thereof: immigrated at age 6+	Thereof: immigrated before age 6	Natives	Natives + foreign born immigrated before age 6
Literacy
Denmark	237.6 (1.96)	232.6 (1.98)	264.3 (5,64)	275.2 (0.66)	275.0 (0.64)
Finland	239.5 (4.13)	230.2 (4.84)	289.2 (8.14)	290.6 (0.65)	290.6 (0.66)
Norway	245.4 (2.55)	241.2 (2.79)	286.2 (6.51)	283.6 (0.62)	283.6 (0.61)
Sweden	235.0 (1.85)	227.7 (2.08)	278.2 (4.93)	288.7 (0.78)	288.4 (0.78)
PIAAC average^a	247.8 (4.09)	243.3 (4.30)	273.0 (7.82)	275.7 (0.78)	275.6 (0.78)
Numeracy
Denmark	245.4 (2.18)	241.4 (2.31)	266.6 (6.04)	282.7 (0.78)	282.3 (0.77)
Finland	233.5 (3.95)	224.4 (4.89)	282.6 (9.42)	285.3 (0.71)	285.3 (0.71)
Norway	238.1 (3.13)	233.4 (3.41)	284.3 (8.03)	284.6 (0.76)	284.6 (0.75)
Sweden	232.7 (1.89)	226.0 (2.10)	272.5 (5.37)	288.9 (0.93)	288.5 (0.91)
PIAAC average^a	241.9 (4.29)	237.6 (4.83)	266.0 (8.74)	272.1 (0.85)	272.0 (0.84)
PS-TRE
Denmark	269.1 (2.30)	266.4 (2.45)	278.8 (4.74)	284.4 (0.70)	284.3 (0.68)
Finland	275.8 (4.67)	270.8 (5.31)	291.5 (8.08)	290.0 (0.87)	290.0 (0.85)
Norway	269.1 (2.61)	266.1 (2.79)	292.2 (6.66)	288.6 (0.61)	288.7 (0.59)
Sweden	260.4 (2.14)	255.1 (2.57)	281.9 (4.59)	292.2 (0.68)	291.9 (0.66)
PIAAC average^a	270.3 (5.61)	264.2 (5.79)	286.7 (9.33)	284.3 (0.93)	284.2 (0.91)

Source: Own computations.

^aAustralia, Cyprus and the Russian Federation are not included in the PIAAC average. Access to data for Australia requires permission from the Australian Bureau of Statistics. The Cypriote data have been withdrawn by the OECD. The data for the Russian Federation are preliminary and do not include the Moscow area. The PIAAC average with respect to PS-TRE skills further does not include Italy, France, and Spain, as these countries (like Cyprus) opted out of the test of PS-TRE skills. Finally, the partition of the PIAAC averages into those that immigrated before and after age 6 does not include Germany, as Germany does not report any information about age at immigration.

The first thing to note in Table 4 is that the group of foreign born, as a whole, consistently had lower scores than the natives and that the differences were larger in the Nordic countries than for the PIAAC average. The largest differences are noted for literacy and numeracy skills; for Sweden and Finland they exceed 50 score points, corresponding to one skill level; cf. note b to Table 3. With respect to skills in PS-TRE the differences are markedly smaller but, again, largest in Sweden. All of the skill differences between natives and foreign born are statistically significant, for all of the Nordic countries as well as for the PIAAC average.

Table 4 also demonstrates that immigrants who have immigrated before school starting age have much better scores than those that immigrated later. Except for Denmark, the Nordic differences with respect to literacy and numeracy amount to between 45 and 60 score points. For Denmark, the differences are on par with those observed for the PIAAC average: around 25–30 score points. However, all the literacy and numeracy differences are statistically significant, including those for Denmark. With respect to PS-TRE skills the differences are more modest: from 12 points in Denmark to 26 points in Norway and Sweden. Only the latter two are statistically significant.

Furthermore, the results of the early immigrants are very close to the average results of natives, especially in Norway and Finland, where the skill differences between the two are less than 5 score points and, in some cases, even to the advantage of the immigrants. About the same holds true for the PIAAC average. Only with respect to numeracy are there statistically significant differences and then only for Denmark and Sweden, in both countries to the early immigrants’ disadvantage.

Thus, apart from the two exceptions just noted, it appears that, on average, immigrants arriving before the school starting age are on par with native born individuals with respect to the three proficiencies assessed in PIAAC. A natural interpretation of this result is that it downplays the importance of origin and emphasizes the role of language skills, as immigrants that arrive at an early age are likely to have better chances to learn the host country’s language than immigrants arriving later.¹¹

One reason why Sweden and Denmark differ from Norway and Finland with respect to the (numeracy) skill gaps¹² between immigrants arriving before school starting age and natives may be that the population shares of the early immigrants are larger in Sweden and Denmark, thus imposing more strains on their educational systems. Table 5 provides information about this issue. The table displays the immigration population shares, in total and broken down by immigrants arriving before the age of six or later, for the Nordic countries and for the average of the countries that participated in PIAAC in 2011–2012. It can be seen that the shares of immigrants arriving before the age of six are largest in Sweden and Denmark. Compared to the corresponding Norwegian share, the Danish share is more than 50% larger (1.9/1.2) and the Swedish share is more than twice as large (2.5/1.2); relative to Finland the differences are even larger. Still, it might seem that the Swedish and Danish shares are not very large. It should be noted, however, that they are population shares, implying that they underestimate the strains imposed on the educational systems by the immigrants that we are considering.¹³ But since we here are interested in relative comparisons across the Nordic countries this underestimation with respect to the levels of the shares is not important.

Returning to the question posed in the first paragraph of this section, i.e., whether or not a country-of-birth criterion yields a good approximation of the residents that have participated in full in a country’s education and training systems, we note that, due to the small population shares of the immigrants that have immigrated before age 6, the differences between the two last columns of Table 4 become negligible. This implies that, for our purposes, natives constitute a good approximation of the residents in a country that are likely to have attended the country’s educational system in full – i.e., natives plus immigrants that have immigrated before age 6. In the following we will thus, for simplicity, consider natives (only).

The Importance of Immigrants: Absolute and Relative Measures

In this section we apply two measures of the importance of immigrants for the results in PIAAC. The first, absolute, measure shows how a country’s mean (population) score relates to the mean score of its natives. The second, relative, measure compares the country’s rank in the distribution of skills over the countries that participated in PIAAC when the ranking is based on its mean population score and its mean native score, respectively.

The Absolute Measure: Mean Scores among Natives vs the (Full) Population

The first column of Table 6, below, shows that the differences between the mean scores among the natives (only) and the (entire) population, are positive, but not very large. The largest differences, observed for Sweden, correspond to, at most, 3.5% of the mean population score; cf. the second column in Table 6. Still, the differences for all of the Nordic countries, as well as for the PIAAC average, are statistically significant.¹⁴

Table 6. Differences in mean scores between natives and (full) populations. Relative differences, in % of population scores, in parentheses. Effect sizes for differences between natives and immigrants.

Skill Country	Difference between natives^c and population^d		Effect size for native – immigrant difference^e	Standard deviation, in score points (for effect size computations)
Literacy				Natives	Immigrants
Denmark	4.4** ^f	(1.6)	0.82	43.5	62.4
Finland	2.9**	(1.1)	1.04	47.2	73.1
Norway	5.1**	(1.9)	0.85	42.0	61.8
Sweden	9.4**	(3.4)	1.16	41.6	63.8
PIAAC average^a	3.3**	(1.2)	0.62	44.0	54.6
Numeracy
Denmark	4.4**	(1.6)	0.75	47.4	64.7
Finland	3.0**	(1.1)	1.02	48.8	74.8
Norway	6.2**	(2.3)	0.90	47.7	73.0
Sweden	9.8**	(3.5)	1.11	45.8	68.6
PIAAC average^a	3.3**	(1.2)	0.61	48.4	59.9
PS-TRE
Denmark	1.8**	(0.5)	0.36	41.8	45.4
Finland	0.8**	(0.2)	0.33	42.1	47.2
Norway	2.6**	(0.7)	0.49	38.6	48.3
Sweden	5.6**	(1.5)	0.74	41.5	48.8
PIAAC average^b	1.8*	(0.5)	0.33	41.6	46.0

Source: Own computations.

^aAustralia, Cyprus and the Russian Federation are not included in the PIAAC averages with respect literacy and numeracy. Access to data for Australia requires permission from the Australian Bureau of Statistics. The Cypriote data have been withdrawn by the OECD. The data for the Russian Federation are preliminary and do not include the Moscow area.

^bRelative to the PIAAC average with respect literacy and numeracy, the PIAAC average for PS-TRE skills further does not include Italy, France, and Spain, as these countries (like Cyprus) opted out of the test of PS-TRE skills.

^cFrom Table 4.

^dFrom Table 3.

^eThe effect size, Cohen’s d, for country j, has been computed according to:$d_j=\left[m\left(P_{j,n}\right)\text{–}m\left(P_{j,i}\right)\right]/\{{\alpha }_j\cdot {\left[s\left(P_{j,i}\right)\right]}^2+\left(1\text{–}{\alpha }_j\right)\cdot {\left[s\left(P_{j,n}\right)\right]}^2{\}}^{1/2}$

where m(P_j,n) and m(P_j,i) denote mean scores among country j’s natives and immigrants, respectively (from Table 4), α_j its immigrant share, and s(P_j,i) and s(P_j,n) denote the standard deviations, in score points, for the immigrants and the natives, respectively. The latter shall not be confused with s[m(P_j,i)] and s[m(P_j,n)] which denote the standard errors in the mean scores of country j’s immigrants and natives, respectively.

^fThe symbols * and ** indicate statistical significance at the significance levels 5 and 1%, respectively.

It should be noted that the variation across the Nordic countries in column one of Table 6 is quite substantial. The largest differences concern Sweden and the smallest Finland – the former are more than three times as large as the latter. This is a direct consequence of Sweden and Finland having the largest and smallest shares of immigrants in the population, respectively, cf. Table 5. It can also be seen that, except for Finland, the differences in the Nordic countries exceed the average difference among all the countries that participated in PIAAC.

For conclusions regarding policy measures it is important to remember that although the differences in the first column of Table 6 are comparatively small, the underlying differences in skills between natives and immigrants are very large. To assess their practical importance, we provide the corresponding effect sizes (Cohen’s d) in the third column. Applying the effect size criteria in Hattie (2009) – small effect: 0.2 ≤ d < 0.4; medium effect: 0.4 ≤ d < 0.6; and large effect: 0.6 ≤ d – we see that for literacy and numeracy all effects are large, for all of the Nordic countries, as well as for the PIAAC average. With respect to skills in PS-TRE, a large effect size is found for Sweden, a medium-sized for Norway, while the effect sizes are small for Denmark, Finland, and the average of the PIAAC countries.

According to Table 6, the effect sizes in all of the Nordic countries are at least as large as the corresponding effect sizes for the PIAAC average. The skill differences between natives and immigrants are thus more important within the Nordic region than outside of it.

The Relative Measure: Rank with Respect to Natives vs the (Full) Population

When the results of international surveys are reported, the country league tables, showing the country rankings, invariably attract the most interest. Unfortunately, little attention is generally paid to the fact that the rankings often are associated with considerable uncertainty. For example, if a country’s rank is 5, that rank may not be significantly different from rank 3, or 7. Moreover, when rank uncertainty is reported it is frequently incorrectly estimated; Lind and Mellander (2016) show that this is the case in PIAAC and that it results in the rank uncertainties being under-estimated.

We apply a method suggested by Leckie and Goldstein (2011). The method makes use of the fact that a country’s skill scores are asymptotically normally distributed, implying that the mean scores and standard errors in Table 3 completely determine the asymptotic skill distribution. For a given skill, we make one random draw from each of the skill distributions of the countries participating in PIAAC. Based on the outcomes of these random draws we rank the countries. We then repeat this process 10 000 times. As a result we get, for each country, a rank frequency distribution. The mean of this distribution constitutes an estimate of the country’s rank. Furthermore, a 95% confidence interval for the rank is obtained by cutting off 2.5% of the distribution’s mass in its left and right tails and using the ranks corresponding to the cut-off points as the lower and upper limits of the confidence interval.

Next, we do the same exercise with respect to the countries’ natives only. Finally, we compare the rankings based on natives with the population rankings. If, for a given country, the confidence intervals for its native and population ranks do not overlap, we conclude that the difference between the corresponding ranks is statistically significant.

Table 7 reports mean ranks and rank confidence intervals for native skills and population skills, by skill type, for the Nordic countries.¹⁵ For example, consider the rankings with respect to numeracy in Table 7. For Denmark, Finland, and Norway the mean ranks for native skills differ slightly from the mean ranks for the skills of the corresponding full populations. However, the differences are not statistically significant – the rank confidence intervals overlap. For Sweden, on the other hand, the mean rank for native skills, 1, is statistically significantly different from its mean rank for population skills, 5, as the confidence interval of the former, [1,1], does not overlap with the confidence interval of the latter, [3,6].

Table 7. Mean ranks and confidence intervals^a, in square brackets, for the Nordic countries among the countries that participated in PIAAC 2011–2012, by skill and for natives and (full) populations, respectively.

Skill Country	Rank based on scores among natives only		Rank based on (full) population scores
Literacy^b
Denmark	11	[8,13]	13	[11,14]
Finland	2	[2,2]	2	[2,3]
Norway	5	[5,5]	5	[4,5]
Sweden	4	[2,4]	4	[4,4]
Numeracy^b
Denmark	7	[5,7]	6	[4,6]
Finland	4	[2,4]	2	[2,2]
Norway	5	[3,5]	6	[4,7]
Sweden	1	[1,1]	5	[3,6]
PS-TRE^c
Denmark	9	[6,10]	8	[5,10]
Finland	3	[2,4]	2	[1,2]
Norway	4	[3,4]	4	[2,4]
Sweden	2	[1,2]	3	[2,3]

Source: Own computations.

^aAccording to the simulation method suggested by Leckie and Goldstein (2011); briefly explained in the main text.

^bThe ranking is based on the 21 countries for which there are literacy and numeracy scores.

^cThe ranking is based on the 18 countries for which there are PS-TRE scores.

It so happens that the statistically significant difference just noted is the only one to be found in Table 7. Accordingly, in general, the country skill rankings are not very sensitive to the observed native-immigrant skill gap.

Conclusions and Discussion

At a comprehensive level, this article addresses, from a Nordic perspective, the question whether immigration matters for adult skills, as measured in PIAAC. This question, in turn, is motivated by the question to what extent the results in PIAAC can be interpreted as indicators of the quality of the participating countries education and training systems.

Regarding the latter question, we examine if, in the PIAAC context, a Nordic country’s foreign-born individuals are likely to provide a good approximation to those of its residents that have not had their entire education in the country (research question R.1). Our analysis answers this question in the affirmative. From this we infer that native born constitute a good approximation to the larger group “native born and immigrants that have immigrated before age six”, which, for practical purposes, is the definition that best corresponds to the residents that have had all their education and training in the country. This result of ours holds for all three of the skills considered in PIAAC, i.e., literacy and numeracy skills, and skills in problem solving in technology-rich environments (PS-TRE).

The reason why we arrive at this result is two-fold. First, the PIAAC scores of those that have immigrated before age 6 are very close to the scores of natives. Only for Sweden is the difference in average results between the two categories statistically significant, to the advantage of the natives, for all skills measured in PIAAC.¹⁶ Second, the population shares of those that immigrated before age six are small, ranging from 0.9% in Finland to 2.5% in Sweden. In consequence, the difference between the average scores of the countries’ natives, on the one hand, and their natives plus foreign born that immigrated before age 6, on the other hand, is negligible – less than 0.5 score points for each skill.

Since, by supposition, the Nordic countries are similar with respect to language, culture and institutions, the finding that native born individuals provide a good approximation of the residents who have had their (entire) education and training in the country yields the following conclusion: In the Nordic countries, the average PIAAC scores of native born individuals can be regarded as indicators of the quality of the countries’ education and training systems.

We proceed to examine if the results for natives differ from the results for the countries’ full populations (research question R.2). We find that the mean score differences between the countries’ natives and their full populations – what we call the absolute measure of the importance of a country’s immigrants for its mean (population) skill score – are positively statistically significant, for all the Nordic countries and all of the skills (research question R.3). Accordingly, the Nordic countries’ average population results in PIAAC cannot be viewed as indicators of the quality of their education and training systems (research question R.4). Compared to the average native scores, which we take as quality indicators, the average population scores constitute under-estimates.

The differences between the average results for the countries’ natives and for their full populations are small. The largest differences, observed for literacy and numeracy skills in Sweden, are just below 10 score points, corresponding to around 3.5% of the mean population scores. However, this is due to the fact that the population shares of foreign-born were comparatively small when PIAAC was conducted – only in Sweden was it over 15%. The underlying native-immigrant skill differences still deserve attention. As a measure of their practical importance, we have computed the implied effect sizes (Cohen’s d). We first note that for all skills and in all of the Nordic countries the effect sizes are at least as large as the effect sizes for the PIAAC average, implying that the differences between natives and immigrants are more important within the Nordic region than outside of it. Furthermore, the effect sizes are large (d ≥ 0.6) for all skills in Sweden, and for literacy and numeracy skills in Denmark, Finland and Norway, indicating that additional resources targeted at immigrants are likely to be worthwhile. Coupled with our findings concerning immigrants that have immigrated before vs. after school starting age, we conjecture that the extra resources should target youth and adult immigrants and focus on developing their proficiency in the host country’s language, as soon as possible upon immigration.

Our relative measure of the importance of immigration for the results in PIAAC – changes in skill rankings based on natives only, compared to population-based, official, PIAAC rankings – demonstrates that the country league tables are quite insensitive to the native-immigrant skill gap. Only one statistically significant difference is established: with respect to numeracy skills, Sweden’s rank is improved from 5 to 1 when the skill ranking is based on the countries’ natives only, instead of on their full populations. Other minor rank changes are observed, too – downwards as well as upwards – but none of them statistically significant. Thus, our relative measure of how immigration influences the performance of the Nordic countries in PIAAC yields a different conclusion than our absolute measure (research question R.5). Specifically, the absolute measure says that immigration matters, with respect to all Nordic countries and all skills, while the relative measure says that immigration matters only for Sweden, and only with respect to numeracy skills.

Table 8 below summarizes the similarities and differences among the Nordic countries with respect to immigration and the skills considered in PIAAC. Notably, the table lists as many differences as similarities. Thus, while the Nordic countries resemble one another in many respects they vary substantially regarding the immigration/skill intersection.

Table 8. Similarities and differences among the Nordic countries with respect to immigration and the adult skills measured in PIAAC.

Similarities		Differences
1	Native born individuals constitute good approximation of residents that have had all their education and training in the country.	Immigrant population shares; vary from 5.7% in FI to 17.5% in SE.
2	The mean scores of all immigrants taken together are markedly lower than the mean native scores. For literacy and numeracy, effect sizes > 0.6.	Mean scores of immigrants that immigrated before age 6, compared to mean native scores; no significant differences in FI and NO, but significantly lower in DK and SE.
3	Mean score differences between the countries’ natives and their full populations are positive and significant, but small (<10 points, or 3.5% of mean population scores).	Country rankings based on natives vs full populations; no significant rank differences in DK, FI, and NO, while for SE rank in numeracy significantly better for natives (1) than for population (5).

Note: The acronyms DK, FI, NO, and SE denote Denmark, Finland, Norway, and Sweden, respectively.

Concerning the last items in Table 8, Similarity 3 and Difference 3, why is it that our absolute and relative measures appear (at least partly) to point in different directions? The reason is simply that these two measures concern different aspects and do not have very much in common. Specifically, that a country’s average score is influenced by its native-immigrant skill gap (the absolute measure) is neither a necessary, nor a sufficient, condition for the country’s rank in the international skill distribution to be sensitive to its immigration (the relative measure). With respect to the relevance for the country’s education and training policies, the absolute measure is the more important one. Yet, the debate about PIAAC, as well as other international skill surveys, is almost entirely focused on the relative measure. Our analysis shows that this is unfortunate and implies a risk for misguided conclusions: While our absolute measure signals the need for enhanced educational efforts targeted at foreign-born in all of the Nordic countries and with respect to all skills considered in PIAAC, our relative measure provides this message only for Sweden and, moreover, only for numeracy skills.

Finally: A few remarks in relation to the differences in immigration and integration policies across the Nordic countries that were mentioned in the Introduction – more lenient in Sweden, tougher in Denmark and Finland, with Norway in between.

Consistent with those differences we find that (i) the population shares of foreign born are largest in Sweden and smallest in Denmark and Finland and that (ii) the measured skills of foreign born in Sweden are lower than the measured skills of foreign born in the other Nordic countries and, moreover, differ more from the skills of natives.

However, some of our findings appear to be inconsistent with the policy differences. First, the scores of foreign born in Denmark and Finland are, in general, not higher than in Norway. Second, like Sweden, Denmark seems to have difficulties equalizing the skill levels of immigrants arriving before the age of six with the skill levels of natives. These latter findings resonate with the conclusions drawn by Jakobsen et al. (2018) regarding immigrants’ employment and earnings, namely that the harsher Danish policy does not seem to have had clear-cut positive effects compared to the softer Swedish and Norwegian policies.

The discussion about immigration and integration policies and, thereby, the immigration-skill nexus will continue, of course. We believe that some observations based on the results in this article provide important inputs to that discussion. The first observation is that the quality of a country’s education and training systems should be measured in terms of the competences of those that have had the chance to attend these systems in full, be they natives or immigrants. The second observation is that when immigration and integration policies are considered, age at immigration is essential for the design of appropriate education and training measures. The last observation is that care should be taken to avoid the discussion being guided by league tables published in connection with international skill surveys, as these league tables are more likely to be misleading than to be informative regarding appropriate policies.

Acknowledgements

Helpful comments from Mary James, Martin Lundin, Anders Stenberg, participants at the Nordic seminar on basic skills in Copenhagen, November 17, 2016, and two anonymous referees are gratefully acknowledged.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Erik Mellander http://orcid.org/0000-0002-7170-9303

Additional information

Funding

This work was supported by NordForsk [Grant No 54681].

Notes

1 Programme for International Assessment of Adult Competencies.

2 International Adult Literacy Survey.

3 Programme for International Student Assessment.

4 Trends in International Mathematics and Science Study.

5 At least to the extent that Swedish can be considered a working language in Finland.

6 Specifically, to all of the countries that participated in the first wave of PIAAC which took place in 2011 and 2012 and comprised 24 countries see OECD (2013a). Since then a second wave has taken place in 2014 and 2015, involving an additional 9 countries, cf. OECD (2016).

7 Cf. van der Linden and Hambleton (1997).

8 It can be added that among all of the countries that participated in PIAAC, this was true only for one additional country, Poland; cf. OECD (2013b, Table 3.10).

9 The statistical significance can approximately be checked by means of the following statistic:$z=\left[m\left(P_j\right)\text{–}m\left(P_a\right)\right]/\{s{\left[m\left(P_j\right)\right]}^2+s{\left[m\left(P_a\right)\right]}^2{\}}^{0.5}$where m(P_j) and m(P_a) denote the mean score points of Nordic country j and for the PIAAC average, respectively, and s[m(P_j)] and s[m(P_a)] denote the corresponding standard errors. The approximation is due to the fact that country j is included in the PIAAC average implying that P_j and P_a are not independent, as presumed by the formula for z. As a Nordic country only makes up 1/21 of the PIAAC average the approximation error is likely to be small, however.

10 Unless otherwise stated, we will take statistical significance to mean statistically significant at the 5 % level. Relying on asymptotic normality, this implies that if z > 1.96 or z < −1.96 then, with 95 % probability, the observed difference m(P_j) – m(P_a) cannot be attributed to random variation.

11 This is not only of importance in general but also specifically for the results in PIAAC, as PIAAC is only conducted in the official language(s) of the participating countries, cf. Table 1.

12 The skill gaps with respect to literacy and PS-TRE skills are also larger for Sweden and Denmark than for Norway and Finland, but not statistically significant.

13 To capture the strains imposed on the educational system, the shares of immigrants arriving before school starting age among adult individuals (16–65) that attended compulsory school in the respective countries should be a more appropriate measure.

14 To test if the differences are statistically significant we make use of the fact that the mean score of the population in country j, m(P_j,p), can be expressed according to(I) $m\left(P_{j,p}\right)={\alpha }_j\cdot m\left(P_{j,i}\right)+\left(1\text{–}{\alpha }_j\right)\cdot m\left(P_{j,n}\right)$(I) where α_j denotes the immigrant population share in country j while m(P_j,i) and m(P_j,n) denote the mean scores of the country’s immigrants and natives, respectively. Using this equality, we formulate the differences in mean scores between the country’s natives and its population as(II) $m\left(P_{j,n}\right)\text{–}m\left(P_{j,p}\right)=\left[{\alpha }_j\cdot m\left(P_{j,n}\right)\right]\text{–}\left[{\alpha }_j\cdot m\left(P_{j,i}\right)\right].$(II) Compared to the left-hand side of (II), the advantage of the right-hand side expression is that it concerns two independent stochastic variables. Accordingly, the test can be conducted by means of a z-statistic, analogous to the one discussed in footnote 9.

15 The results reported in Table 7 are based on simulations for all the countries that participated in PIAAC 2011–2012. To save space, only the mean ranks and rank confidence intervals for the Nordic countries are provided, however.

16 This implies that for Sweden the results for natives over-estimate the average skills among adults that have been educated and trained in the country.