Exploring the OECD survey of adult skills (PIAAC): implications for comparative education research and policy

The Program for the International Assessment of Adult Competencies (PIAAC) was designed to provide broad empirical evidence about adult skills and about the linkages between the formation of such skills and workplaces internationally. The survey measures adults’ proficiency in key information-processing skills – literacy, numeracy and problem solving – and gathers information and data on how adults use their skills at work and outside work. The OECD released PIAAC data and preliminary results to international research communities and policy circles for 24 countries in October 2013, for 9 more countries in June 2016, and for additional 6 countries in November 2019. Since the release of PIAAC, special attention has been paid to the OECD survey by international researchers and policy makers – e.g., European Commission’s Report on PIAAC in 2013, Educational Testing Services’ sponsored international conference in 2013, working papers from the U.S. National Centre for Educational Statistics in 2014, the U.S. National Association of Workforce Boards’ Forum on PIAAC in 2014, the AERA Symposium on PIAAC in 2014, and the U.K. Academy of Social Sciences’ Seminar in 2014, to name a few. These academic or policy meetings focusing on PIAAC reflect the burgeoning demand for comparative studies and empirical research-based debates on adult skills.

The current special issue aims to contribute to these international debates by analysing the potential and limitations of PIAAC data for comparative research on adult skill formation and the associated policy debates. We are particularly interested in what PIAAC can tell us about existing educational and social inequalities between adults across and within countries. The seven empirical studies included in this special issue interrogate skill inequalities in relation to three domains: acquisition of skills through education (Heisig, Elbers, and Solga 2019), returns to education and training (Danner, Lechner, and Rammstedt 2019; Kwon, Park, and Byun 2019) and labour market mismatch (Capsada-Munsech 2019; Choi, Guio, and Escardibul 2019; Fregin, Levels, and van der Velden forthcoming; Sevilla and Farías 2019). The special issue consists of four cross-national comparative studies (Choi, Guio, and Escardibul 2019; Danner, Lechner, and Rammstedt 2019; Fregin, Levels, and van der Velden forthcoming; Heisig, Elbers, and Solga 2019) and three studies delving deeply into a single country analysis focusing on Spain (Capsada-Munsech 2019), South Korea (Kwon, Park, and Byun 2019), and Chile (Sevilla and Farías 2019). The selected papers show the potential of PIAAC not only for studying international patterns but also for analysing how national political economy contexts shape inequalities in adult skills. At the same time, they provide several insights into and reflections on the methodological limitations of this kind of international large-scale assessments for comparative education research and policymaking.

Education system characteristics and skill inequalities

The OECD’s intention to develop the international assessment study of adult skills could be interpreted as an attempt to govern by data, or to generate another type of international league table. At worst, it could be perceived as a neoliberal initiative to stimulate another international horserace. These are valid concerns in light of the recent history of the OECD and Western governments’ use of large international assessment studies for policy. Within neoliberal policy environments, the whole concept of adult learning throughout life is often reduced to a narrow set of competency and skills mainly for employability and upgrading work-related skills (Lee and Friedrich 2011). The OECD Skills Strategy and PIAAC are clearly aligned with this neoliberal framework, given their conceptual emphasis on measurable productive outcomes of adult learning, framed as human capital (Valiente 2014). These criticisms to large-scale international assessments raised by comparative education researchers need to be borne in mind, given that more often than not, the OECD intentionally cherry-picks particular sources of evidence in its recommendations for education policymaking (Lee 2018), often pays little attention to other important social goals of adult learning (Lee and Friedrich 2011), and tends to reinforce the competitive global testing culture among countries (Volante 2017).

Nonetheless, we cannot ignore the value of PIAAC for quantitative comparative research in terms of the availability of nationally representative samples, wide-ranging scope of variables and comparable measurements. The availability of international comparable data on learning outcomes has boosted the rigour of quantitative comparative research in education. It enables researchers to test competing models of organising the provision of educational opportunities and resources across different countries (e.g. public vs private provision; comprehensive vs tracking systems), as well as their relationship with learning outcomes. Probably, the most positive impact of these assessments has been the rediscovery of equity as the most important challenge of education systems globally. With most of the inequality in student performance located not between countries but within countries, and with family background characteristics estimated as the most powerful predictor of these differences, the inconsistency of these findings with the meritocratic agendas in education has been repeatedly unveiled by comparative researchers (Alegre and Ferrer 2010; Duru-Bellat and Suchaut 2005; Gorard and Smith 2004). Although this was not probably the original intention of the OECD, the use of these assessments by comparative researchers has contributed to amplifying international debates around the (hidden) nature of social inequalities in education, the mechanisms of their reproduction, and the more effective education policies and practices to advance the equity agenda in education.

One study in this special issue has directly focused on the relationship between education system characteristics and skill inequalities among the adult population. Heisig, Elbers, and Solga (2019) studied how the relationship between parental education and both educational attainment (qualification) and achievement (skills) varied among adults (aged 30 to 49) across 23 education systems participating in the first two rounds of PIAAC. They selected this age group because most respondents had already completed their initial education by the age of 30 whereas the upper bound of 49 guaranteed a good match with the institutional characteristics of the education system during a given period of time. Through this strategy, the authors managed to use the PIAAC database to compare the level of inequality of opportunity at the end of initial education across countries. They found that parental education had a stronger influence on educational attainment than on educational achievement in all the countries. This result would reinforce the hypothesis that parents may find it easier to influence their children’s level of formal qualifications than their level of skills. This implies that the acquisition of the educational credentials is the primary mechanism of reproduction of social inequalities and not the acquisition of skills.

They also found that the association between parental background and educational attainment was stronger in education systems with early tracking in secondary education, but not for educational achievement. This finding is partially aligned with previous studies (Brunello and Checchi 2007) and suggests that tracking may increase social inequality of opportunity in educational attainment because it can force students and parents to make strategic decisions in educational pathways that require students and parents to have high levels of information and understanding of the education system. In addition to the central focus of the paper on initial education, the authors also analysed the relationship between participation levels in formal and non-formal adult education and training in the country and inequalities by social background. Interestingly, countries with a higher prevalence of formal adult education and training tended to present larger inequalities by parental education while those with higher prevalence of non-formal adult education and training showed the opposite trend. Given the dearth of contextual information to explain this intriguing finding in the paper, we await further research on the contrasting patterns.

Most readers may have noticed that we avoided the use of the term ‘effects’ when commenting on the relationship between system characteristics and skills. The reason for that caution, as the authors adequately explain in the text, is that the cross-sectional character of PIAAC data does not allow researchers to interpret associations between system characteristics and skill measures causally. This may sound like an obvious reminder for comparative researchers, but it is an important limitation of PIAAC data that education policy actors should bear in mind when comparing results between countries and extracting policy lessons.

Inequalities in the returns to education and training

Human Capital Theory (HCT) has been positioned as the mainstream policy approach to understanding the relationship between education and work, from the perspective of individuals, and between education and economic growth, from a societal perspective. The underlying assumption of HCT is that investment in individuals’ education will contribute to the development of individuals’ skills, which will lead to higher earnings for those individuals in the labour market and to greater productivity and growth in the economy. Proponents for HCT assume that the logic of HCT is equally valid for every individual regardless of national contexts, so investment in individuals’ education would be the most efficient strategy for individuals to improve their living conditions and collectively for economies to grow (Becker, 2002). They also assume that the expansion of education would solve social tensions in capitalist societies as everyone has the same capacity and accessibility to be educated and acquire the skills that are valued by the market. This excessive optimism in the individual and collective returns to education has been an ideological driver of the knowledge-based economy discourse and of the worldwide expansion of higher education (Schofer and Meyer 2005). In recent years and inspired by HCT, international large-scale assessments (ILS) like PISA have been used by the OECD to argue that national scores in these tests could explain differences in economic growth among countries (Hanushek and Woessmann 2008). Despite the obvious interest of the OECD for this to be true, independent analyses of PISA and PIAAC data by Rappleye and Komatsu (2019) have consistently refuted this thesis.

The two papers analysing the returns to education and training in PIAAC in this special issue (Danner, Lechner, and Rammstedt 2019; Kwon, Park, and Byun 2019) question the underlying assumptions of HCT and provide empirical evidence on the limitations of HCT by explaining the relationship between education and work in contemporary societies. The two papers also broaden the scope of the debate by considering the role of non-cognitive skills (typically skills that require dealing with emotions, communication and the organisation of work), comparing patterns in subjective measures of job satisfaction, besides earnings, and including participation in non-formal learning in their analysis of the unequal labour market returns to education.

Danner, Lechner, and Rammstedt (2019) analysed the relationship between non-cognitive skills and career success (income and job satisfaction) drawing on data from the OECD’s 2010 PIAAC field trial. The interest in non-cognitive skills has boomed in the last few years due to the difficulties in the traditional measures of human capital (e.g. education level, cognitive skills) to explain unequal economic returns to education among different individuals and groups. In this PIAAC field trial, respondents from 20 countries were randomly assigned to different parts of the background questionnaire, one of which contained a grit scale. The paper focused on the respondents who completed the grit scale, which measured perseverance and passion for long-term goals among participants. The authors hypothesised that ‘persons with a high level of grit should be more motivated and work harder, they should be able to cope with setbacks, remain focused on their goals, and thus be more successful as a result’.

The results showed a positive association between grit and income and job satisfaction, but it was not always significant across all countries. While these results were in line with other studies, suggesting the important role of non-cognitive skills in career success (Heckman and Kautz 2012), they further showed that grit does not seem to benefit individuals with different social backgrounds equally. Again, caution about causality should be exercised in interpreting the finding. Furthermore, in terms of inequalities, it would be important not just to assess the relationship between grit and returns among different social groups, but also to 1) assess the unequal opportunities among these groups to develop non-cognitive skills such as grit and 2) how grit can help disadvantaged people to overcome the labour market consequences of their low levels of education.

Kwon, Park, and Byun (2019) focused on gender inequality in the patterns of non-formal learning participation in South Korea. As the authors highlighted, South Korea presents an intriguing research setting for researching gender inequality in nonformal learning and its possible link to gender inequality in earnings in that the country displays the largest gender gap in earnings among all OECD countries and male-dominated or oriented working culture is perpetuated across many formal organisations. Kwon et al. reported that significantly positive relationships still remain 1) between participation in distance learning and earnings and 2) between workshops and earnings for both males and females whereas the positive relationship between on-the-job training (OJT) and earnings is found only for females, even after controlling for other variables. Based on this finding, they posited that ‘the gender difference in the relationship between OJT and earnings favouring females does not support both the human capital argument and the critical feminist perspective’. Instead, they attributed this important finding to some unique aspects of Korea’s male-oriented or dominated organisational culture, explaining that for Korean male workers, OJT may not be the only source by which they can forge work-related networks and gain work-related information that may positively influence their earnings.

There are various other informal learning opportunities favouring male workers, given the male-dominated working culture in South Korea. However, OJT appears to be a type of non-formal learning opportunities, associated with raising earnings, that is relatively easily accessible to female adults, compared to other types of non-formal learning opportunities from which female adults are often excluded because of male-oriented organisational culture. The finding is theoretically important because it suggests that the broader societal contexts that shape organisational culture need to be examined, and their effects on individual and organisational level outcomes in comparative education research (Lee and Hallinger 2012).

The two papers discussed in this section question one fundamental principle of HCT: its pretended universalism. The two papers convincingly show how important contextual conditions are for the individuals to develop their skills and for the labour market to reward these skills. Non-cognitive skills are a clear example of individual capacities whose acquisition is not always related to the education level of the individual. Similarly, the opportunities to develop individuals’ skills through education and training are not always equally accessible to men and women, and definitively not equally rewarded by the labour market in terms of earnings. Understanding how these relationships are shaped by different political, economic and cultural contexts is one of the main contributions of comparative education research to these debates.

Labour market mismatch

Research on labour market mismatch has traditionally been dominated by economistic and functionalist accounts of education and training that have mainly focused on the supply of skills. The underlying assumption behind this ‘supply side fundamentalism’ is that the educational offer should respond to the needs of employers and that every mismatch between the skills of the workforce and their labour market situation can be explained by failures in education planning and delivery, and/or by the poor investment of individuals in their education and training (McQuaid and Lindsay 2005). Under this perspective, the ‘invisible hand’ of the market should perfectly match the supply and demand of skills in the long term, as both workers and employers will aim to fully utilise the existing skills to maximise their returns. Thus, skill mismatch is considered a temporary problem that can be addressed through better labour market information and a greater labour market orientation of education provision.

The reality of labour markets internationally is far from this prediction. Labour market mismatches depend significantly on social stratification processes and demand-side factors. Educational institutions often operate as mechanisms of social selection that reproduce and reinforce inequalities between individuals and social groups (Althusser 1971). The fact that individuals from different social backgrounds systematically achieve unequal labour market outcomes is substantially explained by pre-existing inequalities passed on through their upbringing and their formal qualifications obtained from the education and training system. In addition to gaining skills, the education system provides credentials that signal to employers what a person is able to do. Skill level, type and quality vary across people with the same credentials or can even be similar to others that do not hold these credentials. However, credentials are used by employers as a proxy for the trainability and productivity of individual workers. This ‘sheepskin effect’ and competition among workers for the best jobs that match their education level creates credential inflation and, in turn, skill mismatch such as over-qualification (McGuinness 2006).

Individual workers have also very limited power to influence their labour market situation, as socioeconomic forces privilege or disadvantage some social groups over the rest. Labour markets are institutionally segmented, offering unequal job opportunities to those that have access to them. On the one hand, the primary segment (firm-specific internal market) offers high-skilled and high-paid jobs, with career progression opportunities and stability to those with better credentials and networks. On the other hand, the secondary segment offers low-skilled and low-paid jobs, with unstable contracts and limited training possibilities to those who did not have access to the primary segment (Doeringer and Piore 1985). Workers are sorted into either one segment of the labour market or the other depending on their education, but also their race, gender and class. Beyond individual education and social background, employment possibilities very much depend on what employers look for, the amount of jobs available and the pace of matching these with the supply of education and skills in different political economy contexts (Lauder et al. 2011).

The four papers focusing on labour market mismatch in this special issue share the understanding that both supply and demand factors influence the configuration of mismatches, that these factors are context dependent, and that mismatches do not affect all individuals and social groups equally. Two of the papers are single-country studies (Chile and Spain), and the other two are cross-national comparisons.

Sevilla and Farías (2019) carried out their single country study of qualification and skill mismatches in Chile, which is a very relevant context for this kind of international research, because Chile is a highly marketised education system that has seen a rapid educational expansion combined with significant stratification of the higher education sector. In this context, the skill levels of the adult population are still low when compared to other OECD countries in PIAAC but, surprisingly, it shows relatively high rates of overqualification and overskilling, especially for the young generations. The authors explain this phenomenon through demand and supply factors. On the demand side, there is a large percentage of positions in the labour market that are linked to low levels of skill requirements. On the supply side, the significant increase in higher education enrolments, accompanied by the hierarchical differentiation of the system, has generated the inflation of qualifications.

An interesting idea emerging from this study is the crowding-out effect in the labour market when those better educated start to occupy low-qualified jobs. Their trend analysis of national data from a household survey shows how the overqualification that was concentrated in the better educated population is starting to affect the less educated youngsters, who are forced to accept lower qualified jobs because the others have been occupied by higher education graduates. The decision of combining the analysis of PIAAC with data from a national source seems a good strategy to follow by other researchers, particularly when willing to focus their work on a single country.

Capsada-Munsech (2019) also conducted a single country study (i.e. Spain) but in this case with a focus on the influence of family background on the occurrence of overeducation. She demonstrated how PIAAC data can be used to compare overeducated workers with similar education and skills levels while controlling for parental education as a source of skill gain via family socialisation. A very interesting aspect of her approach is the consideration of overeducation as another form of social disadvantage of the adult population, which the literature has associated mainly with graduates coming from working-class families. As happened with Chile, the rapid education expansion in combination with a large percentage of low-skilled jobs has caused higher levels of overeducation in Spain than in most OECD countries. When comparing overeducation occurrence among social groups, younger and middle-aged workers are more likely to be overeducated than the older ones, and women are more prone to be overeducated than men. An important contribution of the paper from a social inequality angle is that mother’s education is the most relevant social background indicator to predict overeducation, even when controlling for workplace characteristics and skill levels.

Fregin, Levels, and van der Velden (forthcoming) offered an innovative analytical approach to cross-country comparisons of labour market mismatch. Their study mainly focused on demand-side factors related to the institutional characteristics of labour markets to explain cross-country differences in skill matching. They considered employment protection legislation, unemployment benefits, and enforcing and enabling activating labour market policies as country-level predictors of individual optimal matchings. They included 28 industrial economies in their sample and found that employment protection legislation is positively associated with a higher share of optimal skill matching. The explanation offered to this result was that employment protection legislation makes re-allocation costlier for employers, and that they react to this regulatory environment by carefully recruiting workers with skills that match their job requirements and by upskilling their own employees through investments in training. The authors also found an association between stricter activating labour market policy and higher share of skill mismatch. Although the pattern is not so robust in this case, it suggests potential waste of talent and economic inefficiencies resulting from activation policies that remove employment benefits and force workers to accept jobs below their qualification.

Finally, Choi, Guio, and Escardibul (2019) made a very relevant methodological contribution to comparative studies on labour market mismatch. They assessed the sensitivity of country rankings of labour market mismatches and their effects on wages through the comparison of 18 different measures of qualification and skill mismatch across 20 countries participating in the first two rounds of PIAAC. They reported that the variation across countries in both the incidence and effects of overeducation and overskilling depends largely on the specific skills focused upon and the definitions employed. While it is true that every definition and measure probably accounts for different dimensions of the phenomenon, the results are too volatile to be able to identify a consistent ranking of countries. In their opinion, this lack of consistent patterns in the measures hinders the possibility of grouping countries by their level of labour market mismatches or analysing labour market mismatches by country groups without introducing a discretionary element. A direct implication of their work for comparative education research and policymaking is to avoid making policy recommendations based on rankings of labour mismatch emerging from PIAAC. Sometimes the most straightforward contribution of research to education policy debates is to admit that there is no rock-solid evidence to offer on a particular matter.

Implications for research and policy

The main contribution of this special issue is to show the potential but also the limitations of PIAAC for comparative education research and policy. The articles grouped in this volume have shown that comparative research can benefit substantially from the analysis of PIAAC data, particularly to enhance our understanding of inequalities in adult skills from an international perspective. With PIAAC we can capture the relationship between education and training systems characteristics, not just schools, and the level of skill inequalities among different social groups in different stages of their life cycle. Also, it makes it possible to investigate how new forms of social inequality, like labour market mismatches, affect different populations differentially and to question the policy rhetoric that constantly blames the education system for social problems that have been created by the dominant neoliberal economic model.

The authors contributing to this special issue have largely pointed out and discussed the methodological limitations of PIAAC for many of the burning research and policy questions around adult skill formation that we would like to be able to answer. Disseminating the quantitative comparative work presented in this special issue should help us to fight against the simplistic and mistaken association between quantitative data and ‘hard evidence’. The articles in this special issue have made it explicit that there are limitations of international large-scale assessments like PIAAC in directly extracting the so-called best practices or policy recommendations. International organisations and local policy actors should take note of these limitations, and they should complement the insights emerging from these quantitative comparisons with other sources of data and methodological approaches (e.g. qualitative) when they engage in education policy making and debates.