Education regime and creativity: the Eastern Confucian and the Western Enlightenment types of learning in the PISA test

Abstract

Creativity is the driving force of technological innovation and sustainable economic growth in the knowledge society. A central question, therefore, is how education helps to enhance creativity. As East Asian countries occupy the top of performance tests such as the OECD’s Programme for International Student Assessment (PISA), it is most important to understand the type of creativity they stand for. For doing so, we first investigate the revival of Confucianism and its idea of creativity in these countries in comparison with the Western Enlightenment idea of creativity. Secondly, we scrutinize how far these two types of creativity are represented in PISA’s understanding of problem solving which is assumed to require creativity from students. Thirdly, we conduct multilevel regression analyses with data from the 2009 PISA test to find out which learning strategies help to achieve in the PISA test and to close achievement gaps based on socioeconomic family background and how they relate to the two types of creativity. Two countries representing the Confucian tradition, the Republic of Korea and Singapore, and two countries representing the Western tradition, the United States and Canada, serve as test cases. As a culturally diverse world needs both types of creativity, the fundamental question is whether PISA helps them to flourish both.

Keywords:

• Confucianism
• Chinese learner
• creativity
• Programme for International Student Assessment
• learning strategies

Public Interest Statement

East Asia has achieved to serve as center of economic growth worldwide. China is going to establish itself as leading economic power of the 21st century. At the same time, East Asian school systems excel at the top of international comparative studies on educational performance. The most influential of these studies is the Programme for International Student Assessment (PISA) conducted every three years by the OECD, 2020. Economists argue that worldwide best education is the basis for the extraordinary economic performance of East Asia. In this study, addressing the revival of Confucianism in East Asia, we demonstrate that PISA favors more the Eastern Confucian type of learning and creativity conducive to incremental innovation as against the Western Enlightenment type of learning and creativity, conducive to radical innovation. We recommend that comprehensive knowledge development needs both types of learning in a complementary relationship.

1. Introduction: Culture, education regime, learning and creativity

According to many experts, a top-performing school system is of central importance for the rise of East Asia—always including Singapore in Southeast Asia in the following—to the most prosperous economic region in the world (Barber & Mourshed, 2007; Mourshed et al., 2010; Schleicher, 2018; Tucker, 2011). For education economists, East Asia is the striking proof that good education pays off in hard cash (Hanushek & Woessmann, 2015). It is precisely this connection between education and economic advancement that makes East Asia a challenge to the West (Tan, 2012a; You, 2019). It is also widely claimed that this best performance of East Asian school systems includes that higher performance is not only shown at the top but across all social classes so that family background is less important for educational achievement than in most other countries in the world (Schleicher, 2014; cf., however, for more nuanced analyses Teng et al., 2019; Kim, 2019). Does East Asia’s education regime tell the world how to overcome barriers to educational achievement as well as the continued reproduction of social inequality? Is the so-called Chinese learner a success model for all over the world (cf., Li, 2017; Morrison, 2006; Sellar & Lingard, 2013; Watkins & Biggs, 1996)? Apparently, learning in East Asia is superior to learning in other regions of the world, both in terms of quality and equity (cf., Boman, 2022; Tan, 2012a). Therefore, clarification is needed about the specific features of learning in this region of the world (cf., Kennedy & Lee, 2007). As there is broad consensus that creativity is the crucial source of innovation and innovation the key to sustainable economic growth and wellbeing of everybody in the contemporary knowledge society, we have to scrutinize how learning in East Asia stands with regard to creativity in comparison to the West. This is what we do in this study.

The three questions we want to answer are (1) how the Eastern and Western cultures conceive of learning and creativity, (2) how they are represented in tasks of creative problem solving in the PISA test, and (3) how they help to achieve in the PISA test, both in terms of quality and equity, that is in terms of test performance and reducing achievement gaps based on socioeconomic family background. In doing so, we join Komatsu and Rappleye’s (2017a, 2017b, 2021) plea for “rearticulating PISA” by way of using PISA for reflecting on PISA and its impact on our understanding of creativity in education.

We start with the assumption that learning always takes place in a cultural and institutional setting and applies strategies which are embedded in this setting. They are legitimized by cultural values and set into practice in the institutional context of school systems and schools. An educational regime is a specific arrangement of cultural values, institutions, and institutionalized learning strategies. In this study, we focus on two educational regimes with different cultural roots, institutional practices and learning strategies. These are a Western education regime rooted in the Enlightenment and an Eastern education regime rooted in Confucianism. For reasons of space, we cannot address Hinduism and Buddhism as two other East Asian religions which exert an influence on learning. For highlighting the features of the two chosen regimes in the first step of this study, we start with Max Weber’s (1951) classical comparison of Puritanism and Confucianism in his comparative sociology of religions, and we discuss the current state of research on this subject matter. We spell out the learning strategies embedded in these two cultures and institutionalized school and examination systems.

In the second step, we ask how these learning strategies are reflected in contemporary international educational benchmarking as it is represented by studies like the OECD’s Programme for International Student Assessment (PISA) as it started in 2000 and is conducted every three years (OECD, 2020). We focus on the meaning of creativity in problem solving in the PISA test. The question is how close the meaning of problem solving in the PISA test is to the Eastern Confucian or Western Enlightenment understanding of creativity. Furthermore, we look at the Global Creativity Index of 2015 to identify how the position of countries in the PISA science ranking of 2015 relates to the position in this index. In the third step, we link our findings about learning in East Asia and the West to the school disciplinary climate and to learning strategies as surveyed in the 2009 PISA test to build a bridge to a statistical analysis based on PISA data from this year.

Learning strategies have not been focused to the same extent in other test waves. Therefore, we chose the 2009 PISA test. As learning strategies are deeply embedded in cultural traditions, they do not change for a long time. And the PISA test has not changed its approach of a highly standardized assessment. Therefore, our results should hold not only for 2009 but also for the more recent test waves and for the foreseeable future. We test with a multilevel regression analysis how far learning strategies, which are more or less close to the preferred mode of learning in East Asia or the West, explain the variance of PISA test scores in reading. Furthermore, we test how far learning strategies, which we interpret as more or less close to East Asian or Western learning, help to reduce the impact of socioeconomic family background on PISA performance. Thus, we investigate how far learning strategies advance both, quality and equity.

For comparison, we include two East/Southeast Asian and two Western countries in the analysis, the Republic of Korea respectively South Korea and Singapore, on the one hand, Canada and the United States, on the other hand. The United States stands for the very liberal idea of Western culture, Canada for the extension of this idea to multiculturalism as a paradigm of tolerance and mutual recognition. In the Republic of Korea, the Confucian method of filling administrative positions through a comprehensive examination system to which the entire educational process is directed was introduced in 958 A.D. To this day, this system is set on self-improvement and discipline, on the integration of the individual into the community. This includes a strict disciplinary regime in school, represented by the absolute authority of the teacher over the students. In modern times, the model of status assignment has been extended beyond administrative positions to the entire employment system through a strict examination regime (Kwon et al., 2017; Leighanne, 2019). Singapore is a culturally diverse city state assembling people with a Chinese, Malay, Indian or other background. About three quarters of the people identify as Chinese, about 13 to 14% as Malay, about 9 to 10% as Indian, about 3 to 4% as others (Tan, 2012b, p. 449). With most people of Chinese origin, it is no surprise that the Confucian heritage is visible in Singaporean everyday life, in family, economy, politics and civil society and certainly in the idea of education and the organization of schools. In 1988, the then Deputy Prime Minister Goh Chok Tong emphasized “community and society before the self, family as the basic unit of society, community support and respect for the individual, consensus, not conflict and racial and religious harmony” as common values shared by all Singaporeans (Tan, 2012b, pp. 450–451). These values have been understood as basically Communitarian and Confucian. However, as argued by Tan (2012b) from her interpretation of the Confucian philosophy, there is an overemphasis on community and too little respect for individual rights. Anyway, we can say that schools in Singapore are bearers of a Confucian heritage (cf., Ortmann & Thompson, 2016; Sim & Chow, 2019).

2. The Western Enlightenment and Eastern Confucian modes of learning and creativity

For understanding East Asian education, it is indispensable to go back to Confucius. The fact that Confucianism in China, after half a century of banishment from science, education, politics and business, has regained enormously in importance since the 1980s is undisputed in the research literature. The introduction to an anthology on this revival of Confucianism notes:

“We all know that China’s economy has grown rapidly since the early 1980ʹs … Nevertheless, we often forget that, concomitant with this ‘economic miracle,’ a ‘cultural miracle’ has unfolded. Condemned as a relic of feudalism for more than half a century, Confucianism enjoyed a robust revival during the 1980s and 1990s, as post-Mao China was gradually integrated into the global economy.” (Hon, 2017, p. xi)

Since for Confucianism the right education is the cornerstone of the right conduct of life and of society as a whole, any attempt to understand and explain the rise of East Asia—first Japan, then South Korea, Singapore and China—to the leading economic region of the world in the 21st century must begin with a fundamental analysis of this very specific educational regime. From the point of view of historical studies, Max Weber’s (1951, pp. 226–249) contrasting of Confucianism and Puritanism in his comparative sociological studies on world religions may be lacking historical accuracy at one or the other point. However, his approach does not aim at historical details but at constructing ideal types which help to highlight and understand special cultural and structural features in their uniqueness and their consequences for society, independent of their appearance in concrete historical situations. And looking at contemporary research on the revived Confucian roots of education in East Asia, tells us that Weber can still serve as a most valuable guide for understanding the special Confucian type of rationalism in comparison to the Western type of rationalism as it is rooted in Puritanism (Weber, 2002).

According to Weber, Confucianism essentially produced an attitude of adjustment to the world, while Puritanism created an attitude of mastery of the world (cf., Münch, 2001, pp. 15–33). As Weber (1951) notes, for Confucianism it is important to keep the existing world in harmony, for Puritanism it is about changing an inherently imperfect world according to ethical standards:

“In this is vested the basic difference between the two kinds of rationalism: Confucian rationalism meant rational adjustment to the world; Puritan rationalism meant rational mastery of the world.” (p. 248)

Weber (1951, pp. 227–230) sees in the belief in a Creator God, whose perfection stands in stark contrast to the facticity of the imperfect world, on the one hand, and the absence of such a Creator God, on the other hand, the essential reason for this difference between Puritanism and Confucianism. Puritanism is a religion of redemption with a bourgeois class of adherents that creates a modern order against the traditional order. Confucianism is a cultural religion with a literary civil service class as a carrier stratum that administers an existing order and keeps it in balance. We can also observe this difference in secularized modernity and relate it to the educational process. According to the understanding of the Western Enlightenment, the educational process aims at the mature individual who is capable of thinking beyond the given and changing it according to the standards of reason (Münch, 1986, pp. 172–178). According to the understanding of Confucianism, the educational process consists in the self-cultivation of the individual in order to enable him/her to fit into the existing order and thus to preserve the harmony of the world order (Lai, 2006).

The old Chinese empire was an administrative state dominated by so-called literary officials. The officials had to go through a special educational process according to the writings of Confucius in order to meet the requirements for filling an administrative post. What was already special at that time, was the extensive examination system—reaching from county exams via provincial and metropolitan exams to the palace exams—to which the entire learning process was tailored (Elman, 2013). The system was meritocratic in that sense that every man (but not woman) could try the exams. However, only wealthy families could afford their sons spending years of learning for the exams. Candidates prepared for the exams by exercising sample solutions from previous exams. This was learning in the examination or control mode (Suen & Yu, 2006; Elman, 2013, pp. 33–34). The repetitive and reproductive character of the educational process also included a pedagogy geared to the strictest discipline (Weber, 1951, p. 130). And the civil servant had to continuously prove himself worthy of his position. Weber (1951, p. 129) described this form of “lifelong learning” as “lifelong pennalism”. The performance of the official was recorded meticulously and made public, a precursor of today’s emphasis on comprehensive accountability in New Public Management (Weber, 1951, p. 133).

The educational process consisted in learning literary writings that dealt with questions of the good life, but not with specialist administrative issues. The incumbents of the administrative offices were not supposed to be technical specialists, but educated people, so that they could serve as role models for the good life. The scholar was therefore at the top of the status hierarchy, well above the merchants, craftsmen and farmers. This was because it was the scholars themselves who, based on their leading position, determined which way of life should enjoy the highest respect. It was the view that seemed “natural” to the literary official to imagine the world as a well-ordered whole into which the individual must fit so that it does not fall into imbalance. Disturbances of order were to be avoided as far as possible. Nobody wanted to be the cause of a disturbance. Consequently, great importance was attached to saving face in all situations (Weber, 1951, p. 237). Questioning the knowledge order and the social order built upon it was unthinkable within this world view. Learning as controlled preparation for exams using material from previous exams implied that the Confucian view of the world shaped thought and action without reflecting on it. Learning meant to reproduce the existing knowledge, but not to criticize it in order to renew it. The fact that the literati went through this socialization process made them well-functioning officials in the old Chinese administrative state. And because imperial China was an administrative state, the civil servants held the leading position in society. In their position, they made the administrative view the dominant view of the world (Weber, 1951, pp. 107–141).

The learning process within the Confucian examination system was in the first stage concentrated on pictorial writing (Weber, 1951, pp. 123–124). The reproduction of given things is much more and longer in the foreground here than in the practice of the Latin characters. The meaningful component of writing, the understanding of meaning and the use of writing to formulate contexts of meaning and even more so to formulate one’s own thoughts, remains more and longer in the background than when learning the Latin script. This difference is further accentuated when one compares the teacher–student relationship in the Confucian examination system on the one hand and in the Hellenic academy on the other. In Confucianism, the piety of the student toward the teacher is the decisive basic structure, in the Hellenic academy it is the dialogue. The latter is an open process that drives both, teacher and student, not simply to reproduce given knowledge, but to take it as a starting point for developing new knowledge.

Against this contrasting of the reproduction of given knowledge on the one hand and the acquisition and development of knowledge in thesis and antithesis on the other, one could refer to the Confucian analects as answers of the master to questions of the pupils and see in them a form of dialectic (Weber, 1951, p. 124). The Analects are a collection of conversations between the master and his students which was compiled after Confucius’ death (473 AD) by his students. In the Socratic dialogue, however, the teacher invites the student to critically examine the ideas presented and to develop his own thoughts, while in the Confucian conversation the master teaches the student to respectfully acquire the given knowledge in order to refine his behavior (Gorry, 2011; Lai, 2006; Tweed & Lehmann, 2002). According to the current state of research, Weber rightly identified traditionalism rooted in the absence of dialogue and the associated hostility to renewal as a characteristic feature of Confucianism. But Daoism also had in common with Confucianism the endeavor to maintain an order and the knowledge of it, not to find new knowledge and create a new order (Weber, 1951, p. 205).

However, one could oppose that the imperial examination system was abolished in 1905 and since then, with the interruption of the Cultural Revolution (1966–1976), the Chinese educational system has been opened to Western influences (Zha, 2011), increasingly since 1980, so that it has no significance for the present situation. However, this is not the case. Essential features of this system are still clearly visible in the current predominant role of examinations in the Chinese education system as well as in the other East Asian countries with a Confucian tradition. The National College Entrance Exam (NCEE), which was introduced in 1953 and has since been reformed several times and adapted to Western standards, plays a central role similar to the imperial examination system. However, orientation to Western standards does not exclude that typical features of the imperial examination system remain effective in a new form (Niu, 2012, p. 83). Despite all efforts to modernize, examination as dominant part of education continues unchanged and is deeply rooted in the tradition of Confucianism (Zhao, 2014, pp. 132, 159–160).

It is by no means the case, however, that there is no place for creativity in the tradition of Confucianism and Daoism. Against this view already speaks that China was superior to the West in technological inventiveness—such as paper, letterpress printing, gunpowder, magnetic compass, porcelain, mechanical clockwork, iron foundry or stirrups—until the 15^th century (cf., Needham, 1969/2005, pp. 154–217; Münch, 1984, pp. 212–216; Staats, 2011). It is even argued that the characterization of Confucianism and Daoism as hostile to creativity is merely a Western prejudice and that these teachings simply stand for a different kind of creativity (Tsai, 2013). Therefore, we should not simply oppose Western creativity to Eastern “conformism”. The latter would mean—as is indeed Weber’s above-cited position—not to question established norms and knowledge but rather following them strictly. This would be inadequate for characterizing the Eastern culture. We should understand the difference between East and West as a difference in the type of creativity as rooted in these two cultures instead. Western creativity is anchored in the Judeo-Christian tradition, in the Enlightenment and the bourgeois lifestyle, and finally in capitalist entrepreneurship, science and art. Eastern creativity has its roots in the teachings of Confucianism, Daoism, Hinduism and Buddhism of which our focus is on Confucianism because it is the most relevant teaching for the renewed rise of the East to the top in terms of science, technology and economy (cf., Gorry, 2011; Tweed & Lehmann, 2002).

In the West, the Enlightenment simply transferred the Judeo-Christian idea of the Creator God to the human individual, who is said to have the ability to break with tradition and create something completely new, ultimately to question an existing order of knowledge and rule and to overthrow it in a revolutionary act. This is already present in Jesus’ Sermon on the Mount: “It is written, but I say to you.” Creativity in the Western sense is exactly what Joseph Schumpeter (1943/2013) identified as the central characteristic of the entrepreneur and the source of capitalist economic dynamism: creative destruction. Creativity according to the Confucian teaching, however, consists of learning the teachings of the given tradition and building on it in such a way that it can continue to exist in the future. In Daoism, creativity means to fit into the Dao through meditation in such a way that the human being and nature form a perfect whole. Both conceptions of creativity demand from the human being to work on himself or herself on the way to self-improvement, which is neither an end in itself nor an idealization of the individual but has the sole aim of creating a perfect world order (cf., Lai, 2006; Niu, 2012). An analysis by Charlene Tan (2016) of the analects (conversations) of Confucius summarizes the understanding of creativity contained therein as follows:

“Confucius demonstrated and advocated a moral and social conception of creativity that emphasizes evolutionary rather than revolutionary changes. Confucius supported incremental changes that were built upon the wisdom of the past, and posited the need for everyone to engage in moral self-cultivation within a community.” (p. 84)

Tan (2016, p. 84) admits that she talks about the idea of creativity rooted in the teaching of Confucius from which the practice of teaching and examining may deviate more or less. She understands her interpretation of Confucian creativity as, on the one hand, a demarcation from the Western type of creativity and, on the other hand, as a philosophical guideline for educational reform in East Asia. Along her Confucian idea of creativity, she also shows that there is a special Confucian kind of critical thinking to be found in the analects. It is a kind of critical thinking which guides practical action as incremental advancement of practice in social harmony which also includes steps of extending the framework of the Dao (Tan, 2017).

Of course, the fact that Confucianism was part of an imperial system of rule, for which the preservation of this system was a top priority, played a decisive role in the development of this understanding of creativity. Crucial for the kind of creativity awarded is not the system of ideas alone, but its interweaving with the given institutions of a society (see, Ho & Ho, 2008). Creativity in the radical sense needs an environment which allows for exploration and experimentation. This is, for example, missed with teachers who understand creativity only in cognitive terms as a study on South Korean science teachers shows (Seo et al., 2005; see, Weber, 1951, pp. 107–141).

The great success of East Asian school systems in international performance tests, above all PISA, also raises the question of how far not only reproductive, but also creative abilities are featured. In the research literature, it is seen as paradoxical that Chinese students, like their peers in other East Asian countries, achieve top scores in international performance tests, even though they seem to learn under less favorable conditions and use less effective learning strategies in an examination system that is focused on reproducing canonized knowledge (Watkins & Biggs, 1996, 2001). Keith Morrison (2006) summarizes the paradox of the successful “Chinese learner” as follows:

“The paradox of the Chinese learner is intriguing for Westerners and Chinese alike. In a nutshell, it questions why, despite using rote learning, memorization, repetition, constant testing, large classes, competitive motivation, examination orientation, authoritarian and didactic teaching and learning methods, passivity and compliance - in short the presence of putative negative features of teaching and learning, together with a supposed absence of many positive features of effective teaching and learning, Chinese students consistently achieve more highly than their Western counterparts, who are highly adaptive, prefer high-level meaning-based learning strategies, and engage in deep learning.” (p. 3)

Morrison discusses ten possible explanations for this paradox. Explanation no. 1 considers the characterization of the Chinese learning culture as Western prejudice (see, Li, 2017). Explanation no. 2 simply says that the practices discredited as “putative” are effective, as proven by international performance tests. Explanation no. 3 claims that East Asian students are inherently more intelligent than all other students. Explanation no. 4 sees the main reason for the leading position of East Asian countries in international tests as being the greater diligence of their students. Explanation no. 5 derives from the high effort mentioned in explanation no. 4 that under this condition even less good teaching and learning methods lead to very good test scores because they are compensated by diligence. Explanation no. 6 emphasizes that school and education in the East Asian countries have a high value in the Confucian tradition and form an essential part of the culture. Explanation no. 7 makes the affinity between the typical tasks in international performance tests and the typical tasks in the usual in-country tests responsible for the top results of the East Asian countries. Explanation no. 8 considers East Asian students to be particularly smart “test takers” who, above all, show test intelligence which enables them to achieve top scores in all tests, regardless of their content. Explanation no. 9 draws attention to the so-called Hawthorne Effect, an increase in performance solely due to the high level of attention paid to the test. Explanation no. 10 ultimately recognizes the East Asian culture as the crucial basis for the top results of the countries belonging to this culture.

It is not really possible to distinguish clearly between these ten explanations. They rather revolve around a syndrome in which they meet each other. It is the culture of Confucianism, with its special focus on the piety of the younger with regard to the older, on education as a vehicle for acquiring status, and on a comprehensive examination system with a high degree of standardization that must be passed in order to attain status. The secondary virtues of this syndrome include great diligence, learning in control mode and the acquisition of test intelligence. And it is in the nature of things that, where such a system of testing is practiced, there is a high degree of standardization of test items for reasons of comparability of performance. International performance tests have to proceed in exactly the same way in order to achieve reliable and valid results. This inevitably ends in a high degree of congruence between the national test procedures in East Asia and international performance tests. The international student performance comparisons show therefore an elective affinity with the historically deeply rooted Confucian educational regime and examination system in East Asia (cf., Suen & Yu, 2006). This is the common core of the ten explanations identified by Morrison for the leading position of East Asian countries in international student performance tests. And there is every reason to assume that this is also the basis of the top performance of East Asian students going to school in Western countries with very different regimes of school governance (Feniger & Lefstein, 2014; Hsin & Xie, 2014; Jerrim, 2015).

3. Creativity in problem solving in the PISA test

Regarding stimulation of creativity, there are obviously strong indications that according to the Confucian frame the East Asian educational regime does not favor the kind of radicalism that has so far been at least the playbook of Western education rooted in the European Enlightenment. This judgment could be countered by the fact that it is precisely the countries representing this regime which score highest at issues where problem solving is required. Singapore, the Republic of Korea, Japan, Macao, Hong Kong, Shanghai and Taipei (Taiwan) are also on top of this PISA ranking of 2012 with scores between 562 and 534 (OECD, 2014, Figure V.2.3). It could be argued that problem solving requires creative skills, which would suggest that the educational regime of East Asian countries is obviously not exclusively focused on reproductive learning. To validate this statement, we need to take a closer look at the problem-solving tasks in the PISA study. The OECD’s (2014) report on PISA 2012 defines problem-solving competence as follows:

“ … an individual’s capacity to engage in cognitive processing to understand and resolve problem situations where a method of solution is not immediately obvious. It includes the willingness to engage with such situations in order to achieve one’s potential as a constructive and reflective citizen.” (p. 30)

It is emphasized that problem solving is a non-routine task and requires dealing with an open situation and being inventive enough to be able to deal with it:

“Although some students may be familiar with the context or the goal of a problem situation that refers to a plausible real-world scenario, the particular problem faced is novel and the ways of achieving the goal are not immediately obvious.” (p. 30)

The study argues that the future labor market will increasingly demand problem-solving skills, and that the employment rate for this type of work will increase significantly, while that for routine activities will decrease significantly. However, it is not only about competent employees. According to the OECD definition, problem-solving skills are also a characteristic feature of “constructive and reflective citizens”. This clearly brings “problem-solving competence” close to “creativity”. However, it would go too far to equate the two terms in their meaning. Solving a problem certainly requires a certain amount of creativity. One must be able to recognize the problem and find a solution. The finding part of the task is the creative process.

However, if we recall the distinction between paradigm shift and puzzle solving within the framework of a dominant paradigm in Thomas Kuhn’s The Structure of Scientific Revolutions (Kuhn, 1962/2012), then puzzle solving—which is a form of problem solving—is a practice of normal science. This means that while problem solving is a process of discovery and requires creativity, it does so only to a modest degree in comparison to the paradigm shift. Problem solving does not call existing knowledge into question, but rather mobilizes the available explicit and implicit knowledge. It is part of an everyday practice in which problems are solved by looking for similarities of the new situation with previously experienced situations. The unfamiliar is brought into relation with the familiar and thus becomes understandable and manageable.

If one looks at the tasks that are classified as problem solving in the OECD study, it can be seen that this type of creativity proceeds on existing pathways. These are tasks from everyday life for which it is not school knowledge that needs to be activated but everyday experience. For example, an MP3 player, air conditioning or a ticket machine must be operated (OECD, 2014, pp. 35–40). Another task is to use a network map of traffic routes in a city to determine in which suburb three friends living in different suburbs can meet if none of the three should be on the move for more than 15 minutes (OECD, 2014, pp. 41–44). None of these tasks requires the questioning and criticism of existing knowledge, the exploration of the background of this knowledge and the reflective handling of it.

Rather, the focus is on the activation of existing explicit and implicit everyday knowledge. Finding a solution to a problem can therefore only be described as creative in a limited sense, but not in the radical sense, for example, when Joseph Schumpeter (1943/2013) speaks of the creative destruction by the entrepreneur in the process of economic innovation, or Thomas Kuhn (1962/2012) of the paradigm shift in science, or when Paul Feyerabend (1975/1993) makes a plea against method in order to increase the chances of knowledge progress. In these cases, old practice and old knowledge must be questioned and overcome to be replaced by new practice and new knowledge. In the special awarding of this very creative force, Western culture still differs from the culture of East Asia (cf., Münch, 2001). The top position of the East Asian countries in the PISA ranking of problem-solving competence therefore does not prove that their educational regime is also a model for the stimulation of creativity in the radical sense. The most obvious explanation for the fact that the East Asian countries, as in reading, mathematics and science, are also on top of the PISA league table when it comes to problem solving is, therefore, that they are more accustomed to taking tests, attach greater importance to the test and simply make more effort, which is encouraged by singing the national anthem before the test.

The fact that the one-sided focus on the reproduction of what has been learned and its application to solve problems hinders creativity and innovation in the radical sense has, however, been lamented since the 1990s (Zhao, 2014). Kai-ming Cheng (2011) notes, for example:

“Teaching and learning, especially in secondary schools, still follow the examination syllabi, and school activities are oriented toward exam preparation. Subjects such as music and art, and in some cases even physical education, are often not offered because the exams do not cover those subjects. Students work long hours every day and into the weekends, often taking additional exam-prep classes. As noted earlier, private tutorials, most of them profit-making, are widespread, almost a household necessity.” (p. 30)

In response to criticism referring to too little emphasis on creative learning at school, reforms have been undertaken and are still in process in various East Asian countries (cf., Kennedy & Lee, 2007, chapters 7, 8). The government of Singapore, for example, has geared its teaching guidelines toward promoting creativity and innovation. A number of catchwords express this new objective: the “thinking school”, “the IT master plan”, “national education”, “content reduction”, “project work”, “innovation and enterprise”, “teach less, learn more” (Deng & Gopinathan, 2016, pp. 457–458; cf., Kennedy, 2013). The aim is to consolidate Singapore’s position of a so-called high-performing education system (HPES). However, extensive studies of school practice between 2004 and 2010 by Deng and Gopinathan (2016) reveal that frontal teaching and preparation for performance tests still dominate learning at school and that there is no sign of lessons aimed at stimulating creativity and critical faculties in the radical sense:

“At the policy and rhetorical level, Singapore fits well the description of a high performing system in the HPES literature - ‘unleashing’ the power of ICT for learning … and promoting a ‘thinking pedagogy’ for ‘deep learning’ … However, as noted earlier, at the classroom level, meaningful use of ICT, teaching for deep understanding, and opportunities for critical and creative thinking are still very limited and uncommon.”. (p. 461)

Deng and Gopinathan conclude from this finding that the PISA test must have a high affinity to the learning regime that still prevails in Singapore, as in the other East Asian countries, and which particularly rewards exam-oriented learning, but not critical faculties and creativity (pp. 463–464; cf., Koh et al., 2012; Lim, 2010). For Deng and Gopinathan, the high performance of these countries in tasks that require problem solving, as put forward by the OECD, is only proof that PISA tests standardized problem solving, but not creative thinking in the radical sense (p. 464; cf., Zhao, 2014). Further evidence confirming that the PISA test is at odds with radical creativity and critical thinking is provided by a study which proves in a regression analysis that conservatism (a preference for traditionalism, conformity and security) is the best predictor of higher PISA test scores in all three tested disciplines (Benoliel & Berkovich, 2018).

Ma et al. (2013) have proven that learning in control mode pays off in the PISA test. We understand this learning strategy as representative of Confucian learning as scrutinized so far. In their search for explanatory factors for the PISA success in 2009 of six East Asian school systems—Hong Kong (China), Japan, the Republic of Korea, Shanghai (China), Singapore and Taipei (Taiwan)—using a two-level hierarchical linear model, the authors identified two factors that proved to be the most effective in all six countries for reading as well as for mathematics and science: school discipline as a school characteristic and the application of the control strategy along with using meta-cognitive skills in learning as a student characteristic. Both features characterize an educational regime that places the greatest emphasis on disciplined and controlled learning. School discipline means that the teachers have strict control over what is taught, while the application of the control strategy means learning in examination mode, i.e., learners continually check their own learning progress. This is also closer to the exam situation than mere memorizing or elaborating, both of which have proven to be less effective. It is clear that these two factors reward the reproduction of a knowledge system more than its questioning. Learning in the control mode may be close to what is characterized as combination of motivation to learn, memorization and seeking understanding in deep instead of surface learning as reason for the extraordinary performance of East Asian students (Kember, 2016). Further evidence for achieving in the PISA test through learning controlled by teachers is provided by the fact that teacher-directed but not enquiry-based learning is correlated with higher PISA science scores (OECD, 2016, Figures. II.2.13, II.2.14, II.2.19). As Komatsu and Rappleye (2017b) show, these results prove that the PISA test favors a learning strategy which contradicts the promotion of student-centered learning (see also, Sjøberg, 2018). According to their wording this is teacher-centered learning II as against student-centered learning I (see also, Seo et al., 2005). However, as Jerrim et al. (2020) show for England, inquiry-based teaching, is not effective in generating higher examination scores in general, beyond PISA. This finding reveals a tension between the two, and PISA seems to favor the examination mode of learning.

Further evidence for an insufficient representation of creativity in the more radical sense in PISA is provided by a look at the Global Creativity Index. Success in the PISA test is at odds with a leading position in the Global Creativity Index (Florida et al., 2015; Tienken, 2008). Western countries in the midfield of the PISA test occupy the top positions in the Global Creativity Index, while the East Asian leaders in the PISA league table are only found in a midfield position of this index with the single exception of the special case of the wealthy city state of Singapore . We may object that this effect is due to a typically Western understanding of creativity in the Global Creativity Index. Nevertheless, this type of creativity stands for centuries of progress in science and technology so that a turn away from it by adopting the Confucian learning regime would bring this development to a halt. This assessment of the Confucian learning regime does by no means deny that this regime has its own merits and has made its contribution to the remarkable catch-up technological and economic development of East Asia. However, there are good reasons to take this effect as an indicator of incremental innovation and catch-up economic development but not as an indicator of economic development based on radical innovation. As far as we are interested in radical innovation there must be enough space for explorative and experimental learning and critical argumentation, as paradigmatically explained by John Dewey (1897). And there is reason for concern that an undisputed worldwide rule of the PISA learning regime undermines the institutional basis of a creativity-supporting learning culture in the tradition of the Western Enlightenment (Grey & Morris, 2022; Tienken, 2017).

4. Learning strategies, educational achievement and closing of achievement gaps: A multilevel regression analysis

4.1. Theory

As we have learned, according to Weber and also the current state of research, the basic difference in the idea of learning and its link to creativity between Western Enlightenment and Eastern Confucianism can be spelled out in an ideal-typically pointed emphasis as follows: (1) Learning in the mode of Western Enlightenment means exploring new ways of thinking with the ability to deviate from existing knowledge to replace old thinking with new thinking in a dialogical relationship between student and teacher, preparing for mastering the world in a society of self-responsible individuals. Creativity here is destruction of existing knowledge and practices and their replacement by new ones. This furthers first of all radical innovation in a revolutionary process. (2) Learning in the mode of Confucianism means extending existing knowledge with the ability to adapt it to new situations and challenges to harmonize old thinking with new thinking in a respectful relationship of questions and answers between student and teacher, preparing for the appropriate embedding of the individual in society so that harmony is maintained. Creativity here is the evolutionary development of existing knowledge. This allows first of all for incremental innovation in an evolutionary process.

For Weber, these are two opposing types of rationalism in approaching the challenges of human life in society. It is established thinking that modernity first emerged in Europe based on the Western type of learning with a history that reaches from Hellenic philosophy to the Enlightenment that implies a dynamically changing world driven by the permanent replacement of old knowledge by new knowledge. This does, however, not preclude, that there has always been a counterpart to the revolutionary overturn of knowledge in the West itself. Revolutionary dynamics has always been in interaction with its other, that is incremental innovation in the sense of extending existing knowledge. With the rise of East Asia to the position of outstanding achievement in not only economic but also scientific and technological terms, evolutionary development of knowledge and incremental innovation have gained new recognition as another type of modernity. We see here in the context of learning, creativity, knowledge creation and innovation what Shmuel Eisenstadt (2002) has called multiple modernities. And it might be characteristic for the future that East and West complement each other with two types of modernity both needed for the development of knowledge.

Looking at international comparative tests of learning outcomes like PISA, the question is how the two types of learning and creativity outlined so far are represented in these tests. Our hypothesis is that the ranking of East Asian school systems rooted in the Confucian mode of learning on top of PISA says that PISA favors more the Eastern Confucian type of learning than the Western Enlightenment type. A first prove of this hypothesis is our finding in the previous section that problem solving in the PISA test is closer to adaptive learning than explorative learning.

A further proof is that positions in the PISA ranking are not directly reflected in the ranking of the Global Creativity Index. In order, the test this hypothesis further on we look at the disciplinary climate at school and the learning strategies of students as surveyed along with the PISA test. First, we assume that learning at school within the Confucian culture takes place in a disciplinary climate which is stronger than in countries representing the Western culture. The PISA survey asks principals about how far learning is hindered by teacher or student absenteeism and students about their assessment of the disciplinary climate in class. We take these variables as representing the disciplinary climate in class. As already mentioned, the PISA test of 2009 is the only one in which learning strategies were directly focused, which is why we take data from this test wave to examine the effect of disciplinary climate and learning strategies on the PISA score of students (OECD, 2010a, pp. 48-52; 2010b, 94-98). As the learning strategies are oriented to reading, the reading score serves as dependent variable.

For measuring the disciplinary climate on school level, we use the following variables from the principal survey:

Extent to which student learning is hindered by: Teacher absenteeism: not at all

Extent to which student learning is hindered by: Student absenteeism: not at all

From the student survey an index of the following items is taken for measuring the disciplinary climate in class as experienced by the students:

How often do these things happen in your <test language lessons>? Never or hardly ever, in some lessons, in most lessons, in all lessons:

1. Students don’t listen to what the teacher says

2. There is noise and disorder

3. The teacher has to wait a long time for the students to <quieten down>

4. Students cannot work well

5. Students don’t start working for a long time after the lesson begins

(OECD, 2008, Q36)

In the 2009 survey, three learning strategies were asked from students: (1) memorization, (2) control and (3) elaboration (OECD, 2010a, p. 48). These strategies are defined by the following item parameters:

Student learning strategies: four response categories varying from “almost never”, “sometimes”, “often” to “almost always”

Memorization strategies

“When I study, I try to memorize everything that is covered in the text

When I study, I try to memorize as many details as possible

When I study, I read the text so many times that I can recite it

When I study, I read the text over and over again”

Elaboration strategies

“When I study, I try to relate new information to prior knowledge acquired in other subjects

When I study, I figure out how the information might be useful outside school

When I study, I try to understand the material better by relating it to my own experiences

When I study, I figure out how the text information fits in with what happens in real life”

Control strategies

“When I study, I start by figuring out what exactly I need to learn

When I study, I check if I understand what I have read

When I study, I try to figure out which concepts I still haven’t really understood

When I study and I don’t understand something, I look for additional information to clarify this”

(OECD, 2012, Tables 16.12 to 16.14, p. 293).

A factor analysis reveals, however, that the items regarding learning strategies entailed in Question no. 27 of the student questionnaire are loading on four factors. In order to discriminate clearly between different learning strategies, we therefore include four strategies in our analysis. The first two items of what is called memorization strategy in the OECD report we also take as memorization strategy, the second two we call internalization strategy. Of the OECD’s elaboration strategy, the second, third and fourth items constitute the transfer/elaboration strategy while the first item is deleted in our analysis. The control strategy in our analysis consists of the second, third and fourth items of the OECD’s control strategy.

The items representing the memorization strategy and similarly the items of the internalization strategy unmistakably show students who prepare directly for a test in which they have to reproduce what is written in given texts. Control strategies are also directly oriented to preparing for a test. The students are focused on what exactly is required by the test, what of this required knowledge they already master and what they do not yet master and must concentrate on. This is a learning strategy which is typical for test takers, and which is favored by standardized tests. And as tests have to be all the more standardized the larger the number of students, schools and countries who have to be compared in their competences, this kind of test regime should reward control strategies of learning. The more often such standardized tests are conducted, the more learning takes place as direct preparation for the exam. It is learning in the examination mode complemented by teaching to the test.

In contrast, the transfer/elaboration strategy takes students away from direct exam preparation. It opens new avenues of extending knowledge but entails the danger of dealing with things that are not directly related to the test and may even hinder a straightforward test preparation. We assume that this learning strategy is not conducive to excel in standardized tests like PISA. Taking into account the outstanding role of examination and preparation for exams in the imperial Confucian tradition of recruitment to civil service as well as in its modern counterpart of allocating personnel to occupational positions we may postulate: It seems to be fair to speak of a kind of “elective affinity” between the regime of standardized testing as globally disseminated by PISA and the renaissance of Confucian learning in East Asia. In contrast, what is called “elaboration strategy” in the PISA survey of 2009 or transfer/elaboration strategy in our analysis looks more like learning according to the idea of exploring new fields whose success cannot be predicted exactly. Therefore, it is not a wise strategy of preparing for a standardized test. Here, we may speak of a conflicting relationship between PISA and the Western tradition of explorative learning as paradigmatically explained by John Dewey (1897).

If the hypothesis holds that PISA favors the adaptive Confucian type of learning, particularly control should be more effective in raising PISA scores than transfer/elaboration , not only in East Asian countries but also in Western countries. We therefore look at the effects of these learning strategies in two Eastern and two Western countries: South Korea and Singapore, the United States and Canada. Furthermore, explorative learning and radical innovation might be achieved more by elites but less by the broad mass of students, while adaptive learning and incremental innovation might be more inclusive and attainable for all. We should see this difference in the effect of individual student and average school socioeconomic status on PISA scores in comparing the two Eastern and two Western countries and in looking at the effects of different learning strategies. The question therefore is, how far the emphasis on adaptive learning can reduce the impact of student and school socioeconomic status on PISA scores. If there is an effect, focusing on adaptive learning strategies can help to reduce the impact of socioeconomic family background on learning outcomes everywhere, thus providing for quality and equity.

We carry out a multilevel regression analysis focused on the effect of disciplinary climate and learning strategies on educational performance in comparison to the socioeconomic status of students and schools. The question is whether the disciplinary climate in class and certain learning strategies do not only help to improve PISA outcomes but also to reduce performance gaps based on socioeconomic family background. We compare the United States, Canada, Singapore and South Korea to highlight the differences between highly innovative liberal and highly disciplined Confucian education regimes. The choice of the two Western and two Confucian countries is based on their difference in socioeconomic homogeneity measured by the Gini index of household incomes. Other things being equal, we assume that the impact of socioeconomic background on the PISA score of students should be higher in a country of higher socioeconomic inequality. Of the four countries chosen for our analysis, of the two Western countries, the United States features a much higher socioeconomic inequality than Canada, of the two Confucian countries, Singapore features a much higher socioeconomic inequality than South Korea. According to World Bank data respectively Singapore Department of Statistics data the Gini index of household incomes in 2009 was 40.6 for the United States, 33.8 for Canada, 47.8 for Singapore and 32.3 for South Korea (Singapore Department of Statistics, 2010; World Bank, 2022). For having these contrasts in social inequality in the analysis, we select these two Western and two Confucian countries. This is also the reason why we could not take China for representing the Confucian culture. China is not included as a whole country in the PISA test of 2009, but only with metropolitan areas, that is Hong Kong, Shanghai, and Macao which are not representative for China and for which the Gini index of China would not apply directly.

4.2. Data and methods

The multilevel regression models conducted in this step of our analysis comprise of two levels. The first features variables on student level, the second on school level. More precisely, we employ a random slope, random intercept multilevel linear regression model (Snijders & Bosker, 1994, pp. 120–124). We applied Snijders and Bosker’s (1994) R² to calculate the goodness-of-fit of our models. Z-standardization was applied to ensure comparability of effect sizes, and student weights, provided by the OECD, were included to compensate for missing observations on student level. For descriptive statistics and regression diagnostics see appendices A and B.

Our regression analysis comprises of seven models. We begin with variables on the student level, continue with adding variables on the school level, go on with controls for ESCS, disciplinary climate and learning strategies, and end with including interactions between the four learning strategies and the student ESCS in the full model. Model one addresses the students’ index of economic, social and cultural status (ESCS). The “ESCS is a composite score built by indicators of parental education, highest parental occupation, and home possessions including books” (OECD, 2017, pp. 339–340). The second model estimates the association between students’ learning strategies and the PISA reading score, namely memorization strategies, control strategies, transfer/elaboration strategies, and internalization strategies. The third model introduces variables regarding disciplinary climate and the mean student ESCS value per school. Disciplinary climate is constructed from items that measure whether students listen to the teacher or not, noisiness and disorderliness in class, the time it takes the students to be quiet and to participate in the lessons and the students’ working conditions (OECD, 2017, p. 314). At the same time, the average ESCS is a measure of social inequality between schools. Model four includes all variables and controls for the ESCS, and adds ESCS as random effect, meaning that the slope of ESCS could vary between schools.¹ For better interpretation, the random effects are depicted as slopes in appendix B. Model five includes the random effect of disciplinary climate in addition to ESCS, model six the random effect of the four learning strategies in addition to ESCS and disciplinary climate. Model seven is the full model and includes additionally the interaction of the four learning strategies with the ESCS as fixed effects and the six random effects introduced in models four to six. Results are provided in Tables 1–4.

Table 1. PISA reading score 2009, United States

Variables	Model 1	Model 2	Model 3	Model 4	Model 5	Model 6	Model 7
Intercept	504.02 ***	506.86 ***	496.04 ***	497.09 ***	498.35 ***	500.31 ***	499.65 ***
ESCS	23.39 ***		19.36 ***	16.45 ***	16.24 ***	16.09 ***	16.28 ***
Memorization		−3.44 *	−3.58 *	−3.41 *	−3.45 *	−5.68 *	−4.73
Control		35.28 ***	29.9 ***	29.36 ***	29.11 ***	27.04 ***	26.53 ***
Transfer		−14.78 ***	−14.58 ***	−14.85 ***	−15.12 ***	−11.89 ***	−12.62 ***
Internalization		−14.42 ***	−13.04 ***	−12.91 ***	−12.5 ***	−13.92 ***	−13.18 ***
Disciplinary climate			10.97 ***	10.99 ***	9.04 ***	9.44 ***	9.67 ***
Ave. School ESCS			28.94 ***	33.37 ***	34.54 ***	34.19 ***	35.73 ***
Learning is hindered by student absenteeism: Not at all			23.66	14.09	10.4	7.39	7.99
Learning is hindered by teacher absenteeism: Not at all			−3.53	−3.2	−5.03	−8	−8.35
Interaction: Memorization * ESCS							−4.92 **
Interaction: Control * ESCS							2.89
Interaction: Transfer * ESCS							2.67
Interaction: Internalization * ESCS							−2.38
Variance student level	6517.24	6233.61	5867.3	5687.94	5477.1	4813.51	4803.09
Variance school level	2242.62	2795.24	735.61	825.61	917.17	1081.64	1045.83
ESCS				241.68	246.12	263.43	252.79
Disciplinary climate					242.74	261.73	262.37
Memorization						368.35	336.74
Control						332.5	348.8
Transfer						277.33	265.15
Internalization						266.16	264.23
R2 student level	0.1	0.18	0.27	0.32	0.37	0.51	0.51
R2 school level	0.58	0.35	0.95	0.94	0.93	0.9	0.91
Deviance	39,281,249.49	39,138,001.6	38,902,769.11	38,832,100.53	38,742,647.43	38,449,061.96	38,439,154.39
Number of Students	5233	5233	5233	5233	5233	5233	5233
Number of Schools	165	165	165	165	165	165	165

* p < 0.05, ** p < 0.01, *** p < 0.001

Table 2. PISA reading score 2009, Canada

Variables	Model 1	Model 2	Model 3	Model 4	Model 5	Model 6	Model 7
Intercept	511.08 ***	510.52 ***	510.92 ***	510.21 ***	509.65 ***	507.44 ***	507.28 ***
ESCS	16.49 ***		13.22 ***	12.8 ***	12.85 ***	12.39 ***	12.37 ***
Memorization		0.37	0.1	−0.17	−0.67	−0.08	−0.03
Control		35.39 ***	32.72 ***	32.71 ***	32.93 ***	32.63 ***	32.65 ***
Transfer		−13.13 ***	−13.81 ***	−13.55 ***	−13.47 ***	−16.64 ***	−16.66 ***
Internalization		−12.45 ***	−12.03 ***	−11.74 ***	−11.8 ***	−11.11 ***	−11.04 ***
Disciplinary climate			4.98 ***	5.41 ***	5.21 ***	4.9 ***	4.83 ***
Ave. School ESCS			9.96 ***	10.91 ***	10.92 ***	10.36 ***	10.61 ***
Learning is hindered by student absenteeism: Not at all			−1.6	−0.72	1.46	5.93	6
Learning is hindered by teacher absenteeism: Not at all			6.35 *	6.44 *	5.98	6.93 *	6.59 *
Interaction: Memorization * ESCS							−1.89
Interaction: Control * ESCS							1.39
Interaction: Transfer * ESCS							2.81 **
Interaction: Internalization * ESCS							0.27
Variance student level	6250.7	5751.14	5595.72	5334.77	5049.24	4160.42	4154.66
Variance school level	1670.91	1655.71	1123.87	1146.66	1194.43	1272.32	1274.86
ESCS				329.93	348.81	404.87	399.49
Disciplinary climate					351.47	401.63	401.79
Memorization						450.93	454.22
Control						480.11	482.1
Transfer						412.34	407.36
Internalization						533.48	528.05
R2 student level	0.06	0.21	0.25	0.32	0.39	0.59	0.59
R2 school level	0.34	0.36	0.7	0.69	0.66	0.62	0.62
Deviance	4,180,836.28	4,151,053.56	4,140,099.18	4,127,883.95	4,113,555.46	4,065,791.49	4,065,273.96
Number of Students	23,207	23,207	23,207	23,207	23,207	23,207	23,207
Number of Schools	978	978	978	978	978	978	978

* p < 0.05, ** p < 0.01, *** p < 0.001

Table 3. PISA reading score 2009, South Korea

Variables	Model 1	Model 2	Model 3	Model 4	Model 5	Model 6	Model 7
Intercept	514.92 ***	516.51 ***	543.32 ***	541.92 ***	541.09 ***	521.58 ***	521.18 ***
ESCS	10.63 ***		6.86 ***	12.08 ***	12.43 ***	7.74 ***	7.21 ***
Memorization		−2.12 *	−2	−1.58	−1.52	−9.5	−9.54
Control		21.2 ***	20.48 ***	20.01 ***	20.02 ***	27.47 ***	27.29 ***
Transfer		−0.9	−1.49	−1.31	−1.29	1.89	2.01
Internalization		6.98 ***	6.77 ***	6.63 ***	6.42 ***	14.12 ***	14.47 ***
Disciplinary climate			2.54 **	2.47 **	−0.06	−0.29	−0.24
Ave. School ESCS			23.22 ***	23.53 ***	23.65 ***	23.43 ***	23.32 ***
Learning is hindered by student absenteeism: Not at all			28.44 **	26.45 **	26.82 **	9.43	8.85
Learning is hindered by teacher absenteeism: Not at all			−34.11 ***	−33.43 ***	−32.68 ***	−10.42	−9.64
Interaction: Memorization * ESCS							−3.77 *
Interaction: Control * ESCS							−0.16
Interaction: Transfer * ESCS							1.34
Interaction: Internalization * ESCS							−0.31
Variance student level	4003.99	3635.01	3596.96	3488.35	3392.65	2890.83	2885.49
Variance school level	2387.91	2240.15	1026.14	1040.68	1025.66	2401.8	2394.13
ESCS				180.56	189.06	137.96	141.14
Disciplinary climate					129.98	142.31	140.79
Memorization						1338.39	1326.24
Control						1085.17	1048.97
Transfer						466.39	460.35
Internalization						328.36	356.06
R2 student level	0.04	0.21	0.22	0.27	0.31	0.5	0.5
R2 school level	0.27	0.36	0.87	0.86	0.87	0.26	0.27
Deviance	7,028,862.53	6,968,080.2	6,958,517	6,944,714.46	6,932,657.52	6,863,737.69	6,862,480.49
Number of Students	4989	4989	4989	4989	4989	4989	4989
Number of Schools	157	157	157	157	157	157	157

* p < 0.05, ** p < 0.01, *** p < 0.001

Table 4. PISA reading score 2009, Singapore

Variables	Model 1	Model 2	Model 3	Model 4	Model 5	Model 6	Model 7
Intercept	520.09 ***	520.78 ***	516 ***	516.14 ***	514.53 ***	515.49 ***	515.69 ***
ESCS	22.57 ***		19.53 ***	19.42 ***	19.11 ***	19.22 ***	19.01 ***
Memorization		−4.43 ***	−4.32 ***	−4.35 ***	−4.43 ***	−3.41 **	−3.44 **
Control		27.47 ***	22.96 ***	23.24 ***	22.54 ***	23.36 ***	23.34 ***
Transfer		−11.63 ***	−11.46 ***	−11.27 ***	−10.8 ***	−10.86 ***	−10.82 ***
Internalization		−12.4 ***	−11.8 ***	−11.45 ***	−11.24 ***	−11.71 ***	−11.79 ***
Disciplinary climate			12 ***	12.09 ***	11.51 ***	11.54 ***	11.55 ***
Ave. School ESCS			26.11 ***	26.85 ***	27.07 ***	26.28 ***	26.94 ***
Learning is hindered by student absenteeism: Not at all			8.21	10.1	11.91	10.99	11.15
Learning is hindered by teacher absenteeism: Not at all			10.02	10.45	11.73	11.19	11.1
Interaction: Memorization * ESCS							−2.33
Interaction: Control * ESCS							−3.72 *
Interaction: Transfer * ESCS							0.86
Interaction: Internalization * ESCS							2.46
Variance student level	5637.23	5550.15	5103.39	4899.48	4706.92	4119.9	4112.62
Variance school level	2610.39	2782.38	1130.82	1293.59	1328.8	1331.74	1323.68
ESCS				270.59	270.19	270.55	264.78
Disciplinary climate					250.3	281.47	278.17
Memorization						221.57	218.35
Control						425.27	415.75
Transfer						275.84	274.91
Internalization						325.36	331.07
R2 student level	0.13	0.15	0.28	0.34	0.39	0.53	0.54
R2 school level	0.43	0.35	0.89	0.86	0.85	0.85	0.85
Deviance	596,142.09	595,349.2	590,846.22	589,209.11	587,610.83	582,517.43	582,418.91
Number of Students	5283	5283	5283	5283	5283	5283	5283
Number of Schools	171	171	171	171	171	171	171

* p < 0.05, ** p < 0.01, *** p < 0.001

4.3. Results

Beginning with model one, we see positive, highly significant (p < 0.001) and robust associations of the students’ ESCS with the PISA reading score in all four countries. The strongest association between student-level ESCS and the PISA reading score is given in the two countries with highest inequality of household incomes according to the Gini index, namely in the U.S. and in Singapore. (β = 23.39*** and β = 22.57***). That means, with one unit of standard deviation in the student-level ESCS the PISA reading score rises by 23.39 respectively 22.57 points. The student-level ESCS explains 10% variance on student level and 58% on school level in the U.S., 13% variance on student level and 43% on school level in Singapore. The association remains robust after controlling for all other fixed effects and after including all random effects in the full model. The student-level ESCS is also associated with the PISA reading score with strongest significance and highest robustness in Canada and South Korea, but yields lower effect strength and explained variance, presumably due to lower inequality of household incomes (β = 16.49*** and β = 10.63***; explained variance on student level 6% and 4%, on school level 34% and 27%).

Model two shows strong, highly significant, and highly robust associations of the control strategy of learning with the PISA reading score in all four countries (U.S. β = 35.28 ***, Canada β = 35.39 ***, South Korea β = 21.2 ***, Singapore β = 27.47 ***). Except for a positive significant and robust association of the internalization strategy with the PISA reading score in the case of South Korea, all other learning strategies are either significantly, negatively, and robustly associated or not significantly associated with PISA reading scores. Nevertheless, the internalization strategy shows a smaller effect size throughout the models. The explained variance of the four learning strategies altogether—taking into account the mostly negative association of three of them—is 18% on student level and 35% on school level in the U.S., 21% and 36% in Canada, 21% and 36% in South Korea, and 15% and 35% in Singapore. Hence, except for the internalization strategy in South Korea, of the strategies which have often been associated with Confucianism only the control mode of learning helps to achieve in PISA. More importantly, this holds also true for the two Eastern and the two Western countries. In contrast, we witness that the transfer/elaboration strategy is significantly, negatively, and robustly associated with PISA achievement scores. This implies that a more explorative kind of learning compared to the control strategy is associated with lower PISA-scores. As this learning strategy is closest to the learning type of Western Enlightenment, we may conclude that PISA rewards the Confucian and punishes the Enlightenment type of learning.

In model three, the school ESCS and, except for South Korea, the disciplinary climate but not the variables regarding student and teacher absenteeism show a significant and robust association with PISA reading scores, most so in the U.S. and in Singapore, followed by South Korea regarding school ESCS but not disciplinary climate, least so in Canada. We must consider here that South Korea is characterized by high-school discipline in general, which is why our school discipline variable is neither robust nor yields effects significantly different from zero. The explained variance in the PISA reading score increases, most so in the U.S. and in Singapore, less so in South Korea and Canada. Again, this result is linked to higher income inequality in the U.S. and in Singapore compared to South Korea and Canada.

Controls of ESCS, disciplinary climate and learning strategies in models four to seven add to explained variance in PISA reading scores in all four countries with highest explained variance again in the two countries showing highest income inequality. This is especially caused by the inclusion of random effects, which increase the variance explained on student level significantly, but reduce the explained variance on school level slightly with the exception of South Korea. This is attributable to one outlier school included in South Korea, which combines an extremely high PISA reading score as intercept with a positive slope. This is especially true when random effects of learning strategies are included.

In every country under scrutiny, we see positive values for the random effects included in our models. This indicates that with increasing values for ESCS, disciplinary climate, and learning strategies, the differences between schools increase. In other words: we witness that the effects of social inequality increase between schools, but the effects within schools (fixed effects) remain stable. This means that despite differences among schools, which get even stronger when learning strategies are applied, disciplinary climate and differences in ESCS are considered, the effects of social inequality and control strategies remain stable. Regardless of which school a student goes to in the four countries, the control strategy of learning ideally associated with Confucianism significantly increases PISA scores everywhere. At the same time, our result shows that there are stable effects of social inequality regardless of whether a student goes to a school that has a high ESCS or low ESCS on average. Even more, the disadvantage tends to increase the higher the average ESCS of the school and the lower the ESCS of the students at these schools.

Turning to the explained variance in the full models, we witness that 51% variance on student level and 91% variance on school level is explained in the U.S., 54% and 85% in Singapore, 59% and 62% in Canada respectively. Due to the outlier in South Korea, explained variance is at 31% on student level and 87% on school level in model five to change in models six and seven to 50% on student level and 26 respectively 27% on school level. Nevertheless, the school ESCS shows a significant and highly robust association with the PISA reading score which is nearly as strong as in Singapore, indicating considerable stratification of schools according to average school ESCS and average PISA reading score.

5. Discussion and conclusion

The Confucian type of learning and creativity is different to a type of learning and creativity that originated in Western culture and culminated in the Enlightenment (cf., Münch, 2001). It favors incremental innovation, whereas the Enlightenment type of learning enables radical innovation. On the one hand, learning shall result in the self-improvement of the individual, on the other hand, it shall make the single individual an independent person who can reflect and judge critically any claims of knowledge validity. In order to acquire this capacity, learning has to provide chances of creative exploration of new ways of thinking, experimentation and critical argumentation in collaboration with schoolmates. Yet our findings imply that this type of learning is incompatible with a regime of learning that puts primary emphasis on monitoring through extensive testing and is even punished by lower PISA scores. As it is established on the basis of making achievement in the PISA test the benchmark for best performing school systems, schooling systems with emphasis on creative, out of the box learning will gain lower scores compared to schooling systems which rely on the control-strategy of learning and will become a negative foliage for educational reforms according to the logic of “naming and shaming” introduced by the OECD PISAtest. At the same time, the Confucian mode of learning and creativity is established as role model for schools worldwide.

As our study shows, learning in the control mode is the one and single mode of learning which is significantly positive in its effect on PISA reading scores throughout all countries involved and throughout all models. The memorization and internalization strategies are significantly and negatively associated with PISA reading scores throughout all countries and models except for the internalization strategy in the case of South Korea. Possibly, the students’ understanding of internalization is close to the control strategy in this country. Transfer respectively elaboration strategies are closer to creativity and radical innovation in the Western Enlightenment sense but are significantly negative in their association with PISA reading scores in all four countries, indicating that PISA does not reward creativity in learning in the more radical sense.

What, therefore, remains as the single significantly positive and robust learning strategy is learning in the control mode. We may interpret this strategy as learning in the examination mode. This is a type of learning which includes the procedure of an examination directly in the learning process. We think that this type of learning keeps creative exploring in narrow limits and rewards unquestioned reproduction of the existing order of knowledge. Remarkably, this holds for the two liberal Western countries and the two East/Southeast Asian countries in our analysis alike. This interpretation of our results is supported by the significantly positive and robust effect of the disciplinary climate on PISA scores in all four countries. Apparently, Western students adopt the Eastern mode of learning to perform best in the PISA test. This is true, even after controlled for the student’s socioeconomic status and the differing effects between schools, as indicated by the random effects in our models.

Statistically, we see significant and robust associations of socioeconomic status of individual students and schools with PISA reading scores with considerable explained variance in this score throughout all countries and models, most so in the U.S. and in Singapore, the two countries showing highest income inequality. Disciplinary climate (except for generally highly disciplined South Korea) and learning in control mode add explained variance on the student level but are partly absorbed by socioeconomic status. That is, independent of the respective school under observation, better off students might be more disciplined and apply the more successful learning strategies to achieve in the PISA test. Therefore, we should not expect more than a small significant closing of the socioeconomic achievement gap by exerting more disciplinary pressure and applying the most successful learning strategies across all socioeconomic classes. Despite all efforts, the impact of students’ socioeconomic status on their educational performance has not changed in five decades (Chmielewski, 2019). Socioeconomic status is important for PISA achievement in all four countries, particularly in the U.S. and in Singapore, less so in South Korea and in Canada. The better situation in Canada and South Korea is, however, not so much due to a more favorable disciplinary climate and better learning but due to lower social inequality according to the Gini coefficient, and, in Canada additionally, due to a highly selective migration policy so that there is no significant difference between migrant and native students. And we have to take into account that in Canada, as in the U.S., the impact of family background might be underestimated because of the relatively low coverage of 15 years old youth in the test. It is at 83.6% in Canada and 82.2% in the U.S., as compared to 87.9% in South Korea and 94.3% in Singapore in 2009 (cf., Anders et al., 2019).

We must also take reliability problems of PISA data into account (Jerrim et al., 2017), which are accountable to multiple imputations of the data, the sampling procedure with poor coverage (especially in Canada and the U.S.) and model violations of the underlying latent variable model. The multiple imputation of missing data rests on the family background of the students as well as on the main characteristics of the schools which they attend. In tendence, such an approach might result in smaller deviances between students and thus larger clustering within the classes and schools in regards of the missing values. Additionally, low coverage yields the danger to miscalculate the PISAscores used as dependent variable. Due to this imputation problem, the ESCS measure might be biased to a large extent. Therefore, it is discussed whether the ESCS is appropriate for measuring the socioeconomic status of students (Avvisati, 2020; Jerrim & Micklewright, 2014; Rutkowski & Rutkowski, 2013). An alternative is the international socioeconomic index of occupational status (ISEI) of the father or both parents which is part of the ESCS. According to our previous investigations, differences in outcomes are small, however. And for highly developed countries the complete ESCS seems to be the right index (Muench & Wieczorek, 2022). In combination, both problems yield the possibility of underestimation of the impact of family background on PISA scores in the cases of Canada and the U.S. This means that there might be even greater support for our argument.

Despite these methodological problems, we think that our study contributes to the critical reflection and discussion of the learning regime promoted by PISA (cf., Komatsu & Rappleye, 2021). In order to further prove the robustness of our findings we need more studies that make use of other data such as TIMSS (Third International Mathematics and Science Study) and PIRLS (Progress in International Reading Literacy Study), scrutinize other PISA test years and include other relevant variables in the analysis.

The practical implication of our study for school policy is a warning against an undisputed subjection to a uniform control regime of education as promoted by PISA all over the world (cf., C. Tienken, 2017; Zhao, 2014). Instead, we want to stress that there are two types of learning, creativity and innovation, one more represented by the Western Enlightenment, the other one more represented by Confucianism. Both complement each other as we need both, radical and incremental innovation. The wellbeing of humankind depends on cultural diversity as much as on biodiversity. International benchmarking through standardized testing like PISA replaces cultural diversity by uniformity. We, therefore, should end assessing school systems, schools, and students with one-size-fits-all standardized tests of this kind. Everybody responsible for education should know that learning is more than preparing for standardized tests. This holds particularly for the protagonists of this kind of education, particularly involving the OECD’s PISA network, the education and testing industries, consultancies, philanthropic foundations, advocacy groups, policy makers and administrators working together in this project (Münch, 2020).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Deutsche Forschungsgemeinschaft [461386736].

Notes on contributors

Richard Muench

Richard Muench is Senior Professor of Social Theory and Comparative Macrosociology at Zeppelin University, Friedrichshafen, Emeritus of Excellence at Otto Friedrich University, Bamberg, Germany, and member of the Berlin-Brandenburg Academy of Sciences. He has been a visiting professor at the University of California, Los Angeles, several times, and co-editor of several sociology journals. In 2018, he was awarded for his lifetime achievements by the German Sociological Association. His research focuses on the study of social change in the context of globalization, European integration, and national traditions, with a recent focus on education, schools, and teaching in the competitive state, academic capitalism, and academic careers.

Oliver Wieczorek

Oliver Wieczorek graduated from Otto-Friedrich University, Bamberg, Germany, with a BA, an MA, and a Ph.D. in Sociology. He has been researcher at Otto-Friedrich University, Bamberg, Germany, Zeppelin University, Friedrichshafen, Germany, and is currently senior researcher at the INCHER at the University of Kassel, Germany. His research focuses on the sociology of science and the sociology of secondary and higher education.

Robin Gerl

Robin Gerl graduated from University of Stuttgart, Stuttgart, Germany, with a BA in Social Science, and a MA in Empirical Political and Social Research. He has been a researcher at the University of Stuttgart and the Zeppelin University, Friedrichshafen, Germany. His research focuses on empirical methodology and the sociology of education.

Notes

1. Values greater zero indicate that the range of values “spread” across the schools, whereas values smaller zero indicate that with increasing values of the independent variable, the values of the dependent variable get more similar. For example, a positive value for ESCS on school level indicates that for low ESCS values, PISA scores would be more similar compared to higher ESCS values. This could be read that with increasing ESCS differences, the PISA scores are more unequally distributed among schools.

Appendix A:

Descriptive Statistics

In a first step, we calculated descriptive statistics on the independent variables and the PISA reading score 2009 as dependent variable. We included the minimum, median, mean, maximum, variance, standard deviation, skewness and kurtosis into our calculations. Table A1 provides descriptive statistics on the United States, A2 on Canada, A3 on Singapore and A4 on South Korea.

Table A1. Descriptive statistics USA

Variables	n	mean	sd	median	min	max	range	skew	kurtosis	se
Reading	5,233	497.59	95.60	499.58	156.38	772.46	616.08	−0.11	−0.32	1.32
ESCS	5,233	0.00	1.00	0.02	−3.89	2.98	6.87	−0.23	−0.07	0.01
Disciplinary Climate	5,233	0.00	1.00	−0.13	−3.04	2.88	5.92	−0.41	0.41	0.01
Memorization	5,233	0.00	0.91	−0.20	−1.93	1.52	3.45	−0.08	−0.58	0.01
Transfer	5,233	0.00	0.96	−0.18	−1.40	2.25	3.65	0.37	−0.49	0.01
Control	5,233	0.00	0.93	0.04	−2.02	1.70	3.73	−0.16	−0.45	0.01
Internalization	5,233	0.00	0.90	−0.21	−1.31	1.98	3.29	0.38	−0.54	0.01
Ave. School ESCS	5,233	0.00	1.00	0.00	−2.94	2.84	5.78	0.20	−0.01	0.01
Student absenteeism	5,233	1.05	0.21	1.00	1.00	2.00	1.00	4.21	15.73	0.00
Teacher absenteeism	5,233	1.30	0.46	1.00	1.00	2.00	1.00	0.89	−1.21	0.01

Table A2. Descriptive statistics Canada

Variables	n	mean	sd	median	min	max	range	skew	kurtosis	se
Reading	23,207	511.53	93.56	515.31	45.82	871.12	825.30	−0.24	0.02	0.61
ESCS	23,207	0.00	1.00	0.03	−6.37	3.20	9.58	−0.22	−0.01	0.01
Disciplinary Climate	23,207	0.00	1.00	0.11	−2.81	3.43	6.23	−0.39	0.28	0.01
Memorization	23,207	0.00	0.91	−0.17	−1.96	1.61	3.57	−0.12	−0.40	0.01
Transfer	23,207	0.00	0.95	−0.06	−1.33	2.47	3.79	0.44	−0.38	0.01
Control	23,207	0.00	0.93	0.00	−2.21	1.57	3.78	−0.35	−0.28	0.01
Internalization	23,207	0.00	0.90	0.00	−1.41	1.84	3.25	0.26	−0.66	0.01
Ave. School ESCS	23,207	0.00	1.00	0.01	−6.38	3.36	9.74	0.14	0.79	0.01
Student absenteeism	23,207	1.04	0.19	1.00	1.00	2.00	1.00	4.78	20.85	0.00
Teacher absenteeism	23,207	1.33	0.47	1.00	1.00	2.00	1.00	0.73	−1.46	0.00

Table A3. Descriptive Statistics Singapore

Variables	n	mean	sd	median	min	max	range	skew	kurtosis	se
Reading	5,283	523.27	97.62	529.15	191.81	794.14	602.33	−0.26	−0.23	1.34
ESCS	5,283	0.00	1.00	0.10	−4.12	3.34	7.46	−0.34	0.03	0.01
Disciplinary Climate	5,283	0.00	1.00	−0.11	−3.23	2.75	5.98	−0.46	0.47	0.01
Memorization	5,283	0.00	0.92	−0.23	−2.16	1.73	3.89	0.16	−0.43	0.01
Transfer	5,283	0.00	0.93	−0.14	−1.81	2.03	3.84	0.14	−0.47	0.01
Control	5,283	0.00	0.91	0.10	−2.64	1.47	4.11	−0.25	−0.45	0.01
Internalization	5,283	0.00	0.87	−0.35	−1.53	1.94	3.47	0.39	−0.35	0.01
Ave. School ESCS	5,283	0.00	1.00	−0.28	−2.13	3.19	5.32	0.90	0.40	0.01
Student absenteeism	5,283	1.14	0.35	1.00	1.00	2.00	1.00	2.10	2.40	0.00
Teacher absenteeism	5,283	1.31	0.46	1.00	1.00	2.00	1.00	0.84	−1.30	0.01

Table A4. Descriptive Statistics South Korea

Variables	n	mean	sd	median	min	max	range	skew	kurtosis	se
Reading	4,989	540.98	77.15	545.50	161.27	777.70	616.43	−0.40	0.34	1.09
ESCS	4,989	0.00	1.00	0.01	−4.37	3.07	7.44	−0.16	−0.14	0.01
Disciplinary Climate	4,989	0.00	1.00	−0.05	−3.92	2.55	6.47	−0.34	0.43	0.01
Memorization	4,989	0.00	0.85	0.22	−1.64	2.14	3.78	0.20	−0.08	0.01
Transfer	4,989	0.00	0.95	−0.06	−1.56	2.25	3.81	0.31	−0.40	0.01
Control	4,989	0.00	0.93	0.11	−2.21	1.75	3.95	−0.18	−0.38	0.01
Internalization	4,989	0.00	0.92	0.28	−2.03	1.44	3.47	−0.20	−0.55	0.01
Ave. School ESCS	4,989	0.00	1.00	0.14	−2.32	2.83	5.14	0.04	−0.11	0.01
Student absenteeism	4,989	1.30	0.46	1.00	1.00	2.00	1.00	0.85	−1.27	0.01
Teacher absenteeism	4,989	1.90	0.29	2.00	1.00	2.00	1.00	−2.74	5.50	0.00

Appendix B:

Regression Diagnostics

As with OLS-regressions, there are several assumptions that need to be met for a hierarchical linear model. According to Korendijk et al. (2008), the assumptions investigate the linear relationship between the dependent variable and the independent variables, and of the random effects. It is important to investigate if the relationship between X and Y is linear and does not follow a curvilinear course, or that the standard errors are calculated correctly with the right random effects calculated. Because of the complex nature of hierachical models there is a need to evaluate different assumptions for the fixed part of a model, for the random part of a model and the residuals calculated for each level. For this reason, we begin with the description of the intercept, residuals on student and school levels, ICC, and deviance, as depicted in Table B1

Table B1. Null models

Variables	USA	South Korea	Canada	Singapore
Intercept	507.18^(11.36)	514.31^(8.21)	509.56^(2.52)	520.33^(4.77)
Residuals Level 2	3,456.82	2,798.09	2,062.95	3,447.20
Residuals Level 1	6,874.44	4,081.47	6,459.86	6,031.99
ICC	0.33	0.41	0.24	0.36
Deviance	39,471,114.15	7,041,487.18	4,193,244.43	599,691.96
Number of Students	5,233.00	4,989.00	23,207.00	5,283.00
Number of Schools	165.00	157.00	978.00	171.00

Furthermore, we tested for homoscedasticity using the variance inflation factor (VIF) for the full models. Technically, the VIF regresses each independent variable of a model under investigation on the standard errors of a regression model containing all other variables. For example, in a model with PISA reading scores as dependent variable, the student’s ESCS, the average ESCS at school, and the disciplinary climate at school, the VIF would test if the student’s ESCS could predict the residuals of a regression model consisting of the average ESCS and the disciplinary climate. It is calculated for each variable separately by dividing 1 by 1-R² for each regression on the residuals. Hence, it is minimal at a value of 1 (no multicollinearity). In general, the lower the VIF, the lower the multicollinearity between a given variable and the other variables (O’brien, 2007). A VIF under 4 is considered to be adequate and indicates low levels of multicollinearity.

To calculate the VIF, we conducted linear regression analyses of the full model. The results for the multilevel linear regression models with the PISA reading score 2009 as dependent variable are presented in Table B2. Overall, the models performed well and show only limited multicollinearity.

Table B2. Variance inflation coefficients (full models)

Variables	USA	Canada	Singapore	South Korea
ESCS	1.473	1.304	1.322	1.448
Disciplinary Climate	1.051	1.021	1.044	1.018
Memorization	1.458	1.508	1.291	1.311
Control	1.781	1.594	1.407	1.721
Internalization	1.535	1.465	1.360	1.484
Transfer	1.408	1.241	1.268	1.325
Ave. ESCS per school	1.468	1.288	1.331	1.454
ESCS * Memorization	1.450	1.520	1.298	1.391
ESCS * Control	1.698	1.589	1.395	1.745
ESCS * Internalization	1.500	1.485	1.352	1.555
ESCS * Transfer	1.415	1.255	1.316	1.389
Ave. VIF	1.476	1.388	1.307	1.440

We conducted Shapiro-Francia tests to account for the normality of error-terms assumption (see Ghasemi and Zahediasl, 2012, for specifities and advantages against other tests for normality). As seen in Table B3 in the p-value column, the normality assumption is violated in all cases under investigation. To account for this violation, we applied robust standard errors to our models (Freedman, 2006).

Table B3. Shapiro-francia tests. Full models

Model	W	P-Value	Plausible Value Imputation	Observations	Sampling	Normal Distributed
United States Full Model	0.999	0.001	1	5,233	yes	no
Canada Full Model	0.998	0.000	1	23,207	yes	no
Singapore Full Model	0.993	0.000	1	5,283	yes	no
Korea Full Model	0.997	0.000	1	4,989	no	no

We now continue with a visual depiction of the PISA reading scores on school level and the national average. This is important because this enables us to check whether positive or negative outliers are to be expected—and thus could introduce a potential bias into our calculations on school level. Figures B1–B4 depict the comparisons for the USA, Canada, Singapore, and South Korea.

Figure B1. USA: Intercept of the schools

Figure B2. Canada: Intercept of the schools

Figure B3. Singapore: Intercept of the schools

Figure B4. South Korea: Intercept of the schools

We now continue with the random effects coefficients (Figures B5–B8). For each variable included as random effect, we allow the slope of the effect to vary between schools. This means that we can interpret how the relationship of the independent variable with the dependent variable varies across schools with increasing values. With increasing effect coefficients, we can then assume a positive correlation, which means, using the example of the ESCS, that students at these schools have a higher PISA reading score on average as the ESCS increases. For schools with a negative slope of the effect coefficient, this effect is reversed. Moreover, in terms of slopes and intercepts, it is also possible to identify outliers here and thus determine which models are potentially biased.

Figure B5. Random effects per school, USA.

Figure B5. (Continued).

Figure B6. Random effects per school, Canada.

Figure B6. (Continued).

Figure B7. Random effects per school, Singapore.

Figure B7. (Continued).

Figure B8. Random effects per school, South Korea.

Figure B8. (Continued).

Overall, the slopes of the effect coefficients between schools follow the expectations that can be placed on a multilevel model with random effects. Only in the case of South Korea (Figure B8) we observe a large deviation in the case of one school. This large deviation may in turn lead us to systematically overestimate some effects. Otherwise, we see that for large values of the respective independent variables, the slope coefficients tend to drift apart. This means that we have particularly pronounced differences between schools in our data and the use of a random intercept, random slope model is warranted.