Overclaiming. An international investigation using PISA data

ABSTRACT

This paper investigates the phenomena of overclaiming – the propensity for individuals to claim more knowledge about an issue or topic than they really (or could possibly) do. Using Programme for International Student Assessment (PISA) data from nine Anglophone countries and over 40,000 young people, we examine teenagers’ propensity to claim knowledge of three mathematics constructs that do not really exist. We find substantial differences in young people’s tendency to overclaim across countries, genders, and socio-economic groups. Those who are most likely to overclaim are also found to exhibit high levels of overconfidence and believe they work hard, persevere at tasks, and are popular amongst their peers. Together this provides important new insight into overclaiming, how this differs across groups, and how it relates to other psychological constructs.

KEYWORDS:

• PISA
• overclaiming
• gender gaps

1. Introduction

Overclaiming refers to when an individual states to have more knowledge about a topic or issue than they really – or could possibly – do. It is closely linked to overconfidence, where one believes they are better at something than they really are (Moore & Healy, 2008). Despite their negative connotations, being overconfident or willing to overclaim may have some advantages – such as a willingness to tackle challenging problems or during negotiations in the workplace (e.g. for a pay rise). Of course, individuals may also experience negative consequences if they make a claim that others know to be untrue. Surprisingly, however, we still know relatively little about young people’s propensity to overclaim, including how this – and its correlates – compare across countries. This paper contributes this evidence to the existing literature, investigating the magnitude of overclaiming amongst young people and its correlates with other traits across English-speaking countries.

Early work in this area asked respondents about their knowledge of a mixture and real and fake constructs to understand how people answer surveys. In one of the first studies, Phillips and Clancy (1972) created an index based on how often individuals reported consuming a series of new books, television programmes, and movies, all of which were not real products. They used this index to explore the role of social desirability in survey responses. Stanovich and Cunningham (1992) and Randall and Fernandes (1991) also constructed overclaiming scales containing a mix of real and fake items, the former to examine author familiarity and the latter to investigate self-reported ethical conduct. However, in all these studies, the focus was not overclaiming per se.

Paulhus et al. (2003) focused more directly on the issue of overclaiming and its correlates with other psychological traits. They constructed an overclaiming index based on Paulhus and Bruce (1990) using a set of items, of which one-fifth were non-existent, and employed a signal-detection formula to measure overclaiming and actual knowledge. They found overclaiming to be an operationalisation of self-enhancement and that narcissists were more likely to overclaim than non-narcissists (Paulhus et al., 2003). More recently, both Keller et al. (2021) and Grosz et al. (2017) found a weaker link between overclaiming and narcissism or ‘dark personality traits’ than in previous studies.

Other studies have also explored the correlates of overclaiming. Atir et al. (2015) found people who perceived their expertise in various domains favourably were more likely to overclaim. Pennycook and Rand (2018) argued that overclaimers perceive fake news to be more accurate, making them more receptive to fake news or gullible. Dunlop et al. (2017) reported overclaiming to be related to ‘openness’, but not to honesty and humility. Taken together, this literature reveals gaps in our understanding of who overclaims knowledge.

A relatively small collection of studies has focused upon overclaiming amongst children and adolescents. For instance, Goecke et al. (2022) investigated the link between overclaiming and intelligence, creativity and personality traits in a sample of 897 children in Germany. They failed to find strong evidence that overclaiming is related to honest-humility, cognitive abilities and creative engagement, arguing that future research should consider taking a metacognitive approach to overclaiming. Butler and Nelson (2021) also studied overclaiming amongst a small sample of 94 children, aged between 5- and 10-year-olds. Overclaiming tendencies were found to decline during childhood, though 10-year-olds were still more likely to make exaggerated claims about their knowledge than adults.

Other studies have investigated cross-national differences in overclaiming through analysis of the PISA 2012 data we utilise in our empirical work. These studies have mostly focused on cross-national variation in overclaiming and predictors of overclaiming at the country level. One such example is Fell and König (2020, p. 462), who found that differences across 41 countries in overclaiming was related to ‘the cultural dimensions of humane orientation, in-group collectivism, and gender egalitarianism’. This built upon previous work by the same authors (Fell et al., 2019) who explored whether countries with greater prevalence of ‘rule violations’ (i.e. prevalence of the shadow economy, lack of democracy and lack of control over corruption) had higher levels of overclaiming. They found there to be a positive relationship. Using the PISA data, Vonkova et al. (2018) report there to be cross-country differences in overclaiming, with South Europeans tending to exaggerate the most and Western Europeans the least. They also show there to be a correlation at the country-level between overclaiming, public expenditure on secondary education and levels of corruption. Concluding, they suggest that ‘future research could focus at the more detailed intracountry level analysis to identify the possible differences in response patterns between different groups of respondents distinguished by demographical (gender, age) or socioeconomic (education, occupation) indicators’ (Vonkova et al., 2018, pp. 1,265). Other studies examining overclaiming in PISA have had a more methodological focus (e.g. Jin et al., 2023; Muszyński et al., 2021) or have investigated overclaiming within a single national setting. For instance, Yang et al. (2019) conducted a latent class analysis in the United States, finding that academically disengaged students tended to overclaim the least.

We contribute to this existing literature on overclaiming and the related issue of overconfidence. In particular, we heed the call of Vonkova et al. (2018) and consider within-country differences in overclaiming by demographic (gender, immigrant status) and socio-economic characteristics. This is done using a large sample of around 40,000 young people from nine Anglophone countries, allowing us to dig deeper into the differences between subgroups (e.g. boys versus girls, advantaged vs. disadvantaged young people) – and potential differences across countries in such dimensions – than previous work.

Second, we also examine the relationship between overclaiming and various other personality traits, including overconfidence, self-perceptions of popularity amongst peers, and reported levels of perseverance. Unlike many previous studies, we investigate differences between those who are more and less likely to overclaim conditional on a range of potential confounding characteristics (including a high-quality measure of academic achievement) providing stronger evidence that overclaiming really is independently related to these important traits. Previous research exploring correlations between overclaiming and other psychological constructs amongst adolescents have typically done so either in smaller samples of younger children (e.g. Butler & Nelson, 2021; Goecke et al., 2022) or within a single national setting (e.g. Yang et al., 2019). We extend on such work by considering a rather different set of potential correlates (e.g. whether those who overclaim are more likely to believe they are popular amongst their peers) using a large multi-national sample.

Our findings support the view that young men are, on average, more likely to overclaim than young women, and that socio-economically advantaged teenagers are more likely to overclaim than their disadvantaged peers. There is also important cross-national variation, with young people in North American more likely to make exaggerated claims about their knowledge than those from Europe. We also illustrate how those who are more likely to overclaim display overconfidence in their skills and are more likely to report that they work hard when challenged and are popular at school than other young people.

The paper now proceeds as follows. Section 2 provides an overview of the PISA 2012 data and our empirical methodology. Results are then presented in section 3, with discussion and conclusions following in section 4.

2. Data

Throughout this paper we use data from PISA 2012 (OECD, 2014). Although around 70 countries participated in this study, we focus on countries where English is the most commonly spoken language to minimise concerns about translation of survey items and hence comparability.¹ The United Kingdom is split into its four separate countries – England, Scotland, Northern Ireland and Wales – which are recognised internationally as separate nations by groups such as the International Organisation of Standardisation. The same is not true of, for instance, Australian or US states or Canadian provinces, which are hence treated as being part of a single national entity.

A multi-stage survey design was used, with schools first divided into a series of strata and then randomly sampled with probability proportional to size. From within each school, a sample of around 35 15-year-olds were randomly selected to participate. A total of 2,689 schools and 62,969 pupils took part in the study from across our nine Anglophone countries, reflecting official response rates of around 80%. In all nine countries, the sample was compliant with the standards set by the OECD. Final student senate weights are applied throughout our analysis, with each country being given equal weight. The complex PISA survey design – including the clustering of pupils within schools and the use of ‘plausible values’ as estimates of pupil achievement – are accounted for as recommended by the survey organisers. This is implemented via the use of the ‘repest’ STATA package (Avvisati & Keslair, 2019).

PISA is primarily designed to measure the mathematics, science and reading skills of 15-year-olds across countries via a two-hour achievement test. However, participants also complete a 30-minute questionnaire that gathers information on young people’s demographic background and their knowledge, attitudes and experience of subjects they study at school. Mathematics was the focus of PISA 2012, with most test and questionnaire items centred around this subject. Another somewhat unusual feature of PISA in 2012 was that young people were randomly assigned to complete one out of three different versions of the background questionnaire. We restrict our analysis to the random sub-sample of 40,550² young people from Anglophone countries who completed either form A or form C, which included the following question:

‘Thinking about mathematical concepts: how familiar are you with the following terms?’

A list of 16 items were then given to students, who were asked to indicate their knowledge of each mathematics concept on a five-point scale (ranging from ‘never heard of it’ to ‘know it well, understand the concept’). These constructs were:

1. Exponential function

2. Divisor

3. Quadratic function

4. Proper number

5. Linear equation

6. Vectors

7. Complex number

8. Rational number

9. Radicals

10. Subjunctive scaling

11. Polygon

12. Declarative fraction

13. Congruent figure

14. Cosine

15. Arithmetic mean

16. Probability

The distribution of responses to these 16 items across our analytic sample can be found in Appendix A. Critically, of these 16 constructs, three of them (items 4, 10 and 12) are fake; students are asked about their familiarity with some mathematics concepts that do not exist.

We use participants responses to these three items to form our ‘overclaiming’ scale. This is done via estimation of a two-parameter graded response item-response theory model using STATA 17. Our motivation for combining responses into a scale is a pragmatic one; to identify those participants who were more or less likely to overclaim. It is also consistent with other studies into overclaiming that have also used a latent variable approach (e.g. Yang et al., 2019; Goecke et al., 2022). However, the scale produced is very similar to that formed from a simple summative score (the correlation between the two is 0.94). The overclaiming scale has been standardised to mean zero and standard deviation one, so that all differences between groups can be interpreted in terms of an effect size.

Measures of self-efficacy

Within our analysis, we consider whether young people who score highly on the overclaiming scale also display a series of psychological characteristics. These were captured using questions with either four-point or five-point response options. Throughout our analysis we convert these into binary variables. For consistency, where the question uses a four-point response option, we combine the top two categories and bottom two categories. Similarly, when the question uses a five-point response scale, the top three categories are combined, and bottom two categories are combined.

The first characteristic we consider is the relationship between overclaiming and mathematics self-efficacy. Specifically, as part of the PISA background questionnaire, participants were asked how confident they are in being able to complete the following eight tasks, according to a four-point scale (ranging from very confident to not confident at all):

• Task 1. Using a train timetable to work out how long it would take to get from one place to another.

• Task 2. Calculating how much cheaper a TV would be after a 30% discount.

• Task 3. Calculating how many square metres of tiles you need to cover a floor.

• Task 4. Understanding graphs presented in newspapers.

• Task 5. Solving an equation like 3 × +5 = 17.

• Task 6. Finding the actual distance between two places on a map with a 1:10,000 scale.

• Task 7. Solving an equation like 2(x + 3) = (x + 3)(x-3).

• Task 8. Calculating the petrol consumption rate of a car.

We dichotomise teenagers’ responses, combining the top two responses and the bottom two responses along the four-point scale. This means that we will compare the percentage of young people who said they are confident/very confident to the percentage who said they were not confident/not confident at all. The survey organisers have also created a mathematics ‘self-efficacy’ scale, combining young people’s responses to these eight items into a single continuous index. We standardise this scale so that the mean is zero and the standard deviation one.

Self-belief in problem-solving abilities

Students were asked to indicate how well they believe the following five statements describes them and their problem-solving ability:

1. I can handle a lot of information.

2. I am quick to understand things.

3. I seek explanations for things.

4. I can easily link facts together.

5. I like to solve complex problems.

Responses were providing using a five-point scale (very much, mostly like me, somewhat like me, not much like me, not like me at all). Teenagers’ responses are dichotomised, with the top three responses (very much/mostly/somewhat like) combined and bottom two (not much/not like me at all) responses combined. A total scale score has also been derived by the survey organisers, which we again standardise to mean zero and standard deviation one.

Self-reported popularity at school

To capture teenagers’ self-reported views on their popularity at school, they were asked ‘thinking about your school, to what extent do you agree with the following statements’:

1. I feel like an outsider (or left out of things) at school.

2. I make friends easily at school.

3. I feel like I belong at school.

4. I feel awkward and out of place in my school.

5. Other students seem to like me.

6. I feel lonely at school.

7. I feel happy at school.

8. Things are ideal in my school.

9. I am satisfied with my school.

Responses were to be given on a four-point scale, with our analysis of individual questions combining the top two (strongly agree/agree) categories and bottom two (disagree/strongly disagree) categories into a binary scale.

Self-reported measures of perseverance

A series of five items were used in the background questionnaire to capture teenagers’ self-reported perseverance with challenging tasks. Specifically, they were asked ‘how well does each of the following statements below describe you’ with responses given on a five-point scale (very much like me, mostly like me, somewhat like me, not much like me, not at all like me):

1. When confronted with a problem, I give up easily.

2. I put off difficult problems.

3. I remain interested in the tasks that I start.

4. I continue working on tasks until everything is perfect.

5. When confronted with a problem, I do more than what is expected of me.

We again recode responses to these questions into a binary format, with very/mostly/somewhat like me coded as one and zero otherwise. An overall scale combining information from all five items has also been derived and standardised to mean zero and standard deviation one.

Problem-solving approaches

As part of the background questionnaire, two hypothetical scenarios were set out to students, who were then asked how they would respond. The first scenario asked:

‘Suppose that you have been sending text messages from your mobile phone for several weeks. Today, however, you can’t send text messages. You want to try and solve the problem. What would you do?’

1. I press every button possible to find out what is wrong.

2. I think about what might have caused the problem and what I can do to solve it.

3. I read the manual.

4. I ask a friend for help.

Students were asked whether they would (a) definitely do this; (b) probably do this; (c) probably not do this or (d) would definitely do this, for each of the four statements above. We combine options (a) with (b) and (c) with (d), allowing us to compare young people who said they would probably/definitely use each strategy versus those would definitely/probably would not.

The second scenario followed a similar structure, with participants asked:

‘Suppose that you are planning a trip to the zoo with your brother. You don’t know which route to take to get there. What would you do?’

1. I read the zoo brochure to see if it says how to get there.

2. I study a map and work out the best route.

3. I leave it to my brother to worry about how to get there.

4. I know roughly where it is, so I suggest we just start driving.

Participants were provided the same four response options (a to d) as per scenario 1, which we also convert into a binary format as described above.

Analytic models

Our analysis begins by comparing average overclaiming scores across demographic groups, and across countries. As these results are produced for nine separate countries, one faces the issue of conducting multiple hypothesis tests. Hence, in our results tables, we report whether differences are statistically significant using the conventional alpha cut-off of 0.05, and an alternative alpha cut-off of 0.006 after a Bonferroni correction has been made.

We then investigate the self-reported self-efficacy, problem-solving skills, perseverance and popularity of over claimers. To begin, we divide participants into four approximately equal groups (quartiles) based upon their scores on the overclaiming scale. Those in the bottom quartile are then labelled as having ‘low’ overclaiming scores (i.e. those young people who overwhelmingly said that they had not heard of the fake mathematics constructs) with the top quartile defined as having ‘high’ overclaiming scores (i.e. young people who claimed expertise in the fake constructs). Our decision to focus on the top and bottom quartile is due to these groups being clearly different in their propensity to overclaim (see Appendix A1 for further details). As illustrated in Appendix Tables A2– A4, the situation for those individuals in the middle of the distribution (i.e. the second and third quartiles) is much more ambiguous, with these groups quite likely to overclaim on one item in particular (proper number) but not the other two. This could plausibly be due to these individuals having made a mistake when reading the term (e.g. due to reading through the items too quickly) rather than being due to them overclaiming per se. Our interpretation is that there is only clear evidence of a difference in overclaiming amongst young people that fall in the top and bottom quartiles of the scale. For these two groups, we compare how they responded to each of the self-efficacy, problem-solving, popularity and perseverance questions (and overall scale scores) described above.

A limitation with such summary statistics is that there could be confounding characteristics driving the results. For instance, with respect to self-efficacy, it will be particularly important to consider whether overclaimers are much more likely to believe they can complete each of the eight mathematics tasks than those who do not overclaim after conditioning upon their actual measured academic ability. In other words, do teenagers who overclaim about their mathematics knowledge also display overconfidence in their mathematics skills? For each of the outcome measures described in section 2, we therefore estimate the following OLS regression model within each country³:

(1) $O_{ij}=\alpha +\beta .O{C}_{ij}+\gamma .A_{ij}+\delta .SE{S}_{ij}+\tau .D_{ij}+u_j+{\epsilon }_{ij}$(1)

Where:

$O_{ij}$ = The outcome variable of interest (e.g. teenagers’ self-efficacy).

$O{C}_{ij}$ = A set of dummy variables reflecting quartiles of the overclaiming scale.

$A_{ij}$ = Teenagers’ academic achievement in mathematics, reading, science and problem solving, as measured by the PISA test.⁴

$SE{S}_{ij}$ = Teenagers’ socio-economic status, as measured by the PISA Economic, Social and Cultural Status (ESCS) index.

$D_{ij}$ = A vector of controls for teenagers’ demographic characteristics (e.g. gender and immigrant status).

$u_j$ = School fixed effects.

i = Student i.

j = School j.

The parameters of interest are the estimated $\beta$ coefficients. These will reveal differences between those with high and low levels of overclaiming, conditional on their gender, socio-economic status, mathematics, reading, science and problem-solving skills and school characteristics. Such conditional associations will help reveal whether those young people who overclaim provide different answers to the self-efficacy, perseverance, popularity and problem-solving questions than their peers who do not overclaim of the same demographic background, of equal academic ability and within the same school.

3. Results

Who overclaims?

Table 1 considers how average scores on the overclaiming scale differ between demographic groups. There is an important difference between genders; boys are much more likely to overclaim than girls. This holds true across all nine countries, with all differences statistically significant (even after a Bonferroni correct is made) and equivalent to an effect size of between 0.2 and 0.3 standard deviations in most countries. Consequently, Table 1 provides strong and consistent evidence that teenage boys are more likely to overclaim than teenage girls.

Table 1. The association between demographic characteristics and average scores on the overclaiming scale.

	Girls	Boys	Gap (effect size)	SE
(a) Gender
England	−0.07	0.24	.30**	0.03
Scotland	−0.36	−0.07	.28**	0.04
New Zealand	−0.09	0.18	.27**	0.03
Canada	0.04	0.31	.27**	0.03
Australia	−0.09	0.17	.26**	0.02
Wales	−0.16	0.11	.26**	0.05
Ireland	−0.32	−0.08	.24**	0.03
USA	0.11	0.32	.21**	0.04
Northern Ireland	−0.21	−0.02	.19**	0.04
	Low SES	Q2	Q3	High SES	Gap (effect size)	SE
(b) Socio-economic status
New Zealand	−0.06	0.02	.06	0.24	0.29**	0.06
Scotland	−0.34	−0.21	−.16	−0.08	0.26**	0.07
Wales	−0.14	0	−.01	0.08	0.22**	0.06
Australia	−0.02	−0.03	.05	0.16	0.18**	0.03
USA	0.13	0.23	.2	0.3	0.17**	0.05
England	0.03	0.01	.09	0.18	0.16**	0.05
Canada	0.11	0.11	.14	0.26	0.16**	0.03
Northern Ireland	−0.19	−0.14	−.11	−0.03	0.16*	0.06
Ireland	−0.26	−0.17	−.22	−0.12	0.14**	0.05
		Native-born	Immigrants	Gap (effect size)		SE
(c) Immigrant group
Northern Ireland		−0.14	0.27	.40*		0.17
New Zealand		−0.03	0.25	.28**		0.06
Ireland		−0.23	0.03	.25**		0.07
England		0.05	0.27	.23**		0.06
Scotland		−0.23	−0.05	.18*		0.09
Canada		0.13	0.26	.14**		0.03
Australia		0.01	0.13	.12**		0.03
Wales		−0.01	0.21	.25		0.13
USA		0.21	0.22	.01		0.05

The overclaiming scale has been standardised within each country to mean zero and standard deviation one. The gap refers to the difference between groups in terms of an effect size. SE refers to the standard error of the gap. Northern Ireland excluded from socio-economic status results due to factor scores not able to be calculated. * indicates statistical significance at p = 0.05; ** indicates statistical significance at p = 0.006 after a Bonferroni correction has been made. Data source: OECD (2014).

A similar difference is found with respect to socio-economic status; young people from more advantaged socio-economic backgrounds have higher average overclaiming scores than their less advantaged peers. The magnitude of the difference is again not trivial and varies somewhat across countries. For instance, the difference in average overclaiming scores between the top and bottom socio-economic quartile is almost 0.3 standard deviations in New Zealand, compared to around below 0.15 in England, Canada, Ireland and Northern Ireland. Nevertheless, in eight countries, the difference is statistically significant at an alpha cut-off of 0.006 after a Bonferroni correction has been made (the exception is Northern Ireland, where the difference is statistically significant at an alpha cut-off of 0.05). These results therefore provide strong evidence that young people from more affluent backgrounds are more likely to overclaim than young people from disadvantaged backgrounds.

The final difference considered in Table 1 is between immigrant and native-born citizen groups. In most countries, immigrants having significantly higher scores than young people who are native-born. This is particularly pronounced in New Zealand and Northern Ireland, where immigrants score around 0.3–0.4 standard deviations higher on the overclaiming scale than young people who were born in the country. The association is notably weaker in the United States, with there actually being no difference between immigrants and native-born citizens. Hence, although we find a general pattern of immigrants being bigger more likely to overclaim than native-born citizens, the strength of this association seems to vary quite substantially between countries (and, thus, characteristics and home locations of the immigrant groups).

Finally, in additional analysis, we have also estimated the within versus between school variation of the overclaiming scale within each country. Our motivation was to establish whether overclaiming tends to cluster within the same school, or if overclaiming is roughly equally distributed across schools. We find that the ICCs tend to be very low; in most countries less than four percent of the variance in overclaiming occurs between schools.

Are teenagers in some countries more likely to overclaim than others?

Table 2 provides our comparison of the overclaiming scale across Anglophone countries. The top panel provides the average standardised scale score, while the bottom panel provides t-statistics for pairwise comparisons across countries. Green shading highlights where differences across countries are statistically significant, using an alpha cut-off of 0.05 (light green shading) or 0.006 (dark green) after a Bonferroni correction has been made.

Table 2. International comparison of average overclaiming scores across Anglophone countries.

Country				Mean			Standard error
(a) Average overclaiming scale scores across
USA				0.22			0.024
Canada				0.17			0.013
England				0.08			0.020
Australia				0.04			0.011
New Zealand				0.04			0.022
Wales				−0.03			0.028
Northern Ireland				−0.11			0.028
Ireland				−0.20			0.019
Scotland				−0.21			0.024
(b) T-statistics for pairwise country comparisons

Average overclaiming scale scores have been standardised to mean zero and standard deviation one across the eight Anglophone countries. Wales has been excluded based upon measurement invariance tests. Light green shaded cells in panel b indicate where difference across countries is statistically significant at p = 0.05 (absolute value of the t-statistic is greater than 1.96). Dark green shading in panel b indicates where difference across countries is statistically significant after a Bonferroni correction has been made for eight comparisons being made for each country at p = 0.006 (absolute value of the t-statistic is greater than 2.735). Red shaded cells with italic font illustrates where cross-country differences are not statistically significant at p = 0.05 (absolute value of the t-statistic is less than 1.96). Data source: OECD (2014).

Three broad clusters of countries seem to have emerged. At the top of the rankings are the two North American countries of the United States and Canada. With average scale scores around 0.2, these two countries have significantly higher overclaiming scores than any other country. The next countries (Australia, New Zealand, England and Wales) are in the middle of the rankings. Teenagers in these countries exaggerate less about their prowess, on average, than young people in Canada and the United States – by a magnitude equivalent to an effect size of around 0.15 to 0.20. However, they are also significantly more likely to overclaim than young people from Ireland, Northern Ireland and Scotland who form the final group. The average overclaiming score in these countries ranges between approximately −0.11 (Northern Ireland) and −0.21 (Scotland). Moreover, the difference between these countries and those in North America is sizeable; equivalent to an effect size greater than 0.4. Consequently, despite speaking the same language, and with a closely shared culture and history, we find important variation across Anglophone countries in teenagers’ propensity to overclaim.

A psychological profile of overclaiming

Table 3 turns to how overclaiming is linked to other items included in the PISA background questionnaire. These results are based on the pooled sample including young people from across the Anglophone countries. The top panel refers to their self-confidence in completing the eight mathematics tasks described in section 2, while the bottom panel illustrates how they view their problem-solving abilities. Figures refer to the percentage of young people who believe they could complete the task relatively easily or who believe that they have each specific problem-solving skill. The final rows provide the average score for those most and least likely to overclaim on the self-efficacy and problem-solving scales. These can be interpreted in terms of an effect size. The raw difference in estimates between these groups are then reported, along with the regression model estimates that control for demographic background, prior academic achievement and school fixed effects.

Table 3. The link between overclaiming and self-efficacy.

	Unconditional			Regression results
	Low levels of overclaiming	High levels of overclaiming	Difference	Difference	SE
Self-efficacy
Believe can could complete Task 1	78%	8%	10%	6.9%*	1.1%
Believe can could complete Task 2	72%	85%	14%	9.0%*	1.1%
Believe can could complete Task 3	57%	79%	22%	14.3%*	1.0%
Believe can could complete Task 4	78%	90%	12%	8.9%*	1.0%
Believe can could complete Task 5	78%	90%	11%	5.6%*	1.0%
Believe can could complete Task 6	37%	66%	19%	17.9%*	1.4%
Believe can could complete Task 7	62%	80%	18%	10.5%*	1.2%
Believe can could complete Task 8	40%	67%	18%	17.7%*	1.4%
Scale score (standardised)	−0.34	0.34	0.68	0.42*	0.03
Views of problem-solving ability
Can handle a lot of information	75%	89%	14%	10.3%*	1.2%
Quick to understand things	78%	90%	12%	9.2%*	1.0%
Seek explanations for things	85%	91%	6%	3.8%*	0.9%
Easily link facts together	79%	91%	12%	8.5%*	1.0%
Like to solve complex problems	50%	76%	26%	19.2%*	1.3%
Scale score (standardised)	−0.27	0.27	0.55	0.37*	0.03

Figures refer to percent of young people who agree or strongly agree. Low levels of overclaiming refers to young people in the bottom quarter of the overclaiming scale score distribution. High levels of overclaiming are defined as the top quartile. Regression estimates refers to the difference between low and high overclaiming groups controlling for gender, socio-economic status, immigrant status, PISA reading, maths and science scores and school fixed effects. The ‘scale score’ row refers to results based upon continuous index combining data across all items. This has been standardised to mean 0 and standard deviation 1, and can therefore be interpreted in terms of an effect size. * indicates that the difference between the low and high overclaiming groups is statistically significant at p = 0.05. A full list of the self-efficacy tasks can be found in section 2. Data source: OECD (2014).

Starting with the results for self-efficacy, there are substantial and statistically significant differences between the high and low overclaiming groups on the eight questions asked. For instance, while just 40% of those with low scores on the overclaiming scale were confident that they could work out the petrol consumption of a car (task eight), two-thirds of those with high overclaiming scores claimed they could do this. Moreover, a sizeable difference can still be observed in the regression model results, illustrating how those most likely to overclaim express much higher levels of self-confidence in their skills than those who are the least likely to overclaim, even when they are of equal academic ability. Specifically, the difference in the average self-efficacy scale score is over 0.4 standard deviations; a large and statistically significant effect. Together, these results illustrate how young people who tend to overclaim are also likely to express overconfidence in their skills.

The lower panel of Table 3 confirms these results. When asked about their problem-solving skills, those who are most likely to overclaim are around 10% points more likely to say that they ‘can handle a lot of information’, ‘can easily link facts together’, ‘are quick to understand things’ and ‘like to solve complex problems’. Although controlling for achievement, demographics and school characteristics can explain some of the difference between the high and low overclaiming groups, significant differences remain; we continue to observe an effect size difference of 0.37 standard deviations, even after such characteristics have been controlled. This again demonstrates the overconfidence expressed by those who have a tendency to overclaim.

In Table 4 we report results for young people’s self-reported perseverance. The group most likely to overclaim are much less likely to say that they give up easily when faced with a difficult problem (29% versus 47%) and that they are put off by difficult problems (45% versus 64%). Yet they are more likely to say that they exceed expectations when faced with a difficult problem (80% versus 63% percent). In other words, those who overclaim about their knowledge are also more likely to claim to persevere when faced with challenging tasks. The difference in the average perseverance scale score between those most and least likely to overclaim is 0.52 standard deviations (0.40 once controls have been added) representing a sizeable and statistically significant effect. Table 4 therefore illustrates how overclaimers claim to persevere more with hard-to-solve problems than other groups, independent of a range of background characteristics.

Table 4. The link between overclaiming and perseverance.

	Unconditional (% agree)			Regression results
	Low levels of overclaiming	High levels of overclaiming	Difference	Difference	SE
When confronted with a problem, I give up easily	47%	29%	−18%	−12.9%*	1.5%
I put off difficult problem	64%	45%	−18%	−15.0%*	1.7%
I remain interested in the tasks that I start	79%	88%	9%	7.1%*	1.0%
I continue working on tasks until everything is perfect.	72%	84%	11%	9.8%*	1.3%
When confronted with a problem, I do more than what is expected of me	63%	80%	18%	13.9%*	1.3%
Scale score (standardised)	−0.24	0.28	0.52	0.40	0.03

See notes to Table 3. * indicates statistically significant difference between the low and high overclaiming groups at p = 0.05. Data source: OECD (2014).

How do those overclaimers tell you how they solve problems? Table 5 provides some insight into this issue by summarising how they said they would solve two routine tasks (see section 2 for further details). Interestingly, the most pronounced and statistically significant results are with respect to the most ‘socially desirable’ (or the most ‘obviously sensible’) strategy. For instance, if their mobile phone stops sending text messages, those who overclaim are somewhat less likely to say that they would press all the buttons to find out what is wrong (49% versus 56%) but much more likely to say that they would consult the instruction manual (41% versus 30%). Likewise, if they do not know the route to their destination, those who overclaim are much more likely to say that they would consult a map than other groups (70% versus 57% percent). Although we do not know what strategy these young people would actually use, Table 5 nevertheless provides some indication that those who overclaim are much more likely to say they would take the most obviously sensible approach.

Table 5. The link between overclaiming and problem solving.

	Unconditional (% agree)			Regression results
	Low levels of overclaiming	High levels of overclaiming	Difference	Difference	SE
Task 1.
I press every button possible to find out what is wrong	56%	49%	−7%	−3.7%*	1.4%
I think about what might have caused the problem and what I can do to solve it	85%	90%	6%	4.1%*	0.9%
I read the manual	30%	41%	12%	10.0%*	1.4%
I ask a friend for help	78%	75%	−4%	−1.0%	1.3%
Task 2.
I read the zoo brochure to see if it says how to get there	75%	74%	−1%	0.9%	1.2%
I study a map and work out the best route	57%	70%	13%	8.9%*	1.4%
I leave it to my brother to worry about how to get there	34%	27%	−7%	−5.8%*	1.4%
I know roughly where it is, so I suggest we just start driving	62%	59%	−4%	−1.3%	1.3%

See notes to Table 3. In task 1, participants were asked ‘Suppose that you have been sending text messages from your mobile phone for several weeks. Today, however, you can’t send text messages. You want to try to solve the problem. Which of the following would you do?’ In task 2, participants were asked ‘Suppose that you are planning a trip to the zoo with your brother. You don’t know which route to take to get there. Which of the following would you do?’ Figures refer to the percent of young people who said they would either ‘definitely’ or ‘probably’ use this problem solving strategy. * indicates statistically significant difference between low and high levels of overclaiming group at p = 0.05. Data source: OECD (2014).

Finally, do those who overclaim believe that they are popular at school? Table 6 provides some suggestion that this may be the case. The average ‘school well-being’ scale score is around 0.2 standard deviations higher for individuals who overclaim, and stays at this level even after achievement, demographic and school controls have been added. There is a particularly notable difference in response to the question ‘things are ideal at my school’, to which 74% of the group with high overclaiming scores agree (compared to 64% of those with low overclaiming scores). Therefore, although the evidence is perhaps weaker than for the previous topics considered, we nevertheless find some evidence that students who overclaim are particularly likely to believe that they are popular at school (and certainly believe they are no less popular than their peers who do not overclaim).

Table 6. The link between overclaiming and self-perceived popularity at school.

	Unconditional (% agree)			Regression results
	Low levels of overclaiming	High levels of overclaiming	Difference	Difference	SE
Left out of things at school	14%	13%	−1%	−2.0%	1.1%
Make friends easily at school	85%	89%	5%	4.4%*	1.1%
Feel like I belong at school.	74%	81%	8%	6.9%*	1.5%
Feel awkward/out of place in my school	15%	15%	0%	0.3%	1.1%
Other students seem to like me	91%	93%	2%	2.1%*	0.7%
feel lonely at school.	10%	10%	0%	−0.7%	1.0%
feel happy at school.	78%	85%	5%	6.9%*	1.3%
Things are ideal in my school.	65%	74%	9%	9.5%*	1.5%
I am satisfied with my school.	76%	83%	8%	8.2%*	1.2%
Scale score (standardised)	−0.13	0.08	0.21	0.20*	0.03

See notes to Table 3. * indicates statistically significant difference between the low and high overclaiming groups at p = 0.05. Data source: OECD (2014).

Investigations of possible alternative explanations

There are two primary threats to the validity of our interpretation of the results above. The first alternative explanation is that, rather than capturing young people’s propensity to overclaim, the three fake constructs provide evidence of a careless or extreme response style. For instance, some respondents may not be taking the questionnaire seriously, and are simply ticking the top category for every question. A second possibility is that young people’s responses are reflecting social desirability bias; that they are providing responses that they believe will be viewed as positively by others (e.g. that they know various mathematics concepts, that they work hard at school etc). Both of these possibilities could lead to a spurious correlation between our overclaiming index and the various other psychological traits investigated in the previous sub-section. Similarly, if children with certain characteristics (e.g. boys, immigrants, young people in particular countries) are more likely to provide careless or socially desirable responses, then this could explain why we observe differences between demographic groups.

To explore this possibility further, we investigate how our overclaiming index is related to young people’s responses to two other questions in the PISA background questionnaire: (a) test motivation and (b) truancy at school. Specifically, children were asked to provide the amount of effort they put into the PISA test using a zero to ten scale and how many times they were absent from school over the last two weeks. If respondents are indeed providing high responses consistently across questions – either due to carelessness, response style or social desirability – then we should observe a strong correlation between our overclaiming index and young people’s self-reported truancy and test motivation. These results are presented in Table 7.

Table 7. The association between overclaiming, young people’s test motivation and truancy from school.

	Average		Standard error
(a) Test motivation
Bottom quartile	7.52		0.03
Second quartile	7.70		0.02
Third quartile	7.65		0.03
Top quartile	7.63		0.03
Truancy from school
	Overclaiming index
	Bottom quartile	Second quartile	Third quartile	Top quartile
(b) Truancy from school
None	78%	81%	80%	80%
One or two times	17%	15%	17%	16%
Three or four times	3%	2%	2%	2%
Five or more times	1%	1%	1%	1%
Total	100%	100%	100%	100%

Notes: Figures in panel (a) refer to the average amount of effort children say that they put into the PISA test out of 10. Panel (b) provides column percentages; it refers to the number of times young people said that they skipped school for a whole day over the last two weeks. Data source: OECD (2014).

We find no evidence that the overclaiming index is related to young people’s test motivation; the Pearson correlation is 0.01 while Table 7 highlights how the average test effort reported was very similar within each overclaiming quartile. Similarly, panel (b) of Table 7 illustrates how the overclaiming scale is not associated with self-reported truancy from school. Specifically, around 80% of young people said they were not absent from school at any point during the last two weeks, regardless of how they responded to the questions which form our overclaiming scale.

Together, this provides us with reassurance that the correlations observed in the previous sub-section are unlikely to be driven by social desirability bias or other forms of careless/extreme response.

4. Conclusions

Focusing on 15-year-olds from across nine Anglophone countries, this paper has investigated the characteristics of young people who ‘overclaim’ – state that they have knowledge and expertise in three mathematics concepts that are not actually real. We find that young men are more likely to overclaim than young women, and that overclaiming is somewhat more prevalent amongst those from more advantaged socioeconomic backgrounds. This is in line with previous literature that has shown men to be overconfident in their abilities as compared to women (Niederle & Vesterlund, 2007). Compared to other countries, young people in North America are found to overclaim more than young people in England, Wales Australia and New Zealand, while those in Ireland and Scotland are the least likely to exaggerate their mathematical knowledge.

Strong evidence also emerges that those who overclaim also display overconfidence in their academic prowess and problem-solving skills, while also reporting higher levels of perseverance when faced with challenges and providing more socially desirable responses than more truthful groups. Dunlop et al. (2020) examined when individuals were most likely to overclaim knowledge. They related overclaiming to ‘faking’ in job applications and found that overclaiming was greatest when respondents were asked about job-relevant content. The young people in our study were asked about their mathematical knowledge while taking a mathematics test, which is in line with their findings.

There are of course limitations to this study. First, PISA data are cross-sectional rather than longitudinal. We therefore do not know whether overclaiming is a stable trait that can be consistently observed for an individual over time, or if it is something that changes with age (and the factors associated with such change). Likewise, the implications of overclaiming remain unclear. Making out one is knowledgeable about topic may be useful in certain situations (e.g. job interviews, negotiations, grant applications), yet the social and labour market outcomes of those who overclaim about their abilities are currently unknown. More research is needed to understand the stability of its trait and its implications for actual life outcomes.

Second, our analysis has only considered overclaiming in a single area (knowledge of mathematics concepts). Future work should consider the overlap between overclaiming with respect to different areas of life. Finally, it is important we recognise that our overclaiming scale was based upon three specific items. Ideally, future research should try to include a greater number of fake constructs to maximise precision.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

Shure acknowledges funding from the Economic and Social Research Council [grant number ES/T013850/1]

Notes on contributors

John Jerrim

John Jerrim is Professor of Education and Social Statistics at UCL.

Philip D. Parker

Philip Parker is Professor of Psychology at Australian Catholic University.

Nikki Shure

Nikki Shure is Associate Professor of Economics at UCL.

Notes

1. Our identification of those who overclaim relies upon participants’ responses to some ‘fake’ questions, as shall be discussed below. We are concerned about how well these fake constructs translate to languages outside of English, and hence focus upon the nine Anglophone countries included within the sample.

2. Sample sizes by country are as follows: Australia 9,246; Canada 13,901; England 2,685; Ireland 3,267; Northern Ireland 1,430; New Zealand 2,762; Scotland 1,901; USA 3,193; Wales 2,165. Although sample sizes differ, senate weights are applied when data is pooled across countries to ensure each nation receives equal weight.

3. As we have dichotomised participants’ responses, this is equivalent to estimating a linear probability model for each item, along with a standard OLS model for the overall scale score.

4. Following recommended practice, all models are estimated five times – once for each plausible values – and then combined.

Appendix A.

Distribution of responses to the overclaiming questions

Table A1. The distribution of responses to the mathematics constructs (row percentages).

Constructs	Never heard of it	Heard of it once or twice	Heard of it a few times	Heard of it often	Know it well – understand the concept
Exponential function	45%	16%	16%	12%	11%
Divisor	30%	20%	18%	15%	18%
Quadratic function	21%	13%	17%	21%	28%
Proper number	16%	17%	22%	23%	22%
Linear equation	11%	9%	13%	24%	42%
Vectors	36%	18%	18%	14%	13%
Complex number	22%	21%	23%	19%	15%
Rational number	17%	16%	20%	22%	25%
Radicals	31%	20%	19%	15%	15%
Subjunctive scaling	64%	16%	11%	6%	3%
Polygon	13%	9%	13%	20%	44%
Declarative fraction	59%	18%	13%	7%	4%
Congruent figure	37%	15%	14%	13%	21%
Cosine	36%	8%	8%	13%	34%
Arithmetic mean	42%	15%	13%	12%	18%
Probability	5%	5%	8%	19%	62%

Figures refer to row percentages such that each row sums to 100.

Table A2. Responses to the fake item “proper number” by quartiles of the overclaiming index (column percentages).

	Bottom overclaiming quartile	Quartile 2	Quartile 3	Top overclaiming quartile
Never heard of it	59%	0%	12%	3%
Heard of it once or twice	41%	0%	22%	6%
Heard of it a few times	0%	40%	27%	21%
Heard of it often	0%	32%	25%	31%
Know it well – understand the concept	0%	28%	14%	40%

Table A3. Responses to the fake item ‘declarative fraction’ by quartiles of the overclaiming index (column percentages).

	Bottom overclaiming quartile	Quartile 2	Quartile 3	Top overclaiming quartile
Never heard of it	100%	100%	27%	3%
Heard of it once or twice	0%	0%	58%	17%
Heard of it a few times	0%	0%	14%	39%
Heard of it often	0%	0%	1%	27%
Know it well – understand the concept	0%	0%	0%	15%

Table A4. Responses to the fake item “subjunctive scaling” by quartiles of the overclaiming index (column percentages).

	Bottom overclaiming quartile	Quartile 2	Quartile 3	Top overclaiming quartile
Never heard of it	100%	100%	49%	7%
Heard of it once or twice	0%	0%	43%	23%
Heard of it a few times	0%	0%	6%	38%
Heard of it often	0%	0%	1%	22%
Know it well – understand the concept	0%	0%	0%	10%