Students’ Self-reported Background SES Measures in TIMSS in Relation to Register SES Measures When Analysing Students’ Achievements in Sweden

ABSTRACT

The overall aim of this study was to examine the different self-reported students’ socioeconomic status (SES) measures in the Trends in Mathematics and Science Study (TIMSS) in comparison to national SES measures obtained from Swedish official registers. A further aim was to determine if the same conclusions could be drawn if different student measures were used to define SES when modelling the students’ TIMSS mathematics achievement. The overall results showed that the choice of SES measures matters. The home educational resource index and books at home from the TIMSS data base were good indicators of SES. We conclude that when one has access to SES information from official registers it is recommended to use it because these measures have less missing information compared with the TIMSS variables.

KEYWORDS:

• Socioeconomic status
• TIMSS achievement
• self-reported measures
• register-reported measures

Introduction

Socioeconomic status (SES) can be defined as one’s access to financial, social, cultural, and human capital resources. For a general discussion of different possible definitions of SES when doing research based on register data, see, for example, NFES (2015). A student’s SES has traditionally included the following components: parental educational attainment, parental occupational status, and household or family income, with appropriate adjustment for household or family composition (National Center for Educational Statistics, 2012). In the international large-scale assessment Trends in Mathematics and Science Study (TIMSS), SES is measured by different indicators of the students’ home background, of which home possessions (e.g., having a computer at home, Internet connection, a study desk, and one’s own room), are one indicator. Furthermore, a number of books at home, parental level of education, and spoken language at home are used as indicators of the students’ SES (Broer et al., 2019a; Mullis & Martin, 2013).

An aggravating factor with SES studies based on TIMSS data is that different researchers select different variables from TIMSS to define students’ SES. Another aggravating factor is that TIMSS SES data are based on students’ self-reported data, which might affect the results. This paper contributes to the knowledge of how to perform research on secondary register-based data and how to use self-reported SES data by examining the quality of the self-reported measures of students’ SES in terms of the association between different home background variables and test performance on TIMSS. This is important because TIMSS is distributed to more than 500,000 students in grade 4 and grade 8 in about 60 countries every fourth year (Mullis et al., 2016), and the results are often used to underpin educational investments. This is a unique study in that it combined students’ TIMSS mathematics achievements with other school achievement measures using national register data as well as different measures of the students’ home background. The register data were retrieved from Sweden’s official national registers and contained information about the students’ performance on national school measures, their migration status, and their parents’ educational background. Although the study was conducted in a Swedish context, the results should be of interest for other countries because many countries participate in TIMSS and they might experience similar problems with students’ self-reported SES measures.

SES and Students’ Educational Achievements

Over the years, much research has been conducted focusing on the relationship between educational achievement and students’ home background in terms of SES. Students with high SES background tend to have higher academic achievements than students from low SES background internationally (Erberber et al., 2015; Jurdak, 2014; Sirin, 2005; Wang et al., 2014) as well as in a Swedish context (see, e.g., Gustafsson & Yang Hansen, 2018). In recent years there have been indications of an increase in the achievement gap in relation to SES, and there has been a search for explanatory factors of this gap and a search to find strategies to prevent inequity in education (see, e.g., Broer et al., 2019b; OECD, 2018). However, SES is a complicated and somewhat elusive construct.

A problem when studying SES is that SES might co-vary with other individual factors such as gender (see, e.g., Schoon & Eccles, 2014) and migrant background (see, e.g., Brese & Mirazchiyski, 2010; Hastedt, 2016; Wiberg, 2019; Wiberg & Rolfsman, 2019) and with structural factors such as school choice (see, e.g., Yang Hansen & Gustafsson, 2019). Also, the students’ residential areas have become increasingly important for educational opportunities because educational opportunities are not evenly distributed geographically (see, e.g., Andersson et al., 2018; Ersan & Rodriguez, 2020; Fjellman et al., 2018). Possession of symbolic capital (Bunar & Ambrose, 2016) in terms of being brought up in a home with a large book exposure is a factor that has been shown to have a major impact on academic performance (Evans et al., 2014). Thus, factors in the home environment can overlap, but also transcend, SES status in relation to students’ educational achievements. It is therefore important in large-scale assessments to identify factors that can work as a substitute for what is encapsulated in traditional definitions of SES and to examine which measures of the students’ home background work best to assess SES.

SES and Research Using TIMSS data

Research with TIMSS data has shown that students with high SES in terms of more educated parents and students from more socioeconomic advantaged schools tend to perform better on average on TIMSS mathematics (e.g., Schiller et al., 2002) and TIMSS science (e.g., Caponera & Losito, 2016; Wiberg & Rolfsman, 2019). Kaleli-Yilmaz and Hanci (2016) concluded that there is a positive relationship between students’ TIMSS mathematics achievements, their high school marks, and high parental education level.

Allerup (2008, 2012) used books at home as an SES measure of parental background in research based on TIMSS data, while others have used parental education (Reimer et al., 2018). Also, the home resources for learning scale has been used in several national TIMSS reports (Bergem, et al., 2016; Swedish National Agency for Education, 2016a; Vettenranta et al., 2016).

Research studies performed using TIMSS data from several different countries have concluded that students’ SES in terms of home possessions (e.g., having a calculator, computer, dictionary, and study desk) and a number of books at home have a stronger association with the students’ achievement than school-level factors (e.g., Chiu & Xihua, 2008; Chudgar & Luschei, 2009; Wiberg, 2019; Wiberg et al., 2013). Also, SES in terms of home resources together with parents’ expectations for their child seems to be of significant importance for students’ educational outcome (Geesa et al., 2019). Ersan and Rodriguez (2020) used the home resources for learning scale as a measure of SES and concluded that it has a positive association with achievement in TIMSS mathematics. All of these studies with TIMSS data indicate that SES is important when predicting students’ educational achievements. However, as noted above, a problem when comparing students’ results from large-scale assessments is that different studies use different definitions and different operationalizations of SES. Different results due to a different choice of SES variables were visible in the study by Kaleli-Yilmaz and Hanci (2016) as they found that the relationship between TIMSS achievements was stronger when using the students’ mothers’ education level instead of their fathers’ educational level. Brese and Mirazchiyski (2010) focused on this problem when examining how family background is measured in TIMSS, the Programme for International Student Assessment (PISA), and the Progress in Reading Literacy Study (PIRLS). Nevertheless, except from using different definitions of SES, these researchers did not connect these SES measures to any additionally collected students’ SES information.

SES Data Based on Self-Reports

The use of self-reported SES data from questionnaires is based on the underlying assumption that the students have knowledge about their parents’ education level and/or have the willingness to report such data, which may not always be the case. Due to a number of reasons, the respondents may be unwilling or unable to respond accurately (Aaker et al., 2007), and this may not be evenly distributed in the population. A plausible premise is that some students are unaware of the correct answers, which may be an explanation for why some of the home background variables have a large amount of missing information in TIMSS. Another problem with self-reported students’ answers on education level or family income might also be hampered by a wish for social desirability. For example, in Olivia’s (1986) study a large portion of Hispanic students seeking financial aid overestimated their family income, even though this was detrimental for them. It is therefore of essence to identify which measures of students’ SES provide the highest quality in terms of missing data in order to avoid threats to the reliability and validity.

Gonyea (2005) gives a general overview of studies of self-reported data in higher education and concludes by giving researchers a number of recommendations. Two important recommendations are to back up the accuracy of students’ self-reported data with school records if possible and to use multiple data sources or triangulation instead of only relying on self-reported data when making policy decisions. These two recommendations are in line with our present study in which we compared TIMSS self-reported data with officially gathered data from Swedish national registers.

Aim and Research Questions

The overall aim of this study was to examine the different self-reported SES measures in TIMSS in comparison to national SES measures obtained from Swedish official registers. A further aim was to examine if the same conclusions could be drawn when modelling the students’ TIMSS mathematics achievement using different student measures to define SES. This study focused on the following research questions:

1. Which self-reported measures of students’ home background in TIMSS provide the highest quality with regard to missing data in comparison to register information?

2. Which self-reported measures and register measures of students’ home background have a substantial association with TIMSS mathematics achievement and Swedish national achievement measures?

3. Which of the different self-reported and register SES measures can be used to explain the greatest amount of variance in Swedish students’ TIMSS mathematics achievement?

4. Are the predictive effects for Swedish students’ TIMSS mathematics achievement different if different self-reported and register SES measures are used?

Our hypothesis was that register SES measures contain fewer missing values than TIMSS SES data measures. We also expected to get different results in the prediction of TIMSS 2015 mathematics achievement depending on which student SES measures were used.

The structure of the rest of this paper is as follows. In the next section, we describe briefly the participants, the instruments, and the statistical analyses. This is followed by a results section, and the paper ends with a discussion and a conclusion about which of the SES measures should be used in the future.

Method

Participants

The participants included 4090 Swedish students distributed in 150 schools who took TIMSS 2015 in 8th grade (IEA, 2017). The general TIMSS selection procedure was used to select the Swedish students. In brief, this means that the students were randomly selected in a two-stage procedure to ensure that they were representative of all the 8th-grade students in Sweden. In the first stage, Swedish schools were sampled with probability proportional to their size from the national list of all Swedish schools that are eligible to participate according to the TIMSS selection criteria. In the second stage, one class were selected from the eighth grade of each of the selected schools in the first stage. The Swedish students’ participant rate in TIMSS is in general high, and only about 5% of the originally sampled students did not take part in TIMSS 2015. The students who did not participate in TIMSS 2015 were out of school on the day TIMSS was given at their school according to the Swedish National Agency of Education. When the TIMSS 2015 assessment was distributed to the students, the Swedish students’ social security numbers were also collected by the Swedish National Agency of Education. This collection allowed for a unique possibility to link the information about the Swedish students’ performance in TIMSS 2015 with official Swedish register data.

Sweden has a long history of collecting information about its citizen and keeping this information in official registers. As a citizen you can always get a register extract about yourself, and most of the information is available for the public on an aggregated group level. If you want to get information about specific individuals, this can be made available after ethical review. In this study, we received ethical approval to be allowed to connect the individual students’ TIMSS 2015 data with some official register data. In order to connect data from different official Swedish registers, the social security numbers of the persons of interest are used. In this study, we used Swedish official register data that contained information about the students’ national school measures in terms of subject grades from school years 6 and 9, grades on their national tests in school years 6 and 9, the students’ migration status, and the students’ parents’ education level.

TIMSS mathematics Achievement and Students’ Home Background SES Variables

To limit the time and effort the students devote to answering the TIMSS items, only a portion of all the assessment items are administered to each student. The students’ scores on these items are then transformed into five plausible imputed values representing the students’ mathematics achievements on the entire assessment had they answered all of the TIMSS mathematics items. These five plausible values were used in all of our statistical analyses following the suggested methods and procedures for TIMSS 2015 (Martin et al., 2016) and in line with suggestions on how to use plausible values in secondary analyses (Laukaityte & Wiberg, 2017; von Davier et al., 2009). In the following two subsections, we describe a number of available student home background variables from the TIMSS 2015 assessment and from the Swedish official registers.

Student-Reported Variables from TIMSS

The examined TIMSS sample contained 2104 (52%) boys and 1975 (48%) girls. Different students’ home possessions from TIMSS were examined, and they appear in the left part of Table 1. We also examined immigration variables, which are displayed in the upper right part of Table 1. We did not use language of testing because all test takers answered the TIMSS items in Swedish. We examined the available home possessions, including the number of books a student has at home. Most of the digital home possession variables and those describing study facilities are not that meaningful to use in the Swedish context because most students have them (see Table 1). There were some nation-specific measures in TIMSS 2015 (globe of the earth, a piano, and musical instruments) and they were included for completeness, but they were not used later because they are not used on every testing occasion. We also included the Home Educational Resources (HER) index, which is part of the TIMSS international data base. The HER index consists of a number of books at home, a number of home study supports in terms of own room and Internet connection, and the highest level of education of either parent (see Mullis et al. (2016) for details about the HER index). In the later analyses we also at one point used a categorized version of the HER index (labelled HERC) where HER was categorized into five different categories in order to get a similar scale to the other measures. We chose five categories because a number of books at home had five categories and is part of this index. According to the self-reported TIMSS measures, the sample contained 20% non-native-born students or students who had non-native-born parents, 46% who had mothers with an education level higher than high school, and 37% who came from homes that had more than 100 books. Furthermore, we examined the highest education level of the mother, father, and both parents. As seen in the right part of Table 1, most students in the sample were born in Sweden or had parents born in Sweden.

Table 1. To the left are valid cases, number of students (n), and proportion (%) of students who had each home possession among the valid cases. To the right are valid cases and the number of students who were considered to be immigrated (Imm) or native both in TIMSS 2015 and in the register data.

Home possession	Valid	N	%	Immigration	Valid	Imm	Native
Tablet own	4048	3644	90	TIMSS
Tablet share	4037	2917	72	Parents	4022	1249	2773
Study desk	4050	3860	95	Student	4036	483	3553
Own room	4050	3780	93	Mother	4037	945	3092
Internet	4046	4001	99	Father	3991	952	3079
Own mobile	4050	4010	99	Register
Gaming system	4050	3950	98	Parents/Student	4019	814	3205
Earth globe	4026	1469	36	Parents		361
Piano	4011	1380	34	Students		453
Music instrument	4038	2741	68

Notes: Valid = Number of students with a record or answer among the sampled 4090 students. Tablet = Computer tablet, Internet = Internet connection at home, Imm = Not born in country. Native = Born in country, Parents = If either of the parents have an immigration background, Student = Student’s immigration background, Mother = Student’s mother immigration background, Father = Student’s father immigration background. Parents/Student: If either of the parents or the student had an immigration background.

Register Variables

Similar to the TIMSS data set, there were 48% (1947) girls and 52% (2073) boys in the register sample. We used three different variables to describe the family background (or SES), namely the mother’s highest education level, the father’s highest education level, and the highest education level of both of the parents. We also had information that allowed us to create an immigration variable for the students as seen in the lower right of Table 1. A total of 80% (3205) of the students were born in Sweden with Swedish parents, and 20% (361 + 453) were either born outside Sweden or had at least one parent born outside Sweden.

Instruments

The Swedish Grading System and National Test in Mathematics

The Swedish grading system is criterion-referenced, and the grading scale comprises six levels, A–F, where F is failed and A–E are different levels of passing with A being the top grade. The grades are determined by the teacher at the end of a course or at the end of a school semester by comparing the students’ knowledge to the grade rubrics. No grade is given to students if too little is known about the students’ knowledge. The letter grades correspond to the following numeric grading scale: A = 20, B = 17.5, C = 15, D = 12.5, E = 10, F = 0 (Swedish National Agency for Education, 2017). The Swedish national tests are given in school years 3, 6, and 9 in different core subjects and are comprised of different subtests and are based on the national curriculum and syllabus. The aim of the national tests is to provide information about how the knowledge requirements are fulfilled on a school level and on a national level in order to support an equal and fair grading process (Swedish National Agency for Education, 2016b). In school year 6, the national mathematics test consists of five subtests, and in school year 9 it contains four subtests. The oral subtest is given to groups of 3–4 students and takes about 20–30 (grade 9) or 60–80 min (grade 6) to complete. The three (grade 9) or four (grade 6) written subtests are administered separately and are given on certain tests days. The written subtests contain multiple choice, short answer, and complete solution items. In this study, we had access to the students’ grades and national test scores in mathematics from school year 6 and school year 9. In Tables 2 and 4, the grades from year 6 and year 9 are referred to as G6 and G9, respectively, and the grades from the national tests from year 6 and 9 are referred to as NT6 and NT9, respectively.

Table 2. Valid cases, missing cases (Miss) and correlations between SES measures and TIMSS and Swedish national achievement measures (G9, NT9, G6 and NT6).

SES measure	Valid	Miss	TIMSS	G9	NT9	G6	NT6
TIMSS
Number of home possessions	3938	152	.27	.25	.25	.21	.21
Number of books at home	4026	64	.41	.37	.36	.33	.34
HER index	4030	60	.42	.38	.37	.34	.35
Parents highest education	2460	1630	.24	.27	.26	.23	.25
Mother’s highest education	2228	1862	.27	.29	.28	.25	.26
Father’s highest education	2135	1955	.25	.29	.28	.21	.23
How far student expect to go	3879	211	.30	.38	.33	.28	.29
Immigration	4022	78	−.19	−.10	−.11	−.14	−.11
Register
Parents highest education	4013	77	.39	.39	.37	.34	.34
Mother’s highest education	3943	147	.38	.36	.34	.32	.32
Father’s highest education	3830	260	.37	.38	.36	.31	.31
Immigration	4019	71	−.22	−.11	−.13	−.15	−.12

Notes: G9 = Grades in school year 9, NT9 = Grades on national test in school year 9, G6 = Grades in school year 6, NT6 = Grades on national test in school year 6. HER index = Home educational resources index

Table 4. Linear regression coefficients with TIMSS mathematics achievement as the dependent variable. Independent variables were students’ SES variables from either TIMSS (second column) or registers (third and fourth column) or both (fifth and sixth column) and the students’ grades and national test results from school year 6 in mathematics. The “SSES” columns (columns 2, 3, and 5) show the results when only students’ SES measures were used, and the “All” columns (columns 4 and 6) show the results when students’ SES home background variables, students’ grades, and students’ national test results were used. The standard errors are in parentheses, and the number of cases used (N) and the explained variance (R²) are given.

	TIMSS	Register		TIMSS & Register
Regression variables	SSES	SSES	All	SSES	All
Intercept	333.4 (12.8)	468.9 (3.3)	347.2 (5.2)	340.4 (13.4)	343.9 (5.2)
Grades
G6			5.2 (.6)		4.7 (.7)
NT6			4.8 (.6)		4.9 (.7)
Register
Medu		28.8 (3.1)	8.9 (2.6)	14.3 (3.2)
Fedu		33.1 (2.8)	16.3 (2.3)	19.5 (2.9)	13.2 (2.3)
Imm		−26.3 (5.8)	−19.8 (4.5)	−23.0 (5.6)	−21.1 (4.7)
TIMSS
Fedu	21.9 (2.9)
Imm	−19.1 (4.3)
Expect	29.6 (3.4)			29.5 (3.3)	16.4 (2.4)
Book					15.4 (2.2)
HER	12.8 (1.2)			11.3 (1.3)
R^2	.23	.14	.51	.24	.54
N	3537	3703	3538	3517	3370

Notes: TIMSS = TIMSS variables. Register = Register variables. Grades = Register variables about students’ school achievements. G6 = Students’ grades from school year 6, NT6 = National test score grade from school year 6. Expect = 1 if student expects to go further than high school education, 0 otherwise. Medu = 1 if student’s mother has a higher education than high school education, 0 otherwise. Fedu = 1 if student’s father has a higher education than high school education, Imm = 1 if student or student’s parents is not born in Sweden. Book = 1 if student’s home has more than 100 books, 0 otherwise. HER = Student’s home educational resource index from TIMSS. R²= Proportion variance explained. N = Number of students used in the regression.

Analyses

Missing Values, Comparisons, and Statistical Analyses

We started by examining the amount of missing data in the different home background variables. A commonly used strategy is to exclude students who have missing values for any variable included in the analysis (Alison, 2002). A problem with excluding whole cases is that the effective sample size decreases, and this might influence the results because the sample might lose its representativeness and the estimates might be biased, and thus the wrong statistical conclusions might be drawn that might affect the reliability and validity of the results (McKnight et al., 2007). To be able to use more cases in the different analyses, we used pairwise exclusion, i.e., we included all available values for a particular variable and thus the cases that were included varied in the analyses depending on which variables were used.

The TIMSS 2015 mathematics achievement data were used both separately and together with Swedish official register data regarding parental education level and students’ and parental immigration background. We also examined the association of both self-reported and register-based SES measures of the students’ home background with national test scores and grades from the Swedish school system for year 6. We calculated the correlation coefficients in order to examine the relationship between school measures and TIMSS achievement divided by the different groups. We used linear regressions with the different SES measures when explaining TIMSS 2015 mathematics achievement as well as national Swedish achievement measures in mathematics in terms of subject grades and national test grades. As stated in the introduction, previous research has shown that very few school-level factors can be used to explain variance in TIMSS in Sweden, and thus we restricted our study to student-level variables and did not conduct any multilevel analyses.

The background variables have previously been used in the Swedish context, and to facilitate comparisons with other research studies in the Swedish context we recoded some of the TIMSS variables. How far a student expects to go was recoded to Expect = 1 if the student expected to go further than high school education, 0 otherwise. The TIMSS original scale for this variable ranged from 1 to 6 and was used (ExpectO) when comparing the average TIMSS achievement for different levels (Table 3), and the recoded variable was used in the other analyses. Mother’s education level MeduO, father’s education level FeduO, and parents’ highest education level PeduO were used with their original scales, ranging from low educational level (1) to high educational level (7), when comparing the average TIMSS achievement for different levels (Table 3). For the other analyses that were performed, the mother’s education level and father’s education level were also recoded as Medu (or Fedu) = 1 if the student’s mother (or father) had higher than high school education, 0 otherwise. These educational recoding choices were made because the majority of students in Sweden go to high school and we wanted to single out those who went further. The immigration variable was coded as Imm = 1 if the student or either of the student’s parents were not born in Sweden. Books in the home were originally coded as 0–10 (14.4%), 11–25 (20.5%), 26–100 (27.7%), 101–200 (18.3%), >200 (18.7%) with the percentage of students in parenthesis, and this was recoded as Book = 1 if the student’s home had more than 100 books, 0 otherwise. Note that the original coding was used and named BookO when comparing the levels of SES measures with average TIMSS achievement (in Table 3), otherwise the recoded variable was used. The recoding was done because we wanted to have an indicator for students with high symbolic capital in terms of many books, thus with our recoding 37% were placed in the high book category.

Table 3. Average mathematics achievement (first row) and a number of cases (second row) depending on the category (columns 1–7) within the different students’ home background measures. The lowest level for each measure is 1, and 5, 6, or 7 is the highest level depending on which measure of students’ home background is used.

	Level of students’ home background
	1	2	3	4	5	6	7
TIMSS variables
PeduO	470.0	471.4	490.5	509.1	526.1
N = 2460	45	101	402	589	1323
MeduO	449.0	471.2	494.6	507.4	516.2	525.6	534.2
N = 2228	59	104	418	198	392	650	407
FeduO	480.6	480.2	501.4	506.8	514.8	531.5	532.9
N = 2135	54	186	492	208	316	445	434
ExpectO	404.4	472.2	479.3	503.0	517.7	521.9
N = 3879	63	657	282	733	1227	917
BookO	451.0	479.6	504.7	527.8	539.4
N = 4026	580	839	1116	737	754
HERC	449.0	459.5	492.9	515.1	543.3
N = 4030	133	625	1554	720	998
Register variables
PedcO	416.0	443.4	471.3	486.5	508.0	527.4	559.8
N = 4013	85	98	627	873	744	1478	108
MedcO	432.6	457.5	483.0	494.3	515.3	531.5	574.5
N = 3943	152	212	818	885	619	1213	44
FeduO	433.3	469.7	487.7	502.5	522.3	538.8	560.9
N = 3830	125	343	1228	640	650	762	82

Notes: PeduO = Parents highest educational level, original coding, MeduO = Mothers’ highest educational level, original coding, FeduO = Fathers’ highest educational level, original coding ExpectO = Student expectations of how far they will go in the educational system, original coding. BookO = Number of books at home, original coding, HERC = categorized HER index.

The final models were chosen by the principle of parsimony, and our aim was to find the model that explained the most variance with the fewest variables using only self-reported TIMSS SES variables, using only register-based SES variables, using register SES variables and national achievement measures, and using all available variables from both TIMSS and the registers.

Results

Missing values in home possessions were in general low, as seen in Table 1, where the number of valid cases should be compared with the total sample size of 4090 Swedish students. In general, missing values were higher when using self-reported variables in TIMSS than when using register-reported variables, especially for parental education level and migration.

Correlations with achievement measures varied depending on which measures were used, as seen in Table 2. In Table 2 we also include the number of missing values for each SES measure in order to illustrate how the missing data varied for the different variables. The amount of missing data was in general low, except for self-reported parental education level. Note that the size of the correlation coefficients differed a lot between the different SES measures. Parental education in particular had a stronger association with achievement when more information was used as with the register-based measures. Note also that books at home and the HER index from the self-reported TIMSS measures gave similar correlations with the achievement measures as the register-based variable of parents’ highest education.

We also examined the intercorrelations between register data and self-reported TIMSS SES measures, but instead of displaying them in a separate table, they are given within parentheses in the following text. The highest intercorrelations were found between the register SES measure of parents’ highest education and the TIMSS SES measures of parents’ highest education (.61), HER index (.60), mother’s highest education level (.58), father’s highest education level (.49), and a number of books at home (.43). Within the TIMSS SES measures, the highest intercorrelations were found between the HER index and number of books at home (.89), parents’ highest education level (.70), mother’s highest education level (.63), and father’s highest education level (.57). There were also high correlations within parental education level and mother and father’s education level. Finally, a number of books at home and mother’s education level (.36) had a reasonably high intercorrelation. The intercorrelations with TIMSS mathematical achievement and the national achievement measures were similar for the TIMSS SES measures of books at home and the HER index.

Table 3 shows the parents’ highest education from self-reported and register-reported measures in relation to the average TIMSS achievement within each group from the lowest group to the highest group within each measure of students’ SES. If we compare the overall results in Table 3, we can conclude that regardless of which measure of students’ SES is used, the students’ average TIMSS mathematics achievement is higher for higher SES levels. The only exception was within the TIMSS measure of father’s education level, where the score for the second level was slightly lower than the score for the first level. When examining this variable more closely, we found that there were very few persons in the lowest category, and the standard error for this estimate was actually almost twice the size (12.4) of the standard error of the second category (6.6), which was about the same size as the standard errors for the rest of the categories.

From Table 3 we can conclude that self-reported measures about the students’ parents’ education level seem to be difficult for the students to report because there were many missing values in the self-reported TIMSS variables as compared with the register-based values. Note that the number of categories used differed due to what information is gathered in the official Swedish registers and in TIMSS. The official Swedish registers provide information on education level in seven categories, while the TIMSS data base gives seven categories for mothers’ and fathers’ highest education level but only five categories for the highest parental education level. Also, students’ expectations have six categories while a number of books at home have five categories in the TIMSS data base. Note that the HER index from TIMSS, which we categorized, could of course also have been categorized with either six or seven categories. However, because a number of books at home are part of the HER index and this variable originally only had five categories, we chose to divide the HER index into five categories. Regardless of whether five, six, or seven categories were used for the HER index, the overall conclusion was the same, and thus the uncategorized HER index was used in subsequent linear regressions.

Table 4 shows different linear regressions aiming to explain students’ TIMSS mathematics achievements using the students’ SES home background measures. The best model with significant variables in terms of explaining the most variance was the model using variables from both TIMSS and register data (.54) followed by the model using register SES variables together with national achievement measures (.51), i.e., grades and national test scores. Only using TIMSS self-reported SES variables (.23) gave a better model than only using the available register SES variables (.14). However, the largest improvement was achieved when grades and national test scores were part of the model. The general parental education variable was not used because it was never significant when register variables were used nor when student self-reported TIMSS variables were used. Mothers’ education level from TIMSS was never significant, and fathers’ education level from TIMSS was only significant if we only examined student’ self-reported SES variables from TIMSS. These two variables’ poor performance was probably due to a large amount of missing values (see Table 2). Interestingly, the self-reported book variable was never significant. Not surprising, however, was the fact that the HER index and the book variable were significant when analysed separately, but not when analysed together, because they partly contain the same information.

Discussion

The first research question asked about which self-reported TIMSS SES measure(s) provided the highest quality in terms of low levels of missing data in comparison to Swedish official register information regarding the students’ SES. Not surprisingly, and in line with Brese and Mirazchiyski (2010), TIMSS self-reported home possessions, including a number of books at home, had, in general, a low amount of missing data, probably because it is easy for the students to report what they have at home. Also, the HER index had a low amount of missing data, which is a similar result as in the study by Ersan and Rodriguez (2020). However, TIMSS parental education level variables had large amounts of missing values, which was probably due to students not knowing the correct answer. This result is in contrast to the study by Brese and Mirazchiyski (2010) where these variables only had a small amount of missing data. Our results might be related to the Swedish context and the role of education in Swedish society. In contrast to many other countries, access to education is free of charge in Sweden. Thus, a child’s education is not directly linked to parents’ education level or family income. This might be an explanation for the lack of awareness regarding their parents’ education level. Interestingly, the students seemed to have great difficulties in reporting whether their parents were born within the country or not because their self-reported immigration information differed quite a bit from the register information. Not surprisingly, the students seemed to be quite sure about their own immigration status.

The second research question concerned which of the students’ self-reported and register-based SES measures have substantial associations with TIMSS mathematics achievement and Swedish national achievement measures. The SES register measure of parents’ highest education level correlated relatively strongly with TIMSS mathematics achievement and national achievement measures. However, more information was obtained when using both mother’s and father’s highest education level in the linear regressions. As for the TIMSS self-reported measures, books at home and the HER index gave similar positive correlations with TIMSS achievement and the national achievement measures as the register SES measure of parental education level. The fact that books at home had high correlation with TIMSS achievement is in line with findings in several other studies (e.g., Brese & Mirazchiyski, 2010; Geesa et al., 2019) and is probably a reason why it has worked well when modelling TIMSS mathematics achievement in the past (see, e.g., Chiu & Xihua, 2008; Chudgar & Luschei, 2009; Wiberg, 2019) and in studies based on PISA data (see, e.g., Evans et al., 2014).

A critical question for the future is if books at home can be viewed as a constant variable, because other media tend to compete for a family’s attention today, and if it might even be a more stringent criterion in a society with fewer books. In line with findings from previous studies, our answer to this question is probably yes. Books at home and the HER index were also very highly intercorrelated, which is not surprising because a number of books at home are part of the HER index. The fact that both the HER index and books at home worked well is good news because parents’ highest education level using the TIMSS self-reported measure had many missing values and thus is not very useful. Our conclusion is that the HER index and books at home can be seen as indicators of SES, and our recommendation is thus to use either of these variables to represent SES if one does not have access to additional register information.

It was notable that all of the examined students’ SES measures had the same structure in terms of higher-level categories tending to have higher average TIMSS mathematics achievement. This result is probably a reason why different studies can use different student SES measures and still reach the same overall conclusions, although the size of the coefficients may differ; for example, our overall results from the linear regressions are in line with the results of Wiberg (2019) even though the coefficients differ.

The third research question concerned which of the student self-reported and register-based SES variables could explain the most variance in TIMSS mathematics achievement. From the linear regressions we conclude that it was most useful to use both information from registers about the students’ earlier performance (in terms of grades and national test results) and self-reported information from TIMSS and information from official registers to obtain the best model. This was especially true for the mother’s and father’s education level because the TIMSS variables contained a large amount of missing values, while the register values only had a small amount of missing values. Note that it is not surprising that the HER index and the books at home variable were never entered at the same time because the book variable is part of the HER index. Both of these variables have been successfully used as SES measures (e.g., Allerup, 2008; 2012; Bergem, et al., 2016: Swedish National Agency for Education, 2016a). Our conclusion is that it is best to use a mixture of information from both register variables and self-reported TIMSS variables, and this is in line with previous research on TIMSS mathematics (Wiberg, 2019) and TIMSS science (Wiberg & Rolfsman, 2019).

The fourth research question concerned whether the predictive effects regarding students’ TIMSS mathematics achievement are different if different self-reported TIMSS and different register-based SES measures are used. In this case, our hypothesis was confirmed, and it mattered which measure we used for students’ SES. This is a reason why these results were not exactly the same as, for example, the linear regression results in Wiberg (2019), which also included the students’ sex and either the students’ grades or results from national test scores in the final regression models.

One limitation of the present study was the chosen register variables because they were limited to similar measures as those used in TIMSS 2015. In the future, other variables should also be examined such as the parents’ income level, which is part of other SES definitions (NFES, 2015), and the parents’ occupation as suggested by Brese and Mirazchiyski (2010). Another limitation is that we only used a general immigration variable. In the future, it would be fruitful to have more information about the students and to include information from which country they or their parents have immigrated because immigration can have both positive and negative effects on students’ achievements. The year of immigration might also have an influence (Hastedt, 2016), which should be of relevance in order to prevent inequity in education. Hence, in a changing society, traditional definitions of students’ home background may no longer be sufficient to capture the elusive influence of factors associated with educational achievement. Other variables may therefore be of relevance in the future due to “new” criteria linked to factors of significance for educational outcomes, in accordance with research focusing on the uneven distribution of educational opportunities (see, e.g., Andersson et al., 2018; Bunar & Ambrose, 2016).

Some may argue that this study was too limiting because we only examined students within Sweden. Studies such as this, however, can only be performed within a country and in countries where we have access to register measures about the students. It would be very interesting if other countries were to carry out similar studies in order to examine how the students’ self-reported TIMSS SES variables work within their countries. Even though this study was only carried out within Sweden, we do believe that the results are of interest for those countries taking part in TIMSS studies. First, because it matters which measure one uses, the obtained result can shed some light on the fact that there might be slightly different conclusions from different studies depending on which student home measures they have used, and this might be of relevance for future research as well as when interpreting results as the basis for educational investments. Second, if one has access to register information about students one should consider to use it together with TIMSS data as it may give valuable information. Third, SES variables with a low amount of missing observations were more useful in our study, thus if possible one should try to lower the amount of missing information in the SES variables. For example, one could consider giving instructions to schools participating in TIMSS that their students should find out their parents’ level of education before they answer the TIMSS questionnaires.

Summing up, it is important to be aware that different SES measures may give slightly different results when students’ SES variables are used together with students’ TIMSS mathematics achievement. For future TIMSS mathematics achievement studies, we especially recommend using either the HER index or a number of books at home because both had low amounts of missing data, and similar conclusions were drawn when using them as compared to using register SES measures. Also, if one has access to official register information such as information about the students’ national achievement and the students’ parents educational background, it is highly recommended to use that as additional information. If we only base our conclusions on students’ self-reported data, without ever having examined how these self-reported data relate to other collected data, we might get flawed results. Hence, when self-reported data are used on their own, there might be large amounts of missing data or a response bias, which means that the respondent affects the quality of the measurement by introducing inaccuracy or lack of precision in the data.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The research was funded by the Swedish Research Council grant 2015-02160.Vetenskapsrådet