Disentangling the role of algorithm awareness and knowledge in digital inequalities: an empirical validation of an explanatory model

ABSTRACT

Algorithms have become almost ubiquitous on the internet. They shape the way internet users engage with internet services across diverse online domains and what outcomes they obtain from their internet uses. While scholars have extensively studied the societal implications of algorithms in the digital world, only a few studies investigated the relationship between digital inequalities and algorithm awareness and knowledge. Little is known about how ubiquity of internet access and disparities in skills affect algorithm awareness and knowledge of internet users, and how they affect their internet uses and outcomes. To fill this important gap, this study presents and tests an explanatory model to assert how algorithm awareness and knowledge are related to the three levels of digital inequalities. The data were collected with a face-to-face survey (response rate = 54%) on a representative sample of internet users (N = 802) in Slovenia in 2022. Results of path analysis confirmed the sequential paths between the three levels of digital inequalities, suggesting that ubiquity of internet access strongly determines internet skills and internet uses, which in turn affect tangible internet outcomes. While ubiquity of internet access affected only algorithm awareness, internet skills predicted both algorithm awareness and knowledge. Importantly, algorithm knowledge was a significant determinant of internet uses. Age, education, and income moderated some of the relationships in the model. Overall, the study demonstrates that research on digital inequalities and related interventions need to address algorithm awareness and knowledge, and also consider how social inequalities among internet users shape them.

KEYWORDS:

• Algorithm awareness
• algorithm knowledge
• digital inequalities
• explanatory model
• path analysis
• survey

Introduction

Digital inequalities have long been an issue of concern for researchers. Starting with the so-called digital divide, studies focused on access to computers and the internet (e.g., Helsper, 2021; van Dijk, 2020). However, the field quickly diversified to examine different types and quality of access (first-level digital inequalities) (van Deursen & van Dijk, 2019), and differences in the types and levels of internet skills and internet uses (second-level digital inequalities) (DiMaggio et al., 2004; Robinson et al., 2015). More recently, scholars have been concerned with a new dimension of digital inequalities – whether people are aware of algorithms and what they know about them (Eslami et al., 2015, 2017; Hamilton et al., 2014; Rader & Gray, 2015). In fact, Gran et al. (2021) argue that

[b]eing aware of [algorithms] and navigating consciously on the internet infrastructure could be seen as a new and reinforced level of digital divide. It is more subtle, more difficult to cope with than other skills, and at least as powerful as other skills and usage-based divisions. (pp. 1791–1792)

When examining both general awareness of algorithms, and how much participants understood about how algorithms work (i.e., algorithm knowledge), Fouquaert and Mechant (2022) showed that general awareness, but not understanding of algorithms’ inner workings, was positively associated with a more critical stance toward algorithms. Their findings support recent conceptual propositions of Dogruel (2021), suggesting that algorithm awareness and knowledge represent the two key dimensions of algorithm literacy¹ since they affect the online activities of the largest number of internet users (Dogruel, 2021; Dogruel et al., 2022). Accordingly, we will focus only on these two dimensions of algorithm literacy in this study.

Despite increasing research on algorithm awareness and knowledge (Dogruel, 2021) among users of various online platforms and services (e.g., social networking sites and search engines), only a few studies focus on these two concepts as facets of digital inequality research (e.g., Cotter & Reisdorf, 2020). As such, it is unclear whether and to what extent algorithm awareness and knowledge play a role in the sequential pathways leading from first- and second level to third-level digital inequalities, that is, the outcomes that individuals gain from using the internet (van Deursen et al., 2017).

The limited empirical evidence suggests that higher levels of internet skills are positively correlated with algorithm awareness and knowledge (e.g., Reisdorf & Blank, 2021). Likewise, a positive association was found between internet uses and algorithm awareness and knowledge. Both frequency and breadth (i.e., variety) of internet uses were positively associated with a better understanding of algorithms (Cotter & Reisdorf, 2020; Oeldorf-Hirsch & Neubaum, 2023).

The current study contributes by comprehensively examining the role of algorithm awareness and algorithm knowledge in sequential paths of digital inequalities. It builds on the model of compound and sequential digital exclusion – MCSDE (van Deursen et al., 2017) – to test how ubiquity of internet access and internet skills affect algorithm awareness and knowledge, and how algorithm awareness and knowledge, in turn, affect the variety of internet users’ engagement with the internet, resulting in different levels of achieved tangible internet outcomes. In line with MCSDE, we also investigate whether internet users’ digital inequalities interact with their demographic and socio-economic characteristics. The model was tested with path analysis using a representative subsample of internet users (N = 802) in Slovenia. The results confirm the proposed model, indicating that algorithm awareness and knowledge should be incorporated not only in conceptual models of digital inequalities but should also be part of any effective digital inclusion intervention.

Background

The MCSDE (van Deursen et al., 2017) assumes that the three levels of digital inequalities are related with sequential paths, which determine the level of digital deprivation among internet users. For instance, sequential digital deprivation is indicated when users who lack a particular digital resource (e.g., lack of skills) are also affected by shortage of another digital resource (e.g., low levels of internet use). Following the sequential logic of the MCSDE, we assume that algorithm awareness and knowledge represent two digital resources and thus can be hypothetically related to the different levels of digital inequalities (Hargittai et al., 2020). In this section, we explore how first-level digital inequalities (i.e., ubiquity of internet access) are related to second-level digital inequalities (i.e., internet skills and uses). We then examine algorithm awareness and knowledge, and how these concepts are related to digital inequalities, and in particular to skills, breadth of internet uses, and internet outcomes (third-level digital inequalities).

Internet access, uses and skills

First-level digital inequalities concern themselves with access to both devices and the internet. While access was initially conceptualized along the yes/no axis of having a computer and/or the internet, the field soon shifted to understanding access as a gradation of no access to high levels of access. In particular, Helsper (2021) pointed out the importance of ubiquity of internet access, which refers to whether someone owns a number of (different) devices and has internet access through various means, such as through an internet service provider at home, mobile access through a data plan, and at various other places, such as work, school, or public places (van Deursen & van Dijk, 2019; van Dijk, 2020). As part of access, we also have to consider the quality of a connection or a device and whether users have the means and autonomy to maintain both (Helsper, 2021). All three aspects of access have been shown to be unevenly distributed across populations, with those who are better off socio-economically having more ubiquitous and better-quality access (Robinson et al., 2015; van Deursen et al., 2017).

As researchers realized that digital inequalities are about more than just binary notions of access, they also shifted their attention toward second-level digital inequalities, focusing on digital and internet skills as well as differences in usage (DiMaggio et al., 2004). Internet skills are often higher among younger people, those who have higher educational qualifications, higher incomes, and work in occupations that utilize a lot of digital media (e.g., Scheerder et al., 2017). There has been a plethora of evidence linking access and skills – meaning that those who have more and better access opportunities also have better internet skills (Hargittai et al., 2019; Scheerder et al., 2017; van Deursen et al., 2017). In line with prior research findings and in accordance with the MCSDE, we hypothesize (Figure 1):

H1: Ubiquity of internet access positively affects internet skills.

There has been an equally vast amount of research on the different types, breadth, and amount of internet uses – the second aspect of second-level digital inequalities (Blank & Grošelj, 2014). A range of studies have shown that uses, much like access and skills, are not equal across different populations. Those who are already better off use the internet both more broadly and for more capital-enhancing types of uses (Büchi et al., 2016; Hargittai & Hinnant, 2008; Robinson et al., 2015). Additional studies have demonstrated that those with more access points use the internet for a wider range of activities than those who only access the internet through mobile phones (Reisdorf et al., 2022). Therefore, we hypothesize:

H2: Ubiquity of internet access has a positive effect on breadth of internet uses.

Additionally, various studies based on the MCSDE have demonstrated a positive link between levels of internet skills and the breadth and types of internet uses (Helsper, 2021; Petrovčič et al., 2022; van Deursen et al., 2017). Therefore, we assume that:

H3: Internet skills have a positive effect on breadth of internet uses.

Algorithm awareness and knowledge and digital inequalities

Various studies have examined to what extent internet users are aware that algorithms affect what they are seeing online (i.e., algorithm awareness) and how much they know or understand about how algorithms work (i.e., algorithm knowledge) (Dogruel, 2021).

Early studies have focused on algorithm awareness on social media (Eslami et al., 2015; Hamilton et al., 2014; Rader & Gray, 2015) and search engines like Google (Powers, 2017), often among college students. These studies showed that awareness was low among participants although it appears to be increasing over time, partly due to increased media coverage of the issue. Increased awareness was associated with stronger feelings of control over the features of the specific website and changes in use (Eslami et al., 2017; Rader & Gray, 2015). More recent investigations on algorithm awareness have diversified platforms to include social media like Instagram, TikTok, and more general awareness with reference to algorithms that users encounter in their day-to-day online dealings. Overall, these studies converge to demonstrate that algorithm awareness varies across platforms and that greater awareness often goes along with modified behaviors (Fletcher & Nielsen, 2019; Fouquaert & Mechant, 2022; Siles et al., 2022).

Accordingly, several scholars have recently argued that considering the ubiquity of algorithms across platforms, algorithm awareness and knowledge are new aspects of digital inequalities that need to be considered alongside more ‘traditional’ concepts, such as access, skills, and usage. For instance, Wei and Yan (2023) showed that users who use personal computers alongside mobile devices to go online have a higher level of algorithm awareness than mobile-only users because they can observe the consequences of algorithm curation for content recommendation across different devices. Following this evidence, we hypothesize:

H4: Ubiquity of internet access positively affects algorithm awareness.

H5: Ubiquity of internet access positively affects algorithm knowledge.

Focusing on Google Search, both Cotter and Reisdorf (2020) and Reisdorf and Blank (2021), examined US internet users’ understanding of how algorithms related to search engines work (i.e., algorithm knowledge). Both studies found that those with higher self-rated internet skills had higher algorithm knowledge (Cotter & Reisdorf, 2020; Reisdorf & Blank, 2021). Moreover, in their qualitative study of internet users in the US and four European countries (Germany, Hungary, Bosnia, and Serbia), Gruber et al. (2021) examined whether people were aware of algorithms and what they understood about them with a particular focus on voice assistants. They found that broader previous internet experience and general internet skills increased algorithm awareness and knowledge. In a separate paper based on the same data, Hargittai et al. (2019) make a strong argument about why algorithm awareness and knowledge should be measures of broader digital skills, and how future studies could account for these factors.

We concur that algorithm awareness and knowledge are part of a larger set of proficiencies needed to engage with the internet. However, we also hold that internet skills are distinct from and in fact a prerequisite for being able to encounter, make sense of, and use algorithms online (Pinski & Benlian, 2024). Indeed, internet skills relate to the ability to use the internet efficiently (van Deursen et al., 2016), while algorithm awareness and knowledge enable users to identify the presence and consequences of algorithms online (Dogruel, 2021). Hence, we position internet skills as antecedents of algorithm awareness and knowledge in the MCSDE, proposing the following two hypotheses:

H6: Internet skills have a positive effect on algorithm awareness.

H7: Internet skills have a positive effect on algorithm knowledge.

Cotter and Reisdorf (2020) as well as Lomborg and Kapsch (2020), argued that algorithm awareness affects knowledge. Without awareness, users do not have the basis for building an understanding of algorithms and developing a deeper insight into the principles and methods that underlie algorithms as well as their social and political implications. We hypothesize:

H8: Algorithm awareness has a positive effect on algorithm knowledge.

Although we understand that internet skills and uses are in a recursive relationship – i.e., they influence each other – prior sequential models have placed internet skills as a precursor for broader internet engagement (Helsper, 2021; Petrovčič et al., 2022; van Deursen et al., 2017). In this sense, algorithm awareness and knowledge play the role of mediators between internet skills and uses because, according to DeVito (2021, p. 3), they refer to ‘the capacity and opportunity to be aware of both the presence and impact of algorithmically-driven systems on self- or collaboratively-identified goals, and the capacity and opportunity to crystalize this understanding into a strategic use of these systems to accomplish said goals.’ Hence, we assume that:

H9: Algorithm awareness positively affects breadth of internet uses.

H10: Algorithm knowledge positively affects breadth of internet uses.

Tangible internet outcomes

We assume that algorithm awareness and knowledge do not have a direct association with internet outcomes, but rather that breadth of internet uses mediates the relationship between algorithm awareness and knowledge and internet outcomes (DeVito, 2021). Several studies based on the MCSDE have linked broader internet uses with more beneficial outcomes of using the internet (van Deursen & Helsper, 2015, 2018). For example, van Deursen et al. (2017) found that broader economic internet use, such as selling something online or looking for a job online, resulted in more beneficial economic outcomes, such as saving money or finding a better paid position. Although of course not all broader internet uses are beneficial and overuse can result in reduced well-being (Büchi et al., 2019; Gui & Büchi, 2021), in this study, following the MCSDE, we focus on specific capital-enhancing uses and outcomes related to digital deprivation. Therefore, we suggest:

H11: Breadth of internet uses has a positive effect on internet outcomes.

Figure 1. Explanatory model with hypothesized path relationships.

The moderating role of demographic and socio-economic characteristics

The MCSDE also suggests that disparities among internet users at each level of digital inequalities interact with their demographic and socio-economic characteristics such as gender, age, education, occupation, and income (Helsper, 2021; van Deursen et al., 2017). The level of awareness and knowledge about algorithms has also been correlated with these demographic and socio-economic variables (e.g., Cotter & Reisdorf, 2020; Dogruel et al., 2022; Gran et al., 2021; Wei & Yan, 2023; Ytre-Arne & Moe, 2021). For instance, Gran et al. (2021) found differences in algorithm awareness by age, education, and gender, with younger internet users, those with higher education, and men being more aware of algorithms. Moreover, Wei and Yan (2023) showed that the level of algorithm awareness was positively correlated with occupation and income. However, since the effect of personal and positional characteristics of users can vary across sequential paths of digital exclusion (van Deursen et al., 2017; van Dijk, 2020), we pose the following RQ:

RQ1: How do the hypothesized relationships in the proposed explanatory model interact with gender, age, education, occupation, and income differences among internet users?

Methods

Procedures and data

The data used in this study were collected from the 2022 wave of the Slovenian Public Opinion Survey. A face-to-face survey was conducted between April and August 2022. Participants aged 18 + years were selected from the Central Register of Population (CRP) using two-stage random sampling with stratification by type of settlement and the statistical region of residence. The survey was completed by 1001 respondents, yielding a response rate of 54% after non-eligible units were excluded (AAPOR, 2016). The socio-demographic characteristics of the respondents closely mirrored the characteristics of the general population when compared with data from the CRP. Ethical review and approval were waived for this study by the Ethics Committee of the Faculty of Social Sciences, University of Ljubljana, as the questionnaire did not cover any sensitive issues and did not collect any personally identifiable data. In line with the General Data Protection Regulations, informed consent was obtained from all respondents involved in the study.

The sample characteristics are presented in Table 1. In this study, we analyzed a subsample of respondents who had used the internet in the last three months (N = 802; 80.1%).

Table 1. Socio-demographic characteristics of the sample.

Variable	Categories	Total sample (%)^a	Internet users (%)^b
Gender	Male	45.5	45.9
	Female	54.5	54.1
Age^c	30 years or less	15.6	19.1
	31–45 years	21.9	26.8
	46–60 years	25.0	28.8
	61 years or older	37.6	25.3
Education^d	Primary basic education or lower	14.2	7.6
	Vocational or technical secondary education	19.5	15.8
	General upper secondary education	32.4	35.3
	Higher education	33.6	40.9
Employment status	Working	51.9	62.8
	Not working	42.5	30.4
	Student	5.4	6.6
Occupation^e	Low-skill job	53.9	47.9
	High-skill job	37.2	43.5
Income^f	Low-income (below average)	44.2	44.6
	High-income (above average)	39.7	38.2

Notes: ^aN = 1001. ^bN = 802. ^cM (total sample) = 52.5 years; SD = 18.3 years, M (internet users) = 47.7, SD = 16.2. ^dEducational level reported according to the ISCED 2011 classification (UNESCO, 2012). ^eThe skill content of jobs was coded in accordance with the fourth iteration of the International Standard Classification of Occupations – ISCO-08 (ILO, 2016). ^fAverage individual net monthly income for the total sample was 979.89€ and for internet users 1081.97€. Total percentages may not add up to 100 because of rounding and/or missing values.

Measures

Internet outcomes were measured on a 10-item inventory adapted from Helsper et al. (2015) and validated in Slovenia by Petrovčič et al. (2022). The inventory asserted four types of achieved tangible outcomes that the respondents performed online for private purposes in the previous 12 months: economic, cultural, social, and personal (Table OS1 in Online Supplement). Each item was measured on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). The respondents were also provided with the possibility of choosing the ‘Not applicable’ response option (recoded to missing values in the analysis) if they did not undertake the activity in question. Since internet outcomes were defined as a formative construct, we conducted a principal component analysis (PCA) to extract a single component for all items (Petter et al., 2007). All items had > .50 correlation with the component, indicating that all items sufficiently contribute to the construct (Hair et al., 2018). The scale was not affected by multicollinearity (max[VIF] < 2.0; Table OS1). Accordingly, the scores on the internet outcomes scale were calculated by averaging across the 10 items with higher score values indicating higher achieved tangible outcomes.

Breadth of internet uses was assessed using a 17-item inventory adapted from Blank and Grošelj (2014) and validated by Reisdorf et al. (2021) in Slovenia. Four items assessed cultural, social, and personal uses respectively, and five items measured economic uses. All items were scored on a 6-point ordinal scale ranging from 1 (never) to 6 (several times a day). Since the aim of this scale is to capture the variability – rather than intensity – of internet uses, all answers were recoded into a dichotomous answering scale (‘never’ = 0; any other response = 1), reflecting whether the internet was used for a particular activity or not (Blank & Grošelj, 2014). As internet uses have been defined as a formative construct (Reisdorf et al., 2021), the same validation procedure as for internet outcomes was undertaken. The PCA indicated that the 17 items sufficiently contributed to the construct (Table OS2). There were no issues with multicollinearity (max[VIF] < 2.2). Accordingly, a single formative composite score of internet uses was obtained by calculating the average number of activities per dimension, and then averaging across all four dimensions. Users with a higher composite score engaged in a higher percentage of different online activities.

An adapted short version of the Algorithm Literacy Scale – ALS (Dogruel et al., 2022) was used to assess algorithm awareness and knowledge. Five items were used to measure awareness and five for knowledge. Items measuring awareness asked for which purposes and in what (media) products algorithms are used. Items asserting knowledge tested participants’ understanding of how algorithms work and what implications they can have for users (Dogruel et al., 2022). All items were formulated as a test with true-/false-statements. Correct answers were coded as 1 whereas incorrect answers, including ‘I do not know’, as 0 (Dogruel et al., 2022). Both scales were validated by using the Rasch model (Emberstone & Reise, 2013). We first examined the difficulties of the items. For items measuring algorithm awareness difficulties ranged from P = .39 to P = .64, while for algorithm knowledge from P = .08 to P = .52. We then fitted a Rasch model on both sets individually. Based on item fit statistics and local dependence we excluded one item from each scale (Table OS3). The modified 4-item scales were good fitting with adequate reliability (awareness: KR20 = .83; knowledge: KR20 = .75). The scale scores for both dimensions of the short ALS were obtained by summing the respondent’s correct answers. Higher values of scores indicated higher levels of algorithm awareness and knowledge.

Internet skills were assessed using a short form of the Internet Skill Scale – ISS (van Deursen et al., 2016), validated in Slovenia by Grošelj et al. (2021). In this study, ISS was considered a second-order factor with four first-order dimensions corresponding to operational, information navigation, social, and creative skills (Grošelj et al., 2021). Each dimension was measured with five Likert-type items. The respondents answered on a scale from 1 (not at all true of me) to 5 (very true of me) and included an ‘I do not understand what you mean by that’ response option which was recoded to missing values. Items measuring information navigational skills were reversed before the analysis. Following a confirmatory factor analysis (CFA), we dropped three poorly performing items. The second-order model indicated sufficient fit to the data χ²_(df) = 424.265₍₁₁₅₎, p < .001, CFI = .947, RMSEA = .068 (90% CI = .061–.075), SRMR = .057 (Table OS4 and Figure OS1) and had adequate reliability (Cronbach’s alpha = .90; Composite reliability = .79). Accordingly, the items for each dimension were averaged to obtain subscale scores, and then the mean of the subscale scores was calculated to obtain a single scale score with higher values indicating higher level of internet skills.

Ubiquity of internet access was assessed using an adapted Material Internet Access scale (van Deursen & van Dijk, 2015). Respondents were asked how often they used each of the following five devices to access the internet: desktop or laptop PC, tablet PC, smartphone, smart TV, other internet-based devices such as game console or electronic reader. They provided answers on a 6-point ordinal frequency-based scale ranging from 1 (never) to 6 (several times a day). All answers were transformed into a dichotomous answering scale, reflecting whether each of the five devices was used to access the internet. All recoded items were averaged into a single score that reflects the number of devices used to access the internet with a higher value of the score indicating more ubiquitous access.

Data analysis

Hypotheses in the model were tested with path analysis (Kline, 2015), since some concepts were defined as formative constructs (i.e., internet uses, internet outcomes). Before conducting the path analysis, we screened the data for missing values, variable distributions, and their bivariate correlations (Table 2). As 347 (43.3%) responses had at least one missing value, the model was estimated by using the full information maximum likelihood (FIML) to avoid dropping such units (Kline, 2015). FIML was selected as the variables in the model can explain each other’s missingness (Table OS5) and because FIML provides less biased and more powerful estimates than other missing data techniques (Newman, 2014). To accommodate the non-normal distributions of the observed variables the robust maximum likelihood (MLR) estimator was used (Kline, 2015). All analyzes were conducted in R (R Core Team, 2022) using the eRm package for analysis of the ALS (Mair et al., 2021) and the lavaan package (Rosseel, 2012) for CFA and path analysis.

Table 2. Descriptive statistics and correlations among observed variables.

Observed variables	N	M	SD	Scale	Skewness	Kurtosis	[1]	[2]	[3]	[4]	[5]	[6]
[1] Internet outcomes	537	2.96	0.90	1–5	−0.14	−0.46	1
[2] Breadth of internet uses	786	0.70	0.23	0–1	−0.90	0.10	.56*	1
[3] Algorithm awareness	793	1.97	1.60	0–4	−0.03	−1.57	.26*	.40*	1
[4] Algorithm knowledge	783	1.56	1.43	0–4	0.19	−1.47	.31*	.47*	.62*	1
[5] Internet skills	678	3.93	0.79	1–5	−1.05	0.70	.33*	.60*	.39*	.48*	1
[6] Ubiquity of internet access	784	0.54	0.23	0–1	0.42	−0.58	.31*	.56*	.30*	.32*	.43*	1

Notes: Pairwise deletion used for correlations (N = 479–779 depending on variable pair). * p < .05.

To answer the RQ1, we conducted a multi-group analysis (MGA) to test if path estimates differ across gender (male and female), age (18–44, 45–64, 65+), educational (high school or less and college or more), occupational (low- and high-skill job), and income (above and below average) groups. Following Hair et al. (2018), we first tested the model fit for each of these groups. Next, to assess potential differences between groups, we compared an unconstrained model (i.e., where path estimates in each group are freely estimated) to the constrained model (i.e., where path estimates are set to equal across groups) using the chi square difference (Δχ²) test. A non-significant result indicates that the constraints do not worsen the model fit and hence the estimates can be considered equal across groups. Otherwise, the fit is worse, and estimates cannot be considered equal. In such cases, we tested each individual path estimate for (in)equality and subsequently assessed a modified constrained model where invariant estimates were constrained while non-invariant estimates were freely estimated.

Results

The tested model demonstrated an excellent fit to the data (N = 802): χ²_(df)= 8.395₍₄₎, p = .078, CFI = .996, RMSEA = .046 (90% CI = .000–.095); SRMR = .024 (Figure 2 and Table 3). Ubiquity of internet access had a significant positive effect on internet skills (β = .452), breadth of internet uses (β = .319), and algorithm awareness (β = .136). Results also demonstrated significant direct positive effects of internet skills on breadth of internet uses (β = .409), algorithm awareness (β = .366), and algorithm knowledge (β = .280). Algorithm knowledge was positively affected by algorithm awareness (β = .482) and had a significant positive effect on breadth of internet uses (β = .115). Internet outcomes were positively affected by breadth of internet uses (β = .602). All statistically significant effects were demonstrated at p ≤ .001 level. The effect of ubiquity of internet access on algorithm knowledge and the effect of algorithm awareness on internet uses could not be confirmed.

Figure 2. Path analysis results (standardized estimates).

Table 3. Summary of parameter estimates and hypotheses testing (direct effects).

#	Hypotheses	Estimate	SE	Standardized	Z	p	Validation
H1	Ubiquity of internet access → Internet skills	1.585	0.123	.452	12.912	<.001	Yes
H2	Ubiquity of internet access → Breadth of internet uses	0.314	0.029	.319	10.962	<.001	Yes
H3	Internet skills → Breadth of internet uses	0.114	0.010	.409	11.381	<.001	Yes
H4	Ubiquity of internet access → Algorithm awareness	0.932	0.244	.136	3.825	<.001	Yes
H5	Ubiquity of internet access → Algorithm knowledge	0.271	0.196	.044	1.383	.167	No
H6	Internet skills → Algorithm awareness	0.713	0.072	.366	9.908	<.001	Yes
H7	Internet skills → Algorithm knowledge	0.489	0.057	.280	8.517	<.001	Yes
H8	Algorithm awareness → Algorithm knowledge	0.432	0.028	.482	15.429	<.001	Yes
H9	Algorithm awareness → Breadth of internet uses	0.008	0.005	.056	1.641	.101	No
H10	Algorithm knowledge → Breadth of internet uses	0.018	0.006	.115	3.341	.001	Yes
H11	Internet uses → Internet outcomes	2.479	0.162	.602	15.348	<.001	Yes

Notes: N = 802.

Results of MGA showed that the model had acceptable fit in all groups (Table OS6). There were no differences in path estimates between men and women or between users with low- and high-skill jobs, but there were some significant differences (at p < .05 level) in the case of age, educational, and income groups (Table OS7 and OS8). Specifically, the effect of ubiquity of internet access on internet skills significantly differed across all three groups, being the weakest among younger, well educated, and high-income users and the strongest among older, less educated, and low-income users. Age also moderated the effect of internet skills on breadth of internet uses, which was weaker among younger compared to middle-aged and older users, and the effect of algorithm awareness on algorithm knowledge, which was stronger among younger and middle-aged users compared to older users. Interestingly, although income moderated the effects of ubiquity of internet access on algorithm knowledge this effect was non-significant within both high-income and low-income user groups (Table 4).

Table 4. Moderated unstandardized parameter estimates from (modified) constrained models across age, educational, and income groups.

	Age			Education		Income^a
Paths	18–44	45–64	65+	Lower	Higher	Below average	Above average
Ubiquity of internet access → Internet skills	0.632*	1.468*	2.409*	1.831*	0.951*	2.091*	0.803*
Internet skills → Breadth of internet uses	0.045*	0.112*	0.116*
Ubiquity of internet access → Algorithm knowledge						−0.237	0.450
Algorithm awareness → Algorithm knowledge	0.429*	0.472*	0.275*

Notes: * Statistically significant estimates at p < .05. ^aAverage individual net monthly income was 1081.97€.

Discussion

Main findings

Various hypotheses in our model (H1, H2, H3, and H11) confirmed results of the MCSDE in previous empirical research, that is, ubiquity of access positively affects internet skills and breadth of internet use, internet skills positively affect breadth of internet use, and breadth of internet use positively affects internet outcomes (e.g., Petrovčič et al., 2022; van Deursen et al., 2017). What is new in our expanded model is the inclusion of algorithm awareness and knowledge. Based on prior research in this area (Gruber et al., 2021; Hargittai et al., 2020), we situated algorithm awareness and knowledge as digital resources akin to internet skills that mediate between access and uses in the MCSDE. Following that logic, we tested the effects of both ubiquity of internet access and internet skills on algorithm awareness and knowledge. Both the strong model fit, and the validation of the tested hypotheses confirm that this approach makes sense, with some minor caveats.

For example, to our knowledge, there has been no prior research that examined the relationship between ubiquity of internet access and algorithm awareness and knowledge. Our results confirm the findings of Wei and Yan (2023) that ubiquitous access has a positive effect on algorithm awareness (supporting H4); however, we did not find a statistically significant effect on algorithm knowledge (H5 was not supported). Even though a higher number of access devices may increase someone’s awareness of the presence of algorithms, it does not necessarily mean they understand more about how these algorithms work. Similarly, whereas algorithm awareness has no significant effect on breadth of use (H9 was not supported), algorithm knowledge is positively related to breadth of use (supporting H10). This highlights an important facet of research in this area. While both algorithm awareness and knowledge are part of broader algorithm literacy – and in fact, algorithm awareness has a strong effect on algorithm knowledge (supporting H8) – the differences in results support the notion that these concepts are different and should be considered separate parts of algorithm literacy (Dogruel et al., 2022), similar to the differentiation of other types of internet skills, such as the five dimensions in the ISS model (Grošelj et al., 2021; van Deursen et al., 2016).

This brings up another important argument posed by Hargittai et al. (2020) and Gruber et al. (2021). Both works argue that given the ubiquity of algorithms in our online experiences, algorithm literacy or skills, including the two dimensions of algorithm awareness and knowledge examined in this study, should be considered an integral component of the broader understanding of digital skills at large. Our analysis confirms that overall higher internet skills have a strong positive effect on both algorithm awareness and knowledge (supporting H6 and H7). Since our results indicate that internet skills affect internet uses directly and indirectly through algorithm knowledge, future conceptual models should consider defining algorithm literacy as an integral aspect of second-level digital inequalities. This is important not only for a better understanding of the mediating roles of algorithm awareness and knowledge between internet skills and uses, but also for developing initiatives aimed at improving the behavioral dimensions of algorithm literacy such as coping strategies that internet users need to limit and/or avoid the negative implications of algorithmic curation (Dogruel, 2021).

Our analyzes also found the moderating effect of age, education, and income but not gender or occupation (RQ1). The moderation effects indicate that addressing digital inequalities among disadvantaged groups (older, less educated, lower income) will have a disproportionally greater effect on digital inclusion than among better-off groups. For example, a one-point increase in internet skills will increase algorithm knowledge by 0.6 points among those with below average income, but by 0.3 points among those with above average income. The only exception was the weaker effect of algorithm awareness on algorithm knowledge among older users, compared to their younger counterparts. However, this finding is in line with prior literature (Fletcher & Nielsen, 2019; Ytre-Arne & Moe, 2021) suggesting that older users who gain awareness about algorithmic curation show less motivation to proactively acquire more knowledge about algorithms than younger users. Therefore, targeted interventions for older users are more so needed to improve their understanding of algorithms on the internet.

Overall, our study tested an expansion of the MCSDE with algorithm awareness and knowledge. The strong model fit and the overall results confirm the importance of algorithm awareness and knowledge in sequential models of digital inequality, echoing calls from other researchers (e.g., Gran et al., 2021) to include these (and possibly other) dimensions of algorithm literacy as an integral part of digital equity in a world that increasingly relies on algorithms and artificial intelligence to create, maintain, and run uncountable platforms and apps that internet users engage with on a daily basis (e.g., Cotter & Reisdorf, 2020; Gruber et al., 2021). Whether internet users are aware of algorithms and know how they work – to the extent that the latter is possible – is heavily dependent on their ubiquity of access and their general levels of internet skills. Not only that, but algorithm awareness and especially algorithm knowledge appear to influence how broadly they engage with the internet. Prior research has shown that algorithm awareness and algorithm knowledge can be tied to modified behaviors, such as avoiding or trying to manipulate platforms in certain ways (Fouquaert & Mechant, 2022; Siles et al., 2022). Our study confirms these results and supports the notion that algorithm awareness and knowledge affect the way that people utilize the internet, which, in turn, has an effect on the tangible outcomes of their internet use.

Limitations and future research

There are some limitations that need to be considered. While Slovenia is an average performing EU country in terms of internet skills and uses (Eurostat, 2022), no comparable data exists in terms of algorithm awareness and knowledge in the EU. Moreover, this study focused on two dimensions of algorithm literacy (i.e., awareness and knowledge). While this is appropriate as they affect the online activities of the largest number of internet users (Dogruel, 2021; Dogruel et al., 2022), future studies examining additional dimensions are nevertheless warranted. In addition, while the sequentiality of the paths in the explanatory model was derived directly from the assumptions of the MCSDE, our research design could not verify these causal claims. Relatedly, the current study could also not test the potential bidirectional causal relationship between algorithm awareness and knowledge and internet uses as proposed by some scholars (e.g., Swart, 2021). Indeed, a post-hoc analyzes of reverse (Table OS9) and mediational (Table OS10) models showed that the fit of such models was equal or close to the proposed model, indicating that longitudinal or experimental studies should elucidate the causal and bidirectional effects among the concepts in the explanatory model.

Conclusion

Algorithms have become ubiquitous, and they are here to stay. This also means that policymakers and practitioners in the area of digital inequalities need to take algorithm awareness and knowledge seriously and include them in efforts to improve digital engagement among the population. Internet users need to understand how their content is curated, what this means for the content they see online, and how this affects their lives both online and offline. To that extent, it is not enough to integrate algorithm awareness and knowledge into the conceptual models of digital inequalities. They should be also part of any serious digital inclusion intervention. Our findings suggest that awareness-raising initiatives regarding the implications of algorithmic curation would be particularly valuable for digitally disadvantaged groups, such as older adults or low-income groups. Not only would this address low levels of algorithm awareness, but it would also enable them to gain knowledge about the consequences of algorithmic curation across various online domains (e.g., social media, online news aggregators). Increasing the levels of algorithm awareness and knowledge among the population is critical for both coping with algorithmically curated online environments on an individual level and for raising the public’s understanding of potential risks and benefits that algorithmic curation and decision-making bring for society in general.

Acknowledgement

We thank Nuša Cekuta for her support in data analysis. We also thank both reviewers for their constructive and insightful comments that helped us improve the article.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study are openly available in ADP – Social Science Data Archives at https://doi.org/10.17898/ADP_SJM221_V1

 Google Scholar

, reference number SJM221.

Additional information

Funding

This work was supported by the Slovenian Research Agency under Grants no. BI-US/22-24-055, J5-2558, P5-0399, V5-2275, and the Young Research fellowship of the last author.

Notes on contributors

Andraž Petrovčič

Andraž Petrovčič (Ph.D. University of Ljubljana) is an associate professor of social informatics in the Faculty of Social Sciences, University of Ljubljana, Slovenia. His research focuses on socio-technical aspects of older adults’ interactions with internet technologies in the context of digital inequalities [email: [email protected]].

Bianca C. Reisdorf

Bianca C. Reisdorf (D.Phil. University of Oxford) is an associate professor in the Department of Communication Studies, University of North Carolina Charlotte, USA. Her work focuses on the intersection of inequalities and digital media and the internet among marginalized populations, as well as proxy internet use and how internet users look for and evaluate information from various media sources.

Vasja Vehovar

Vasja Vehovar (Ph.D. University of Ljubljana) is a full professor of statistics in the Faculty of Social Sciences, University of Ljubljana, Slovenia. His research focuses on internet research, information society indicators, statistics, missing data, sample collection, social science methodology, and web survey methodology.

Jošt Bartol

Jošt Bartol is a doctoral student of statistics at the University of Ljubljana, Slovenia, and a research fellow at the Centre for Social Informatics at the Faculty of Social Sciences of the University of Ljubljana, Slovenia. His research centres on information privacy on the internet and scale development.

Notes

1 Due to space limitations, we do not provide a discussion of other dimensions of algorithm literacy (e.g., critical evaluation, coping tactics, creation and design). For a detailed discussion of the concept of algorithm literacy, see Dogruel (2021) or Shin et al. (2022).