Development and testing of the advertising literacy activation task: an indirect measurement instrument for children aged 7-13 years old

ABSTRACT

The aim of this study was to develop and test an indirect measurement instrument that is able to assess children’s advertising literacy activation. Study 1 (N = 24, age: 10–12) describes the development of the Advertising Literacy Activation Task (ALAT) in which participants sort successive words in two categories: advertising or news. Studies 2 and 3 tested the ALAT on usability, reliability, predictive, and construct validity. In study 2 (N = 38, age: 10–12), participants were primed with a story about advertising or about news. Study 3 (N = 114, age: 7–13) used a similar design but with actual television commercials and news broadcast. Frequency and speed of categorizing conceptual and attitudinal words were analyzed with Bayesian statistics investigating effects of priming, dispositional advertising literacy, and their interactions. Results indicate that the ALAT is a usable and reliable measurement instrument with good predictive validity, but with limited construct validity.

KEYWORDS:

• advertising
• children
• advertising literacy
• indirect measurements
• measurement development

Introduction

For decades, there has been a strong scientific and societal debate about the appropriateness and fairness of advertising targeting children. One of the primary concerns is that children, as compared to adults, are particularly susceptible to the effects of advertising because they still have immature levels of advertising literacy (John, 1999; Kunkel, 2010). Advertising literacy includes a broad range of knowledge, attitudes, and skills needed to critically process advertising messages. Increasing children’s advertising literacy is often seen as the solution for reducing children’s susceptibility to advertising (e.g., An & Stern, 2011; Nairn & Fine, 2008). The general assumption is that children with better developed advertising literacy are more likely to critically process advertising messages and, therefore, have less desire for advertised products (De Jans, Van de Sompel, Hudders, & Cauberghe, 2019; Opree & Rozendaal, 2013). The central idea underlying this assumption is that children will activate their advertising literacy once they are confronted with advertising (De Jans, Cauberghe, & Hudders, 2019; Hudders et al., 2017) and use it to process the advertising message and form their attitude toward the advertisement and advertised product.

However, several studies have shown that a higher level of advertising literacy does not necessarily protect children against advertising effects (e.g., Naderer, Matthes, Marquart, & Mayrhofer, 2018; Van Reijmersdal, Rozendaal, & Buijzen, 2012). The question remains if this is because increased advertising literacy indeed does not decrease susceptibility, or if this is due to the fact that although children have a certain level of advertising literacy, they may notactivate and applyit when they are confronted with advertising. An answer to this questions is still lacking, because there is no adequate measurement tool available to assess children’s ability to activate and apply their advertising literacy.

The process of activation and application of advertising literacy during advertising exposure is also referred to as advertising literacy performance (Rozendaal, Lapierre, Van Reijmersdal, & Buijzen, 2011). While application of advertising literacy cannot exist without activation, it is possible that advertising literacy is activated but not applied during exposure to an advertising message. In order to investigate children’s advertising literacy performance, we must first get an understanding of children’s advertising literacy activation. The aim of this study is to develop and test a measurement tool that is able to examine the extent to which children activate their advertising literacy during exposure to advertising-related material.

Children’s advertising literacy

Advertising literacy includes a conceptual and an attitudinal dimension (Rozendaal et al., 2011). Conceptual advertising literacy gradually develops throughout life, evolving from simple to more sophisticated knowledge and beliefs about the nature and working of advertising (Friestad & Wright, 1994; Hudders et al., 2017; Rozendaal et al., 2011). From the age of five, children start to distinguish advertising from noncommercial media content (Livingstone & Helsper, 2006). Around the age of eight most children have acquired a general understanding of advertising’s selling intent (i.e., knowledge that advertising is made to sell products, for reviews, see John, 1999; Kunkel et al., 2004). Between the age of eight and eleven, children also start to understand advertising’s persuasive intent (i.e., knowledge that advertising is made to generate favorable views toward products). Persuasive intent is harder to understand for children than selling intent because persuasive intent relates to changes in beliefs, while selling intent relates to changes in behavior. In relation to theory of mind, the latter is easier for children to understand than the former (Lapierre, 2015; Moses & Baldwin, 2005). For this reason, knowledge of persuasive intent develops at a slower pace than knowledge of selling intent (Carter, Patterson, Donovan, Ewing, & Roberts, 2011).

Over the years, children also develop their attitudinal advertising literacy, which consists of skepticism and disliking (Rozendaal et al., 2011). Skepticism is the tendency to disbelieve advertising, while disliking is a general negative attitude toward advertising (Rozendaal et al., 2011). Critical beliefs about advertising start to develop around the age of eight and become more negative as children move into adolescence (John, 1999). Typically, children’s attitudinal advertising literacy develops in parallel with their understanding of advertising’s persuasive intent. The more they become aware of the persuasive nature of advertising, the better they understand that advertising is sometimes biased and untruthful, which results in a more negative view toward advertising in general (Rozendaal et al., 2011).

Both conceptual and attitudinal advertising literacy are dispositional (Hudders et al., 2017), meaning that even when a child is not actively using it, it is still present in the child’s mind. Children’s dispositional advertising literacy is clustered in advertising schemas. Schemas are mental structures that people use to make sense of the world (Aronson, Wilson, & Akert, 2005). In our advertising schema we cluster, for instance, all knowledge and beliefs toward advertising in general, but also toward specific advertisements (Dahlén & Edenius, 2007). According to information-processing theory, incoming information (e.g., an advertising message) can result in the retrieval of related information (e.g., activation of the advertising schema). The incoming (new) information thus serves as a cue for information retrieval (Roedder, 1981). As such, exposure to advertising could (but not necessarily will) lead to the activation of children’s dispositional advertising literacy (see Buijzen, Van Reijmersdal, & Owen, 2010).

Measuring activation of children’s advertising literacy

In recent years, scholars have worked on developing valid and reliable questionnaire measures to assess children’s advertising literacy (D’Alessio, Laghi, & Baiocco, 2009; Rozendaal, Opree, & Buijzen, 2016). Although these self-report questionnaires are valuable measurement tools for capturing children’s dispositional advertising literacy, they may not be suitable to measure children’s advertising literacy activation for several reasons (see also Hoek, Rozendaal, van Schie, van Reijmersdal, & Buijzen, 2020).

First, in order to answer questions, children have to retrieve information from memory. This may be difficult for them because they have to introspectively search for, evaluate, and formulate the answer (Dunham, Baron, & Banaji, 2006). Second, the way questions and answer options are worded or framed may prompt the answers children provide (Brace, 2004). It may result in children choosing the option they like best, or the option they think matches the researcher’s objective. Third, and most importantly, questionnaires often ask participants to critically think about and evaluate a past experience. This may activate posthoc rationalizations (Vandeberg, 2014), which means that asking children to consciously think about their response to an advertising message leads to a reevaluation of the cognitive and affective processes that were actually activated. Therefore, their answers are not a representation of what truly happened, but of what they think happened. To summarize, the self-report questionnaire measurements that are currently used to assess children’s dispositional advertising literacy have a number of disadvantages that may make them unsuited to validly and reliably assess children’s advertising literacy activation.

A more suitable way to assess children’s advertising literacy activation is by using indirect measures (i.e., reaction time measures). Indirect measures have often been used in research on schema activation (e.g., Kim & Hancock, 2016). As indirect measures do not require participants to consciously reevaluate a past experience (Vandeberg, Wennekers, Murre, & Smit, 2013) they are better able to capture sub-conscious processes (De Houwer, 2006), such as children’s activation of dispositional advertising literacy. Even though using indirect (and implicit) measures in studies with children is not very common, previous research has shown that using these types of reaction time measures is possible even when doing research with children as young as 6-years old (e.g., Banse, Gawronski, Rebetez, Gutt, & Bruce Morton, 2010; Baron & Banaji, 2006; Cvencek, Meltzoff, & Greenwald, 2011).

One approach to developing an indirect measurement tool for assessing the activation of advertising literacy is to select a task in which the activated advertising literacy influences the processing of words that are associated with advertising. The activation of a certain concept (i.e., schema) makes the processing and categorization of words related to this concept (i.e., schema) easier. Several studies on categorization tasks within priming research have shown this effect (e.g., Fazio, Sanbonmatsu, Powell, & Kardes, 1986; Ortells, Kiefer, Castillo, Megías, & Morillas, 2016). In the context of the current study, in the categorization task children would be asked to decide whether a target, in this case a word related to advertising (e.g., product) or a word unrelated to advertising (e.g., journalist) is part of the concept of advertising or not. This makes the categorization task an indirect task, since it indirectly assesses advertising literacy activation from children’s responses to the task (Fazio & Olson, 2003).

The present study

In this paper, we describe three studies in which we develop (study 1) and test (study 2 and 3) the Advertising Literacy Activation Task (ALAT). The ALAT is a word categorization task that aims to measure the extent to which children activate their dispositional advertising literacy during exposure to advertising-related material. Study 1 describes the development of the ALAT. We examine which words children associate with advertising in order to develop the task. In study 2 the usability, reliability, and validity of the ALAT is tested in a controlled lab-like setting. In study 3 we build on the results of the second study and test the ALAT in a more naturalistic setting.

Usability is operationalized in this study as the extent to which children like the task, and to what extent they think the task is difficult or easy. We hypothesize that usability of the ALAT is high when children (1) like the task and (2) think the task is easy. Reliability is determined by looking at the split-half reliability, which assesses the internal consistency of a measure (Nosek, Greenwald, & Banaji, 2007). We specifically look at the internal consistency regarding children’s overall performance on the task (i.e., if their reaction times remain consistent during the task). For split-half reliability, the scores on a scale or measure are split in half. Then, the correlation between the two halves of the scale or measure is calculated (Drost, 2011; Peter, 1979). We hypothesize that reliability of the ALAT is high when there is a strong correlation between the reaction time scores on the first half of the trials and the reaction time scores on the second half of the trials.

Validity is tested by looking at both predictive validity and construct validity. Predictive validity is the extent to which a score on one measure predicts the score on another measure (Bergkvist & Rossiter, 2007). A study by Greenwald, Poehlman, Uhlmann, and Banaji (2009) shows that predictive validity of an indirect measure can be assessed by looking at self-report measures. Children’s dispositional conceptual and attitudinal advertising literacy, as assessed with a self-report questionnaire, should predict the scores on the ALAT, because in order to activate certain knowledge and attitudes these have to be present in the first place. Therefore, we hypothesize that predictive validity of the ALAT is high when the scores on dispositional advertising literacy predict the scores on the ALAT. This is tested in two ways. We expect that children with higher levels of dispositional advertising literacy are (1) more accurate and (2) faster in categorizing advertising-related words in the ALAT than children with lower levels of dispositional advertising literacy.

Construct validity is described as the degree to which a particular operationalization measures the concept it is supposed to measure (Bagozzi, Yi, & Phillips, 1991). It requires a theoretical argument (Cook, Campbell, & Shadish, 2002). In this paper, the theoretical argument is based on information processing and schema theories. These theories predict that exposure to a cue (i.e., advertising) results in the retrieval and activation of related information (i.e., advertising schemas, including advertising literacy). In other words, these theories predict that exposure to advertising will lead to advertising literacy activation (which is also the general assumption in the advertising literacy literature; e.g., De Jans et al., 2019; Hudders et al., 2017). Therefore, we hypothesize that construct validity of the ALAT is high when the ALAT is able to measure advertising literacy activation in response to exposure to advertising. This is tested in two ways. We expect that children who are exposed to advertising-related material are (1) more accurate and (2) faster in categorizing advertising-related words in the ALAT than children who are exposed to non-advertising-related material.

Furthermore, we test the construct validity by looking at the interaction between dispositional advertising literacy and exposure to advertising-related material. When children are confronted with advertising, activating their knowledge and critical attitudes should be easier for children with higher levels of dispositional advertising literacy than for children with lower levels of dispositional advertising literacy. Therefore, we expect that children who are exposed to advertising-related material and have higher dispositional advertising literacy are (1) more accurate and (2) fasterin categorizing advertising-related words in the ALAT than children with lower levels of dispositional advertising literacy. We do not expect this effect when children are exposed to non-advertising-related material.

Prior to collecting the data this project received approval from the Ethics Committee of the Radboud University Nijmegen, the Netherlands. All data for this study were collected in Dutch.

Study 1

The aim of the first study was to develop the Advertising Literacy Activation Task (ALAT). In this task, children had to categorize words into two categories. One of the categories was advertising, therefore we needed to know which words children associate with advertising. Furthermore, we needed a similar category, in order to create a balanced task and a control condition. This way we were able to look at the specific effects of the task. The other category we chose was news, since this is a concept that is also easily understandable for children and part of their (daily) media diet (De Leeuw, Kleemans, Rozendaal, Anschütz, & Buijzen, 2015). Thus, we also needed to know which words children associated with news.

Method

Participants

The study was conducted in an elementary school in the Netherlands. After the school gave permission for participation, parents were informed about the study with an information letter containing a consent form. Only children with active parental consent were able to participate (N = 24). The children were aged 10–12 years old.

Procedure

Half of the children were asked to do an open association task for words related to advertising and the other half were asked to do an open association task for words related to news (“when you think about advertising [news], what other words do you think of? Please write down all words that come to mind.”). After the open association task, half of the children from each group (n = 12) were asked to perform a closed association task. We had chosen 76 words; 20 words were related to advertising (e.g., product, price, buy), 30 words were related to news (e.g., studio, event, important), 14 words were affective words (e.g., funny, stupid, boring), and 12 words were neutral words (e.g., face, house, baker). The words were printed on cards and children were asked to categorize the words into one of four categories: 1) advertising, 2) news, 3) both, or 4) neither. The advertising, news, and affective words in the closed association task were chosen by looking at words that are used in questionnaire research with children on the topics of advertising and news (e.g., Rozendaal et al., 2016). Furthermore, we discussed these words in the research team in order to determine if the list was exhaustive. The neutral words were chosen based on their comprehensibility and the fact that they were unrelated to advertising.

Results

The advertising-related words that were written down most often in the open association were buy (7x), product (4x), toys (3x), and store (3x). The advertising-related words that were chosen most often in the closed association were product (10x), advertisement (9x), discount (8x), and price (7x). The news-related words that were written down most often in the open association were weather (8x), countries (8x), event (4x), and information (3x). The news-related words that were chosen most often in the closed association were Jeugdjournaal¹ (9x), really happened (9x), NOS² (8x), and information (8x). A more complete overview of the words that were chosen most often as being related to either advertising or news, in both the open and closed association, can be found in Table 1.

Table 1. Results of association tasks study 1

Advertising – OPEN	Advertising – CLOSED	News – OPEN	News – CLOSED
Buy	Product	Weather	Jeugdjournaal
Product	Advertisement	Countries	Really happened
Toys	Discount	Event	NOS^1
Store	Price	Information	Information
Fake	Persuade	Jeugdjournaal^2	News broadcast
Lies	Pay	People	Journalist
Boring	Buy	Funny	Anchor
Fun	Sell	Important	Event

¹The NOS is the biggest Dutch news broadcasting agency.

²Jeugdjournaal is the name of a Dutch news broadcast especially made for children.

Development of the ALAT

In the ALAT, participants are presented with words related to advertising, words related to news, and words denoting positive and negative attitudes. Each word has to be categorized to either advertising or to news by pressing one of two response keys. Based on the results of this study, we selected 10 words as the advertising-related words and 10 words as the news-related words. These 20 conceptual words were selected based on the frequency with which they were mentioned in the open and closed association tasks combined. If two words were mentioned equally often, the word that was mentioned more often in the open association task (as opposed to the closed association) was selected. Furthermore, we selected 10 attitudinal words, which could be related to both advertising and news. These 10 words were selected based on the results of the open and closed association tasks, as well as on the words that are generally used in advertising literacy research to (negatively) describe advertising (Rozendaal et al., 2016). Note that for the attitudinal words, only negative words are considered to be advertising-related, since dispositional attitudinal advertising literacy is the extent to which one has a negative and skeptical attitude toward advertising (Rozendaal et al., 2011). See Table 2 for an overview of the selected words. The ALAT was constructed with these 30 words.

Table 2. Words used in the advertising literacy activation task

Practice	Advertising	News	Attitudinal
Order	Product	Jeugdjournaal	Annoying
Purchase	Advertisement^3	NOS	Boring
Brand	Discount	Journaal^4	Fun
Pay	Price	Reporter	Interesting
Journalist	Buy	Event	Funny
Studio	Persuade	Weather	Stupid
Domestic	Offer	Informative	Fake
Foreign	Stuff	Countries	Mislead
	Tempt	Information	Lies
	Store	Anchor	Lying

Note: The words really happened were not included in the final task, even though over a third of the children chose these words as related to news in the closed association. The reason for this is that we only wanted to include singular words.

³In the original language, the way of writing the translations of Advertising and Advertisement differ strongly from each other.

⁴Journaal is the name of a regular Dutch news broadcast.

It was hypothesized that following priming with advertising, conceptual words related to advertising (i.e., conceptual advertising literacy) would be categorized more accurately and faster than following priming with news, and that words denoting a negative attitude (i.e., attitudinal advertising literacy) would be categorized more frequently with the advertising category. Although choosing between advertising and news categories certainly requires activation of the concept of advertising, we reckoned that advertising literacy would be activated to a stronger extent, more concretely and more specifically (Kiefer & Pulvermüller, 2012) in the context of exposure to explicit advertising literacy-related information (study 2) and actual television commercials (study 3).

Description of the ALAT

The ALAT started with an instruction in which it was made clear that the goal was to categorize the words as quickly and as accurately as possible as either advertising- or news-related. This was followed by a practice block with eight words (four related to advertising and four related to news). In the practice block, children were shown if they correctly categorized the word. After this, the real task started and children were asked to categorize the 60 words (note that the 30 chosen words were each presented twice). For the main task, children were not shown if they categorized the word correctly. Each trial was in the same order. First, a fixation point was shown for 500 ms (a blue dot with a two cm diameter). Directly after the fixation point, the target (word) was shown for a maximum of 5000 ms (or until the child pressed one of the two buttons). The target was always black, had font style Arial, and font size 30. The background throughout the task was white. Two seconds after the child pressed one of the response buttons, the next fixation point was shown.

After categorizing 40 words there was a short break, where children were told they were almost finished. During the categorization of the words, children saw the words “advertising” and “news” in the lower left and right corner of the screen, corresponding to the side they had to push the button. For all trials, the correctness of the response and the response time were recorded. If children did not respond within 5 s, the response was labeled “missing.” The ALAT was made and executed in PsychoPy version 1.84.2. In order to get the most accurate response time measures, we used a button box instead of a regular keyboard.

Study 2

The aim of the second study was to test if the ALAT is a good measurement instrument to assess children’s advertising literacy activation. Testing the ALAT was done by looking at usability, reliability, predictive validity, and construct validity. In order to activate children’s dispositional advertising literacy, we presented them with a story, either about advertising or news. As this was the first time that the ALAT was tested in the field, we chose a more clinical approach in this study. With the use of a story, we constructed a strong manipulation that was supposed to activate children’s conceptual and attitudinal advertising literacy. This way we could establish the sensitivity of the ALAT in response to exposure to advertising-related material. Although this approach may not be comparable to how children are exposed to advertising-related material in a natural setting, we believed this clinical approach was needed as a first step to test the ALAT.

Method

Study design

The second study was an experiment with a 2 × 2 (activation material: advertising vs news; words: conceptual vs attitudinal) repeated measures design. In the experimental condition, children’s advertising literacy was activated with a story about advertising. In the control condition, children read a story about news. Children were exposed to both conditions with a one-week interval between conditions. The order of conditions was counterbalanced between participants.

Participants

Children were recruited from an elementary school in the Netherlands. Forty children participated (52.5% boys, Age_range = 10–12 years old, M_age = 10.95, SD_age = 0.71) at measurement time 1 and 38 children participated (50% boys, Age_range = 10–12 years old, M_age = 11.00, SD_age = 0.70) participated at measurement time 2, resulting in a drop-out rate of 5.0%.

Stimulus material: stories

In a separate pretest (N = 24, aged 10–12 years old), half of the children were asked to read the advertising-related story and write down what it was about. The other half were asked to read the news-related story and write down what it was about. Almost all (91.7%) of the children indicated the advertising story was about advertising, while only 66.7% indicated the news story was about news. Based on these results, the stories were adjusted. The final versions of the stories were about two children who were watching television. They talked about what they saw, and one of the children gave factual information (e.g., on advertising: “advertising tries to tempt you to buy the product”). The factual information was designed to activate children’s dispositional conceptual advertising literacy. Furthermore, there was an attitudinal component in the stories (e.g., in the advertising-related story, one of the children said “advertising is stupid and boring”), designed to activate children’s dispositional attitudinal advertising literacy. The stories (translated) can be found in Appendix A.

Procedure

After the school gave permission, parents were informed with an information letter containing an active consent form. Only children with active parental consent could participate in the study. The study took place at school. Children gave verbal consent before the start of the study. Before they started, they were told the study was about what they see on television and that the researcher was interested in their opinion; therefore, there were no right or wrong answers. Children then put on headphones to limit distractions. In the first part of the study, children were exposed to the stimulus material, by reading a story (either related to advertising or news). Then, they performed the ALAT. Finally, they filled out a questionnaire. In the advertising condition, children only answered questions about the task, their age, sex, school, and grade. In the news condition, they additionally answered questions regarding their dispositional advertising literacy. We chose to only assess children’s dispositional advertising literacy (i.e., literacy that is present regardless of advertising exposure) once, because it is considered a trait variable. This was done in the control condition, so answers would not be influenced by exposure to the advertising-related material. The time interval of a week was chosen based on previous research with within-subject designs in which children were exposed to different types of (non-)advertising-related materials (e.g., Boyland, Kavanagh-Safran, & Halford, 2015; Moore & Lutz, 2000). Each child’s participation lasted for approximately 20 minutes. In the second week, the setup was exactly the same, only children were assigned to the other condition. Thus, children performed the ALAT twice, once after exposure to advertising-related material and once after exposure to news-related material. This way, we were able to compare the ALAT scores for the two conditions for each child.

Measures

The following variables were measured within this study.

ALAT

For the ALAT, we recorded both the accuracy and speed of the response. The accuracy of the response was registered as 0 (incorrect response) or 1 (correct response). The speed of the response was recorded in seconds with millisecond accuracy. We converted seconds to milliseconds in the final dataset. Response time outliers were removed via the method described by Leys, Ley, Klein, Bernard, and Licata (2013). For the conceptual words the outliers were calculated per child, per measurement time, only looking at the correct and “real” trials (practice trials were disregarded). For the attitudinal words the outliers were calculated per child, per measurement time, and only for the “real” trials. Because there was no right or wrong categorization for the attitudinal words, correctness of categorization was not used in calculating the outliers.

To test our expectations, we looked at (1) categorization and (2) speed of the categorization of the advertising-related words. We argue that a higher level of advertising literacy activation will be indicated by more accurate and faster categorization of the advertising-related words (Fazio & Olson, 2003). The conceptual advertising words are considered separately from the negative (attitudinal) advertising words. Four new variables were constructed for the speed of categorizing the advertising words:(1) a mean reaction time score on the advertising words in the advertising condition, (2) a mean reaction time score on the advertising words in the news condition, (3) a mean reaction time on the negative (attitudinal) words in the advertising condition, and (4) a mean reaction time on the negative (attitudinal) words in the news condition. However, because the negative (attitudinal) words could also be categorized as belonging to the news category, we only used the negative (attitudinal) words that were categorized as words related to advertising, disregarding the negative (attitudinal) words that were categorized as words related to news.

Questionnaire measurement I: background measures

After children performed the ALAT, they were asked to fill out a short questionnaire. In the standard questionnaire, children received six questions regarding the usability of the measure (see below) and three questions about their age, sex, and grade.

Questionnaire measurement II: usability

Based on Van Reijmersdal et al. (2012), six questions were used to assess the usability of the ALAT. Answers were given on a four-point Likert scale, ranging from 1 (No, not at all) to 4 (Yes, very much). The scores of measurement time 1 and measurement time 2 were combined by calculating the mean. Some items were recoded, so that a higher score on likability would indicate stronger liking of the task, and a higher score on difficulty would indicate the child found the task more difficult. The scores on four items (did you think the ALAT was (1) fun, (2) stupid – R, (3) exciting, (4) boring – R) were combined to calculate ALAT Liking. The scores on two items (did you think the ALAT was (1) difficult, (2) easy – R) were combined to calculate ALAT Difficulty.

Questionnaire measurement III: dispositional advertising literacy

In the news condition, children received additional questions related to their dispositional conceptual and attitudinal advertising literacy, based on Rozendaal et al. (2016). Conceptual advertising literacy was divided into (knowledge of) persuasive intent (3 items, α = .69, M = 3.51, SD = 0.54) and selling intent (2 items, α = .58, M = 3.65, SD = 0.57, r(38) = .45, p = .004). These items were measured on a scale ranging from 1 (No, not at all) to 4 (Yes, very much), with higher scores indicating a higher level of knowledge. Attitudinal advertising literacy was divided into disliking (7 items, α = .51, M = 3.81, SD = 0.46) and skepticism (3 items, α = .85, M = 4.01, SD = 0.58). These items were measured on a scale ranging from 1 (Never) to 5 (Always), with higher scores indicating a higher level of disliking/skepticism. For the complete scales used in this study, see Appendix B.

Plan of analysis

The ALAT was tested on usability, reliability, and predictive and construct validity.

Usability

To determine the usability of the ALAT, we look at the mean scores of ALAT Liking and ALAT Difficulty.

Reliability

The reliability of the ALAT is determined by looking at split-half reliability (Drost, 2011; Peter, 1979). This reliability measure gives an indication of the consistency with which children performed the task. We calculated the Spearman-Brown correlation between the reaction time scores on the first 30 trials and the reaction time scores on the second 30 trials. It is important to note that the order in which the words were presented to the children was completely random. Therefore, the split-half reliability was calculated for a different combination of the first 30 words and the second 30 words for each of the children. The split-half reliability was only calculated for the reaction time scores on the first measurement, since this was the first time children worked with this task and reaction time could not be influenced by external factors (e.g., familiarity with the task).

Validity

We used Bayesian statistics to test our expectations related to predictive and construct validity. There are two main reasons why we chose a Bayesian approach over a frequentist approach. First, Bayesian statistics provide different degrees of support for each of the expectations and allows for a more balanced decision in comparison to frequentist statistics (Konijn, Van de Schoot, Winter, & Ferguson, 2015). For instance, Bayesian analyses can show whether there is very strong evidence for the proposed hypothesis, or merely anecdotal evidence. The strength of the evidence determines how much value we can derive from the results found. Second, with Bayesian statistics it is not only possible to quantify support for the alternative hypothesis, but also for the null hypothesis (Wagenmakers et al., 2018). This means that Bayesian statistics can quantify support for the absence of an effect, something that is impossible to do with frequentist statistics (Schönbrodt, Wagenmakers, Zehetleitner, & Perugini, 2017). The fact that Bayesian statistics can also provide an estimate of support for the null hypothesis makes this type of analysis more suitable than frequentist statistics for a study that focuses on measurement development. As this is the first study to investigate children’s advertising literacy activation by using an indirect measurement approach, we do not only want to know if there is any support for our hypotheses but also how strong this support is, or whether there is stronger support for the null hypothesis. Therefore, we chose to use Bayesian statistics in this study. The analyses were done in the statistical program JASP (version 0.9.0.0, JASP Team, 2018).

For interpreting the Bayes factors, we adhered to Wagenmakers et al. (2018) who published a table with guidelines for interpreting Bayes factors. Bayes factors give an indication of how much more probable one (e.g., the alternative) hypothesis is as compared to the other (e.g., the null) hypothesis. For instance, when BF₁₀ equals 5 this means that the alternative hypothesis is five times more likely than the null hypothesis. When the Bayes factor is smaller than 1 (e.g., 0.20) this means that the null hypothesis is more likely (Wagenmakers et al., 2018). The higher the Bayes factor, the stronger the support for one hypothesis over the other. When the Bayes factor is (close to) 1 this means there is equal support for both hypotheses Bayes factors can, thus, be interpreted on a continuous scale. In order to provide clarity in our interpretation, we use Bayes factors of 3 or higher as showing support for the alternative hypothesis. This cutoff point was chosen because a Bayes factor above 3 indicates at least moderate evidence for the alternative hypothesis (Wagenmakers et al., 2018).

Predictive validity

To test the predictive validity of the ALAT, we performed four regression analyses with the default prior. Accuracy of categorizing and speed of categorizing the (a. conceptual, b. attitudinal) advertising words were the dependent variables (independent of condition) and the four dispositional advertising literacy dimensions (selling intent, persuasive intent, skepticism, and disliking) were the independent variables. We report the values for BF_Inclusion, which provide the posterior inclusion odds for each variable (Rouder, Morey, Verhagen, Swagman, & Wagenmakers, 2017; Wagenmakers et al., 2018). If the value for an independent variable is 3 or higher, this is an indication that including this variable in the model improves the fit considerably in comparison to the null model and all other potential models without this specific variable (Wagenmakers et al., 2018).

Construct validity

To test the construct validity of the ALAT, we performed four paired samples t-tests with a Cauchy prior width of $\frac{\sqrt{2}}{2}$ (default prior in JASP). The first two t-tests were used to determine if children in the advertising condition were more accurate in categorizing the advertising-related words (a. conceptual, b. attitudinal). We report the BF₊₀ (the data support our hypothesis) because it is a one-sided test with the expectation that the score in the advertising condition is higher than the score in the news condition. The second two t-tests were used to determine if children in the advertising condition were faster in categorizing the advertising-related words (a. conceptual, b. attitudinal). We report BF₋₀ (the data support our hypothesis) because it is a one-sided test with the expectation that the score in the advertising condition is lower than the score in the news condition.

To test the interaction between exposure to advertising-related material and dispositional advertising literacy (also part of construct validity), we performed four regression analyses with the default prior. To be able to test the interaction effect on accuracy of categorization, we first subtracted the score on accuracy of categorization in the news condition from the score on accuracy of categorization in the advertising condition. A positive score on this accuracy difference score, therefore, indicates that the child categorized more (a. conceptual, b. attitudinal) advertising words as advertising words in the advertising condition than in the news condition. A negative score indicates that the child categorized more (a. conceptual, b. attitudinal) advertising words as advertising words in the news condition than in the advertising condition. To test the interaction effect on speed of categorization, the same difference score was calculated, only on the speed of categorization instead of accuracy. The accuracy difference scores and the speed difference scores were used as dependent variables in the regression analyses and the four dispositional advertising literacy dimensions were used as independent variables. We report the values for BF_Inclusion. If the value for an independent variable is 3 or higher, this is an indication that including this variable in the model improves the fit considerably.

Results

Usability

Children scored just above the midpoint of the scale regarding likability of the ALAT (M = 2.73, SD = 0.43) and thought the task was easy (M= 1.63, SD = 0.57).

Reliability

A split-half reliability analysis showed that the Spearman-Brown correlation between the first 30 trials and the second 30 trials is .93.

Validity

Only the hypotheses that are supported by the data are described in greater detail. Table 3 gives an overview of all Bayes factors and whether there is support for the hypotheses. Table 4 gives the means and standard deviations for the accuracy and speed of categorization, broken down into condition and word type.

Table 3. Overview of expectations regarding validity, Bayes factors and whether or not the data support the expectations for study 2

Expectation	Bayes factor	Support	Additional information
PV 1a	≤0.47	Inconclusive	Inconclusive support for including dispositional advertising literacy in the model
PV 1b	13.01	Support	Support for including Disliking
PV 2a	≤ 2.51	Inconclusive	Inconclusive support for including dispositional advertising literacy in the model
PV 2b	≤ 1.39	Inconclusive	Inconclusive support for including dispositional advertising literacy in the model
CV 1a	0.11	No support	It is (1/0.11 =) 9.09 times more likely that there is no effect vs. that there is an effect
CV 1b	3.09	Support	It is 3.09 times more likely that there is an effect vs. that there is no effect
CV 2a	0.10	No support	It is (1/0.10 =) 10.00 times more likely that there is no effect vs. that there is an effect
CV 2b	0.12	No support	It is (1/0.12 =) 8.33 times more likely that there is no effect vs. that there is an effect
I 1a	≤0.69	Inconclusive	Inconclusive support for an interaction between exposure and dispositional ad literacy
I 1b	≤0.54	Inconclusive	Inconclusive support for an interaction between exposure and dispositional ad literacy
I 2a	4.74	Support	Support for including Disliking
I 2b	≤ 2.58	Inconclusive	Inconclusive support for an interaction between exposure and dispositional ad literacy

Note 1: PV = Predictive Validity, CV = Construct Validity, I = Interaction between exposure difference score and dispositional advertising literacy (part of CV).

Note 2: 1a: effect on accuracy of categorizing conceptual advertising words, 1b: effect on accuracy of categorizing attitudinal advertising words, 2a: effect on speed of categorizing conceptual advertising words, 2b: effect on speed of categorizing attitudinal advertising words.

Table 4. Mean and standard deviation scores for categorization and speed of categorization for the advertising words in study 2

	Advertising condition		News condition
	M	SD	M	SD
Categorization of conceptual words	1.91	0.13	1.93	0.12
Speed of categorizing conceptual words	1205.62	358.36	1150.98	261.80
Categorization of attitudinal words	1.81	0.23	1.70	0.33
Speed of categorizing attitudinal words	1392.60	421.04	1351.17	347.95

Note: Categorization can vary between 0 and 2, since each word is presented twice in the task.

Predictive validity

A Bayesian linear regression analysis showed that children with a higher level of dispositional attitudinal advertising literacy (disliking) were more accurate in categorizing the attitudinal (i.e., negative) advertising words. There is strong evidence that including disliking will improve the model (BF_Inclusion = 13.01). A regular regression analysis showed that the effect was in the expected direction (see Table 5).

Table 5. Regression analysis for the effect of dispositional advertising literacy on the accuracy of categorizing attitudinal advertising words in study 2

Variable	b	SE b	β	t	p
Persuasive Intent	0.07	0.08	.17	0.94	.353
Selling Intent	−0.07	0.07	−.17	−0.95	.349
Disliking	0.24	0.08	.49	2.95	.006
Skepticism	−0.01	0.07	−.03	−0.19	.848

Note: R² = .24.

Construct validity

A one-sided Bayesian paired samples t-test with the default Cauchy prior showed that the data support our expectation that children exposed to advertising-related material are more accurate in categorizing attitudinal (i.e., negative) advertising words, BF₊₀ = 3.09. Furthermore, a Bayesian linear regression analysis showed that there was an effect of dispositional advertising literacy on the speed difference score of categorizing conceptual advertising words. There is moderate evidence that including disliking will improve the model (BF_Inclusion = 4.74). A regular regression analysis showed that the effect was in the expected direction (Table 6). Finally, a Bayesian linear regression analysis showed that there was an effect of dispositional advertising literacy on the speed difference score of categorizing attitudinal (i.e., negative) advertising words. There is very strong evidence that including persuasive intent will improve the model (BF_Inclusion = 30.94). A regular regression analysis showed that the effect was in the opposite direction of our expectation (Table 7).

Table 6. Regression analysis for the effect of dispositional advertising literacy on the speed difference score of categorizing conceptual advertising words in study 2

Variable	b	SE b	β	t	p
Persuasive Intent	85.55	107.69	.15	0.79	.433
Selling Intent	13.83	103.08	.03	0.13	.894
Disliking	−275.52	115.57	−.42	−2.38	.023
Skepticism	−18.11	90.66	−.04	−0.20	.843

Note: R² = .20.

Table 7. Regression analysis for the effect of dispositional advertising literacy on the speed difference score of categorizing attitudinal (i.e., negative) advertising words in study 2

Variable	b	SE b	β	t	p
Persuasive Intent	405.74	127.43	.54	3.18	.003
Selling Intent	−1.68	131.98	−.00	−0.01	.989
Disliking	−281.74	136.76	−.32	−2.06	.047
Skepticism	−5.44	107.28	−.01	−0.05	.960

Note: R² = .36.

Discussion

The results showed that the ALAT was perceived as an easy task by the children who participated in this study. Furthermore, children seemed not to like nor dislike the task. In terms of usability of the measure this is positive. The split-half reliability was good (r_{Spearman-Brown} = .93), indicating that the ALAT is internally consistent. For both predictive and construct validity, the results were less evident. The results showed that disliking was a predictor for categorizing the attitudinal advertising words, which is an indication for predictive validity. Furthermore, we found that children who were exposed to the advertising-related material categorized more negatively valenced attitudinal words as advertising words, which is an indication of construct validity. We additionally found evidence for the interaction between exposure to advertising-related material and dispositional advertising literacy, with one effect in the expected direction and one effect in the opposite direction. In terms of usability and reliability, this study showed that the ALAT is suitable to assess children’s advertising literacy activation. However, in terms of both construct and predictive validity, this study had mixed results. The fact that we did not find support for any of the other expectations regarding construct validity may be due to the stimulus material. Even though the stories were pretested, it is possible that the advertising story was not able to activate children’s advertising literacy, which then in turn could not be assessed with the ALAT. The story was rather short and factual and is in no way a representation of how advertising literacy will be activated in real life. Therefore, we chose to do another study, in which we used real advertising.

Study 3

In the third study, we replicated the experiment in study 2 by using more ecologically valid stimulus material: real child-directed commercials and real clips of children’s news broadcast. We started out with collecting data of children in the same age-range as in study 2 (10–12 year-olds). The full description of this sample and results can be found in Appendix C. After collecting and analyzing the data, we again did not find support for predictive validity, which was contrary to our expectations. We believed that a lack of variance in the dispositional advertising literacy scores (see Table 8) obscured any effects with regard to testing the predictive validity of the ALAT. To increase variance in children’s scores on the dispositional advertising literacy variables, we decided to collect more data among a younger sample. When using frequentist statistics, collecting additional data is considered questionable, as it increases the rate of false-positive results (Schönbrodt, Wagenmakers, Zehetleitner, & Perugini, 2017). For Bayesian statistics, however, collecting additional data is not a problem, as Bayes factors simply provide a numerical value that quantifies whether the data are more compatible with one hypothesis or the other. When the results are inconclusive (as was the case after the first part of the data collection for this study), the researcher can collect additional data until there is support for one of the two hypotheses or until the researcher runs out of time or money (Schönbrodt et al., 2017; Wagenmakers, Lee, Lodewyckx, & Iverson, 2008).

Table 8. Mean and standard deviation scores for the four dispositional advertising literacy constructs in study 2, study 3 (data collected among 10–13 year-olds), and study 3 (data collected among 7–13 year-olds)

	Study 2 (10–12y)		Study 3 (10–13y)		Study 3 (7–13y)
	M	SD	M	SD	M	SD
Persuasive Intent	3.51	0.54	3.31	0.57	3.10	0.71
Selling Intent	3.65	0.57	3.68	0.57	3.38	0.73
Disliking	3.81	0.46	3.73	0.62	3.72	0.64
Skepticism	4.01	0.58	3.76	0.88	3.74	0.85

We chose to collect data among younger children because they have less well-developed dispositional advertising literacy than older children (Moses & Baldwin, 2005; Rozendaal et al., 2011) which would result in more heterogeneous scores on dispositional advertising literacy. Both data collections were performed in the same time of the school year (end of March/beginning of April) with a year interval, guaranteeing the children in both samples were comparable regarding their developmental stage. After collecting the second round of data, the datasets were merged. The method and results of the combined dataset are described below.

Method

Study design

This study was an experiment with a 2 × 2 (activation material: advertising vs news; words: conceptual vs attitudinal) repeated measures design, exactly as in study 2. Children’s advertising literacy was activated with a two-minute video of real advertisements. In the control condition, children saw a two-minute video of a children’s news broadcast.

Participants

Children were recruited from several elementary schools and aftercare facilities in the Netherlands. In this study, 118 children (47.5% boys, Age_range = 7–13 years old, M_age = 10.10, SD_age = 1.32) participated at measurement time 1 and 116 children (48.3% boys, Age_range = 7–13 years old, M_age = 10.18, SD_age = 1.32) participated at measurement time 2. Data were excluded for children who did not participate at both measurement times. This resulted in a drop-out rate of 5.0%, with 114 children remaining in the sample.

Stimulus material: videos

During the pretest mentioned in study 2, we also tested the clips we wanted to use as stimulus material in study 3. All children saw eight television commercials and eight clips of a children’s news broadcasting show which they had to rate on three aspects. First, video identification (Q1: is this a. a commercial?/b. part of a news broadcast?). Second, video aim (Q: is the aim of this commercial to make you a. buy the product?/b. think positively about the product?). And third, video liking (Q3: how much did you like the video?). For the advertising [news] condition, we wanted to select those videos that were strongly identified as advertising [news], and not identified as news [advertising]. It was also important that the videos in both conditions scored equally high on liking.

Finally, we also measured product liking and product desire for the products that were in the eight television commercials. We selected products that scored average on these two scales, because we did not want to select products that were extremely desirable, nor extremely undesirable. Based on the pretest we selected four advertising videos and four news broadcast clips. In the advertising video, we added a banner before and after the commercials, so it looked like a real commercial block. In the news condition, we added the intro and outro for the regular news broadcast for children. The advertising and news videos were constructed with Windows Movie Maker and both lasted 2 minutes and 16 seconds.

Procedures

The procedure was the same as described in study 2, except for the stimulus material.

Measures

The following variables were measured within this study.

ALAT

The ALAT was exactly the same as in study 2. To test the hypotheses, we took the same approach as described in study 2.

Questionnaire measurements I/II

See description in study 2.

Questionnaire measurements III: dispositional advertising literacy

As in study 2, children in the news condition received additional questions related to their conceptual and attitudinal advertising literacy, as used in Rozendaal et al. (2016). Conceptual advertising literacy was divided into (knowledge of) persuasive intent (3 items, α = .65, M = 3.11, SD = 0.71) and selling intent (2 items, α = .56, M = 3.39, SD = 0.73, r(114) = .40, p< .001). Attitudinal advertising literacy was divided into disliking (7 items, α = .74, M = 3.70, SD = 0.65) and skepticism (3 items, α = .86, M = 3.74, SD = 0.84).

Plan of analysis

All analyses are the same as in study 2.

Results

Usability

Children scored just above the midpoint of the scale regarding likability of the ALAT (M = 2.80, SD = 0.46) and thought the task was easy (M= 1.75, SD = 0.62).

Reliability

A split-half reliability analysis showed that the Spearman-Brown correlation between the first 30 trials and the second 30 trials was .96.

Validity

As in study 2, only the hypotheses that are supported by the data are described in greater detail. Table 9 gives an overview of all Bayes factors and whether there is support for the hypotheses. Table 10 gives the means and standard deviations for the accuracy and speed of categorization, broken down into condition and word type.

Table 9. Overview of expectations regarding validity, Bayes factors and whether or not the data support the expectations for study 3

Expectation	Bayes factor	Support	Additional information
PV 1a	22.15	Support	Support for including Selling Intent
PV 1b	≤ 1.35	Inconclusive	Inconclusive support for including dispositional advertising literacy in the model
PV 2a	15.31/5.34	Support	Support for including Persuasive Intent and Skepticism
PV 2b	23.58	Support	Support for including Skepticism
CV 1a	0.30	No support	It is (1/0.30 =) 3.33 times more likely that there is no effect vs. that there is an effect
CV 1b	0.13	No support	It is (1/0.13 =) 7.69 times more likely that there is no effect vs. that there is an effect
CV 2a	0.14	No support	It is (1/0.14 =) 7.14 times more likely that there is no effect vs. that there is an effect
CV 2b	0.94	Inconclusive	It is equally likely that there is an effect as that there is no effect
I 1a	≤0.54	Inconclusive	Inconclusive support for an interaction between exposure and dispositional ad literacy
I 1b	≤0.83	Inconclusive	Inconclusive support for an interaction between exposure and dispositional ad literacy
I 2a	3.52	Support	Support for including Persuasive Intent
I 2b	≤0.31	No support	No support for an interaction between exposure and dispositional ad literacy

Note 1: PV = Predictive Validity, CV = Construct Validity, I = Interaction between exposure difference score and dispositional advertising literacy (part of CV).

Note 2: 1a: effect on accuracy of categorizing conceptual advertising words, 1b: effect on accuracy of categorizing attitudinal advertising words, 2a: effect on speed of categorizing conceptual advertising words, 2b: effect on speed of categorizing attitudinal advertising words.

Table 10. Mean and standard deviation scores for categorization and speed of categorization for the advertising words in study 3

	Advertising condition		News condition
	M	SD	M	SD
Categorization of conceptual words	1.80	0.22	1.77	0.25
Speed of categorizing conceptual words	1308.26	369.03	1321.42	319.42
Categorization of attitudinal words	1.61	0.45	1.60	0.42
Speed of categorizing attitudinal words	1495.51	486.76	1567.18	446.50

Note: Categorization can vary between 0 and 2, since each word is presented twice in the task.

Predictive validity

A Bayesian linear regression analysis showed that children with a higher level of dispositional advertising literacy (selling intent) were more accurate in categorizing the conceptual advertising words. There is strong evidence that including selling intent will improve the model (BF_Inclusion = 22.15). A regular regression analysis showed that the effect was in the expected direction (Table 11). Furthermore, a Bayesian linear regression analysis showed that children with a higher level of dispositional advertising literacy (persuasive intent and skepticism) were faster in categorizing the conceptual advertising words. There is strong evidence that including persuasive intent (BF_Inclusion = 15.31) and moderate evidence that including skepticism (BF_Inclusion = 5.34) will improve the model. A regular regression analysis showed that both effects were in the expected direction (Table 12). Finally, a Bayesian linear regression analysis showed that children with higher levels of dispositional advertising literacy (skepticism) were faster in categorizing the attitudinal (i.e., negative) advertising words. There is strong evidence that including skepticism will improve the model (BF_Inclusion = 23.58). A regular regression analysis showed that the effect was in the expected direction (Table 13).

Table 11. Regression analysis for the effect of dispositional advertising literacy on the accuracy of categorizing conceptual advertising words in study 3

Variable	b	SE b	β	t	p
Persuasive Intent	0.01	0.04	.03	0.25	.805
Selling Intent	0.09	0.03	.29	2.63	.010
Disliking	0.07	0.03	.19	2.11	.037
Skepticism	0.02	0.02	.06	0.68	.497

Note: R² = .14.

Table 12. Regression analysis for the effect of dispositional advertising literacy on the speed of categorizing conceptual advertising words in study 3

Variable	b	SE b	β	t	p
Persuasive Intent	−116.39	45.07	−.28	−2.58	.011
Selling Intent	−58.06	43.56	−.14	−1.33	.185
Disliking	37.34	41.25	.08	0.91	.367
Skepticism	−82.12	31.18	−.23	−2.63	.010

Note: R² = .21.

Table 13. Regression analysis for the effect of dispositional advertising literacy on the speed of categorizing attitudinal advertising words in study 3

Variable	b	SE b	β	t	p
Persuasive Intent	−60.22	64.33	−.10	−0.94	.351
Selling Intent	−98.88	62.41	−.17	−1.58	.116
Disliking	2.56	58.91	.00	0.04	.965
Skepticism	−141.05	44.59	−.29	−3.16	.002

Note: R² = .15.

Construct validity

We found no support for our expectations that children exposed to advertising-related material are more accurate or faster in categorizing advertising-related words. However, we did find an effect of dispositional advertising literacy on the speed difference score of categorizing conceptual advertising words. There is moderate evidence that including persuasive intent will improve the model (BF_Inclusion = 3.52). A regular regression analysis showed that the effect was in the expected direction (Table 14).

Table 14. Regression analysis for the effect of dispositional advertising literacy on the speed difference score of categorizing conceptual advertising words in study 3

Variable	b	SE b	β	t	p
Persuasive Intent	−114.87	55.86	−.24	−2.06	.042
Selling Intent	2.62	53.99	.01	0.05	.961
Disliking	18.44	51.13	.04	0.36	.719
Skepticism	−15.75	38.65	−.04	−0.41	.684

Note: R² = .06.

Discussion

The third study showed that the ALAT can be considered as a usable and reliable measurement task. The task was not liked, nor disliked, by the children participating in this study. Children also indicated that the task was not very difficult. This is in line with the results in study 2. Reliability of the measure is also in line with the second study and indicates that the ALAT is internally consistent. This study did provide much support for the predictive validity of the ALAT. The results showed that children who scored higher on dispositional conceptual advertising literacy (persuasive or selling intent) were more accurate in categorizing conceptual advertising words and also faster in doing so. Furthermore, children who scored higher on dispositional attitudinal advertising literacy (skepticism) were faster in categorizing both conceptual and attitudinal advertising words.

In terms of construct validity, the outcomes of study 3 provided little indication that the ALAT was able to measure advertising literacy that was activated by the block of television commercials. That is, Bayesian analyses of the factor prime indicated moderate support for the null hypothesis, suggesting that the ALAT was not sensitive to pick up the more subtle effects of advertising literacy activation in this experiment. Notably though, analyses of the interaction between dispositional variables and the prime indicated an effect of understanding of persuasive intent on the speed difference score of categorizing conceptual words. Children with higher scores on understanding of persuasive intent were faster in categorizing conceptual advertising words after being exposed to a block of television commercials. This finding is in line with the expectation that individual variations in advertising literacy may influence if, and to what extent, advertising literacy is activated in response to the viewing of advertising.

General discussion

The aim of the current study was to develop and test a measurement instrument that is able to assess children’s advertising literacy activation. Overall, the results showed that the ALAT is a usable and reliable measurement tool. However, in terms of validity, the results are mixed. With regard to predictive validity, we expected that children’s dispositional advertising literacy scores would predict their score on the ALAT. In the second study, we indeed found that children who had higher levels of dispositional attitudinal advertising literacy (disliking) categorized more negatively valenced attitudinal words as advertising words, but we did not find any other support for predictive validity. The third study showed more extensive support for predictive validity of the ALAT. As expected, children who had higher levels of dispositional conceptual and attitudinal advertising literacy showed a stronger activation of their literacy. The differences between study 2 and 3 can be explained by looking at the sample in both studies. The children who participated in study 2 were fairly homogeneous with regard to their dispositional advertising literacy. There was probably not enough variation on the four advertising literacy dimensions to detect any effects. The results of study 3 are more robust, due to a sample with a broader age range and, therefore, more heterogeneous scores on dispositional advertising literacy.

It is important to note here that, although the ALAT was developed with children with an older age range (i.e., 10–12 years-old) in study 1, the ALAT can also be used with younger children (as was done in study 3). We considered the suitability of the ALAT for younger children in three ways. As a first step, we only collected data among a small subsample of ten children aged 7–10 years old. During this data collection, we observed these children attentively to check whether they performed the task correctly. Second, we asked these 10 children afterward what they thought about the task and they indicated that they thought the task was easy. Third, after data collection was complete, we analyzed the data for the younger children and saw that these children also performed well on the task (which was derived from the fact that they were very accurate in categorizing the conceptual words). Therefore, we are confident that the ALAT can also be used among younger children.

With regard to construct validity, we expected that exposure to advertising would lead to the activation of children’s advertising literacy and that this would be reflected in the accuracy and speed of children’s word categorization performance. In study 2 we found a main effect of priming on the accuracy of categorizing attitudinal words, but no effects on the other three indicators of advertising literacy activation. Similarly, in study 3 we did not find any effects of priming. In other words, children’s performance on the ALAT was unrelated to the stimulus material they were exposed to. The fact that we did not find strong priming effects in both studies may be due to the construction of the task. The current version of the ALAT includes the word “advertising” several times, which may also have activated children’s advertising schema in the control condition. This is a limitation that should be addressed in future research.

Although we did not find a strong main effect of priming, both studies showed effects of priming for a subset of participants. In study 2 we found that children with higher levels of advertising disliking were faster in categorizing conceptual words after they were primed with a story about advertising. In study 3 we found that children with more knowledge of advertising’s persuasive intent were more accurate in categorizing conceptual words associated with advertising after they were primed with actual television commercials. These findings suggest that the ALAT may be sensitive to pick up activation of advertising literacy when children are exposed to advertising-related material. However, this sensitivity appears to be limited to children with relatively high levels of dispositional advertising literacy.

Suggestions for future research

Although our research shows promising results regarding the usability, reliability, and predictive validity of the ALAT, more research is needed to further refine and improve the measurement instrument. Most importantly, future research is needed to determine if the specific findings reported in study 2 and 3 replicate and if these reflect a true sensitivity of the ALAT to measure aspects of advertising literacy activation, or whether these findings reflect false positives caused by the substantial number of statistical tests that were run.

Furthermore, future research may focus on one of the following four directions. First, to further develop the ALAT, future research could adapt the ALAT in such a way that children are not instructed to relate the words to advertising or news, but to relate the words to the stimulus material they were exposed to (i.e., in order to not prime them with the concept of advertising which is a limitation of the current version of the task). The instruction of the task would then be to ask children whether they think the word they see on the screen is related to the video they just saw and to press Yes if they think it is, or No if they think it is not. Second, to further develop the ALAT, future research could focus on variables that may be related to ALAT scores, such as reading ability. Children who can read faster may also respond faster to words presented on the screen. It is important to gain more insight into the factors associated with task performance. This way, alternative explanations for the scores can be ruled out and it is possible to see whether the ALAT really measures activation.

Third, for further application of the ALAT, future research could look at other contexts in which an activation task, such as the ALAT, can be used. For instance, by adapting the words that are used in the task, it could be used to assess media literacy or news literacy activation. Of course it is important to first get an overview of the words that are related to these types of literacy by pretesting among the intended target group before constructing an adaptation of the ALAT. Fourth, it can be argued that the disadvantages of using self-report questionnaire measures are also valid for research with adolescents and adults. Therefore, it would be interesting to test the ALAT among an adolescent or adult sample. As their advertising schema is probably more sophisticated a pretest should first be conducted to get an understanding of which words they associate with advertising and news (or another reference category). The words that are used in the ALAT may then differ based on this pretest.

Scientific and societal implications

The current research has both scientific as well as societal implications. From a scientific perspective, we have not (yet) been able to develop a valid measurement instrument that can be used to assess children’s advertising literacy activation. Although the ALAT is sensitive for children’s level of dispositional advertising literacy (i.e., children with better developed dispositional advertising literacy show better scores on the ALAT), it is not sensitive for exposure to advertising. As indicated earlier, this could be the result of the design of the task. However, another explanation for the lack of sensitivity for exposure to advertising-related material could be that children actually do not activate their advertising literacy when they are exposed to advertising. Although it is a common assumption that exposure to advertising leads to the activation of advertising literacy (e.g., De Jans et al., 2019; Hudders et al., 2017), it is by no means certain that this is the case. The studies in this paper prove that more research on children’s advertising literacy activation is needed.

Researchers who are interested in using the ALAT should know that this task is relatively easy to use in various kinds of research settings (e.g., research lab, school). In order to use the ALAT, one needs a computer with PsychoPy (or similar) software and a device that is able to very accurately register reaction times (e.g., a button box). The task can be presented as a game, which makes it more appealing for children than, for instance, filling out a questionnaire. Total administration of the current version of the task takes approximately five to seven min. However, it needs to be noted that some adaptations may be necessary in order to use the ALAT correctly. In the current version of the task, the concept “news” was chosen for the control condition, because in the country where the study took place child-friendly news programs are offered to children. As a result, news is part of many children’s media diet and, therefore, easily understandable for them. This may not be true for other countries. Therefore, we advise researchers to adapt the task if they think the concept of news would be unsuitable, thereby taking into account the following conditions: the concept relates to some form of media genre or type, the concept is easily understandable, and the concept is clearly distinguishable from and unrelated to advertising.

From a societal perspective, a measurement instrument that is able to assess whether children can activate their advertising literacy is highly relevant because it can be used to evaluate advertising literacy programs. For instance, the ALAT could be used as a pre- and posttest measure for the effectiveness of an advertising education program, to see whether the program increased children’s ability to activate their advertising literacy. This gives the measurement instrument practical value, but of course the measurement instrument first has to be tested more thoroughly.

To conclude, the current study is a first step toward the development and testing of a research measure that can reveal the extent to which children activate their advertising literacy when they are exposed to advertising. One of the most prevalent assumptions in both the scientific and societal debate about the fairness of child-targeted advertising is that children’s advertising literacy can help them defend against unwanted advertising effect. In order to do this, children have to be able to activate their advertising literacy when they are confronted with advertising. However, the question is whether children actually do so. The ALAT can help answer this question, thereby making an important contribution to the child and advertising debate.

Acknowledgments

The authors would like to thank Annemarie Nanne for her assistance with programming the indirect measurement task in PsychoPy.

Disclosure statement

All authors declare that they have no conflict of interest.

Notes

1. Jeugdjournaal is the name of a Dutch news broadcast especially made for children.

2. The NOS is the biggest Dutch news broadcasting agency.

Appendix A.

Stories used in study 2

Note: in the original language the stories have the exact same length and contain the words used in the ALAT (i.e., in the advertising story each advertising-related word in the ALAT is used once, in the news story each news-related word in the ALAT is used once).

Advertising story

James and Lisa come home after a long day at school. Get inside quickly, it is cold and wet outside. They sit on the couch and turn on the television. Yes! Their favorite program is on. When the program is finished, it is time for the commercials. “That looks cool and it is on offer!” says James during one of the commercials, “I want to have that!” Lisa laughs “Haha, the commercial persuaded you”. “Not at all!” says James. “But it did” says Lisa. “Sometimes it is hard to see, but commercials try to tempt you to buy the product”. “Is that really true?” asks James. “Yes” Lisa responds, “they pretend the price is suddenly lower, like they have a discount, to make you go to the store.” Lisa tells more: “Commercials are also called advertisements. They only show the good things don’t always tell the truth”. James asks: “But are commercials lying?” “Yes, some commercials lie and are fake” says Lisa. “You have to pay close attention, so they won’t mislead you”. “You are absolutely right” says James, “commercials are stupid and boring, and very often also annoying.” James turns off the television. “Can we go play soccer?”. James runs outside and grabs the ball. “Can you give that to me?” Lisa asks. “You also want to play?” asks James, “that is peculiar”. They play outside briefly, because soon their father calls “Dinner is ready!”

News story

James and Lisa come home after a long day at school. Get inside quickly, it is cold and wet outside. They sit on the couch and turn on the television. Yes! Their favorite program is on. When the program is finished, it is time for the news. “Look!” says Lisa. “That anchor, he works for the NOS. They always broadcast news.” “But what is news?” asks James. “In the news they tell you about events that happen all over the world. As a result, you will learn things about the Netherlands and other countries” Lisa explains. “Oh” says James “that is very informative”. Lisa nods “Indeed! There is a reporter who gathers information and they broadcast it in the news.” “I think the news is interesting” says James. “But sometimes it is also exciting”. Lisa agrees with him. “But news is also very important. Luckily there is also a news broadcast especially for children, such as the Jeugdjournaal. They explain everything very well and make it less scary.” “Yes, that is correct” says James, he agrees with Lisa. Suddenly he laughs: “And sometimes it is also funny, when they show a fun video”. After the weather forecast James turns the television off. “Can we go play soccer?”. James runs outside and grabs the ball. “Can you give that to me?” Lisa asks. “You also want to play?” asks James, “that is peculiar”. They play outside briefly, because soon their father calls “Dinner is ready!”

Appendix B.

Dispositional advertising literacy questionnaire

The questionnaire is based on the Conceptual Advertising Literacy Scale for Children and the Attitudinal Advertising Literacy Scale for Children by Rozendaal et al. (2016).

Table

Component	Items	Response categories
Understanding selling intent	Is the goal of advertising to make you buy the advertised products? Is the goal of advertising to make you ask your parents to buy the advertised products?	1 = No, definitely not 2 = No, I don’t think so 3 = Yes, I think so 4 = Yes, for sure
Understanding persuasive intent	Is the goal of advertising to make you want to have the advertised products? Is the goal of advertising to make you think positively about the advertised products? Is the goal of advertising to make you feel positively about the advertised products?	1 = No, definitely not 2 = No, I don’t think so 3 = Yes, I think so 4 = Yes, for sure
Disliking	How often do you think advertising is fun? (R) How often do you think advertising is funny? (R) How often do you think advertising is beautiful (R) How often do you think advertising is boring? How often do you think advertising is interesting? (R) How often do you think advertising is stupid? How often do you think advertising is annoying?	1 = Never 2 = Almost never 3 = Sometimes 4 = Almost always 5 = Always
Skepticism	How often do you think advertising is honest? (R) How often do you think advertising tells the truth? (R) How often do you think you can believe advertising? (R)	1 = Never 2 = Almost never 3 = Sometimes 4 = Almost always 5 = Always

Appendix C.

Description of study 3a

Description of the sample and results for study 3a, the first part of data collection of study 3.

Participants

Children were recruited from an elementary school in the Netherlands. In this study, 57 children participated (50.0% boys, Age_range = 10–13 years old, M_age = 11.14, SD_age = 0.81) at measurement time 1 and 58 children participated (51.7% boys, Age_range = 10–13 years old, M_age = 11.17, SD_age = 0.80) participated at measurement time two.

Results

Usability Analyses showed that children scored just above the mean of the scale regarding likability of the ALAT (M = 2.81, SD = 0.44) and that they thought the task was very easy (M= 1.80, SD = 0.65).

Reliability A split-half reliability analysis showed that the Spearman-Brown correlation between the first thirty trials and the second 30 trials is.94.

Validity Only the hypotheses that are supported by the data are described in greater detail. Table C1 gives an overview of all Bayes factors and whether there is support for the hypotheses. Table C2 gives the means and standard deviations for the accuracy and speed of categorization, broken down into condition and word type.

Predictive Validity A Bayesian Linear Regression analysis showed that children with a higher level of dispositional attitudinal advertising literacy (Disliking) were more accurate in categorizing the conceptual advertising words. There is very strong evidence that including Disliking will improve the model (BF_Inclusion = 39.93), which is in line with our expectation. A regular regression analysis showed that a one-point increase on Disliking leads to.10 more conceptual advertising words being categorized as advertising words, b* = .41.

Furthermore, a Bayesian Linear Regression analysis showed that children with a higher level of dispositional attitudinal advertising literacy (Skepticism) were faster in categorizing the attitudinal advertising words. There is moderate evidence that including Skepticism will improve the model (BF_Inclusion = 6.23), which is in line with our expectation. A regular regression analysis showed that a one-point increase on Skepticism leads to 157.70 ms decrease in reaction time for categorizing the attitudinal words, b* = −.35.

Construct Validity A one-sided Bayesian paired samples t-test with the default Cauchy prior showed that the data support our expectation that children exposed to advertising-related material are faster in categorizing the attitudinal (i.e., negative) advertising words, BF₋₀ = 13.59. We did not find support for our other expectations regarding construct validity.

Table C1. Overview of expectations regarding validity, Bayes factors and whether or not the data support the expectations for study 3a

Expectation	Bayes factor	Support	Additional information
PV 1a	39.93	Support	Support for including Disliking
PV 1b	≤ 1.94	Inconclusive	Inconclusive support for including dispositional advertising literacy in the model
PV 2a	≤ 1.63	Inconclusive	Inconclusive support for including dispositional advertising literacy in the model
PV 2b	6.23	Support	Support for including Skepticism
CV 1a	1.11	Inconclusive	It is equally likely that there is an effect as that there is no effect
CV 1b	1.50	Inconclusive	It is equally likely that there is an effect as that there is no effect
CV 2a	0.86	Inconclusive	It is equally likely that there is an effect as that there is no effect
CV 2b	13.59	Support	It is 13.59 times more likely that there is an effect vs. that there is no effect
I 1a	≤ 1.21	Inconclusive	Inconclusive support for an interaction between exposure and dispositional ad literacy
I 1b	≤0.49	Inconclusive	Inconclusive support for an interaction between exposure and dispositional ad literacy
I 2a	≤0.72	Inconclusive	Inconclusive support for an interaction between exposure and dispositional ad literacy
I 2b	≤0.49	Inconclusive	Inconclusive support for an interaction between exposure and dispositional ad literacy

Note 1: PV = Predictive Validity, CV = Construct Validity, I = Interaction between exposure difference score and dispositional advertising literacy (part of CV).

Note 2: 1a: effect on accuracy of categorizing conceptual advertising words, 1b: effect on accuracy of categorizing attitudinal advertising words, 2a: effect on speed of categorizing conceptual advertising words, 2b: effect on speed of categorizing attitudinal advertising words.

Table C2. Mean and standard deviation scores for categorization and speed of categorization for the advertising words in study 3a

	Advertising condition		News condition
	M	SD	M	SD
Categorization of conceptual words	1.88	0.13	1.83	0.23
Speed of categorizing conceptual words	1174.24	293.54	1241.61	317.56
Categorization of attitudinal words	1.68	0.43	1.61	0.41
Speed of categorizing attitudinal words	1370.50	447.47	1522.45	438.71

Note: Categorization can very between 0 and 2, since each word is presented twice in the task.