Development of the Persuasion Knowledge Scales of Sponsored Content (PKS-SC)

Abstract

Despite the popularity of the Persuasion Knowledge Model, and its persistent relevance given the omnipresence of sponsored content (e.g. brand placement in TV programmes and video games, or paid product reviews in blogs), the way scholars measure persuasion knowledge varies widely. This study aims to develop valid and reliable scales for standardized measurement of consumers' persuasion knowledge of sponsored content. In three phases, we developed the Persuasion Knowledge Scales of Sponsored Content (PKS-SC) that measures nine components: (1) recognition of sponsored content, (2) understanding of selling and persuasive intent, (3) recognition of the commercial source of sponsored content, (4) understanding of persuasive tactics, (5) understanding of the economic model, (6) self-reflective awareness of the effectiveness of sponsored content, (7) skepticism toward sponsored content, (8) appropriateness of sponsored content, and (9) liking of sponsored content. All scales have good to appropriate validity and reliability. Recommendations for future research are discussed.

Keywords:

• Persuasion knowledge
• sponsored content
• scale development
• advertising
• skepticism
• appropriateness
• liking

Introduction

Since its introduction in 1994, the Persuasion Knowledge Model (PKM; Friestad and Wright 1994) has received a lot of scholarly attention. The model describes people's personal knowledge about the goals and tactics of persuasion agents, and about how people use this knowledge to cope with persuasive attempts. With 2508 cites on Google Scholar to date, the model has proven relevant in numerous contexts such as interpersonal sales (Campbell and Kirmani 2000), word-of-mouth (Hamilton, Vohs, and McGill 2014; Packard, Gershoff, and Wooten 2016), brand placement (Matthes, Schemer, and Wirth 2007; Matthes and Naderer 2016), advergames (Van Reijmersdal et al. 2015), corporate social responsibility marketing (Pomering and Dolnicar 2009), religious and pharmaceutical marketing (McGraw, Schwartz, and Tetlock 2012), and various types of online advertising (Boerman, Willemsen, and Van Der Aa 2017; Ham 2017; Tutaj and van Reijmersdal 2012; Wojdynski and Evans 2016).

Persuasion knowledge is believed to develop throughout life, but in general, adults are assumed to be able to distinguish ads from non-commercial content and to understand advertisers' intent and tactics. In addition, adults are generally able to form their own beliefs about persuasive messages, such as advertisements, and use their cognitive and attitudinal resistance strategies to cope with persuasive messages. These assumptions are challenged by covert marketing tactics such as sponsored content.

Sponsored content, which is also referred to as embedded advertising, native advertising, content marketing, and brand or product placement, is the purposeful integration of brands and products into non-commercial media content paid for by a sponsor (Van Reijmersdal, Neijens, and Smit 2009). Examples include brand placements in TV programmes or video games, paid product reviews on blogs, sponsored news articles, and social media posts commissioned by a brand. Because sponsored content is made to resemble entertainment or editorial content, it blurs the lines between non-commercial and commercial content. As a result, adults may have difficulties recognizing such content as advertising. Research provides evidence for adults' inability to activate and use their persuasion knowledge in response to different varieties of sponsored content, and shows that transparency tools (e.g. sponsorship disclosures) developed by the industry and regulators can help consumers recognize advertising (e.g. Boerman, Van Reijmersdal, and Neijens 2012; Wojdynski and Evans 2016).

Despite the popularity of the PKM, and its persistent relevance given the omnipresence of sponsored content, the way scholars use and measure persuasion knowledge varies widely. Overall, there is no standard measure of persuasion knowledge, and researchers use a variety of concepts, scales, and methods (Ham, Nelson, and Das 2015; Wojdynski, Evans, and Hoy 2018). It is important to have standard measures to be able to gain insights into people's knowledge of sponsored content, to compare knowledge between different types of sponsored content, between media, or between people. Moreover, standard measures of persuasion knowledge are essential to create an accumulating body of research that allows comparisons between studies.

This study aims to develop valid and reliable scales for standardized measurement of consumers' persuasion knowledge of sponsored content (i.e. the Persuasion Knowledge Scales of Sponsored Content, PKS-SC). The scales intend to measure dispositional persuasion knowledge (see Ham, Nelson, and Das 2015), meaning that they focus on measuring consumers' persuasion knowledge of sponsored content in general, instead of measuring their persuasion knowledge of a specific instance of sponsored media content (e.g. a particular brand placement in a video game). Researchers have suggested that persuasion knowledge involves both the conceptual understanding of the intent, source, and tactics of advertising, and evaluative beliefs about advertising (Boerman, Van Reijmersdal, and Neijens 2012; Ham, Nelson, and Das 2015; Rozendaal, Opree, and Buijzen 2016). Therefore, we aim to develop scales that can be used to measure both the conceptual and the evaluative dimensions – divided over nine components – of dispositional persuasion knowledge of sponsored content.

To develop the PKS-SC, we tested the measurement properties for three different media formats in which sponsored content is frequently embedded: video games, television programmes, and blogs (PQ Media 2015). These media formats differ on various elements that can affect people's processing of and responses to sponsored content, namely modality (i.e. video games and television programmes are audio-visual, blogs are visual), pacing (i.e. television programmes are externally paced, blogs internally, and video games both externally and internally), and interactivity (i.e. video games are interactive, while television programmes and blogs are not). Testing the scales for various media formats allowed us to ascertain their applicability to sponsored content in the diverse contemporary media landscape.

Components of persuasion knowledge

The original PKM (Friestad and Wright 1994) is applicable across a variety of persuasion contexts and is not restricted to advertising formats such as sponsored content. The PKM proposes that receivers of a persuasion attempt have knowledge about persuasion, the persuasion agent (the one responsible for designing and constructing the persuasive attempt), and the topic of the persuasive message. The receiver can use this knowledge to cope with the persuasion attempt. This coping usually involves maintaining control over the outcome of the attempt and achieving personal goals.

Several theoretical and review articles provide insights on the conceptualization of persuasion knowledge in the context of advertising (see Ham, Nelson, and Das 2015; Hudders et al. 2017; John 1999; Rozendaal et al. 2011; Wright, Friestad, and Boush 2005). In their theoretical essay, Rozendaal et al. (2011) argue that dispositional persuasion knowledge encompasses two dimensions: a conceptual dimension and an evaluative one. The conceptual dimension includes the recognition and understanding of advertising in general, and of certain formats in particular (e.g. sponsored content). The evaluative dimension includes holding a critical attitude toward advertising in general and specific advertising formats. The authors state that although most existing empirical studies have concentrated on the two basic components of conceptual persuasion knowledge (i.e. recognition of advertising and understanding of its selling intent), conceptual persuasion knowledge also encompasses a more sophisticated understanding of advertising, such as an understanding of its persuasive nature and tactics. Besides that, they emphasize the importance of the evaluative dimension of persuasion knowledge, including disliking of and skepticism towards advertising as the most important components. Hudders et al. (2017) add to this conceptualization by introducing a moral component to the evaluative dimension of advertising, namely the perceived appropriateness of advertising. The extensive review of research by Ham, Nelson, and Das (2015) adds to these insights on the conceptualization of persuasion knowledge by showing that in the advertising literature, persuasion knowledge has been conceptualized, operationalized, and measured in various ways. This indicates that researchers indeed perceive persuasion knowledge as a broad theoretical concept including different underlying components that are either conceptual or evaluative in nature and vary in level of complexity.

Based on a careful study of these theoretical and review articles on persuasion knowledge, we identified six components of persuasion knowledge that are conceptual in nature (i.e. knowledge and understanding) and together may encompass the conceptual dimension of persuasion knowledge. The components differ in their level of complexity, meaning that they reflect relatively simple (e.g. recognition of sponsored content) to more complex and abstract competences (e.g. self-reflective awareness of the effectiveness of sponsored content). In the process of identifying these components, we specifically focused on components that are relevant in the context of sponsored content.

The first four conceptual components reflect people's basic understanding and recognition of sponsored content, its intent, its source, and tactics, which are common components used to measure conceptual persuasion knowledge (Ham, Nelson, and Das 2015; Rozendaal et al. 2011). We added a fifth component reflecting people's understanding of the economic model behind sponsored content which is based upon a measure of people's objective knowledge about the industry (Moreau, Krishna, and Harlam 2001). This component reveals how much a person actually understands about the financial model and industry behind sponsored content. The sixth component addresses self-reflective persuasion knowledge. This component is based on prior studies addressing the concepts ‘self-confidence of persuasion knowledge’ (Bearden, Hardesty, and Rose 2001), ‘perceptions of advertising effectiveness’ (Celsi and Gilly 2010), and ‘perceptions of product placement on self and others’ (Nelson and McLeod 2005), and assesses people's confidence in inferring advertisers’ intent and knowing how to cope with that intent. Prior studies have shown that the perceived effects of sponsored content on the self vary from the perceived effects on others (third-person perception, Ham and Nelson 2016; Nelson and McLeod 2005). Therefore, we divided this component into perceptions of the effect of sponsored content on the self, and on others. The final six conceptual components are:

1. recognition of sponsored content – differentiating sponsored content from other media content, which is an important prerequisite for consumers to activate the other components of persuasion knowledge (i.e. without awareness of the persuasion attempt, activation of persuasion knowledge is unlikely);

2. understanding of the selling and persuasive intent of sponsored content – understanding that the aim of sponsored content is to sell products and that it attempts to influence consumers' behaviour by changing their mental states, for instance their attitudes and cognitions about a product;

3. recognition of the commercial source of sponsored content – understanding who pays for the showing and the mentioning of a brand in media content;

4. understanding of the persuasive tactics in sponsored content – understanding specific strategies underlying sponsored content, such as the hiding of the persuasive intent, and the linking of a brand to an emotionally pleasing context;

5. understanding of the economic model of sponsored content – understanding that media content (e.g. video games) would not be free or would be more expensive, or even not have been made at all, without brands paying for showing their product or brand in the media content;

6. self-reflective awareness of the effectiveness of sponsored content – perceptions of the effectiveness of sponsored content on self and others (e.g. thinking about and reflecting on the effect of sponsored media content on one's own feelings, thoughts, and behaviour).

   Additionally, we distinguished three components that are evaluative in nature. Skepticism and (dis)liking are distinctive evaluative components that are commonly used to measure people's attitudinal responses to advertising (Ham, Nelson, and Das 2015; Rozendaal et al. 2011). As suggested by Hudders et al. (2017), we add a moral component named ‘appropriateness’:

7. skepticism toward sponsored content – the tendency toward disbelief of sponsored content;

8. appropriateness of sponsored content – beliefs about the moral appropriateness of sponsored content;

9. (dis)liking of sponsored content – a general attitude toward sponsored content.

These nine components form the basis of the scales developed in this study, which we refer to as the PKS-SC.

Phase 1: developing the scales and testing face validity

Based on existing measures, as reviewed by Ham, Nelson, and Das (2015), we created a first draft of the PKS-SC. To be able to measure people's ability to recognize sponsored content, the survey included three screenshots of video games that showed cases of video games including no brand, a subtly placed brand, or a prominently placed brand.

To test this first draft on face validity, language, and clarity, we conducted a small online focus group in December 2015. We presented the scales, applied to sponsored content in video games, to a small group of native English speakers (N = 11), who were faculty of or graduate students in Communication Science at the University of Amsterdam. This online focus group was asked to fill out the complete survey and to provide their suggestions and feedback about the specific questions, items, and screenshots. Furthermore, the respondents were asked to indicate whether the questions and response categories were clear, understandable, and grammatically correct, had no spelling mistakes, and did not involve jargon. We also asked them to indicate whether they had any suggestions for improvement.

As the respondents were faculty or graduate students in Communication Science, and can thus be perceived as experts in the field, the focus group provided us the opportunity to test the face validity of the scale qualitatively. Although the respondents reported some textual mistakes and recommendations, overall the respondents agreed that the test was about persuasion knowledge and sponsored content, and agreed on the components and items of the scale.

Based on the responses, we rephrased some introductory texts, questions, and response categories. Moreover, the first draft of the PKS-SC only included one general question to measure recognition of sponsored content (i.e. ‘Have you ever seen advertising in a game?’). Respondents remarked that this causes some confusion with respect to what we meant by ‘advertising.’ To address this confusion, we decided to add a second general question asking about brands within video games (i.e. ‘Have you ever seen brands within video games?’). In this way, we hoped to develop a more nuanced measure of the recognition of sponsored content. Furthermore, based on some suggestions from the focus group, we decided to change the stimuli that were shown for the three cases from screenshots to short video clips of a video game being played by someone else.

In addition, one respondent rightfully stated that it was unclear what was meant by ‘games’ (‘I’m not totally sure what you mean by game. Game could be a board game, like monopoly, or it could be a sports match’). Therefore, we narrowed down ‘in a game’ to ‘within video games’ and included a short introductory text explaining what we meant by ‘video games’ (i.e. ‘The questions in this survey are about video games. Video games are all games you play on a computer, phone, or tablet, or using a console [such as Xbox, Play-station, etc.]. This can be online and offline.’)

Phase 2: pre-testing the scales

To test the second draft of the PKS-SC applied to video games, we conducted Pre-test 1 amongst 62 college students (82% female, age M = 20.35, SD = 1.55) in March 2016. Overall, the items' descriptive statistics, preliminary factor analyses, and Cronbach's alphas for this pre-test revealed several minor issues, which were corrected by rephrasing some sentences. For instance, one of the semantic differential items measuring skepticism (biased – not biased) was reverse-coded, causing erratic scores.

After having tested two drafts of the PKS-SC applied to video games, we also created and pre-tested the PKS-SC applied to sponsored content in television programmes and to sponsored content in blogs. In May 2016, Pre-test 2 was filled out by 100 college students (83% female, age M = 21.45, SD = 1.93) who were randomly assigned to one of the two medium types (TV n = 48, blogs n = 52). For both television and blogs, the descriptive statistics, preliminary factor analyses, and Cronbach's alphas were comparable to those for video games.

Furthermore, for all medium types, the two general questions to measure Recognition of Sponsored Content (i.e. ‘Have you ever seen advertising/brands within video games?’) resulted in different open answers. This assured us of the importance of asking both questions to tap into people's ability to recognize sponsored content.

Phase 3: item and scale analysis and validation

Validation of the PKS-SC

The two pre-tests resulted in 76 items aiming to measure the nine components. We administered a two-wave survey with the aim to shorten the scales based on psychometric analyses, and to validate these shorter scales. We first tested the item properties, structural validity, and reliability for each scale. Furthermore, we tested convergent and discriminant validity via several hypotheses regarding correlations between PKS-SC and constructs that are assumed to be (un)related: level of education, general beliefs about advertising, ad skepticism, advertising avoidance, and gender.

Persuasion knowledge is assumed to continue to develop throughout life, and to increase with experience and education (Friestad and Wright 1994). Higher education has been shown to be associated with more critical thinking and more critical evaluations of sponsored content (De Gregorio and Sung 2010). Therefore, we believe that education is positively related to the evaluative components (i.e. convergent validity). We could not find studies that found evidence of a relation between education and understanding of advertising (i.e. discriminant validity). Thus, we propose the following hypothesis:

H1: Education is (a) positively related to the evaluative components, and (b) not likely to be related to the conceptual components of the PKS-SC.

The evaluative components of the PKS-SC capture people's feelings and beliefs about sponsored content. Therefore, we would expect these scales to correlate with more general evaluative scales such as general advertising beliefs and skepticism toward advertising in general. General advertising beliefs are people's beliefs about whether advertising in general is informative and entertaining (Smit and Neijens 2000). Skepticism toward advertising in general (from now on: ad skepticism) is defined as the tendency toward disbelief of advertising claims (Obermiller and Spangenberg 1998). Thus, it is expected that people who hold less positive beliefs and are more skeptical about advertising in general, also have higher evaluative persuasion knowledge of sponsored content.

Additionally it is expected that general advertising beliefs and ad skepticism are related to conceptual persuasion knowledge of sponsored content. Specifically, we expect that people who hold more critical attitudes toward advertising in general have a more developed conceptual persuasion knowledge of specific advertising formats, such as sponsored content, because their critical attitude is likely to function as a motivator to think about the source and intent of media content in more depth. Due to their critical attitude, these people tend to be more suspicious and therefore more likely to adopt an analytical processing style to evaluate the source and intent of media content (Wentzel, Tomczak, and Herrmann 2010). Therefore, we propose the following hypotheses:

H2: General advertising beliefs are negatively related to (a) the conceptual components, and (b) the evaluative components of the PKS-SC.

H3: Ad skepticism is positively related to (a) the conceptual components, and (b) the evaluative components of the PKS-SC.

Finally, we expect that the evaluative components of persuasion knowledge are related to coping tactics, such as people's general tendency to avoid advertising. In response to a persuasion attempt, people can cope with the message in many different ways, such as counter arguing its content, doubting its appropriateness, or ignoring it (Friestad and Wright 1994). People who are more skeptical toward advertising are more prone to actually use such resistance strategies and avoid advertising in general. Therefore, we propose that dispositional evaluative persuasion knowledge is positively related to advertising avoidance.

Friestad and Wright (1994) explicitly argue that people do not invariably or typically use their persuasion knowledge to resist a persuasion attempt. People can use their persuasion knowledge to cope with the attempt in a way that matches their goals. This means that people with more conceptual persuasion knowledge do not necessarily resist or avoid persuasion attempts. In the context of sponsored content, this would mean that people with a good understanding of sponsored content, its intent, source, and tactics, do not necessarily resist or avoid such advertising. Therefore, advertising avoidance should be positively related to the evaluative components (i.e. convergent validity), and less likely to be related the conceptual ones (i.e. discriminant validity):

H4: Advertising avoidance is (a) positively related to the evaluative components, (b) and less likely to be related to the conceptual components of the PKS-SC.

A construct that the PKS-SC components should be unrelated to is gender. We do not expect an individual's gender to predict the different components of PK. Thus, to test discriminant validity, we propose:

H5: The components of the PKS-SC are unrelated to gender.

Sample

The data for Waves 1 and 2 were collected via Prolific, a crowdsourcing community set up by researchers at the University of Oxford, UK, offering high quality participant pools. Respondents were recruited via the website which linked to our survey in Qualtrics. Wave 1 was administered on 31 May 2016. In total, 638 people finished the survey. The survey included three test questions (i.e. ‘Please tick 2 here’) to make sure respondents paid attention. Removing all respondents who incorrectly answered two or three of the three test questions (n = 11), and those who participated twice (n = 13), left a final sample of 614 respondents randomly assigned to one of the three medium types (Video games n = 210, TV n = 189, blogs n = 215). This sample had a good distribution of gender (52% female), age (M = 36.34, SD = 10.99, range 19–75), and educational level (24% had completed high school or less, 8% had completed lower professional education, 5% had completed higher professional education, 48% has completed a bachelor college degree, 15% had a masters or doctoral degree).

To analyse the test-retest reliability of the PKS-SC, we invited half of the people who finished the first wave to fill out the second wave five weeks later, and assigned them to the same medium. In total, 293 (54% female, age M = 37.57, SD = 11.17) filled out the complete second wave (Video games n = 98, TV n = 96, blogs n = 99).

Measures

Most of the items were based on existing measures, primarily drawn from the overview of measures by Ham, Nelson, and Das (2015). To minimize order-effects, we carefully selected the order of the questions, as follows: recognition of sponsored content, understanding of source, liking, intent, skepticism, appropriateness, tactics, self-reflective, economic knowledge, followed by convergent and discriminant validity measures, and demographic variables.

Recognition of sponsored content

Recognition of sponsored content (REC) was measured in two ways: first, by asking two general questions (‘Have you ever seen advertising/brands within [medium]?’ Yes, No, Not Sure), and second, by examining people's recognition of sponsored content in three specific cases. By trying out different measures used in prior research (e.g. Rozendaal, Buijzen, and Valkenburg 2010; Rozendaal, Opree, and Buijzen 2016; Wojdynski and Evans 2016), we aimed to find the most reliable and valid measure that can capture whether people can generally recognize sponsored content within a medium's content.

We expected that one of the two general questions would provide sufficient information to reveal someone's capability to distinguish sponsored content from non-commercial content. When respondents answered Yes or Not Sure to the two general questions, they were asked to describe what this advertising looked like. The answers to these open- ended questions were coded by two of the authors (no double coding), using the following categories: (1) description does not reflect sponsored content or unclear, (2) description of traditional advertising (e.g. banners, pop-ups, commercials), (3) description of other types of advertising (e.g. sport sponsoring, in-game purchases), (4) mention of brand names or ‘brands’, (5) recognition of sponsored content: (partly) correct description of sponsored content, or mention of brand/product placement.

As an alternative, we presented to all respondents three real-life cases of one medium (i.e. three one-minute video clips from a television programme, or three 40-second video clips from video games, or three blog posts; in randomized order). One of these cases did not include a brand, one included a subtle brand placement, and one a prominent placement. For each case, we asked respondents whether they had seen any advertising in the presented materials (Yes, No, Not Sure). When respondents answered Yes or Not Sure, they were asked to describe what this advertising looked like. These open ended answers were coded into dichotomous variables (0 = No, or incorrect answer, 1 = (Partly) correct description of the sponsored content). With these cases, we hoped to capture whether a person is able to recognize sponsored content with different levels of prominence. We expected that there was an order with respect to the difficulty of recognizing the sponsored content, with no brand being the easiest, followed by the prominent placement, and the subtle placement being the most difficult to recognize. This order suggests that respondents that would correctly recognize the subtle placement would also recognize the prominent and no placement. Those who answered the prominent placement correctly should also answer no placement correctly.

Understanding of the selling and persuasive intent of sponsored content

The measure of understanding of selling and persuasive intent (INTENT) was a combination of measures of persuasive and selling intent used in prior studies (e.g. Oates, Blades, and Gunter 2002; Rozendaal, Opree, and Buijzen 2016; Tutaj and van Reijmersdal 2012; Van Reijmersdal, Rozendaal, and Buijzen 2012). We asked people to indicate on a 7-point Likert scale the extent to which they agreed (1 = Strongly disagree, 7 = Strongly agree) with 14 statements starting with ‘The reason brands are mentioned or shown in [medium] is to…’ followed by six correct (e.g. ‘sell products’) and eight incorrect (e.g. ‘entertain people’) reasons. The final scale consists of a mean score of the answers to the six correct statements, with high scores indicating a better understanding of selling and persuasive intent of sponsored content.

Recognition of the commercial source of sponsored content

Our scale to measure the recognition of the source (SOURCE) was based on a measure used by Van Reijmersdal et al. (2012). We introduced the notion that sometimes brands are shown in a medium (a television programme, a blog, a video game) and asked to what extent they agreed (1 = Strongly disagree, 7 = Strongly agree) with five statements proposing different sources (e.g. for blogs: ‘The blogger pays for mentioning a brand in a blog post’, ‘The brand pays for mentioning a brand in a blog post’). Only one of the statements represents a correct answer (the brand is paying), and all other statements were incorrect. Because the correct item best represents a person's understanding of the source of sponsored content, we decided to only use this item as a measure of recognition of source.

Understanding of the persuasive tactics in sponsored content

To tap into people's understanding of the tactics used with sponsored content (TACTIC), we asked respondents the extent to which they agreed (1 = Strongly disagree, 7 = Strongly agree) with nine statements starting with ‘Brands try to influence me by…’, followed by six correct (e.g. ‘…hiding the commercial purpose of showing the brand’) and three incorrect (e.g. ‘… making the game more entertaining’) options. This measure was based upon a scale used by Boush, Friestad, and Rose (1994). The TACTIC score was calculated as the mean of the answers to five correct statements, with higher scores indicating a better understanding of the tactics.

Understanding of the economic model of the sponsored content

Our measure of understanding of the economic model (ECO) was based on a scale by Moreau, Krishna, and Harlam (2001) measuring knowledge of the retail industry. We created four statements in which the same notion was applied to sponsored content. We asked respondents to report the extent to which they agreed (1 = Strongly disagree, 7 = Strongly agree) with statements starting with ‘If a brand does not pay for (a part of) the production of a [medium]…’, followed by four consequences (e.g. ‘…the [medium] would be more expensive’). The ECO score was calculated as the mean of the answers to the four statements, with higher scores indicating a better understanding of the economic model of sponsored content.

Self-reflective awareness of the effectiveness of sponsored content

Following scales of Self-confidence of Persuasion Knowledge (Bearden, Hardesty, and Rose 2001), and Perceived Effects of Product Placement on the Self and on Others (Nelson and McLeod 2005) we created two scales of self-reflective awareness. We created one scale reflecting the effects of sponsored content on the self (SELF), and one reflecting the effectiveness of sponsored content on others (OTHER). Both scales involved five statements (1 = Strongly disagree, 7 = Strongly agree) about the effects of seeing brands in a medium (e.g. ‘Seeing brands in a [medium] influences me,’ and ‘Seeing brands in a [medium] influences others’).

Skepticism toward sponsored content

The next three components of the PKS-SC were all measured with 7-point semantic differential scales. To tap into people's skepticism toward sponsored content (SKEP), we introduced the fact that brands sometimes pay the producers of a [medium] to show their brand within the [medium] and asked for their opinion about this. The statement said: ‘I think that showing brands (for which the brand has paid) in [medium] is…’ followed by six items (e.g. dishonest–honest, insincere–sincere) based on prior studies (Boerman, Van Reijmersdal, and Neijens 2012; Campbell and Kirmani 2000; Obermiller and Spangenberg 1998; Ohanian 1990). The mean score of the five best performing items were chosen as a measure of skepticism, with high scores corresponding to more critical evaluations, and thus high skepticism.

Appropriateness of sponsored content

The measure of appropriateness (APPR) was based on several prior studies on the acceptability of brand placement (d’Astous and Séguin 1999; d’Astous and Chartier 2000; Gupta and Gould 1997; Wei, Fischer, and Main 2008), and asked: ‘I think that showing brands (for which the brand has paid) in [medium] is…’ followed by 13 items (e.g. inappropriate–appropriate, unacceptable–acceptable). The APPR score was calculated as the mean of the seven best performing items, with high scores corresponding to perceptions of sponsored content being inappropriate.

(Dis)liking of sponsored content

Finally, (dis)liking of sponsored content (LIKE) was based on regular attitude measures often used in advertising research (Bergkvist and Langner 2017) and a scale measuring children's advertising literacy (Rozendaal, Opree, and Buijzen 2016). The statement ‘I think that showing brands in [medium] is was followed by eight items (e.g. negative–positive, irritating–pleasant). The LIKE score is calculated as the mean of the six best performing items, with high scores representing more disliking of sponsored content.

Convergent and discriminant validity measures

To be able to assess convergent and discriminant validity, the survey also included measures of ad skepticism in general (9-item scale with high scores corresponding to more skeptical attitudes (Obermiller and Spangenberg 1998); a = .96, M = 5.04, SD = 1.29), general advertising beliefs (6-item scale with high scores corresponding to beliefs that advertising is informative and entertaining (Smit and Neijens 2000)); a = .86, M = 3.50, SD = 1.20), and advertising avoidance (3-item scale (Fransen, ter Hoeven, and Verlegh 2013); a = .93, M = 5.04, SD = 1.51). In addition, we asked for respondents' highest level of education completed, ranging from less than High School to a PhD degree, and we asked them to indicate their gender (0 = male, 1 = female).

Statistical analyses

Structural validity and reliability of the scales was tested with a recently developed R- based protocol for scale validation, from which we used the following five steps (Dima 2018; R Core Team 2017; Peters et al. 2016). Step 1 performed basic descriptive statistics at item level. Step 2 examined item properties according to non-parametric item response theory requirements using Mokken Scaling Analysis. Step 3 examined the structure of the item set according to factor analysis (exploratory and confirmatory). Step 4 applied classical test theory for unidimensional item sets to examine scale reliability and item properties. Step 5 computed total scores and score statistics for each subscale, and examined their distributions. These steps were followed for each conceptual component, except for REC, which only involved cross tabulations. The evaluative scales were analysed simultaneously as the items were conceptually similar and we wanted to test whether all items would form three scales (i.e. SKEP, APPR, and LIKE) as predicted. Finally, we ran confirmatory factor analyses (CFA) using structural equation modelling to test the overall model fit of the final PKS-SC. All analyses were performed for the video games, blogs, and television samples, as well as for the three media together.

Based on the results of these initial analyses, we excluded items that showed problems with monotonicity and invariant item ordering, that did load on the same scale or had divergent distributions. After excluding the items, we reassessed structural validity of the resulting scales.

Test-retest reliability of all nine components was assessed amongst the respondents who filled out both wave 1 and wave 2. We used IBM SPSS Statistics 21 to calculate Pearson's or Spearman's correlation coefficients between scores of the same scale in the two waves. Correlation coefficients between .10 and .29 were considered small, between .30 and .49 medium, and between .50 and 1.0 as large effect sizes (Cohen 1988).

Results: structural validity

Based on our analyses, we shortened the survey to 47 items divided over the nine components (with one to seven items per component). Table 1 presents the final scales. Detailed reports of all analyses, for all samples (TV, blog, video games, and total sample), can be found in the Online Appendix (http://bit.ly/2wmuUrB). For reasons of conciseness and clarity, we only report the results of the total sample.

Recognition of sponsored content

General questions

In response to the question ‘Have you ever seen advertising within [medium]?’, 77% said yes (Games: 68% Yes, 20% No, 12% Not sure; Blogs: 85% Yes, 5% No, 10% Not sure; TV: 78% Yes, 7% No, 15% Not sure). However, when asked for a description of this advertising, only 43% correctly described sponsored content (see Table 2). Many described traditional advertising instead. With respect to the question ‘Have you ever seen brands within [medium]?’, 61% said yes (Games: 40% Yes, 40% No, 20% Not sure; Blogs: 60% Yes, 16% No, 24% Not sure; TV: 85% Yes, 11% No, 4% Not sure). On average, 48% correctly described sponsored content.

The two questions led to similar results. A majority of the respondents seemed to have experience with advertising or brands within television programmes, blogs, and video games. When asked to describe this, almost half of them correctly described sponsored content. This result may mean that many people have difficulties in describing sponsored content, especially for video games and blogs, and thus have little persuasion knowledge. However, the results probably indicate a problem in the question itself. Despite our efforts in coding the open answers, we cannot be sure that people who did not correctly describe sponsored content truly do not recognize this type of advertising, or that they just did not understand that we were asking them about this. For instance, when people were asked to describe the brands they had seen within a medium, often they would just mention the brand names. Such answers do not provide us any insight into whether people understand that the brand's presence is a form of advertising.

In addition, we found a large difference with respect to people's knowledge of sponsored content in television programmes compared to games and blogs. Although this suggests that people's persuasion knowledge is more developed for television, it may also indicate that this question is easier to understand and thus more suitable in the context of television programmes.

Altogether, the face validity for the two general open questions aiming to measure recognition of sponsored content was low. People often misunderstood the abstract questions, and the open-ended answers might not reflect their knowledge of sponsored content. Therefore, we decided to exclude these two questions from the PKS-SC.

Three cases

For the specific questions, we showed three cases to the respondents (three short videos of the video games or television programmes, and three screenshots of blogs): one example without sponsored content, one prominent placement of a brand, and a more subtle one. The correct responses to the three cases are presented in Table 3. The percentages of correct responses were in line with the expected difficulty of the examples in terms of recognition as sponsored content. Most people (average recognition for all medium types 92%) correctly answered that there was no advertising in the examples that did not include a brand. In addition, prominent sponsored content was more likely to be recognized (average recognition 82%) than more subtle sponsored content (average recognition 62%).

Table 1. Final Persuasion Knowledge Scales of Sponsored Content (PKS-SC).

Component	Items	Response categories
Conceptual dimension (1) Recognition of sponsored content	Please [watch/take a look at materials] and answer the following questions.	(1) Yes, No, Not sure
	Did you see any advertising in the [medium]? You indicated that you saw/may have seen advertising in the [medium]. Can you please describe what this advertising looked like?	(2) Open (when Yes/Not Sure): coded wrong or right
(2) Understanding of the selling and persuasive intent of sponsored content	Please indicate the extent to which you agree with the following statements.	1 = Strongly disagree, 7 = Strongly agree
	The reason brands are mentioned or shown in [medium] is to …
	…stimulate people to want the advertised brand …encourage people to buy the brand …sell products …make people think positively about the brand …attract attention to the brand …make people remember the brand ( + fillers)
(3) Recognition of the commercial source of sponsored content	Sometimes brands are shown in [medium]. To what extent do you agree with the following statements?	1 = Strongly disagree, 7 = Strongly agree
	The brand pays for showing a brand in a [medium]. (+4 fillers)
(4) Understanding of persuasive tactics in sponsored content	Brands sometimes pay [the makers of a game/the producers of a TV programme/blogger] to show their brand in the [medium]. How do you think they are trying to influence you with this? Please complete the following statement:	1 = Strongly disagree, 7 = Strongly agree
	Brands try to influence me by …
	…hiding the commercial purpose of showing the brand …making sure it does not look like advertising …making sure I am exposed to the brand …placing the brand in a context that I like …placing the brand in a context that people trust ( + fillers)
(5) Understanding of the economic model of sponsored content	The following statements refer to [medium] that show brands.	1 = Strongly disagree, 7 = Strongly agree
	To what extent do you agree or disagree with the following statements:
	If a brand does not pay for (a part of) the production of a [medium]…
	…the [medium] would not have been made. …the [medium] would be more expensive. …there would be fewer [medium]s. …the [medium] would not be for free.
(6) Self-reflective awareness of the effectiveness of sponsored content	To what extent do you agree or disagree with the following statements:	1 = Strongly disagree, 7 = Strongly agree
	SELF a1. Seeing brands in a [medium] influences me. a2. Seeing brands in a [medium] influences me without me realizing it. a3. I have bought a brand or product after I had seen it in a [medium]. a4. I liked a brand more after seeing it in a [medium). a5. I know certain brands because I have seen them in a [medium].
	OTHER b1. Seeing brands in a [medium] influences others. b2. Seeing brands in a [medium] influences others without them realizing it. b3. Others have bought a brand or product after seeing it in a [medium]. b4. Others have started liking a brand more after seeing it in a [medium]. b5. Others know certain brands because they have seen them in a [medium].
Evaluative dimension (7) Skepticism toward sponsored content	Brands sometimes pay the producers of a [medium] to show their brand within the [medium]. What is your opinion about this?	1–7 semantic differential
	I think that showing brands (for which the brand has paid) in [medium] is …
	Dishonest – Honest
	Not trustworthy – Trustworthy
	Incredible – Credible
	Not truthful – Truthful
	Insincere – Sincere
(8) Appropriateness of sponsored content	I think that showing brands (for which the brand has paid) in [medium] is:	1–7 semantic differential
	Inappropriate – Appropriate
	Unacceptable – Acceptable
	Wrong – Right
	Bad taste – Good taste
	Undesirable – Desirable
	Unfair – Fair
	Illegitimate – Legitimate
(9) Liking of sponsored content	Sometimes brands are shown in [medium]. For instance, [example specific to medium].	1–7 semantic differential
	I think that showing brands in [medium] is:
	Negative – Positive
	Unattractive – Attractive
	Boring – Interesting
	Not amusing – Amusing
	Irritating – Pleasant
	Obtrusive – Unobtrusive

Note: Depending on the medium participants were assigned to, [medium]said ‘videogame’, TV programme’,or ‘blogpost’.

Table 2. Answers to general questions measuring Recognition of Sponsored Content.

	Have you ever seen advertising within [medium]?			Have you ever seen brands within [medium]?
	Games	Blogs	TV	Games	Blogs	TV
Answer did not reflect PK	10%	4%	8%	47%	23%	7%
Traditional advertising	61%	55%	11%	4%	15%	2%
Other forms of advertising	4%	10%	2%	1%	7%	1%
Mentioned brands	1%	2%	3%	12%	25%	12%
Correct description of sponsored content	24%	30%	76%	36%	31%	78%

Table 3. Recognition of Sponsored Content in the three specific cases.

	Game	Blog	TV	Total (average)
No brand	96%	87%	94%	92%
Prominent	94%	63%	90%	82%
Subtle	66%	47%	73%	62%

Furthermore, most respondents behaved as expected: Correct responses to the more difficult examples corresponded with correct responses to easier ones (responses as expected: 94% games, 82% blogs, 92% TV). Based on these findings, we concluded that a sum score for the three cases can be used as a measure of people's ability to recognize sponsored content as advertising in a specific medium (REC). These sum scores range from 0 (no recognition, low PK) to 3 (recognition of all three cases, high PK), M_game = 2.55, M_Blog = 1.97, M_TV = 2.56.

Other components

Table 4 shows a summary of psychometric results for each component of the PKS-SC.

The INTENT scale originally included 14 items. Four (items 3, 7, 11, and 13) were excluded because of problems with monotonicity and invariant item ordering. The three incorrect statements (items 8, 14, and 15) were excluded because they formed their own scale, and one item (item 6) was excluded because it had too much conceptual overlap with another item. The final six-item INTENT scale was homogeneous, H(se) = 0.60, (0.03), and reliable (α = .89). A CFA with two item error covariances (items 2 and 4, items 9 and 10) had good model fit: χ²(7) = 19.28, p = .007; CFI = 0.99; RMSEA = 0.05 [0.03–0.08].

The four incorrect statements and one correct statement (item 3) of the SOURCE scale did not form one homogenous and reliable scale in the blog and total sample. Therefore, we decided to choose only the one correct statement as a measure of the recognition of the commercial source of sponsored content.

The TACTIC scale was created by excluding the three incorrect items (items 8, 9, and 10), and excluding one item (item 3) that violated monotonicity and invariant item ordering in the video games and total sample. The final five-item TACTIC scale was homogeneous, H(se) = 0.41, (0.03), and reliable (α = .76). A CFA with two item error covariances (items 1 and 2, items 5 and 7) had good model fit: χ²(3) = 3.56, p = .313; CFI = 1.00; RMSEA = 0.02 [0.00–0.07].

The results with regard to the ECO scale varied between samples. The four items formed one scale in the total and the TV sample. Item 3 caused violations in the video games sample, and item 1 did so in the blog sample. These differences were probably due to factual differences in economic models between media. Despite these variations between media, our analyses allowed us to create one homogeneous H(se) = 0.40, (0.03), and reliable (α = .72) ECO scale with four items, which had good model fit: χ²(1) = 3.13, p = .077, CFI = 1.00; RMSEA = 0.06 [0.00–0.14].

The 10 items measuring self-reflective awareness of sponsored content were clearly divided in two scales, SELF and OTHER. Both were homogeneous and reliable (SELF H(se) = 0.63, (0.02) and α = .89; OTHER H(se) = 0.69, (0.02) and α = .91). A two-factor CFA had good model fit: χ²(27) = 111.82, p < .001, CFI = 0.98; RMSEA = 0.07 [0.06–0.09].

Table 4. Summary of outcomes for each component of the PKS-SC

	Item descriptives	Scale descriptives	Homogeneity (Mokken scaling)	CFA fit	Reliability	Test-retest reliability
(1) REC	Sum score min 0–max 3	M = 2.34 (SD = 0.80)	n.a.	n.a.	n.a.	ρ = .46***
(2) INTENT	min M = 6.03, max M = 6.28	M = 6.15 (SD = 0.82)	H(se) = 0.60, (0.03)	CFI = 0.99; TLI = 0.97; RMSEA = 0.05, 90% Cl 0.03–0.08; χ^2(7) = 19.28, p = .007.^a	α = .89	ρ = .58***
(3) SOURCE	min 1-max 7	M = 6.11 (SD = 1.20)	n.a.	n.a.	n.a.	ρ = .53***
(4) TACTIC	min M = 4.96, max M = 6.01	M = 5.43 (SD = 0.94)	H(se) = 0.41, (0.03)	CFI = 1.00; TLI = 1.00; RMSEA = 0.02, 90% Cl 0.00–0.07; χ^2(3) = 3.56, p = .313^b	α = .76	ρ = .69***
(5) ECO	min M = 3.61, max M = 4.07	M = 3.85 (SD = 1.34)	H(se) = 0.40, (0.03)	CFI = 1.00; TLI = 0.98; RMSEA = 0.06, 90% Cl 0.00–0.14; χ^2(1) = 3.13, p = .077.^c	α = .12	ρ = .57***
(6a) SELF	min M = 2.93, max M = 3.94	M = 3.40 (SD = 1.49)	H(se) = 0.63, (0.02)	CFI = 0.98; TLI = 0.97; RMSEA = 0.07, 90% Cl 0.06–0.09; χ^2(27) = 11 1.82, p < .001.^d	α = .89	ρ = .79***
(6b) OTHER	min M = 5.18, max M = 5.44	M = 5.30 (SD = 1.12)	H(se) = 0.69, (0.02)	(see SELF)	α = .91	ρ = .65***
(7) SKEP	min M = 4.07, max M = 4.49	M = 4.29 (SD = 1.40)	H(se) = 0.82, (0.01)	CFI = 0.97; TLI = 0.96; RMSEA = 0.07, 90% Cl 0.06–0.07; χ^2(129) = 486.87, p < .001.^e	α = .95	ρ = .73***
(8) APPR	min M = 3.55, max M = 4.61	M = 4.02 (SD = 1.34)	H(se) = 0.76, (0.02)	(See SKEP)	α = .95	ρ = .80***
(9) LIKE	min M = 3.79, max M = 4.59	M = 4.25 (SD = 1.26)	H(se) = 0.65, (0.02)	(See SKEP)	α = .91	ρ = .76***

Note. CFA = confirmatory factor analysis, n.a. = not applicable, CFI = comparative fit index, TLI = Tucker-Lewis index, RMSEA = root mean square error of approximation. The item descriptive statistics refer to the minimum and maximum (mean) scores for the items in the scales. The scale descriptive statistics present the mean and standard deviation of the final scale of that component.

^a Six-item one-factor model with two item error covariances (items intent2-intent4, intent9-intent10).

^b Five-item one-factor model with two item error covariances (tactic1-tactic2, tactic5-tactic7).

^c Four-item one-factor model with one item error covariance (eco2-eco3).

^d 10-item two-factor model with seven item error covariances (five identical items about self and other; two within factor items self1-self2, other1-other2).

^e 18-item three-factor model with three item error covariances (skep1-skep2, app1-app2, app5-app7).

*** p < .001.

The analyses of the three evaluative scales showed that items should be dropped in all scales. For SKEP, we excluded item 3 because it had a divergent distribution, low correlations with the other items in the scale, and because alpha increased when dropping this item. The final five-item SKEP scale was homogeneous H(se) = 0.82, (0.01) and reliable (α = .95).

Of the 13 items measuring APPR, we excluded item 4 because it had divergent distribution and low correlations with the other APPR items, items 6, 9, and 11 because they violated invariant item ordering, and item 13 because it did not load to the APPR factor in the CFA. The final seven-item APPR scale was homogeneous H(se) = 0.76, (0.02) and reliable (α = .95).

Finally, we excluded items 2 and 8 from the LIKE scale because they caused problems in monotonicity and invariant item ordering. The final six-item LIKE scale was homogeneous H(se) = 0.65, (0.02) and reliable (α = .91). A CFA of the three final evaluative factors (with several covariances) had good model fit: χ²(129) = 486.87, p < .001, CFI = 0.97; RMSEA = 0.07 [0.06–0.07].

Correlations between components and CFA

To understand how the components relate to each other, we calculated the correlations between all components (see Table 5). We calculated Spearman's rho (ρ) for skewed conceptual components (INTENT skewness = −1.02, Kurtosis = 0.91; SOURCE skewness = −1.77, Kurtosis = 3.71; TACTIC skewness = −0.48, Kurtosis = 0.55; SELF: skewness = 0.19, Kurtosis = −0.75; OTHER: skewness = −0.68, Kurtosis = 0.86), and Pearson's correlation (r) for all other components. The matrix shows that most relations are significant, but small or medium in size. Only the three evaluative components have large correlations. This emphasizes that the components are distinct and unique.

To examine the overall fit of the final PKS-SC, we tested several CFA models. We started with a model including all components as first-order factors, and no second-order factors or error variances. This model had a poor fit: χ²(902) = 3030.57, p < .001, CFI = 0.89; RMSEA = 0.06 [0.06–0.06], and also showed that the evaluative components were strongly correlated.

To test the theoretical proposition that the structure of PKS-SC consists of two higherorder dimensions (conceptual and evaluative), we ran a second model including all components and two second-order factors (the conceptual and evaluative dimensions). This resulted in estimation problems, with the APPR scale showing negative variance, likely due to the strong associations between the evaluative components combined with the weak associations between the conceptual ones.

Table 5. Correlations between all components of the PKS-SC.

	1	2	3	4	5	6a	6b	7	8	9
(1) REC	1
(2) INTENT	ρ = .15***	1
(3) SOURCE	ρ = .20***	ρ = .37***	1
(4) TACTIC	ρ = .12**	ρ = .42***	ρ = .15***	1
(5) ECO	ρ = −.08†	ρ = .08†	ρ = .06	ρ = .03	1
(6a) SELF	ρ = −.11**	ρ = .08†	ρ = −.15***	ρ = .14**	ρ = .12**	1
(6b) OTHER	ρ = .00	ρ = .34***	ρ = .09*	ρ = .40***	ρ = .06	ρ = 48***	1
(7) SKEP	ρ = .05	ρ = .06	ρ = .00	ρ = .31***	r = −.18***	ρ = −.24***	ρ = −.01	1
(8) APPR	ρ = .11**	ρ = .06	ρ = .05	ρ = .22***	r = −.16***	ρ = −.31***	ρ = −.07†	r = .82***	1
(9) LIKE	ρ = .20***	ρ = .14***	ρ = .18***	ρ = .16***	r = −.14**	ρ = −.45***	ρ = −.14***	r = .57***	r = 69***	1

Note: Total sample wave 1, N = 614. Correlation coefficients involve Spearman's rho (ρ) for skewed variables, and Pearson's correlation (r) for other variables.

*** p < .001, **p < .01, *p < .05, †p < .10.

Therefore, we ran a third model, including all components and a second-order factor for only the evaluative dimensions. This model had a comparable fit with the first model: χ²(916) = 3183.55, p < .001, CFI = 0.89; RMSEA = 0.06 [0.06–0.06]. In a fourth model, to account for local dependencies between items identified in previous analysis steps, we added the 15-item error covariances modelled in the previous subscale analyses based on content similarities and modification indices (i.e. intent2-intent4, intent9-intent10, tactic1- tactic2, tactic5-tactic7, eco2-eco3, self1-other1, self2-other2, self3-other3, self4-other4, self5-other5, self1-self2, other1-other2, skep1-skep2, app1-app2, and app5-app7). This model had the best model fit compared to the previous models: χ²(901) = 2244.39, p < .001,CFI = 0.93; RMSEA = 0.05 [0.05–0.05].

To test the structure of the conceptual components individually, we ran a CFA with only the conceptual components. This model had poor model fit, χ²(319) = 1816.45, p < .001, CFI = 0.82; RMSEA = 0.09 [0.08–0.09], and again revealed that the conceptual components, especially REC, SOURCE, and ECO, do not load on a common factor.

Based on these CFA models, we conclude that the conceptual components are not a unitary construct and should be treated as separate scales measuring different types of knowledge, whereas the evaluative components belong to one evaluative dimension.

Results: test–retest reliability

The test-retest correlations (see Table 4) were all high (coefficients above .57) and significant at p < .001, except for REC. This implies that all scales, except for REC, yielded relatively stable scores over time. REC yielded medium correlations (ρ = .46) between the two waves, and thus we cannot be sure of the stability of this measure over time. The low correlations could be explained by the fact that we exposed the respondents to the same cases in the two waves. The repeated exposure to these materials and the questions about advertising may have caused a learning effect. People's ability to recognize the sponsored content in these cases may have increased, resulting in different scores in the retest. In line with this reasoning, REC did indeed increase significantly (M_wave1 = 2.34, SD = 0.80; M_wave2 = 2.60, SD = 0.62), t(288) = −4.79, p < .001.

Results: convergent and discriminant validity

Table 6 presents an overview of the outcomes of our tests of convergent and discriminant validity of the PKS-SC. Regarding convergent validity, the results show that education has positive, but small (range .09; .12), correlations with the three evaluative components, providing weak evidence for H1a. With respect to people's general ad beliefs, the results for the conceptual components are mixed (H2a), but, as expected, the evaluative components have significant, medium-sized (range .39; .49), negative correlations with general ad beliefs (H2b). For general ad skepticism, opposite to our expectations (H3a), our results show significant, small negative correlations (range −.13; −.18) with most conceptual components, and medium-sized positive correlations (range .44; .48) with the evaluative components (H3b). Finally, ad avoidance showed positive correlations of medium size (range .27; .37) with the evaluative components as expected (H4a).

Table 6. Convergent and discriminant validity for each component of the PKS-SC.

	Education	Ad beliefs	Ad skepticism	Ad avoidance	Gender
Conceptual components
(1) REC	ρ = .07	ρ = −.10*	ρ = −.14***	ρ = .04	U = 45132.00
(2) INTENT	ρ = −.08*	ρ = −.05	ρ = −.17***	ρ = .10*	U = 42952.00
(3) SOURCE	ρ = −.01	ρ = −.04	ρ = −.13**	ρ = .10*	U = 45957.50
(4) TACTIC	ρ = .00	ρ = −.07†	ρ = −.18***	ρ = .07†	U = 43920.00
(5) ECO	r = .07†	r = .22*	r = .11**	ρ = −.08*	t = −.36
(6a) SELF	ρ = .02	ρ = .47***	ρ = .36***	ρ = −.43***	U = 43920.00
(6b) OTHER	ρ = −.00	ρ = .18***	ρ = −.02	ρ = −.11**	U = 44297.50
Evaluative components
(7) SKEP	r = .11**	r = −.39***	r = .47***	r = .27***	t = .78
(8) APPR	r = .12**	r = −.42***	r = .44***	r = .30***	t = −.48
(9) LIKE	r = .09*	r = −.49***	r = .48***	r = .37***	t = .62

Note: Total sample wave 1, N = 614. Correlation coefficients involve Spearman's rho (ρ) or Mann–Whitney test (U) for skewed variables, and Pearson's correlation (r) or independent t-tests (t) for other variables.

*** p < .001, **p < .01, *p < .05, †p < .10.

Overall, convergent validity was supported for the evaluative persuasion knowledge components and mixed for the conceptual components. The mixed findings for the conceptual components may be because of the evaluative nature of the constructs ad skepticism and ad beliefs. These findings signal the need for additional convergent validity testing for the conceptual components in the future, with other constructs that are more cognitive in nature. In addition, the fact that the correlations were small- to medium-sized indicates that the persuasion knowledge components are related to ad beliefs, ad skepticism, and ad avoidance, but do measure different concepts. If the correlations were high, the persuasion knowledge components would have too much overlap with the other constructs.

With respect to discriminant validity, the results show non-significant or very small (r = −.08) correlations between the conceptual components and education, supporting H1b. In addition, we expected that the conceptual components were not likely to be related to ad avoidance. The results show significant but small correlations between ad avoidance and the conceptual components (all [very] small coefficients [range .04; −.11], except for one medium sized correlation with SELF). Hence, H4b was supported: the conceptual components have low correlations with ad avoidance. To compare the means for females and males, we did a Mann–Whitney test (U) for skewed conceptual components and independent t-tests (t) for all other components. Results show no significant relations between gender and the PKS-SC components, supporting H5. Altogether, this means that we find evidence for discriminant validity regarding education, ad avoidance, and gender.

Discussion

This study was the first to develop scales for standardized measurement of consumers' persuasion knowledge of sponsored content (PKS-SC). These measurement instruments are essential to examine people's understanding and evaluation of sponsored content and to investigate the role of persuasion knowledge in sponsored content effects. Based on the theoretical and review articles on persuasion knowledge (Ham, Nelson, and Das 2015; Hudders et al. 2017; John 1999; Rozendaal et al. 2011; Wright, Friestad, and Boush 2005), the scales reflect nine components of persuasion knowledge.

The PKS-SC was developed and tested in three phases. The nine scales had good structural validity, reliability, and face validity, and good to appropriate convergent and discriminant validity. Moreover, the test-retest reliability was good for all components, except for Recognition of Sponsored Content.

Recommendations for researchers

This study showed that persuasion knowledge comprises nine separate components. The conceptual components should be used as separate scales, whereas the three evaluative components do form one evaluative dimension. Although Mokken analysis showed that some appropriateness and skepticism items may be interpreted as two separate components of persuasion knowledge at higher thresholds of homogeneity, the components are highly correlated. It should be noted that when measured in the same study, the evaluative components may lead to multicollinearity. However, when researchers are interested in measuring one of the components, the subscales developed in this study can be used. Regarding the conceptual components, future research needs to investigate to what extent and under what circumstances they could be described within the same term of ‘conceptual persuasion knowledge’ or need to be seen as stand-alone aspects of different types of knowledge about sponsored content.

The nine subscales offer researcher a more refined choice when measuring persuasion knowledge. For example, when studying effects of specific persuasion knowledge interventions, researchers may be interested in whether specific elements of persuasion knowledge are enhanced, rather than whether the overall persuasion knowledge has improved (Nelson 2016). Similarly, in studies on disclosing sponsored content, the content of the disclosure may activate specific components of persuasion knowledge, for example, the understanding of persuasive intent or skepticism (Rozendaal, Buijs, and Van Reijmersdal 2016). In these situations, a differentiation in subscales is pivotal.

Moreover, there may be individual differences in the extent to which the persuasion knowledge components are developed or are activated by sponsored content. For example, due to past negative experiences with sponsored content, the activation of disliking of sponsored content is easier for some individuals than for others (Darke and Ritchie 2007). Or, subtle brand placements may activate understanding of the economic model of sponsored content among people with professional experience with advertising, whereas for people who lack this experience, subtle placements do not elicit this knowledge.

The PKS-SC scale was successfully tested for video games, blogs, and television programmes, providing a valuable tool to measure persuasion knowledge for sponsored content in these media. Based on the similarities in performance of the scales between the media, we expect that with small adjustments in wording, the PKS-SC can also be used in research on sponsored content in other media.

To measure recognition of sponsored content, we used three videos or three pictures of content that included no brands, a subtly placed brand, or a prominently placed brand. The same cases can be used in future research, but researchers can also find their own (national) examples for the medium that they want to study. To find appropriate examples, the subtly placed brands should be small in size, in the periphery, and not central to the information presented, whereas the prominently placed brands should be larger and more centrally portrayed.

The current PKS-SC measures dispositional persuasion knowledge for sponsored content. However, if researchers want to measure situational persuasion knowledge, the items can be applied to specific instances of sponsored content by using specific names of games, blogs, or television programmes. For example, the item for dispositional economic persuasion knowledge ‘If a brand does not pay for (a part of) the production of a video game, the video game would be more expensive’ could be rephrased to ‘If Brand X did not pay for showing the product in video game Z, the video game would be more expensive’ to measure situational economic persuasion knowledge. Similarly, the statement for dispositional liking of sponsored content ‘I think that showing brands in blogs is…‘ could be rephrased to ‘I think showing or mentioning brands in blog X is…’ to measure situational liking of sponsored content.

Suggestions for future research

By developing the PKS-SC, we hope to fuel future research on persuasion knowledge concerning sponsored content in various media. The scales can be a foundation to measure all components of persuasion knowledge in one study, or to measure specific components. We present four more specific suggestions for future research using the PKS-SC.

First, future research could use the PKS-SC to determine and compare people's level of persuasion knowledge for various types of sponsored content within one medium (e.g. subtle versus prominent sponsored content in blogs), but also for sponsored content between media (e.g. television versus blogs).

Similarly, the scales can be used to study persuasion knowledge from a developmental perspective and compare people of different ages. For example, emerging adults who grew up with sponsored content may show higher levels of persuasion knowledge than elderly who grew up with traditional forms of advertising.

Second, several studies showed that the activation of persuasion knowledge is an explanatory mechanism in the effects of sponsored content on persuasion (Boerman, Van Reijmersdal, and Neijens 2012; Van Reijmersdal et al. 2015; Wojdynski and Evans 2016). For future research it would be interesting to see how the components of persuasion knowledge concerning sponsored content explain the persuasion process. The components may operate in opposite directions, for example, higher levels of economic persuasion knowledge may positively affect brand attitudes, whereas higher levels of understanding selling and persuasive intent may negatively affect brand attitudes. Moreover, the components may differ in their ability to explain the persuasion process: some components may be stronger predictors of persuasion or resistance than others.

Third, although the nine components of persuasion knowledge that are included in the PKS-SC are based on extensive literature reviews and previous empirical work, there may be other components that are part of the concept of persuasion knowledge that are not included in the scale. Future conceptual studies may uncover relevant components that can be added to the PKS-SC.

Fourth, despite our efforts to be as complete as possible, the development of the nine scales representing different components of persuasion knowledge does not include all possible validity measures. For instance, we did not test the scales' predictive and nomological validity. The final PKS-SC should also be validated on an independent sample. In addition, further research could do additional convergent validity testing for the conceptual components with constructs that are more cognitive in nature, and could do more elaborate testing of discriminant validity. Thus, there is a need for further validation and testing of the scales.

The R protocol of scale validation we adapted to develop the PKS-SC is freely available in the Online Appendix. Future research that employs these scales can use this procedure to test item and scale properties as part of their data analysis process. We believe that scale development and validation is a perpetual and ongoing process that needs to accompany theory development and testing, particularly in fast-moving fields such as that of persuasion knowledge and new forms of advertising. We believe that the scales would benefit from further research, with more data collected, studying different types of sponsored content in different media, in different countries. It would be good to validate the PKS-SC again after several studies by merging data from different studies to also look at measurement invariance across different socio-demographic variables.

In sum, this study fulfilled a strong need for standardized measures of the different components of persuasion knowledge that was fuelled by the increasing popularity of persuasion knowledge in the current literature on consumers and persuasion (Ham, Nelson, and Das 2015). The PKS-SC enables accumulation of knowledge as true accumulation can only occur when standardized and validated measures are used.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Notes on contributors

Sophie C. Boerman

Sophie C. Boerman (PhD, University of Amsterdam, 2014) is an assistant professor of persuasive communication at the Amsterdam School of Communication Research (ASCoR), Department of Communication Science, University of Amsterdam. Her research mainly focuses on the persuasive effects of embedded and personalized advertising, and how informing consumers about advertising may influence the use of persuasion knowledge and its persuasive outcomes.

Eva A. van Reijmersdal

Eva A. van Reijmersdal (PhD, University of Amsterdam, 2007) is an associate professor of persuasive communication at the Amsterdam School of Communication Research (ASCoR), Department of Communication Science, University of Amsterdam. Her research focuses on the effects of embedded advertising on adults and children and on the effects of disclosing advertising.

Esther Rozendaal

Esther Rozendaal (PhD, University of Amsterdam, 2011) is an associate professor in communication and media at the Behavioural Science Institute at the Radboud University, the Netherlands. She is an expert on young people's media and consumer behavior. In her research she mainly focuses on the role of advertising literacy in children's susceptibility to advertising.

Alexandra L. Dima

Alexandra L. Dima (PhD, University of Edinburgh, 2010) is a health psychologist and Marie Curie Fellow in the Health Services and Performance Research laboratory (HESPER), University Claude Bernard Lyon 1. She has extensive experience in psychometrics and questionnaire validation, and has contributed to the development and adaptation of measurement tools in various research areas.