What makes you click? The effect of question headlines on readership in computer-mediated communication

Abstract

This paper reports the results of two field experiments that investigate the effect of using question headlines to enhance readership in computer-mediated communication. The results from both experiments suggest that question headlines are significantly more effective than declarative headlines in generating readership. The results from the second experiment also indicate that question headlines with self-referencing cues are particularly effective and generate higher readership than question headlines without self-referencing cues and rhetorical question headlines. However, results also suggest that the effect of question headlines varies across message topics. Limitations and future research opportunities are discussed.

Keywords::

• Question headlines
• Readership
• Internet
• Computer-mediated communication
• Advertising

The Internet and the attendant media pose one of the largest upheavals to the manner in which communicators of all types attempt to reach their audiences. The Internet provides fast distribution of messages to wide audiences. Yet, generating message readership, i.e., interest in reading beyond the headline, is the main objective of most communicators and a prerequisite for conveying persuasive messages, whether in marketing, politics, science communication, public relations, or other areas. Marketers have long recommended the use of question headlines, i.e., headlines with interrogative elements, to enhance readership, and question headlines have been prevalent in print advertising and other media since the beginning of the last century (Howard & Barry, 1988; Starch, 1914). Likewise, question headlines are also widely used in computer-mediated communication, such as Internet advertising (e.g., pop-ups and banner campaigns), online news media (e.g., Internet newspapers and magazines), and other online communication forums (e.g., blogs and shopping websites). Despite the frequent use of question headlines in computer-mediated communication, however, little is known about their effectiveness in terms of generating readership in this medium. Do question headlines indeed increase the likelihood of generating readership in computer-mediated contexts, and to what extent are the effects generalizable across domains and topics? The research reported here attempts to address these questions by exploring the effectiveness of using different types of question headlines to enhance readership in two domains of computer-mediated communication.

The Power of Questions

According to Howard and Barry (1988, p. 18), “the most commonly cited potential benefit of a question headline is its ability to capture the attention of a reader.” The power of question headlines seems to be rooted in their ability to initiate cognitive processes associated with recall, accessibility, and elaboration of information (Moore, Neal, Fitzsimons, & Shiv, 2012; Munch & Swasy, 1983). As such, question headlines are used to generate attention, interest, and curiosity in a particular communication, to establish agreement or concessions with this communication, and, in turn, induce certain types of behaviors.

Many different types of questions may be used as communication headlines, such as hypothetical questions, rhetorical questions, leading questions, and tag questions. Hypothetical questions, which reflect thought experiments, have been studied in domains such as political polling, jury selection, and market research. Their persuasive effect tends to be attributed to their ability to make relevant knowledge more accessible (Moore et al., 2012). Rhetorical questions, in which the answer is implicit within the question, have also been studied in a variety of settings, including politics, law, media, and advertising. The effectiveness of rhetorical questions seems to be rooted in their ability to enhance the likelihood of message- and/or source-based elaboration, depending on the salience of the rhetorical format (Ahluwalia & Burnkrant, 2004). Earlier research suggests that the effect of rhetorical questions depends on issue involvement and argument strength (Munch & Swasy, 1988), and that rhetorical questions enhance message processing primarily when the question is placed after the arguments to which they refer (Howard, 1990, 1991). Leading questions, which give the impression that the questioner has an evidentiary basis for the questions, often elicit responses that provide behavioral evidence for what is implied by the question (Swann, Giuliano, & Wegner, 1982). Studies of media effects have shown that leading questions such as “Is P a criminal?” tend to be more effective than denial statements such as “P is not a criminal” (Wegner, Wenzlaff, Kerker, & Beattie, 1981). Tag questions, which are declarative or imperative statements turned into questions by adding an interrogative fragment (the tag), such as “aren't you?” or “don't you think?,” have been found to affect speaker and message quality perceptions, cognitive responses, perceptions of the speaker's power and credibility, and more indirectly, have an effect on attitude change (Hosman & Siltanen, 2011).

Question headlines are frequently combined with self-referencing cues that are intended to trigger the reader to relate the message to his or her own knowledge and beliefs. Research suggests that self-referencing is particularly effective for enhancing cognitive processing (e.g., Howard, 1991; Klein & Loftus, 1988). In particular, self-referencing is assumed to facilitate elaboration of information because “the self” is a motivating and complex memory structure (e.g., Greenwald & Banjai, 1989). Furthermore, self-referencing is generally assumed to enhance the likelihood of perceiving messages as arousing or as positive or negative in valence, which in turn increases the level of attention allocated to messages (Fujioka, 2012). In-line with such theorizing, empirical studies indicate that self-referencing enhances the persuasive effects of advertising (Debevec & Iyer, 1988), and the use of self-referencing second-person pronouns (e.g., “you” and “your”) in marketing communication generates perceptions that the messages communicated are more personally relevant, and the products presented more personally useful, compared to when third-person pronouns (e.g., “they”) are used (Chang, 2011; Chang & Lee, 2011; Debevec & Romeo, 1992). In contrast, however, other studies indicate that self-referencing cues may have a “boomerang effect,” i.e., lead to biased and distorted message elaboration (Chen, Alden, & He, 2010).

Despite the apparent value of question headlines, previous research findings are conflicting with respect to the general effectiveness of question headlines in enhancing readership compared to other headline strategies. Early research suggests that question headlines generate more attention when the majority of headlines are declarative rather than interrogative, but that question headlines are not remembered any better than declarative headlines (Myers & Haug, 1967). Soley and Reid (1983) did not find that any specific headline formats, including question headlines, enhanced industrial ad readership, although question headlines were initially noticed more frequently by prospective readers. A recent study suggests that question headlines can be more effective than other headline strategies for some products, but not all, when using SMS advertising (Wu & Luo, 2011). However, no known research has addressed the effectiveness of question headlines in generating readership in computer-mediated communication.

Based on the research outlined above, and the lack of research concerning question headline effectiveness in computer-mediated communication, two experiments were conducted to investigate whether question headlines enhance readership in this medium. The purpose of the first experiment was to see whether question headlines were more effective than traditional, declarative headlines in generating readership of computer-mediated communication, and whether question headlines with self-referencing cues were more effective than questions without self-referencing cues in generating readership. The purpose of the second experiment was to test whether the results could be replicated in another domain of Internet-based communication, and to compare the effect of rhetorical questions to non-rhetorical question headlines.

Experiment 1

Method

A field experiment was performed using the Internet-based micro-blog service Twitter as the research laboratory. The second author is a professional science communicator in Norway and actively manages a Twitter account that is dedicated to communication of research news from a wide array of disciplines. The profile had approximately 6350 followers during the period in which the experiment was conducted. The account is the second largest within the “Education & Research” category in Norway, and the majority of followers work within different types of news media.

The experiment was performed over a 4-month period in which the followers of the profile were exposed to messages (“tweets”) that were presented by headlines that were either formulated in a declarative manner based on suggestions from the scientist (control condition), or as questions that were either without self-referencing cues (experimental condition A) or with self-referencing cues (experimental condition B). Question headlines were formulated based on the key findings of the research and with no specific question format in mind. Yet, in order to reduce error-variance, attempts were made to avoid using hypothetical, rhetorical, leading, or tag questions.

A mixed design was used, that is, a combination of a within-subjects design and a between-subjects design. The main motivation for using this approach was the need for reducing biases associated with differences in the a priori popularity of message topics, as some topics were generally more interesting and popular than others. In order to sufficiently control for variations in a priori popularity, a very large sample of messages would be called for if using a between-subjects design. A between-subjects design would also require a second Twitter account with comparable followers, and no available accounts matched these criteria.

Therefore, for the purpose of the present study, an alternative approach was chosen based on pretests performed to determine the life cycle of Twitter messages posted on the account. The pretests revealed that most “clicks” occurred within the first hour after the message was posted. Accordingly, each message was posted twice, first by using the control condition (declarative headline) and subsequently, after 1 hour, by using one of the two experimental conditions (question headline with and without self-referencing cues). By adopting this approach, differences in the relative popularity of topics could be controlled for by calculating a base rate index. However, in order to reduce the risk of biases associated with order effects or repeated exposure to treatments that may produce inflated responses to the experimental conditions, efforts were made to formulate question headlines in a manner that did not yield direct recognition of the original message. Examples of alternative headline formulations are, “The hunt for status in the advertising business” (control condition) and “Why are advertisers so obsessed with winning prices?” (question headline without self-referencing cues); “Power corrupts” (control condition) and “Is your boss intoxicated by power?” (question headline with self-referencing cues).

The tool “Bit.Ly” was used to create short and unique links to target articles as well as to calculate the number of clicks on each link. Only clicks on the links in the posted Twitter messages were counted, whereas access to the target articles via other channels, such as by accessing directly through the web page, were excluded from the analysis. Moreover, in order to reduce limitations and biases associated with the number of potential recipients at different hours of the day, all messages were posted at the same time of day and with the same time span between the control condition and the following experimental condition.

Results

The effect of using question headlines without self-referencing cues is illustrated in Figure 1, whereas Figure 2 illustrates the effect of using question headlines with self-referencing cues.

Figure 1 Changes in readership when using question headlines without self-referencing cues (across 16 message topics).

Figure 2 Changes in readership when using question headlines with self-referencing cues (across 18 message topics).

The results reveal that in all of the cases observed, regardless of the topic communicated, question headlines generated higher interest in the target message compared to non-question (control) headlines. The effect size varies between 10% and 533% (M = 150%) for question headlines without self-referencing cues, and between 49% and 350% (M = 175%) for question headlines with self-referencing cues.

A one-way between-groups analysis of variance was conducted to determine whether the observed differences were statistically significant. The results reveal statistically significant differences in readership scores between the three conditions investigated (F = 19.155, p < .001). Post hoc comparisons using the Tukey HSD test indicate that mean scores in the two experimental conditions differ significantly from the control condition. Readership scores from the two experimental conditions were, however, not significantly different. Yet, the results offer support to the hypothesized positive effect of question headlines in generating readership of computer-mediated communication.

However, an important limitation of the present study relates to the lack of a sufficiently separated control condition and, hence, the risks associated with repeated exposure to treatments. Therefore, a second experiment was conducted to account for these risks. The purpose of the second experiment was also to explore the generalizability of results across different domains of computer-mediated communication, as well as to compare the effectiveness of rhetorical versus non-rhetorical questions in generating readership.

Experiment 2

Method

A field experiment was conducted using the Norwegian auction and shopping website FINN^TM (www.finn.no), which resembles the international eBay^TM website. FINN^TM, which in English translates to “Find,” offers several services, including a virtual marketplace where individuals and businesses can advertise products for sale. When potential buyers search for a product, only the headline and a photo are displayed, and readers need to “click” on the announcement to read more about the product. Advertisers can monitor the number of clicks made to read about a product at any given time.

Ads were published in Finn's virtual marketplace for four types of frequently advertised, gender-neutral consumer products, including an Iphone4^TM, a Trademax^TM Portugal couch, a Philips^TM LCD TV, and a Miele^TM laundry machine, all of which were presented as “used only once/a few times” or “as good as new.” Several fictitious advertiser profiles were created to publish the ads in a way that seemed authentic to readers. In order to avoid confounding effects, ads were published in four rounds over a period of 4 weeks. Each week, an ad for each of the products was published, based on random rotation of the following conditions: (a) a nonquestion headline (control condition), (b) a question headline without self-referencing cues, (c) a question headline with self-referencing cues, and (d) a rhetorical question headline (without self-referencing cues). The ads were deleted after 1 week. Accordingly, each week, any given product and any given experimental condition would occur only once. At the end of each 1-week period, readership was measured using the accumulated number of clicks to read the advertisement during that period.

The headlines were formulated in a similar manner across the four products. Examples of headlines are “For sale: Black iPhone4 16GB” (control condition), “Anyone who needs a new iPhone4?” (question headline without self-referencing cues), “Is this your new iPhone4?” (question headline with self-referencing cues), and “We all do agree that iPhone4 is the best phone available?” (rhetorical question headline).

Results

The results (see Figure 3) show that question headlines, whether with or without self-referencing cues or rhetorical in nature, generate higher readership for three of four products advertised. For the laundry machine, there is a negative effect of between 20.29% and 28.99% when using question headlines compared to declarative headlines. For the three other products, there is a positive effect of between 53.85% and 361.11% when headlines are without self-referencing cues, and an effect size between 101.71% and 744.44% (M = 257.03) when question headlines include self-referencing cues. Rhetorical questions are found to increase readership between 53.85% and 257.41%. Across all four products, the mean increase in ad readership is 137.30% when question headlines are without self-referencing cues, 257.03% for question headlines with self-referencing cues, and 103.21% for rhetorical question headlines.

Figure 3 Results from Experiment 2 (bars reflect readership in number of “clicks”).

The results of Experiment 2 replicate the patterns indicated in Experiment 1 in offering support to the main hypothesis that question headlines are more effective than declarative headlines in generating readership of computer-mediated communications. The results also support the hypothesis that question headlines with self-referencing cues are more effective than questions without self-referencing cues. Yet, the effect of question headlines on readership is reversed (i.e., negative rather than positive) for one of the products investigated (laundry machine).

Nevertheless, even when including the product for which the effect of question headlines was reversed, differences are statistically different in the hypothesized direction. The experiment resulted in 3256 responses (clicks) across the four conditions. These data reflect aggregated readership scores, however, rather than individual scores based on a predefined sample. There are also variations with respect to the relative popularity of the products presented. Hence, in order to estimate whether differences were statistically significant, the data were transformed to reflect the proportion of scores relative to the highest number of clicks for each of the products. A one-way between-groups analysis of variance reveals statistically significant differences in readership scores between the four conditions investigated (F = 442.07, p < .001). Post hoc comparisons using the Tukey HSD test indicate that mean scores in each of the four conditions differ significantly from all of the other conditions, which offer robust support for the hypothesized relationships.

Discussion

The purpose of the two experiments reported here was to explore the effectiveness of question headlines in enhancing readership in computer-mediated communication. The findings from Experiment 1 suggest that question headlines represent a more effective headline strategy compared to traditional, declarative headlines. In Experiment 2, these findings were replicated in three of four types of consumer products investigated. General question headlines with self-referencing cues were also found to be more effective than rhetorical question headlines for three of four products investigated.

Experiment 2, however, also reveals an instance of a reversed effect, that is, a negative rather than positive relationship between question headlines and readership for one of the products investigated (the laundry machine). There are no apparent explanations as to why readers respond differently when looking for a laundry machine compared to the other types of products (e.g., a cell phone, TV, or couch). Overall, there are small differences in the readership of the laundry machine advertisements across the control condition and the three experimental conditions. These findings suggest that those who are looking for household appliances, such as a laundry machine, through an Internet marketplace service, could be less susceptible to the way headlines are framed compared to consumers who are searching for other types of products. Differences between products with respect to the effectiveness of question headlines in generating interest have also been found in previous studies (e.g., Wu & Luo, 2011). As the present study only investigates a limited set of question headlines, one cannot eliminate the possibility that consumers who are searching for a laundry machine would respond more positively to other types of question headlines or other types of headlines with self-referencing cues. The number and type of products investigated in this experiment is also limited, and a much broader span of products needs to be systematically investigated before the generalizability of findings can be assessed. Future research is therefore called for to unravel contingencies as well as possible interaction effects between the specific domains (and message topics/products) of communication on the one hand and different types of question headlines and self-reference cues on the other.

Modern technology offers a multitude of opportunities for testing the effects of question headlines and other types of headlines by means of easily available, secondary data. For example, inspection of the articles published in the present journal (Social Influence) reveals that 16 of the first 140 original articles utilize question headlines. The journal's website provides information about the number of views from the website for each article. In this case, the sample size (N = 140) is too small to control for differences in the relative popularity of topics. However, data on readership and, perhaps more importantly, actual impact (i.e., citations) for large samples of articles are available through a series of other databases. Data on readership and headline types can also be harvested from other types of websites, providing unlimited opportunities for research on the effect of headlines to enhance readership and different types of impact. Studies that utilize large, pre-existing samples of cases can also easily control for the relative frequencies with which question headlines have been used, and hence systematically investigate the contingencies and dynamics associated with the effects of different types of question headlines as well as other headline strategies. Further experimental research, based on a theoretical framework, is also called for to explore the interaction effects between different topics, particular headline strategies, and individual reader characteristics such as topic involvement and prior expertise.

Conclusion

The results from the two experiments reported here suggest that question headlines represent an effective strategy for enhancing readership in computer-mediated communication. The results from Experiment 2 also suggest that question headlines with self-referencing cues are particularly effective. Provided that the findings presented here indeed reflect actual effects, the combined strategy seems to represent a useful tool for practitioners in attracting readers to their Internet-based communications. This research also contributes to the literature on triggering attention in computer-mediated communication, where we hope it promotes additional inquiry.

Acknowledgements

The authors thank the associate editor, Dr Daniel J. Howard, as well as two anonymous referees for their valuable comments and suggestions. The authors are also indebted to Master of Science students Eivind Haugstad, Tanya Wolter, and Aksana Tikhonova for their assistance in executing the second experiment reported here.