![Sample our Humanities journals, sign in here to start your FREE access for 14 days]({"type":"image" , "src" : "/sda/5939/TOC-Subject-Sample-2021-Humanities.jpg"})
Abstract
In light of a recent exchange between Prior (2013a) and Dilliplane, Goldman, and Mutz (2013), we evaluate the new American National Election Study program-count measures of news exposure using a unique dataset that tracks self-reports as well as actual exposure to news collected via passive tracking devices. We bring these data to bear on concerns raised by Prior (2013a) about the construct and convergent validity of the new ANES measures. Our results add nuance to previous findings showing respondents’ propensity to overreport exposure to news, and also demonstrate that on average, self-reported measures reflect relative levels of exposure quite well. Additionally, we show that the more unique news programs a person watches, the more total time he or she is exposed to political news. Very few people watch only one program but watch it repeatedly. The data also reveal an increase in the number of programs watched leading up to election day, and a concomitant increase in the amount of time per capita spent with political news as elections approach. We conclude, however, that the program-count measure is not without its weaknesses. Shortening the list of programs affects construct validity by introducing noise into the low end of the scale. Expanding the list of programs in the survey to include local news and special reports will improve fidelity at the low end of this new measure.
Notes
1. 1. The principal investigators of the 2012 project were Vince Hutchings, Simon Jackman, and Gary Segura. The committee evaluating the existing media, participation, and campaign items consisted of Matt Barreto, Sunshine Hillygus, Diana Mutz, and Lynn Vavreck, who served as the committee’s chair.
2. 2. This ignores the possibility of correlated errors with dependent variables (see Vavreck Citation2007), but this problem is not part of the critique in question.
3. 3. The company’s technology was subsequently purchased by Arbitron in 2009. These data were obtained through a consulting agreement between IMMI and Lynn Vavreck in 2006.
4. 4. YouGov started with a list of consumers in each market obtained from a commercially available file. Panelists were randomly selected and were replaced with the nearest matching consumer on the list if need be.
5. 5. Nearly 20% of the sample completed the initial profile survey. Actual enrollment in the panel, which was done via a second phone call, was roughly half (10%).
6. 6. To ease concerns about the way media viewership or the media landscape has changed since 2006, we note that the IMMI method accounts for time-shifting viewing and can code content viewed or heard online as well.
7. 7. For more on the IMMI method and validation of its measurement techniques, see Jackman, LaCour, Lewis, and Vavreck, (Citation2012) and LaCour (Citation2013). We use IMMI data from the weeks leading up to the 2006 midterm elections in the New York and Chicago media markets, which both had a competitive Senate election and several house elections in 2006. A replication dataset for the figures drawn here is available upon request.
8. 8. To convey a sense of the fidelity of the IMMI measures, the IMMI panelists in these data watched an average of 28.4 hours of television per week, compared with the Nielsen 2006 national estimates of 31 hours of television per week. Similarly, in terms of radio exposure the IMMI panelists averaged 18.8 hours of radio a week compared with Nielsen 2006 national estimates of 19 hours of radio per week. More rigorous tests of the accuracy of these data are reported in Jackman et al. (Citation2012).
9. 9. We selected the universe of news programs based on programs that the 2006 TV Guide categorized as “General News” or “Politics.” The table in the Appendix (available online) shows the list of 188 unique television programs included in our analysis.
10. 10. This pattern is consistent with previous illustrations of overreporting (Vavreck, Citation2007) showing that people with higher levels of interest and participation exaggerate their exposure to political media more than those with lower levels of political interest and participation.
11. 11. Dilliplane et al. (Citation2013) question the validation exercises previously performed on the IMMI tracking data (Jackman et al., Citation2012; LaCour, Citation2013), saying that no analyses were done to evaluate the extent to which exposure was underestimated or overestimated by the tracking software. Jackman et al. (Citation2012) investigated instances of suspected technology overreporting when they noticed the software registering hits for political ads that were not aired during the time people allegedly saw them (the master set of media showed the ads were not on the air at these times). Upon further investigation, they discovered that the campaign ads were being played on Sunday morning political talk shows that the panelists were watching—and the tracking software, appropriately, counted that media as an ad viewing.
12. 12. In the ANES administration in 2012, respondents of Hispanic ethnicity were shown an additional set of 16 programs aimed at Hispanic audiences. These shows were mixed in with the regular programs.
13. 13. They are correlated in the 2012 ANES at .76 regardless of mode of interview.
14. 14. It is worth pointing out at this point that the program-list measure, the old ANES days-per-week measure, and the IMMI hours-per-day measure are all tapping slightly different concepts, albeit all about news exposure. The list method requires that respondents count individual programs they watch, while the other measures ask people to track the number of days or hours they do something. Given that all of the measures are memory recall measures about news programming, it is unlikely that the differences in question wording render the items incomparable. In fact, the high levels of correlation across the measures we are able to compare suggest that each item does about as well as the other in measuring actual exposure, which is another reason that the list method may be preferable as it also provides the opportunity to measure content as well as frequency.
15. 15. Because the exposure data are substantially skewed, in this graph and the ones that follow, we transform the data for ease of visualization by taking the natural logs of exposure (we add one unit to every case to avoid having to take the log of zero). Essentially, we are “zooming in” on the part of the graph where most of the cases are found—the lower left corner—and “zooming out” on the high end of the scale. Units on the graph represent actual hours of exposure or number of programs, untransformed.
16. 16. Including more shows comes at a price, but it seems a good deal of leverage can be gained by the inclusion of local news, perhaps at the expense of one of the more nuanced cable programs, thus keeping the change cost-neutral.
17. 17. Of course, if one is interested in the marginal results on frequency of viewing a particular show in a week or month, the list measure cannot deliver those. But these data show that the measure taps total levels of exposure well conceptually, with the bonus of being able to measure the content of exposure, too.
18. 18. As of this date, the preliminary release of the ANES 2012 data did not include a variable for date of interview; thus, we cannot check for this pattern in 2012.
19. 19. Although surveys, like ANES, that are not designed to have representative replicates of the sampling frame released at a daily or weekly level are not ideal for assessing temporal changes in marginal results. The NAES, however, is designed to be analyzed this way (see Johnston & Brady, Citation2002).
Additional information
Notes on contributors
Michael J. LaCour
Michael J. LaCour is a PhD candidate in the Department of Political Science, University of California, Los Angeles.
Lynn Vavreck
Lynn Vavreck is Professor in the Department of Political Science, University of California, Los Angeles.