Confronting Measurable Journalism

Abstract

This introductory essay introduces “measureable journalism” as a term that encapsulates the cultural and material shift to digital platforms capable of providing real-time, individualizable, quantitative data about audience consumption practices. Measurable journalism is not a break from the past as much as it is an enactment of industrialized journalism’s desire to monitor its audience. As a conceptual framework, measurable journalism adopts a broad, anti-reductionist perspective to avoid privileging technology, economics, or professionalism as having ultimate explanatory or agentic dominion over the other facets. What is proposed is a mutually constitutive approach that considers how elements of materiality, practice, culture, and economics converge and diverge under the banner of measurable journalism. This approach both informs future research and helps identify the social implications of measurable journalism.

Keywords:

• measureable journalism
• metrics
• analytics
• digital platforms
• news audiences

Introduction

The front page of the October 30, 1884, edition of the New York World features a large editorial cartoon attacking the lavishness of a dinner held to honor Republican presidential candidate James Blaine. Running the length of the front page, the cartoon juxtaposes the decadent feast attended by wealthy and powerful New Yorkers with a starving family begging for scraps. The twin message of anti-elitism and anti-corruption epitomized the emerging power of the crusading press. A week later, Grover Cleveland prevailed over Blaine in the state of New York by just over a 1000 votes, ensuring his presidential victory. The cartoon is certainly eye-catching, but above the headline, in twin boxes positioned on either side of the Gothic-lettered nameplate, the newspaper announced:

CIRCULATION OVER 100,000 EVERY DAY

ADVERTISE YOUR WANTS

Cheapest Rates in the City

The combination of a political cartoon and an announcement seeking advertisers speaks to the efforts of publisher Joseph Pulitzer to transform the recently purchased newspaper from one of many indistinguishable daily New York papers into a leading title in the city. A few years later, Pulitzer would partake in perhaps the fiercest battle for circulation in US history against William Randolph Hearst’s New York Journal in what became known as the Yellow Journalism era.

The New York World example reminds us that as long as the news has been an industrialized product, it has been a measured one. Accounts of newspaper circulation serve a dual function. Instrumentally, they foster the economic success of newspapers by providing a quantitative measure to would-be advertisers intent on efficiently reaching a wide audience. Only with circulation measurement can this exchange take place (which led the advertising industry to form the Audit Bureau of Circulations in 1914). One of Pulitzer’s innovations as a publisher was to aggressively sell advertising at low rates, shifting the subsidy from the newspaper purchaser to the advertiser. In the second year of his ownership, each edition of the World contained twenty times as many advertisements as it had before Pulitzer took over (Juergens 1966, 337). Across the newspaper industry, advertising revenues would eclipse circulation revenues, making newspapers a lucrative business as editorially independent papers appealing to wide audiences replaced the partisan press. All of this depends on metrics.

Economic imperatives aside, circulation numbers—and news audience measures generally—perform the public-facing cultural role of equating reach with influence. Gauging the actual authority of any news outlet eludes any easy calculation. Some specialized news organizations will tout the quality of their audience, but, for the most part, audience size comes to stand in for cultural significance. Pulitzer’s World and Hearst’s Journal proudly announced their rocketing circulations to the entire public, not just to interested advertisers that would show up to their offices. Well after the newspaper wars, television news outlets, first with broadcast and later with cable, have continually sparred over audience size, with the ratings victors displaying their numbers (including occasionally through newspaper advertisements). To be sure, ratings correlate with advertising revenues (although the make-up of the audience certainly confounds these calculations), but slight variations correspond to only slight differences in advertising costs. Instead, news organizations’ public display of their audiences has for more than a century been a particular kind of performance used to index cultural influence in lieu of direct measures of influence.

An awareness of audience measurements—whether publicly touted or simply shared among the newsroom staff—complicates normative understandings of journalism. The journalist as gatekeeper (Shoemaker and Vos 2009) presupposes a type of professional judgment in which decisions of newsworthiness are made on behalf of audiences but also made without direct audience involvement. From this vantage point of normative detachment, large news audiences may be considered a form of proof that the journalists are fulfilling journalism’s public service role. Yet such rhetoric cannot be neatly divided from economic motivations of news organizations to attract larger audiences, particularly in competitive markets (Hamilton 2004). Moreover, a normative view that narrows journalistic content to an idealized subset of hard news leaves open all non-hard news content to accusations of being there purely to drive up profits. Such arguments reveal the normative limits of much talk about journalism and ignore the social value of a variety of news formats and topics. But they also hint at a deep unease residing within competing conceptions of journalism as both a social good and an economic one. Robust profits once kept this tension at bay (Hallin 1992), but it continues to reemerge in the present with the profit demands of corporate media ownership, the shrinking advertising market for news, and the bevy of digital-native news formats.

The issues that are emerging around digital news audience measurement are both old and new. Certainly, any accounting of data metrics or analytics needs to acknowledge the longer history of audience measurement as the longstanding tensions surrounding journalism’s mandate to inform and its need for subsidy unfold in a new century (Napoli 2011). Yet, while caution is needed to avoid unduly asserting the novelty of the present, the distinct capabilities of digital platforms need to be recognized and explored. This introduction provides a foundation that attempts to balance between the long view of industrial news production and an appreciation of what new developments have emerged in the present to provide perspective on what I am calling “measurable journalism.”

Measurable Journalism in the Digital Age

As argued above, the economic backbone of the news industry has long relied on the development of elaborate audience measurement techniques employed to attract advertisers, justify funding, and allow for comparisons with peers. Audience measurement numbers provide agreed upon metrics allowing for the coordination of various economic actors. These measurements aggregate news consumption behaviors and attitudes, ideally making journalists more responsive to their audiences. But measuring mass communication audiences has always been an approximation, close enough to suit its purposes while reducing the audience to an imagined concept, either massed into an absolute number or sliced according to broad demographic segments (Carlson 2006). The problems of measurement are practical. Perhaps the best symbol of mass communication is the broadcast antenna, rising above cities to indiscriminately emit messages. With news consumption cut-off from production, audience measurements become approximations at best, collated through sampling techniques to extrapolate the behaviors of an unseen whole. Even if imperfect, they are functional in providing a common stock of information to advertisers and news organizations. Newspaper and magazine circulation numbers might provide a more accurate accounting of the production of texts, but gauging readership relies on similar methods of extrapolating from small samples. Moreover, circulation numbers only reveal the quantity of texts, and not the quality of their consumption by the actual audience.

Digital media platforms alter the audience measurement dynamic when they allow news organizations to monitor, track, and interact with their audiences directly. Audience metrics and analytics eliminate both the lag time for audience measurement and the need for complex sampling practices. Once elusive, real-time tracking provides granular data on any digital item—a video clip, a news story, an interactive feature, etc.—as well as information about the individual behavior of audience members. These newfound capabilities mask more complicated measurement difficulties, such as being able to determine who news users are as they move across digital platforms or distinguishing human users from bots—not to mention more difficult to measure attributes like affect. Nevertheless, these types of data stem from news organizations’ interests in tracking audiences, but with new possibilities being explored and implemented in a variety of ways.

Researching the digital news environment begins with embracing complexity to avoid oversimplification (Lewis and Westlund 2015). In this spirit, this essay introduces “measureable journalism” as a term that encapsulates the cultural and material shift to digital platforms capable of providing real-time, individualizable, quantitative data about audience consumption practices. As a conceptual framework, measurable journalism adopts a broad, anti-reductionist perspective to avoid privileging technology, economics, or professionalism as having ultimate explanatory or agentic dominion over the other facets. What is proposed is a mutually constitutive approach that considers how elements of materiality, practice, culture, and economics converge and diverge under the banner of measurable journalism. Examples of the dimensions of measurable journalism include:

•	Material: Digital analytics and metrics software; the infrastructure underlying this software, digital news platforms.
•	Organizational: New newsroom positions monitoring and/or reacting to metrics; new types of vendors interacting with journalists and news managers.
•	Practice: Use of data in story assignment/placement decisions; new modes of audience engagement.
•	Professional: Acceptance or resistance to audience metrics being used in the newsworthiness decisions; concern over professional autonomy; hope for better connections to the audience.
•	Economic: Shifts in monetizing content; basing resource and personnel decisions on data; altering advertiser relations with data; facilitating clickbait.
•	Consumption: Personalized content or content recommendations for news audiences.
•	Cultural: Concern over popularity deciding decisions about newsworthiness; debates over individualized news versus collective news.
•	Public policy: Concerns over audience privacy and data-tracking.

Of course, it is too much to ask any single study to capture all of these dimensions, but any accounting of measurable journalism needs to be aware that this is a multidimensional construct. This broad view works against assigning too much explanatory power to any one element—e.g. that because the technology is available, it must be used in a particular way. It is a holistic concept that orients our attention to how different parts work together. This broad view also helps identify points of strain among competing understandings of what measurable journalism should look like.

What Measureable Journalism Means for News

Having established measurable journalism as a conceptual orientation to connect a number of interlinked phenomena, it becomes important to trace how researchers have pursued different parts of the issue. Much of the scholarly attention has focused on how the quantifiable audience affects journalism, particularly in considering the real or potential clash between audience metrics and journalists’ understandings of their professional role (Anderson 2011; Hanusch 2017; MacGregor 2007; Tandoc 2014; Usher 2013). This research reveals that metrics have not smoothly or uniformly slid into newsroom practices. Instead, different accounts reveal a range of reactions and practices (Cherubini and Nielsen 2016). These findings dispel a conception of technology as the ultimate driver of newsroom practices in their pursuit of professional orientation and role conceptions across different sites.

The adoption of metrics across newsrooms does not fit a neat pattern. Petre’s (2015) study of analytics in different journalistic contexts asserts that new kind of journalistic cultures are developing, albeit in different ways at different news organizations. These cultures are not wholly endogenous, but form in relation to new players in the analytics industry, like Chartbeat or Parse.ly. The adoption of a metrics-oriented newsroom leads to new kinds of dynamics, such as gamification (Ferrer Conill 2017). Other scholars locate metrics within the economic conditions of newswork. Cohen (2015) situates metrics within larger economic processes shaping the new industry. As many news organizations face shrinking budgets and staff, technological innovations that promise efficiency becomes imperative (Vu 2014). This occurs as the fields surrounding journalism (e.g. advertising) transform and adapt to the availability of metrics (see Wang, this issue).

Moreover, Bunce (2017) shows how news metrics can be used as disciplinary devices by news managers concerned with shaping news output in ways that fit desired business models. This use of metrics to identify and reward certain reporters while rebuking others becomes absorbed into newsroom cultures and affects how reporters understand what good work looks like. These various forces underscore the need to understand measurable journalism as an interaction among competing forces of technology, economics, and professional identity. We cannot look at the technology behind metrics as inevitable. Instead, understanding the interconnected logic of metrics as they get taken up in newsrooms remains vital.

Measurable journalism extends outside the newsroom to mediate the journalist–audience relationship. New measurement tools produce data that reveals a “gap” between journalist and audience preferences (Boczkowski and Mitchelstein 2013). This environment is one in which direct feedback from audiences affects traditional gatekeeping roles of the news media (Coddington and Holton 2014), leading to a feedback loop that shapes what news looks like. The development of direct audience metrics are upending previous understandings of audience measurement and revealing granular trends among the audience (Nelson and Webster 2016). Editors respond to audience actions as they are mediated through metrics data (Lee, Lewis, and Powers 2014), albeit not without newsroom resistance (Welbers et al. 2016). While claims of turning over judgments of newsworthiness might be overstated (Zamith 2016), metrics affect the dispersal of news when news organizations reorient their distribution practices around the audience (Hanusch 2017). This is a process of deritualization (Broersma and Peters 2013) as news audiences that once arranged their schedule around news consumption now find that the news meets them in their daily patterns.

The rise of metrics owes to the rise of digital platforms where the act of news consumption also entails the transmission of data about news consumption (Diakopoulos 2016). In particular, social media platforms have become important sites for news consumption, even if both social media companies (Carlson 2018) and news organizations (Bell and Owen 2017) are wary of this role. The streams of audience data become objects of struggle between social media distribution platforms and news providers. Moreover, the opacity of algorithmic practices constrains news distribution in ways that require new conceptual approaches (Napoli 2014). Most notably, it opens up digital news spaces to the clutter of clickbait and “fake news” fabricated to earn clicks and reap advertising dollars.

The irony of measurable journalism is that at the moment when audiences have gained the capability of participating through digital platforms, they are also the most commodifiable because of the data traces they leave (Anderson 2011). As journalists speak more of “engagement” with the audiences and encourage the sharing of news content, they simultaneously build traceable relationships that can eventually lead to individualized news or target marketing. This results in the quantified audience—the reduction of a news user to a set of monitored and recorded characteristics in order to extract economic value (Couldry 2018).

Existing research has made important contributions connecting digital media to measureable journalism and the quantified audience. Studies reveal a confusing news landscape as various actors inside and outside of journalism react to changing practices and differing ideas of what news distribution ought to look like. Even this short summary reveals complicated issues involving journalists’ professional judgment, normative ideas of autonomy, economic pressures, and privacy questions about audience surveillance, among others. Future empirical work will no doubt continue to refine the contours of measurable journalism by examining different types of news organizations, different platforms for news distribution, and various modes of news consumption—all of which will clarify the heterogeneity of the news industry. But we also need to step back and take stock of what these findings indicate. When it comes to research on measurable journalism, the familiar adage of losing the forest in the trees is an apt one. Greater effort needs to generate connections across this research and think about the larger implications for journalism and society.

What are We Most Concerned About?

The constellation of activities taking place under the umbrella of measurable journalism draws attention from scholars not merely because the technologies and practices being implemented introduce new modes of audience measurement. Yes, journalism pedagogy requires keeping abreast of newsroom developments to train future news staff, but the study of journalism more broadly begins from a social concern regarding the power of news to produce widely shared representations of the real to audiences. It is from this position that journalism studies turns its critical eye toward the vast array of forces that shape news practices. If measureable journalism is to continue to occupy scholarly attention, what is needed is a specific articulation of concern.

As a way of formulating a statement of concern pertaining to measureable journalism, we can start with a basic assumption: Digital media platforms simultaneously alter journalistic practices while creating the conditions to imagine journalism’s relationships with others in new ways. These are inseparable processes in that a shift in practice is embedded within shifts in thinking about practice (Carlson 2016). To introduce a new technology into the newsroom, for example, requires an articulation of what the technology will do, how it connects to existing understanding of news, how it promotes new practices—all of which is open to contestation based in pre-existing conceptions (Powers 2012). These moments matter. Much of news work is regimented and routinized to allow for the swift production of news according to unceasing and grueling demands for content (Boyer 2013), all of which are exacerbated when resources shrink. The different machinations of measurable journalism being introduced become moments for practitioners to confront their work, as well as moments for journalism scholars to look afresh at news processes.

In this moment, what is needed is a prioritizing of conceptual development about measurable journalism that moves beyond descriptions of practice to the implications of the shifts taking place. In short, our shared concern should be about how measurable journalism alters the shared imagining of what journalism is and what it ought to do. This reimagining needs to be made explicit. A challenge is to move beyond individual cases to consider the larger patterns that are developing. This returns us to the model of measurable journalism posited above, which privileges a multiplicity of interlinking factors over any single determining factor. We need to theorize about how the system works together and what its implications are.

A central question for measureable journalism involves how the audience is being imagined (Anderson 2011). One reaction to the instantiation of metrics, as uneven as it has been, is that it journalism is being reimagined in ways that chip away at journalistic authority by abdicating news selection to audience data that come to stand in for audience wants. Tandoc and Thomas (2015) warn of the ethical implications of this scenario. Their concern is that metrics allow for the individualization of the news audience, robbing journalism of its cultural and political role in providing the public a shared stock of knowledge. In place of professional assessments of newsworthiness, what is feared is an “audience-centered free-for-all governed by market logic” (251). This is a serious concern that echoes long-running criticisms of journalism for content choices that are more about attracting audiences than informing them. But this argument also reveals ambiguity about how audiences are imagined. Advocates for a greater adherence to metrics promote, to an extent, shaping news content to fit audience interests in the pursuit of larger news audiences to increase journalism’s reach.

The debate around measureable journalism raises deeper questions about how anything should become news. Those wary of audience metrics having a role in news selection or placement worry that popularity should not be the main determinant of news. Yet having a closed off system in which journalists make decisions without regard for the audience raises its own issues around the concentration of symbolic power into the hands of a limited few. Ultimately, this is about the social power of journalism as a system for producing knowledge about the world. But someone must make judgments as to what is news, and journalism scholars will need to advocate for what types of configurations best serve the social interest.

Formulating a critical perspective that takes measurable journalism not as inherently good or bad but as a set of practices that can be deployed in any number of ways provides a lens for assessing what is being measured. That is, audience data, however collected, are proxies for the audience and not the audience member herself. After all, metrics are constrained. Journalists only receive what is measurable. Audiences, through their clicking and scrolling, can only reply to the choices offered to them. Sophisticated audience measurements have become standard, but scholars need to recognize that what is measured is what is measurable, and what is not measurable is often ignored. Audience metrics and analytics are always partial data, even when they are treated as complete.

Ultimately, what we are concerned about is the whole apparatus of audience feedback that gets built into these systems, and how this feedback becomes part of how journalism works and how it is understood. In one scenario, journalistic judgment is being diminished, with the arrival of metrics curtailing newsroom autonomy. Metrics elevate economic imperatives above all else by enabling minute tinkering aimed at extracting larger audience numbers. In another view, judgment is being augmented, and audience feedback via metrics improves the selection of news and builds better connections between the audience and journalists. Journalists are able to sharpen their reach, putting more news in front of more people. Even more optimistically, measurable journalism can provide novel ways to measure the impact and social value of journalism. Metrics can be put to use to enhance journalism’s social position and indispensability at a time when journalists face uncertainties about the continued economic viability of news. After all, measurable journalism produces data, and data can be used as evidence.

With metrics not likely to disappear, vigilance is needed to work toward a journalism that works for audiences and the best interest of communities. This includes advocating for ways of using metrics for the social good and not merely the economic well-being of news organizations. Doing so requires thinking holistically about what research should look like.

Building a Research Agenda for Measurable Journalism

As news organizations and audiences adapt to the conditions of measurable journalism, so too must journalism studies. At this nascent stage, much of the work in this area is empirical with an orientation toward explaining the dynamics that are developing and questioning blanket assumptions being tossed around. Many of the studies above demonstrate how standard methodological procedures within journalism studies—e.g. surveys of journalists or ethnography—can be utilized to explore emerging dynamics. Other studies employ innovative data analysis techniques to examine news content flows. This body of research has continued to keep pace with the state of change in digital journalism, and has begun to shed light on changes in practices and attitudes overtime as the technologies of measurable journalism and their institutionalization within newsrooms develop.

As this research has matured from early exploratory studies, it is time to consider, more directly, what the future research agenda for measurable journalism ought to look like. This essay and the special issue as a whole are meant to push this conversation forward. Certainly greater conceptual clarity will be necessary so that studies in this area are working together. In addition, methodological innovation promises to deepen understandings of how people use news (Picone 2016). As part of this, there is certainly room for mixed methods approaches, including those that combine qualitative and quantitative designs. Another research trajectory needs to be greater attention to the audience side of measurable journalism. Much of this research takes a view from inside of journalism to ascertain how journalistic working conditions are changing, how new positions are forming, and examining the tensions that arise. Certainly, audiences are represented through various metrics, but this cannot account for the richness of how audiences experience various practices associated with measureable journalism. This becomes particularly important as news content on digital platforms becomes more responsive to user interactions, such as with article recommendations or the customization of news layouts.

Finally, research needs to do more to conceptualize the social significance of findings on measurable journalism. Empirical work too narrowly focused on innovation risks being descriptive at the cost of providing insight (Carlson and Lewis, 2018). The instability of new technologies and lack of consistent application across news organizations makes this task difficult. In response, scholars need to work toward the creation of a critical apparatus that places the elements of measurable journalism within larger questions of journalism and social power.

Inside this Special Issue

The articles in this special issue provide important steps forward for the study of measurable journalism. The breadth of the work presented here underscores the need to appreciate the different elements that comprise measurable journalism. To this end, Zamith offers a state of the literature on audience metrics and analytics, demonstrating the need for a careful approach toward the use of audience analytics and metrics that neither asserts the wholesale transformation of journalism’s relationships with its audiences nor denies that these technologies have made a difference. Instead, he positions the gathering of information about news audiences in a longer history of journalism.

The other articles all examine the combination of digital technology that makes measurable journalism possible, the work of various actors involved in aspects of measurable journalism, and more abstract ideas about what the data of measurable journalism can accomplish normatively. An example of the latter running throughout the issue is “engagement”—an idea that is much touted while remaining conceptually ambiguous. For example, Ferrer Conill and Tandoc find a global emphasis on engagement coupled with the increased use of audience metrics has resulted in news organizations creating new positions. These audience-oriented editors are expected to mediate between audience data and the rest of the newsroom, giving them power to dictate the direction of news content. The emphasis on engagement grounds their work in something more normative than just a mandate to increase traffic, but the tension for news organizations between increasing income and fulfilling a public mandate persists. Powers turns from engagement to impact in his study of journalists working in a variety of media in a single metropolitan area. Like engagement, impact is both a lauded goal and an ambiguous one. Journalists speak of the need for impact, but find few ways to measure it. Moreover, journalistic objectivity norms inhibit journalists from trumpeting the impact their stories have. Wang employs field theory as a mode of analyzing the use of audience metrics within newsrooms. Rather than starting from an assumption of newsroom autonomy, her article highlights the heteronomy of news production by showing how shifts in other fields like advertising affect how the news operates. The resulting “dimensional field model” underscores the need to attend to a variety of interlocking elements rather than reduce metrics to a single dominant one.

Three of the articles in this volume direct their focus outside the newsroom at the organizations that have sprung up to facilitate contemporary measurable journalism practices. Belair-Gagnon and Holton examine the engagement of web analytics managers in news work. Through in-depth interviews with managers from across the analytics industry, they focus not on practice, but on how these managers articulate their vision of journalism and what their products can do for news. These managers find themselves selling a vision of how journalism should work while simultaneously maintaining a distance from the news. Petre’s ethnographic study of the analytics firm Chartbeat reveals how the company must develop an audience metric product that negotiates the apprehension journalists experience when their work is being monitored with their desire to monitor online traffic. Chartbeat markets itself as useful for journalism, but also raises the specter of a new form of digital Taylorism. Finally, Nelson conducts an ethnographic study of Hearken, a start-up specializing in providing news organizations with digital tools to improve engagement. The company touts its version of engagement as a means of listening to and connecting with news audiences, with the belief that doing so will lead to a more devout audience. Yet Hearken’s inability to conjure clear measurements for its vision of engagement runs up against news organizations looking for a quantifiable justification not only for the expense of Hearken, but as justification for altering news practices.

Taken together, these studies illustrate the variety of research sites and methods necessary to formulating a better understanding of measurable journalism. They all point to a time of unsettledness across ways of imagining journalism inside and outside of newsrooms. They balance hopes for increased engagement or impact with fears that economic prioritization will hurt journalism’s standing in the public sphere. Finally, they provide many paths for much-needed future research, as we have no shortage of questions about measurable journalism.

Disclosure Statement

No potential conflict of interest was reported by the author.