Research Report by Ryan Leach
Created July 22, 2017
Scholars have long theorized the impact of mass media on society. Only recently, however, have researchers attempted to construct metrics for calculating media impact. Rather than solely focusing on these current developments, this report puts the contemporary metrics approach within the long history of assessing and conceptualizing the impact of the mass media on social and political life. In so doing, media impact appears less as a contemporary phenomenon brought about through digitization, and more as part of a broader field of study encapsulating multiple, often conflicting, perspectives and goals. After recounting the history of media impact, the report focuses on the contemporary journalism industry’s metrics and tools for assessing media impact, and evaluates the usefulness of these metrics and tools for the WhatEvery1Says (WE1S) project.
Overview of the Topic
Although media impact often appears as a new paradigm specific to the digital age, as Fergus Pitt and Lindsay Green-Barber point out, it is “a suitably complex topic that has preoccupied researchers and journalists alike for almost as long as the industry has existed.”Rather than confining ourselves to thinking media impact only in relation to the rise of donor-based journalism, we might instead consider digital approaches to media impact as a relatively new “turn” in a scholarly field beginning at the latest with the rise of the “mass media” in the late nineteenth and early twentieth centuries.
History and Current State
Over the course of the past century or so, journalistic and academic perspectives on media impact have varied considerably. Although contemporary journalists attempt to reconcile considerations of media impact with notions of supposed journalistic objectivity, early American newspapers had no such qualms; they entirely existed as public mouthpieces for specific political parties, and their function was to manipulate public perception to accord with their party’s platform.However, from 1870 to 1920, technological development in the printing industry, as well as the rise in advertising revenue, enabled the predominance of independent newspapers lacking ties to any specific political party.With the rise of independent newspapers, news agencies portrayed themselves as credible sources, and professional journalism associations began developing principles of journalistic objectivity.
From the 1920’s onwards, mass media scholars developed a new strain of media impact analysis to understand the rise of fascism in Europe and, in some cases, the maintenance of capitalist hegemony. In so doing, these newer notions of media impact challenged the supposed objectivity of (even independent) journalism, revealing the news media as a means through which powerful government and corporate groups solidify control and manufacture consent. One of the earliest theoretical approaches of this sort, Antonio Gramsci’s hegemonic model of culture and media analyzes dominant ideological formations and discourses within such institutions as the news industry to expose the process by which the working class becomes immersed (or interpolated) in capitalist ideology.Similarly, although outside the Marxist tradition, Paul Lazarsfeld and Robert Morton’s seminal “Mass Communication, Popular Taste, and Organized Social Action” (1948) conceptualizes the function of the mass media as the enforcement of existing societal norms and narcotization of the masses, thus inhibiting political and social change. This hegemonic turn within the field of media impact extends to the present day, notably including Edward Herman and Noam Chomsky’s Manufacturing Consent: The Political Economy of the Mass Media(1988).
Perhaps most related to WE1S, Max McCombs and Donald Shaw’s “The Agenda-Setting Function of Mass Media” (1972) developed an empirical basis for studying how the media directs public attention, namely through granting more attention so some topics than others.According to the authors, increased news coverage of a topic correlated with greater perceptions of importance among public audiences. In so doing, McCombs and Shaw established that the positioning and coverage of certain news articles over others enables news organizations to frame the debate (or set the agenda) regardless of the objectivity of the reporting.
Leaving aside for a moment these academic approaches, the advertising industry has also long attempted to provide an empirical basis for studying media impact. In the 1950s and 60s, advertising agencies evaluated the success of an advertisement purely in relation to increases in sales and profit.However, over time, these metrics became more complex, evaluating also customer satisfaction, customer loyalty, and brand equity. These new metrics required new methodologies for assessment. According to Fergus Pitt and Lindsay Green-Barber, “Marketing and advertising firms traditionally used surveys and focus groups to measure these attitudinal factors, although the advent of social media has produced a set of tools promising to deliver customer insights and measure the impact of marketing and communications.”
In its contemporary state, media impact attempts to quantitatively assess the effects of journalism on public policy through the employment of digital tools. Like all big data projects, the current study of media impact heavily relies on recent developments in digital technology; however, as others point out, the emergence of the media impact paradigm does not merely evolve out of new tools, but also wider social and economic shifts in the journalism industry. In particular, authors such as Anya Schiffrin and Ethan Zuckerman perceive the transition from advertising-based journalism to philanthropy or donor-based journalism as crucial to the birth of new media impact methodologies. For them, advertisers only wanted to quantify the exposure of their advertisements, limiting the assessment of impact to what they call “reach.” In contrast, philanthropists and donor approach journalism from a business perspective that wants to evaluate the effectiveness of journalism in order to justify their investments.Others, such as Dana Chin (who runs the Media Impact Project) acknowledge the persistence of media impact questions in the pre-digital journalism studies, claiming that digital tools have provided us the opportunity where “we can get one level above where we’ve been in the past, which is throwing up our hands and saying, ‘It can’t be done.’”However, as with any paradigm shift, obviously technological development and social and economic changes both contribute to the development of these new methodologies. In essence, media impact attempts to develop metrics for quantifying the effect of news media on public opinion and policy.
In this longer historical view of the paradigm, the meaning of “media impact” expands to include multiple, oftentimes conflicting, perspectives and goals for studying the impact of media. Rather than a single homogenous field, media impact covers both anti-capitalist and anti-fascist approaches to identify state and corporate control of the media and metrics for establishing return-on-investment for advertising agencies and billionaire donors. Only recently has the term “media impact” come to designate the quantitative assessment of the effects of a news article or outlet on public opinion and policy within the journalism industry.
Broader Research Context
In its contemporary state, media impact is primarily relevant to multiple components of what we might think of as a journalism network, including journalism academic departments, media industries, philanthropic donor groups, nonprofit organizations, and public policy institutes. Articles on media impact tend to draw from all of these sources in order to conceptualize the work taking place within the field. In addition, there’s significant cross-collaboration between all of these components—e.g. academic departments funded by donor groups providing media impact tools to news corporations.
Statement of Relevance and Limitations of the Paradigm to the WE1S Scoping Problem
Metrics: Reach, Influence, Impact
Both “Can We Measure Media Impact? Surveying the Field” and “Can We Measure Media Impact? Between the Lines” categorize these methodologies within three overarching dimensions: Reach, Influence, and Impact. Although, confusingly, Chip Giller and Katharine Wroth prefer to call Anya Schiffrin and Ethan Zuckerman’s conception of influence “impact” and vice versa. For clarity’s sake, the below descriptions use Schiffrin and Zuckerman’s terminology.
Perhaps the most simplistic metric, reach quantifies the level of audience engagement with a news article or source. This includes tracking the number of individuals accessing the content, the length of time the audience views the content (attention minutes), and the sharing of content via social media. One of the primary tools for measuring reach is NewsLinx, which provides assistance to news agencies and their funders in tracking the spread of specific stories across the internet.
Delving deeper into media engagement than reach, influence attempts to reveal the effect of specific content on public discourse. Although metrics for this dimension are still under dispute, researchers currently measure influence through recourse to social media interaction (beyond mere sharing, as in reach) and tracking hyperlinks to the article or publication (inlinks) and out from the article or publication (outlinks). In so doing, influence attempts to view an article or media source in relation to the wider sphere of published material. Media Cloud provides an exemplary tool for tracking these relationships at various levels of scale, from sentence to article to media outlet. In addition to quantifying inlinks, outlinks, facebook shares, and bit.ly links, Media Cloud enables researchers to identify themes and keywords, and to map the relationships between these key words and between links in Gephi. For more about Media Cloud’s relevance to WE1S, see below.
Whereas influence quantifies the effect on discourse, impact attempts to correlate news content with concrete changes in public policy and movement building. Of course, it is incredibly difficult to determine causation between the publication of a news article and the social and political changes with which it has been correlated. Nevertheless, researchers are currently in the process of developing metrics for measuring such impact, and projects such as the Participant Index currently attempt to causally connect news or documentary exposure with the audience’s later political engagement (such as signing petitions, making donations, or joining organizations).
Relevance to WE1S
As “Can We Measure Media Impact? Surveying the Field” notes, “Tools that can measure not just “reach” but also “influence” and “impact” are in their infancy.”Impact, as defined above, seems almost impossible for WE1S to measure, at least in the beginning stages of the project. It seems like it would be very difficult to determine causation between articles on the humanities and academic or governmental policy changes. In addition, as Lindsay Thomas points out, impact measurement might only apply to the wrong level of scale—that is, impact measures the representativeness of a given article, but we are perhaps primarily interested in the representativeness of a publication or media outlet.
Influence might prove more useful, but we might have to evaluate the metrics of usefulness in relation to our study. As mentioned above, Media Cloud is an exemplary tool for measuring influence, and one we might want to consider for WE1S. In so doing, we could track not only the influence of a particular article on humanities discourse, but also the influence of a larger media organization. But how useful are inlinks, outlinks, facebook shares, and bit.ly clicks in measuring the influence of articles on the humanities? Inlinks seem to provide the most reliable means of assessing influence; however, they have the same downfall as the impact factors of academic journals: it assumes that linking (or citing) a work means that the work proved influential. But perhaps the citation was merely for further study, or to single out the article as particularly useless. Was it still influential? Nevertheless, considering the irrelevance of the other Media Cloud metrics, inlinks might provide at least the best method of approximating article and media outlet influence.
In addition, Media Cloud provides a variety of other useful tools that may be of use to WE1S, provided we manage to use them on our own corpus. The Attention tool creates a chart of the number of sentences containing a keyword (e.g. “humanities”) over time, enabling researchers to identify spikes of interest in the topic and correlate these spikes with issues during that time period. Similar to topic modeling, the Top Themes function will eventually employ machine learning to not only aggregate words into coherent topics, but also name these topics (something topic modelers have to do themselves). It also reveals the count and percentage of a theme’s occurrence. However, as of now, this feature is still under development. In line with more basic forms of text analysis, the Top Words function provides an ordered word cloud based on word counts within a sample of the corpus. Although it is not clear how this sample is chosen, the site claims, “We have done extensive testing to validate that the sample size is representative of the entire set of results.” Researchers can also click on each word in the ordered word cloud to reveal: (1) articles it has appeared in, (2) words that generally come before and after it, (3) another ordered word cloud of the selected word in relation to the corpus, and (4) the attention section for that particular word (based on sentence numbers over time). Finally, Media Cloud creates word and link maps as downloadable Gephi files for visualizing the network of related words and of links between media sources, respectively. As mentioned above, Media Cloud currently analyzes only its own corpus. Is there perhaps some way we might employ these tools on our own corpus, which will probably exceed their collection of sources on the humanities?
Scope and Media Specificity
During one of our previous meetings, we discussed some of the difficulties in addressing the media specificity of print and online news within the context of WE1S. As yet, I do not have answers to this problem, but reading several reports from the Pew Research Center lead me to further complicating this print/online binary, as well as to consider other media sources. After all, the majority of Americans still get their news from television and many others from radio.
In addition, the print vs online assumption rests on the idea that online news is always read. However, as the Pew Research Center points out, many online news accessors prefer listening or watching their news:
Although the preference for watching or listening does not necessarily indicate watching and listening online, it’s impossible to ignore the multi- or hypermediality of online news sources, where even written articles are accompanied by videos on the story, or audio clips. All of this leads to two questions: (1) do we want to include TV, radio, and podcasts as sources, and (2) how should/can we include them?
One way (and perhaps the only way) would be through transcripts. The Internet Archive provides access to the TV News Archive which contains TV News transcripts from 2009 to the present. However, the search functionality is currently quite limited and makes it difficult to search for a single word. For instance, a search for “humanities” produces results for “human,” “humanity,” “humane,” etc.—and therefore plenty of useless results. In addition, LexisNexis provides transcript sources for broadcast TV and radio.
Considering the large audiences of TV and radio news, these sources would theoretically have a significant impact on the public discourse surrounding the humanities. Of course, reducing these news sources to transcripts ignores the audio and visual components of TV and radio, rending the source material mere text. However, one could argue that we are already doing that with online news articles—i.e. ignoring the videos, images, sound recordings, etc. that often accompany the text.
The media impact paradigm introduces serious considerations for the WE1S project. While none of the above approaches evaluates the impact of media technologies themselves, the paradigm provides multiple perspectives for thinking the impact of mediated content on political and social action and discourse. Although metrics for quantifying impact (in the specific sense above) remain in their infancy, tools and metrics for influence, particularly Media Cloud, may prove useful in analyzing the effect of particular articles and media outlets on the public discourse of the humanities. However, the project might need to scale upward to include impact analysis not only on single articles or news outlets, but on media technologies themselves. As shown above, the majority of Americans receive their news from television programs. Although it’s outside the scope of WE1S to consider the impact of television as a technology, the predominance of TV viewers may require including television (and potentially radio) transcripts in the corpus.
Herman, Edward S., and Noam Chomsky.Manufacturing Consent: The Political Economy of the Mass Media. Reprint edition. New York: Pantheon, 2002. Print.
Giller, Chip and Katharine Wroth. “Can We Measure Media Impact? Reading Between The Lines.” Stanford Social Innovation Review. Fall 2015. Accessed July 22, 2017. https://ssir.org/articles/entry/can_we_measure_media_impact_reading_between_the_lines?_ga=2.53219758.723755759.1499186117-744132522.1497402763
Lazarsfeld, Paul F. and Robert K. Merton. “Mass Communication, Popular Taste, and Organized Social Action.” Mass Communication and American Social Thought: Key Texts, 1919-1968. eds. John Durham Peters and Peter Simonson. Lanham, MD: Rowman & Littlefield, 2004, pp. 230–241.
McCombs, Maxwell E., and Donald L. Shaw. “The Agenda-Setting Function of Mass Media.” The Public Opinion Quarterly, vol. 36, no. 2, 1972, pp. 176–187. JSTOR, www.jstor.org/stable/2747787.
Media Cloud. Homepage. MIT Center for Civic Media and Berkman Klein Center for Internet & Society at Harvard University. Accessed July 22, 2017. https://mediacloud.org/
Mitchell, Amy, Jeffrey Gottfried, Michael Barthel and Elisa Shearer. “The Modern News Consumer.” Pew Research Center. July 2016. Accessed July 22, 2017. http://www.journalism.org/2016/07/07/the-modern-news-consumer/
NewsLynx. Homepage. Tow Center for Digital Journalism at Columbia University. Accessed July 22, 2017. http://newslynx.readthedocs.io/en/latest/
Pitt, Fergus and Lindsay Green-Barber. “The Case for Media Impact: A Case Study of ICIJ’s Radical Collaboration Strategy.” Tow Center for Digital Journalism. Columbia University Academic Commons. June 2017. https://doi.org/10.7916/D85Q532V.
Schiffrin, Anya and Ethan Zuckerman. “Can We Measure Media Impact? Surveying the Field.” Stanford Social Innovation Review. Fall 2015. Accessed July 22, 2017. https://ssir.org/articles/entry/can_we_measure_media_impact_surveying_the_field
TV News Archive. Homepage. Internet Archive. Accessed July 22, 2017. https://archive.org/details/tv
Sources for Further Study
Alcorn, Jason and Lauren Furhman. “A 2-in-1 Guide: Impact Assessment & Metrics Guide.” Ed. Veronica Jauriqui. Media Impact Project. https://learcenter.org/publication/impact-assessment-metrics-guide/
Green-Barber, Lindsay. “Offline Impact Indicators Glossary.” Media Impact Project. September 18, 2014. http://www.mediaimpactproject.org/offline-impact-indicators.html
Media Impact Project Data Repository. Media Impact Project. http://www.mediaimpactproject.org/data-repository.html
- Knight Foundation
- Harvard Nieman Lab
- USC Norman Lear Center guide
- Columbia University’s Tow Center for Digital Journalism
Ryan Leach is a Ph.D. student in English at the University of California, Santa Barbara. His research interests include media theory, digital humanities, speculative fiction, and science and technology studies. He can be contacted via email (email@example.com), read via blog (https://frettingsontheblank.wordpress.com), or merely followed via Twitter (@ryankleach).
We could potentially go back even further. In some sense, concerns over impact have arose with the development of any new media technology, from Plato’s critique of writing onwards. However, for the purposes of this report, I have confined our consideration to the impact of journalism in the mass media.
Admittedly, I’m a bit skeptical of this claim: it’s not as if advertisers only cared about the exposure of their ads and not the impact on sales, for instance. In addition, advertisers are not politically neutral and probably were also concerned with the political impact of journalism in which their ads existed.