It’s a series of questions we get often at the Center for Cooperative Media: How many news organizations operate in New Jersey? How many are print versus radio versus television? How many people do they employ? So what parts of the state have no local news source? How can we help those places?

The problem is, we can’t answer these questions with complete accuracy. In fact, we don’t know anyone who can. Despite the volume of research currently under way about news ecosystems, there is no gold standard; many studies to date have critical flaws, such as focusing on only one type of media, using too few sources to feed underlying databases, or considering news only through a strict geographic lens.

We believe that knowing the true landscape of the local news ecosystem in this digital age could help our work, and others, in so many ways.

More than a year ago, the Center began work on a new methodology to address this urgent question.

We’ve been considering the work a three-phase process. Phase 1, now complete, was a literature review that summarized the entire ecosystem/ecology literature to date, organized the studies according to eight different typologies, and fleshed out a new method that will allow for both depth and scale in the study of local news ecosystems. Phase 1 was co-authored by myself; Magda Konieczna, assistant professor of journalism at Temple University; and Jesse Holcomb, assistant professor of communication arts and sciences at Calvin College.

This is the first of a series of posts that will address different issues and aspects of news ecosystem mapping. In this first post I will discuss one aspect of the research addressed in Phase 1: The problem of depth versus scale. This is, in some ways, the fundamental issue with which all research must grapple, but it is especially acute in ecosystem mapping because of the amount of data needed to understand a local news ecosystem, and the desirability of a comparative element, which requires a larger sample of ecosystems.

Combining depth and scale requires either a nearly unlimited amount of resources or an incredibly sophisticated research design utilizing the latest digital tools. Because we do not have a nearly unlimited amount of resources, we are striving for a brilliant research design using the latest digital tools (more on the research design in a later post).

By depth I mean building out a comprehensive census of outlets, an accounting of news flow that includes key producers and amplifiers, and an assessment of corresponding social media. Depth may also, but need not always, include understanding consumption patterns and effects. By scale I mean producing this knowledge for many local news ecosystems, not just a few or a handful. Obviously, doing both is the gold standard for understanding a regional, state, or national local journalism landscape.

Ecosystem studies that aim for depth are usually case studies. Case studies examine a specific place or event and use data collection tools that cannot be easily scaled, like on-the-ground explorations or analog studies of particular news stories flowing through the ecosystem. The scaled/scalable approach, on the other hand, employs methods capable of spanning geographies or news events: surveys, large datasets, or scraping software. Some such ecosystem studies are not technically scaled but are scalable.

Ecosystem case study methods may include fieldwork, ethnography or manual content analysis. The Pew Research Center 2010 study of news diffusion in Baltimore, for instance, reconstructed specific local news events by manually analyzing iterations of stories as they evolved in print, broadcast and online. The actors were enumerated through reportorial investigation —  digital groundwork, email messages and phone calls to journalists and other stakeholders. Another good example is Anderson’s 2010 study of the Philadelphia news ecosystem, which came out of time spent in Philadelphia newsrooms and activist communities, combining network ethnography and qualitative newsroom analysis. These are classic case study designs. The amount of labor involved in replicating these works at a larger scale is outside the realm of possibility for any academic or industry study.

In 2015, Pew Research Center also sought depth when it looked at the local news ecosystems of three American cities — Denver, Macon, and Sioux City — to understand how local news was transforming in the digital age. They conducted surveys of residents, content analyses, built outlet censuses, and performed social media analysis in each city. What resulted was a detailed analysis of the cities both individually and in comparison, from which they produced much insight. However, the methodology was so detailed and labor-intensive that it could never be replicated for a large number of cities.

Aiming for scale, Napoli, Weber, McCollough, and Wang sought in 2018 to compare the quantity and quality of local news for 100 communities around the U.S. Using digital methods that included scraping more than 700 local news websites, they were able to both catalogue all outlets with an online presence in each community and study whether their output lived up to its democratic mission of providing original news about the community that fulfilled a critical information need. They included only outlets that were physically located within each community, leaving out any that might serve the community from elsewhere. This method produced important comparative results, but sacrificed some depth and nuance because it defined the local news ecosystem according to strict geographic parameters.

Another example of a scaled study of local news ecosystems is Abernathy’s 2018 study of the closure of local newspapers. As struggling locals are acquired by non-media corporations, Abernathy found, they are being closed or turned into “ghost” papers, which provide little original hard news about the communities they purport to serve. This study covers the entire United States at the county level — clearly a scaled effort — but because it only looks at newspapers, does not give us an accurate picture of the local journalism landscape in the digital age.

A number of published studies may be scalable, allowing proliferation of data while keeping research labor in check. These include network analyses, such as Graeff et al.’s 2014 study of the diffusion of news about the killing of Trayvon Martin, or Gordon and Johnson’s 2011 and 2012 studies of the Chicago digital news ecosystem. Neither were scaled, but their leveraging of computational tools to collect data programmatically opens up broader possibilities.

As we can see, case studies offer more nuance and depth, but often sacrifice generalizability. Scalable studies can offer generalization, but often sacrifice detail and depth. The goal of the Center for Cooperative Media’s local news ecosystem mapping project is to devise a methodology that does both.