If your only source of news is your local TV news station on Facebook, will your news and information needs around what’s going on in your community be met? The forecast isn’t good.

Stations’ tag lines make strong pronouncements about being “on your side” and “community-driven” and an area’s “best place for local news.” Facebook going to make local news a priority everywhere, is it drawing on a universe of entirely locally relevant content?

I was curious how much of the news that local TV stations posted to Facebook was really local news. So I analyzed 28 TV stations’ Facebook posting activity for the period between March 23 and March 30 — around 8,000 total posts — downloaded via CrowdTangle. (Caveat: CrowdTangle is only able to pull public posts, so our analysis doesn’t include targeted posts intended for a subset audience. I was also only able to look at English-language stations.)

Stations in our sample posted an an average of just over 50 percent local content, using an an extremely generous threshold for “local” content that didn’t require the content be original work produced by the station itself. For five of the stations in this sample, I tallied local news posts over a longer, 30-day period to spot-check whether their seven-day percentages held steady, and they they did.

For comparison, I also downloaded the Facebook posting activities of the Boston Globe, our local paper here: 65 percent local content for that same week. (This “local” percentage would actually be higher if I’d excluded the Globe’s extensive original politics coverage out of its D.C. bureau, as well as non-local but original reviews and essays by its many critics and columnists.)

The overall range was pretty wide. The most local-focused station in our sample posted more than 91 percent local content (for this story, we won’t get into how much of this was weather and crime). The station posting the lowest proportion of local to non-local content shared around 29 percent locally relevant stories in the week examined.

Most of the TV station social media managers I checked in with for this story said they had no specific “local” quota they were trying to hit on Facebook, and just worked off intuitions on what would elicit engagement among their Facebook crowd. Often, they were an overtaxed staff of one — sometimes, no one on staff specifically had “social media” in their title — though many people had admin rights to post to or stream from the station’s page. Digital staff across the board seem to have sensed the continued success of Facebook Live on the local level — as well as the reliable boosts in reach that come from soliciting puppy photos, asking survey questions, or weighing in using the station’s official account below a post.

The stations in our sample represented nearly all of the cities in Facebook’s local news test, plus a few others that have cultivated reputations for higher quality local information (WCPO, WRAL), two station pages in New Orleans (WLWT, WDSU), and a Boston station (WCVB).

Many are under the same ownership; Hearst, Sinclair, Scripps, and Tegna were the common reoccurring parent companies. Stations with shared ownership crossposted from a pool of videos and stories that seem to be getting good engagement at sibling stations. The same viral stories came up over and over again, so predictably that it felt like they were being pushed automatically to sibling station Facebook pages, with or without individual station staff input.

Hearst-owned pages all drew on videos from the feel-good social video service Stitch, also a Hearst property. Scripps-owned station pages posted a lot of evergreen, not very locally related content from a lifestyle site Scripps owns called Simplemost. Sinclair-owned station pages posted content from Circa. Tegna stations drew from its “Verify” video series, which pairs a journalist and a non-journalist with a question to be fact-checked.

WCVB of Boston, with more than 750,000 followers, was the biggest Facebook page I looked at (followed by the Boston Globe’s, at more than a half million. KIDY of San Angelo, Texas had the smallest page. I checked the sizes of the markets these stations covered, the sizes of the Facebook followings, and the size of their digital staff (as far as I could find on these station websites). Some stations in our sample were unaffected by Facebook’s algorithm change to favor friends-and-family interactions in the News Feed; others were. None of these factors seemed to be tied to how local their Facebook pages turned out to be.

Local, but not local and original

It was often difficult for me to determine where the news being posted about had originally took place. Location information would’ve helped viewers, too:

For this experiment, I drew the “local” circle around the state — or, if a station was based in, for instance, a tri-state area, the states that station’s signal touched. If a station’s Facebook post covered a statewide law or the state governor or a state senator, or streamed remarks given by state senator on an issue specific to the state, I counted it as local. A post that was mostly aggregation, but pulled out information relevant on a local level (e.g., how much the state’s senators received in NRA donations, taken from a database compiled by a national news publisher), counted as local. A post involving a prominent senator from the state “calling on Congress or Donald Trump to make X or Y national policy change” didn’t count as local. All weather and traffic counted as local (several stations in our sample had separate weather-only Facebook pages; I didn’t analyze these).

A story written by a reporter at the station that covered a national topic with no local-level interviews didn’t count as local, but when a story interviewed a local voice — such as local legal or medical experts — it did. An AP story that covered a local topic also did. A common tactic (recognizable to anyone who’s had to produce quick hits for the internet under some duress) is writing one or two paragraphs around a tweet from a celebrity with roots in the area and embedding reaction tweets from more local-level celebrities such as radio hosts or former mayors. I gave those a pass and counted them as local as well. (I relied on this paper for a lot of my thinking around how to classify locally relevant information.)

Stations often posted photos that didn’t link out to anything, usually of a local landmark or scenic point a viewer had sent in, the weather forecast, a call for photos or audience questions about a timely topic, or awkward memes featuring local sports teams or players, all of which I counted as local. Calls to discuss a “favorite margarita place in the city” or “best crawfish in the region,” tied to some news event, however loosely, fell under my definition of local. But: A video of station talent giving a rundown of the “trending topics on social,” when none of the topics were local? No. Facebook text statuses announcing an upcoming local news broadcast or a time change counted as local; statuses that only read “enjoy your Monday! TGIF!” without any tie-in to events or other local information didn’t. Generic calls like “post a photo of your pet” didn’t count either. Neither did posts linking to instructions for how to really delete your Facebook account. I discarded paid posts with a local sponsor attached from the sample entirely.

SHOCKING!

Between local newspapers and TV stations covering similar geographical terrain, there’s often been mutual disdain (just recall the Cincinnati Enquirer staff’s reactions to this “newspaper war” TV station WCPO unilaterally declared). Ongoing work from the News Measures Research Project on news deserts suggests that the thinness of local content promoted by local news outlets isn’t limited to the local TV sphere. When researchers analyzed the home pages of news sources including TV, radio, online-only outlets, and local newspapers in 100 municipalities around the U.S., they found that 20 of those communities saw no local news stories at all, at least not during the time periods the researchers looked at. (These initial findings are from a draft of the paper “Assessing Local Journalism: News Deserts, Journalism Divides, and the Determinants of the Robustness of Local News,” submitted by researchers for an upcoming meeting of the Association for Education in Journalism & Mass Communication.)

But local TV’s continued influence is undeniable. Pew Research Center data has shown that while the percent of Americans who regularly get their news from local TV is shrinking, it’s still a top source of news for many. A recent sweeping analysis of local TV by the Knight Foundation found that in many smaller markets, local TV stations’ websites actually outstrip local newspapers’ sites as the dominant source of news. Moreover, in the latest iteration of the Facebook News Feed, local news is getting some help, and local TV news has a leg up when it comes to video material.

Local TV has also faced a pile-on of scrutiny lately, due to concerns over the right-leaning Sinclair Broadcasting Group (scroll through some comments on Sinclair-owned stations’ Facebook pages after the Deadspin video came out and you’ll see a new flood of protest).

Sinclair already owns or operates 193 stations, making it the largest owner of TV news stations in the country (though its quest to acquire the 40 Tribune Media stations has stumbled). In Circa, it has another social media-friendly content mine. Researchers have found that stations Sinclair bought do tend toward airing less local politics and more national news than other stations in the same market. The Sinclair-owned stations in our sample were posting half of less of local content. But so were many non-Sinclair-owned stations.

There’s a lot more to tease out. Critically, I couldn’t tell from this experiment, for instance, which posts were reaching the most Facebook users, and whether those most popular posts were local in nature (I tried this for a few stations in our sample, and got different results depending on whether I was judging reach by volume of shares, comments, likes, or video views. Matt Karolian of the Boston Globe tried an early version of this experiment with Boston-based TV stations using CrowdTangle performance scores; his results were similar to mine.) There’s also the question of staffing, and directives from corporate: How much control do individual digital staffers really have over their organizations’ social presence? Are stations being given impossible Facebook reach targets?

To their credit, many stations in our sample made compelling use of Facebook Live, especially for weather and traffic, and to bring viewers into the TV studios before a broadcast. Every station in our sample covered the March for Our Lives protests on a local level, and livestreamed the national event in D.C. Some have used Facebook video to help make the findings in an investigative report more concrete for viewers. Others offer Facebook Groups as a space for readers to discuss a very specific local issue. Examples of smart use of Facebook would span a whole separate story (stay tuned).

But if your main source of local news is the Facebook page of your local TV news station, and if less than half of your local news TV station’s posts on Facebook each week are locally relevant, then it’s a good bet that, in the limited real estate of the News Feed, you’ll actually see an informative story about your own community.