Nieman Foundation at Harvard
Editor’s note: Our friends at NPR Digital Services, Eric Athas and Keith Hopper, wrote about an experiment of theirs using Facebook’s geotargeting features on the Digital Services blog. We thought their findings were interesting — geotargeting’s a tool not a lot of news organizations use — so they’ve kindly agreed to let us cross-post their story here.
NPR’s Facebook page and its 2.3 million-like audience is made up of users from thousands of cities across the world. We wondered: What if we focused on just one city?
The question arose after identifying a somewhat obscure Facebook feature that allows anyone with a Facebook page to customize posts by location. This means, for example, that you can post a story about Boston and modify it so that only users in Boston will see it in their Facebook feed.
Last October NPR Digital Services and Digital Media used this tool to launch an experiment with member station KPLU, in which we shared selected KPLU.org content on NPR’s Facebook page, but only for the eyes of the Seattle region (KPLU’s market). Four months into this experiment, we’ve made some unexpected discoveries around Facebook communities and the power of localization on a national platform.
From a technical standpoint, geofocusing page posts on Facebook is easy — just click the Public button next to Post prior to publishing. But before launching into this experiment — which is ongoing — we outlined how it would work, what it would look like and how we’d measure it.
The coffee shop test
We work with KPLU’s Online Managing Editor Jake Ellison each day to determine which KPLU.org story we’ll post. It must pass what we call the content coffee shop test: the conversation-style that NPR’s Social Media Desk has developed, mixed with a splash of local flavor. We want stories that will raise curiosity and be talked about in a Seattle coffee shop. Jake works with reporters to produce stories that will hit this sweet spot. If it does, Digital Services posts it to NPR’s Facebook page. If you live in the Seattle region and like NPR’s Facebook page, you may have noticed more Seattle-oriented stories from NPR, linking to KPLU.org. If you live outside of Seattle, this experiment hasn’t touched you.
Pacing
NPR’s Social Media Desk has a steady cadence — about every hour or so — for its Facebook posts. In a given day you may only see 10-12 NPR posts in your Facebook news feed. We don’t want to disrupt this pace and overflow Seattle users with too many stories, so we only post up to one KPLU story on NPR’s Facebook page per day.
Measurement
To our knowledge, no other news organization has used Facebook to geofocus content in quite this way. So before diving in we needed a way to closely measure the experiment. We wanted to know how this test would impact KPLU.org’s audience growth. For this we decided on campaign tracking — tacking a unique tag onto the end of each link we post. We also wanted to monitor engagement on Facebook to determine how users would interact with these local stories and whether or not the number of likes, shares and comments would differ from globally shared posts. For this we dug into Facebook Insights.
We knew posting a KPLU story to NPR’s Facebook page would result in a traffic bump to KPLU.org, even if only a small piece of the 2.3 million audience would see it. But we didn’t know how big of a boost this would give the site.
As it turns out, a big one. During the first four months of this experiment, we posted about 50 geofocused KPLU links — a fraction of all KPLU content — on NPR’s Facebook page. These posts accounted for 12 percent of KPLU.org’s sitewide visits during this four-month period. The test helped KPLU achieve three milestones: record traffic for a single day (January 19), second-highest traffic for a single month (October 2011) and the highest traffic for a single month (January).
When a story is posted to NPR’s Facebook page the usual way (globally visible), the result is an instant flurry of likes, shares and comments from across the world. When a Seattle-targeted story is posted, it gets a high volume of likes, shares and comments, but usually fewer than a global post. This of course is because only a fraction of the NPR Facebook audience can actually see it.
Using data from Facebook Insights we were able to measure the relative engagement of stories, allowing for an apples-to-apples comparison. In other words, we looked at the number of likes, shares and comments on a Facebook post as a percentage of the number of unique people who viewed it. We call this a post’s engagement rate — of the people seeing it, how many likes, shares and comments are they generating.
We found that geofocused posts to the Seattle region usually had a much higher engagement rate than links shared to the global NPR Facebook audience.
For example, the KPLU story “Is Seattle a great but lonely place to live?” was posted to NPR’s Facebook page on January 6 to a geofocused Seattle audience. This story achieved relatively high levels of engagement compared to other local posts.
The NPR story “The Public Respects Civility, But Rewards Rudeness” achieved relatively high levels of engagement compared to other global posts and was posted to NPR’s Facebook page on January 26 to the entire global audience.
As is clear for these specific stories, the local post outperformed the global post in relative engagement across likes, shares and comments. After noticing this same trend for other individual posts, we wanted to know if this was the case more generally across local posts, so we rounded up the full body of posts and did the math. We found that during the first four months of this experiment, the average engagement rate across all geofocused post was six times higher than all global posts.
Throughout the course of this test we’ve had a lot of conversations about possible explanations for why this is happening. One concern we kept a lookout for was the possibility of confusion over where the content originated and where it linked to. However, steady growth in engagement throughout the experiment coupled with a lack of curious comments in the story threads seem to suggest that this is likely not a problem.
Just a few days into this experiment, we noticed something different was happening. It was after we posted a KPLU story about Amanda Knox — a Seattle native — being flown back to her home town after Italian authorities freed her. The story focused on how SeaTac Airport was dealing with the media hysteria that would likely ensue.
This story sparked the normal gut-reaction comments, but the users also reacted to how the story related to them — the Seattle resident. They began talking to one another, sharing information as local residents.
“Would have been a lot smarter to SAY they were flying into Sea-Tac, but instead, arrive in Portland and drive up I-5 to get home.”
“Glad I’m flying in tomorrow.”
“It’s kind of like when it snows here. Everyone freaks out over 3 inches, schools close, nobody can drive, and the east coast sits back laughing at us.”
We saw this trend continue throughout the experiment and develop as the content became more pointedly about Seattle. For example, one KPLU story tackled a question Seattleites know well: Why don’t people in Seattle use umbrellas?
Residents of New York, Boston, or D.C. wouldn’t have much to contribute to a conversation around this question — or even understand why the question was being posed. But Seattle users — the only ones who saw this post on NPR’s Facebook page — had a lot to say.
“Because we spend more on rain gear than most people spend on computers.”
“It never rains hard enough to run makeup….you just get a misty glow. And umbrellas are for quitters ;)”
“Because it’s hard to open and close an umbrella with Coffee in your hand.”
We’ve told you a lot about what we know from this experiment, but there are plenty of things we’re still investigating. We’re curious if this can be replicated in other markets and are exploring options for scaling it to more member stations. Some questions about this test will be answered when the experiment grows — something we’re looking to pursue. Although we’re still analyzing the results, we’re confident about the potential of this as a powerful journalism tool.
