Nieman Foundation at Harvard
Paywalls are nothing new. But using advances in machine learning to make paywalls “smarter” could help resource-strapped local newsrooms up reader engagement — and in turn, hopefully revenue.
The crux of the idea is to put an ask in front of the right reader at the right time. While many newsrooms might already have a metered paywall, the lab will try to take that a step forward: the technology might give some readers more articles, others fewer before they’re prompted to engage. The goal is to “find the readers that are at the cusp of subscribing and help the newsroom get them over the edge,” said Al Johri, the lab’s engineering lead.
“Our core objective is to really help local news organizations improve the revenue that they’re getting from readers,” said Johri, who was previously a data scientist at The Washington Post, where he was working on a project predicting a user’s propensity to subscribe.
The Local News Lab is actively recruiting small- to- medium-sized newsrooms as initial “development partners” that are willing to try new things. Those newsrooms can be nonprofit or for-profit companies, and their business models may include subscription, membership, or donation.
“It excites me to be talking about paywall modeling and content-gating in a way that is going to help local newsrooms be more sustainable — but with a very watchful eye on equity and access to information,” said Hannah Wise, the Lab’s news partnerships lead. “We are looking at it from a lens that will hopefully ensure that people who need information get the information they need.”
The team is thinking about the “smart paywall” broadly — it could adapt to a reader’s behavior or the content. Machine learning might also be able to suggest to editorial staff which stories to put or not put behind a paywall.
“One incarnation of a smart paywall might estimate your propensity to subscribe given your browsing history or what’s known about you — and then adjust the metering in more of a personalized way, maybe to give you a few more articles or bring the paywall up sooner depending upon what your likelihood to subscribe turns out to be,” said Mark Hansen, director of the Brown Institute at Columbia Journalism School.
Part of the challenge of implementing this sort of machine learning is that a newsroom might need event-level behavioral data on its audience: When a user comes to a website, what articles do they click on and what interactions do they have with an existing paywall? The Local News Lab’s team is aware that while some newsrooms already have the digital infrastructure in place to implement these experiments, others don’t. So are thinking of ways to address various scenarios.
“As we’ve talked to some of these newsrooms, we’ve heard really interesting things being tried. The machine learning layer is simply there to help optimize some of that or to maybe codify that in code so the model can be repeated in other newsrooms,” Hansen said.
That’s also why the Lab plans to open-source the code they’re writing, so additional newsrooms can replicate it down the road. Wise, who previously managed the cross-functional news labs at the CBC in Canada, is also planning on building a community around what they’re doing with the initial development partners so they can share learnings from the experiments with others.
“When you’re a small- to medium-sized local newsroom, there’s sometimes not a lot of time or space to connect with your counterparts at other organizations across the country, so we’re hoping to facilitate that,” Wise said.
In addition to Wise and Johri, each member of the Local News Lab has a connection to journalism and news, including former machine learning engineers from BuzzFeed and The Wall Street Journal.
“Everyone has an understanding of and respect for journalism and how to talk to journalists and where the ethics lie,” Johri said. “As we’re having conversations with newsrooms, we can start on the same page. We won’t make suggestions to them that are completely out of the blue — that wouldn’t fly with the higher-ups.”
A board of representatives from 11 news organizations are also advising the project, including leaders from The Philadelphia Inquirer, Newsday, The Post and Courier, The Tennessean, the Detroit Free Press, and the Chicago Sun-Times.
Hansen envisions that in the same way a newsroom might have an editorial strategy to report on elections or a social media strategy to disseminate stories, newsrooms could have more robust strategies around encouraging conversions or unique engagement around content, “so that paywall strategies become something that is routinely thought about and aligns nicely with your content.”
“Bringing teachings from different disciplines really helps broaden the creativity in the field,” Hansen added. “That applies equally to the way we find stories, tell stories and now the way those stories are circulated.”
Though this is the Local News Lab’s first initiative, the team hopes there will be future projects with open-source software or easily deployable systems that small- and medium-sized newsrooms can use.
“We understand how important local news is to our democracy overall but also to larger news organizations — local news is where the news really starts from,” Johri said.
Vignesh Ramachandran is a freelance journalist and co-founder of Red, White and Brown Media. He has written for the Colorado Sun, Knight Foundation, and NPR and previously worked for ProPublica, the Stanford Computational Journalism Lab, NBC News Digital, and Mashable.
