The Online News Association is holding its annual conference this week. One early highlight was Sonali Verma — senior project manager at The Globe and Mail, the largest newspaper in Canada — leading an absolute masterclass in dynamic paywalls on Tuesday.

The Globe and Mail’s artificial learning technology, called Sophi, uses deep-learning techniques to automate and optimize as many publishing decisions as possible. Sophi, which was developed in-house at The Globe and Mail and has since been adopted by more than 50 outlets across 11 different publishers, helps curate homepages, choose which articles to post on Facebook, and more.

One of the most crucial decisions Sophi makes is determining when — or if — to present a paywall.

[Read: After years of testing, The Wall Street Journal has built a paywall that bends to the individual reader]

Verma walked through a couple of examples. A reader who reads mostly general news and recipes might be less likely to subscribe than one reading a lot of business-related content. Still, Sophi might present this general news reader with a paywall. If they don’t reach for their wallet, the model wouldn’t hit them with the same message again and again. Instead, Sophi might pivot, and try asking the reader to register with an email instead.

This approach is possible because The Globe and Mail has no evenly applied article limit. Some readers might never encounter a paywall (Verma mentioned a hypothetical visitor who primarily reads car reviews — a strong source of ad revenue) while others might see one every time they visit the site (depending on their device, location, and propensity to read, say, articles about investing). In other words, Sophi uses analytics to make decisions that balance the potential for ad revenue vs. the potential for subscriber revenue.

“It knows when to give up,” Verma said. “It knows when not to bug someone because they’re not going to get money from them anyway.”

Sophi has been successful, according to the numbers that Verma presented. The Globe and Mail has seen email registrations more than double and subscription conversion increase by 51%. In April, company executives credited Sophi with helping the news organization reach 170,000 digital-only subscribers. Verma said the model also increased return visits and engagement among registered users.

Sometimes, though, the technology just needs to get out of the newsroom’s way. Verma said that editors can override the algorithm at any time — no questions asked. When wildfires in Western Canada were dominating the home page, for example, the newsroom chose to put evacuation information in front of any paywall and also removed ads to make sure nothing slowed down load times.

For Covid-19 content, Sophi has learned to distinguish between different types of coverage. An article about when vaccines will be available in your province? Free for everyone. An article with advice on which vaccine makers you should invest in? Paywalled.

Verma was open about the mistakes The Globe and Mail has made along the way, including a hard paywall attempt back in 1998 and a messy experiment with meter indexes (in which particular stories count as half a story or two stories toward a paywall count). Another mistake? Relying on intuition rather than data — or what Verma described as deferring to “the highest-paid person in the room.”

Data scientists and data engineers now make up roughly 10% of The Globe and Mail’s staff. It’s an investment Verma said has made all the difference. “Each data scientist we have hired has paid for the next one,” she noted.

ONA conference lasts through June 25 and sessions — including Verma’s — are being recorded. You can view the schedule or register here.