When the Bureau of Labor Statistics released its jobs report at the beginning of the month, news organizations unleashed their push notifications.

On Friday morning, the Wall Street Journal tested live mobile push alerts for their jobs coverage, working closely with the Guardian Mobile Innovation Lab, which has been for the past year tirelessly testing a range of ideas for distributing news that make the most of people’s phone-reading preferences.

Readers who arrived at the Journal’s mobile site or its Android or iOS apps were able to read its live coverage of the jobs numbers for July — but were also alerted with preview push notifications on updates as they read the existing analysis on the page (readers could dismiss and keep reading, or jump to the update from the push alert).

Journal developers built the infrastructure for the live notifications, and its markets team reported on the event and sent the pushes. The Mobile Innovation Lab provided guidance — based on learnings from its own past experiments and user testing — throughout the process, from evaluating design prototypes for the alerts to crafting an effective survey for users who encountered the Journal’s experiment.

The Journal has its own internal live coverage tool, built ahead of the Iowa Caucuses coverage in time for last year’s elections, but hadn’t dealt with live push notifications, according to Jennifer Hicks, deputy managing editor of digital at the Journal.

“We had a highlights feature where we could pin key posts, but we couldn’t notify readers within the live reading experience,” she said.

The Guardian’s Mobile Innovation Lab had been hosting some get-togethers and roundtables with various news organizations after the November 2016 election around news notifications, and the Journal expressed interest in trying out an experiment with the Lab. Work on this project started in June.

“There were lots of experiments the Guardian group was doing, so we talked about what we could bite off and pull off in a short amount of time,” Hicks said. “For us, it was also an opportunity to change our culture and talk directly to readers about testing a new feature.” (The Journal and Mobile Lab teams had a joint Slack channel going morning of the live notifications project for potential troubleshooting in implementation.)

The Journal plans to use the live notifications feature in future live coverage (with tweaks as necessary), according to Journal mobile editor Phil Izzo: “From jobs reports to the Olympics to terrorist attacks, we use live coverage a lot, and that’s one of the reasons we really wanted to build this out, since we knew there were so many use cases for it,” he said.

Both the Guardian and Journal teams emphasized the project’s experimental nature; it’s the first partnership of this kind for both organizations. The Lab is welcoming similar partnerships with other interested outlets.

“In the Lab we’re working for the industry and not just for ourselves — if we were to experiment in silence for two years and not share tips and tricks that we’ve experimented with, that wouldn’t be fulfilling the mission of the Lab,” Sarah Schmalbach, the Lab’s senior product manager, said. “We have been flexing our notifications muscle, then when we felt more confident in what we’d learned, we began to host events to ask other organizations what they were doing, where we’d then make a point to say, please come talk to us if there’s anything we can do to help, any data we can provide. Maybe we can launch something together.”

“We really relied on Sarah and [Mobile Innovation Lab editor] Sasha Koren to provide expertise in terms of, how do you talk to your audience directly, how do you conduct a real-time experiment, how do you offer a survey to audiences that gets you useful and actionable feedback,” Hicks said. “We had a lot of guidance on how to set up an experiment, which is not something we’ve done regularly at the Journal.”

Data points the Journal will evaluate for this jobs report experiment center around engagement, and include time spent on the live coverage, whether readers dismissed the notifications or clicked into the post, and bounce rate during the live event.

“Another thing we’re thinking about is, does this tell us anything about experimentation at the Journal?” Izzo said. “Did we make the job reports live blog better, because we put more attention to it, and should we push to do more things like this in the newsroom in general?”