Nieman Foundation at Harvard
With more than 2.5 million paid members, The Wall Street Journal is the biggest it has ever been. While we will continue to sharpen and evolve our member acquisition strategy, we believe that the next chapter of growth will be fueled by retention. Data clearly shows that the best way to reduce churn is to increase engagement — but the path to driving product use and building loyalty amongst members has not always been as obvious.
Over the past year, a cross-functional group here at the Journal has worked together to identify retention-driving actions and reinvent the way we promote those habits to our member base. We call it Project Habit.
We’ve known for some time that if a member downloads our mobile app or signs up for an email newsletter, they’re more likely to stay with the Journal. Project Habit began when someone said: “There must be more.” Members of our data analytics, optimization, and membership teams set out to confirm that assertion.
Our first step was to make an exhaustive list of all the things a member could do on our site. The list was quite long, including actions from Email Article and Play Puzzle to Build Watchlist and Comment. Next, we built a model to ingest all of these actions or “habits.” We decided to borrow techniques commonly used in medicine by applying the Kaplan-Meier estimator to member retention. Essentially, the model compares the “survival rate” of members who have taken a particular action against the rate of those who did not. It looked at the impact of performing these actions within the first 100 days of membership and how it affected retention in 30-day increments over the course of their first year of tenure.
Why did we only look at actions taken within the first 100 days? To try to eliminate the causation vs. correlation question. Focusing on our newest members ensures we’re looking at those who aren’t yet habituated. Any behavior they display marks the emergence of a new pattern rather than the presence of an existing one. The trends we observed held true for members of all tenures and pack types.
Our big finding: There are many more actions beyond app downloading and newsletter subscribing that drove retention amongst members. We saw, in fact, that there are dozens of habits that clearly reduce churn.
We plotted the actions based on how dramatically they affect retention and their current adoption rate amongst all members. Here are some of the top line trends we found:
Armed with the lessons of Project Habit, we were able to make data-driven and meaningful adjustments to our engagement strategy. We used the newfound data to diversify and optimize our existing communications, as well as champion a longstanding need for more prominent member messaging within our products.
A critical insight from Project Habit was that habits are formed early on. If you look at a member’s likelihood to adopt a new habit, that percentage is halved by Day 50 and plateaus soon after Day 100.
With this understanding, we knew it was critical to prioritize onboarding to capitalize on a member’s most impressionable time. We optimized and expanded our onsite onboarding experience to include several new habits. We also extended our email welcome series from 12 days to 100, adding touch points we knew to drive meaningful engagement. Today, 8 out of 10 new members take at least one retention-driving action during onboarding.
At the same time, we partnered with our product, design, and engineering (PDE) team to launch Project Canvas. We now had data to support exactly what we should be promoting to members, but a very limited canvas on which to do it. The Canvas initiative focused on creating more and better spaces to communicate with members. Historically, we had relied almost solely on email as a communication channel. While email remains an important part of many campaigns, the channel’s reach is, at best, limited to those who are opted into marketing comms. We knew it was imperative to build new and effective placements to promote our lengthy list of churn-busting habits.
At the Journal, we always look for the lowest-hanging fruit, and in this case, that was on WSJ.com. We grew the number of onsite member messaging placements from 1 to 5. We worked with PDE and the newsroom to ensure the messages looked like a cohesive part of the product and enhanced the member experience without compromising our journalistic or brand standards. Each placement has proven to be at least 2× — and in some cases 55× — more engaging than email.
Project Habit also helped us plan several in-life marketing campaigns around our stickiest content and product features. One campaign, Daily Dose, leveraged the trend revealed about cadence to program a member’s week by assigning a sticky action to each day of the week. In addition to owned media placements, we can now spend confidently in paid channels since we’re able to more precisely understand the value associated with a member action. We found that some of the rarely adopted actions quickly drew the most engagement amongst members. This proved that low adoption didn’t necessarily mean lack of interest, but lack of awareness.
Our data-driven strategy is starting to pay off. Over the past year, we’ve seen active days — our key engagement metric — steadily rise. While there’s no silver bullet to soundly eliminate churn, our quantifiable understanding of what actions drive retention gives us powerful ammunition in the fight against it. Coupling this insight with bigger, better space to communicate with members has allowed us to build an engagement strategy that drives better product usage and retention.
Anne Powell is associate director of engagement, John Wiley is associate director of data science and analytics, and Peter Gray is vice president of product optimization at The Wall Street Journal.
