Upworthy wants to be known as the company that shares a massive amount of socially conscious content across the web. What Upworthy is widely known for, however, is perfecting the art of the clickbait, curiosity gap headline. (On the front page now: “The Perfect Reply A Girl Can Give To The Question ‘What’s Your Favorite Position?'” “What If Everyone Who Reacted Negatively To A Super Bowl Ad Knew The Facts? They’d Learn This.” “A 16-Year-Old Explains Why Everything You Thought You Knew About Beauty May Be Wrong. With Math.”)

The company denies that they’re the main culprit, and some initial research by The Atlantic’s Robinson Meyer suggests that may be true. But now, they’re taking steps toward measuring their performance through something deeper than clicks.

This morning, Upworthy announced it would be using a new metric — “attention minutes” — as its primary tool to determine how they’re doing. The goal is to blend traditional eyeball-counting metrics with figures that more accurately measure engagement or how much the audience actually likes the content that they’re making. And they’ll be releasing their source code so other publishers can use it.

We love thinking this way because it rewards us for sharing content that people really enjoy and find valuable — not just stuff they click on a lot. It may mean that we don’t do quite as well on uniques or pageviews, but that’s a tradeoff we’re happy to make because this is a metric focused on real user satisfaction.

Upworthy says they decided to build attention minutes in the fall out of frustration with available analytics, either because they measure the wrong things or because they measured them poorly. For example, while Google Analytics’ time-on-site metric seems like it should provide something like what Upworthy was looking for, it was often misrepresentative, calculating at one point that the average Upworthy user spent 21 minutes on site. (“We’re good, but we know we’re not that good,” they write.)

Of course, Upworthy isn’t the only publisher feeling disatisfied with the mighty pageview and its many successors. Former Knight-Mozilla OpenNews fellow Brian Abelson, who now works at Enigma, spent part of his fellowship at The New York Times working on “pageviews above replacement,” a method of looking at basic traffic results without the confounding factor of internal promotion. James Robinson, director of news analytics at the Times, had this to say about metrics in his Nieman Lab prediction for news in 2014:

But for all types of newsrooms, developing a more qualitative understanding of audiences will be increasingly important. As publishers continue to invest in longform journalism and experiment with new types of storytelling, questions will naturally arise about the ways in which people actually read and engage with these pieces, beyond the feel-good tally of how many visitors they attract.

As Upworthy acknowledges, other media companies have also begun to make moves towards more meaningful metrics. “There are definitely others looking at how to improve metrics,” writes Upworthy CEO Eli Pariser in an email, citing YouTube’s “Time Watched,” Medium’s “Total Time Reading,” and Chartbeat’s “Average Engaged Time.” (Chartbeat responded to Upworthy’s announcement on their own blog, calling the metric “another big win.”)

“They all have a similar hunch,” Pariser said. “As we’ve been building it out it’s been nice to see additional traction on this front. It gives us a sense we’re not way out in left field.”

Attention minutes are calculated using an assemblage of what Upworthy calls “signals.” Pariser says they started out by asking, “What are the ways you can know if it’s a sad, lonely, ignored tab or if someone is watching intently?”

Some of these signals include the length of time a browser tab has been open, how long a video player has been running, and the movement of the mouse on screen. “This method doesn’t store raw information — it follows a stream of events and every few seconds asks, ‘Is this person still paying attention?'” Pariser said. “If they are, that sends a signal to a data warehouse that says, ‘Chalk up a few more attention seconds for this post.'” Together with other information, including pageviews, Upworthy hopes to get a sense of what content is most delighting and engrossing viewers, not just briefly capturing their attention.

This chart is meant to illustrate how the signals that make up attention minutes present a significantly different portrait of a story than pageviews do:

Upworthy plans to release the source code “in the coming months.” Pariser says Upworthy has already begun to gather valuable information about how people consume content — like that “the relationship between the length of a video and how much people watch isn’t what we though it would be,” he says. They’ll be publishing more of that information in 2014, too.

The overarching hope is that by improving the quality of your metrics, you can improve the quality of your work. Campbell’s Law puts it well, if academically:

The more any quantitative social indicator (or even some qualitative indicator) is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.

If you measure performance in pageviews, you encourage slideshows. If you measure performance by social shares, you encourage clickbait headlines and giant Like buttons. Finding a metric that lines up with a publisher’s goals is one of the most important things it can do to encourage better work, and Upworthy thinks this is a step closer to that alignment for their needs.

“The reason that we’re focused on this metric is not just because we think it’s good for us, but because we think it would serve everybody better if we moved away from some of the metrics that people have been using,” Pariser says. “Pageviews can be bought, but this…is a truer measure of whether you’re delivering something meaningful and valuable.”