Second-guessing a play call is one of the great joys of being a football fan. Because you know — sitting there on the couch, never having played a single minute of professional football — you just know that you could run an offense made to score points. And yet, lacking the wisdom your keen football mind would provide, your team is facing the reality of an overtime tie with the Packers.

Fortunately for you, @SuperFan99, The New York Times, of all people, has your back. Meet the 4th Down Bot.

The 4th Down Bot performs realtime analysis of NFL teams’ fourth-down plays and determines if the right call was made. “Right,” in this case, is subjective. Judging by the ball’s location, the distance required for a first down, and the time remaining, the bot examines historical data to figure out whether punting, kicking a field goal, or going for the first down makes the most sense.

Here’s a scenario from the Saints/Seahawks game in Week 13: With 2:14 remaining in the first half, the Saints had 4th and 2 on their own 30-yard line. Coach Sean Payton chose to trust the rocket leg of punter Thomas Morstead and kick it away.

The Bot disagreed and said he should’ve gone for it. Why? Teams that go for it in that situation win about one percent more often than those who punt. Teams that go for it on fourth down in that position get the first 60 percent of the time, according to the Bot’s data, and possessions are valuable!

It should be pointed out that the Saints lost. The Bot does not gloat, but when it disagrees with a call, like the rest of us, it takes to Twitter:

The #Saints had a 4th and 2 on their own 30. They punted. I would have gone for it. http://t.co/2M4q5gSkek

— NYT 4th Down Bot (@NYT4thDownBot) December 3, 2013

The bot is a collaboration between the Times and Brian Burke of Advanced NFL Stats, who originally built the code behind the bot for a 4th Down Calculator tool on his site. How exactly does it work? Here’s an explanation from Burke and Kevin Quealy, a graphics editor at the Times:

NYT 4th Down Bot uses thousands of N.F.L. plays since 2000 to calculate the average number of points each situation is worth, a measure called expected points. (Expected points and its application to fourth downs is not new. It was created in 1971 by former N.F.L. quarterback Virgil Carter and Robert E. Machol and has been improved and refined in various ways since, notably with the book “The Hidden Game of Football” and David H. Romer’s signature 2002 paper. NYT 4th Down Bot’s model is similar to Mr. Romer’s, but has more seasons of data behind it.)

With about 10 minutes left in the fourth quarter, NYT 4th Down Bot switches from maximizing points to maximizing win percentage. Win percentage measures how often teams who punted, attempted a field goal or went for a first down won the game.

But the formula is only part of what makes the bot work. In order for the analysis to work in real time, the Times is pulling in live play-by-play data from SportsDirect. Plugging that into an API of Burke’s historical fourth-down data, the Bot assesses whether the smart money is on punting, kicking a field goal, or going for it. The system is not without its problems. Sometimes the time and yardage scenario is too perplexing for the bot. (As with relationships on Facebook, sometimes the best you can say is: “It’s complicated.”)

The bot will run through the rest of the NFL season, starting weekly with Thursday games (it dinged the Texans for cowardice last night) running through Monday Night Football. At the moment, the system is semi-automated, with a healthy dose of human oversight. Quealy told me they try to verify the down and distance data for plays in order to avoid errors. They don’t want the bot to suggest going for it on 4th and 7 when your team actually gained 20 yards and a first down on the previous play. (That’s easier when there’s only one game on at a time, as there has been since the bot’s debut. Sunday will change that.) When the system is fully automated, it will run analysis and tweet by itself. As for the look of the bot, which progressed from ASCII to 3D, that’s the work of Shan Carter and Jennifer Daniel.

Burke’s statistical model supports an idea known in the academic literature for more than a decade: Coaches keep it conservative. They’re far more likely to punt than the model says they should. Burke, who has contributed writing and analysis to the Times’ sports section for several years, said the data backs up many fans’ instinct that coaches are too often wedded to old conventions. Despite the fact that today’s players are bigger and quicker, and today’s playbook is more expansive, coaches keep it simple. “What I think coaches are doing is overthinking the harm that not converting will bring to their team and undervaluing what a successful conversion can bring,” Burke said.

(For example, Sean Payton, the Saints’ coach, is considered a risk taker in the profession, having made the stunning decision to try a surprise onside kick in the Super Bowl. But the bot thinks even Payton should have gone for it on 23 occasions when he chose to punt so far this season. On average, the bot argues there should be around two fewer punts per team each week than there are.)

Traditionally, Burke’s type of analysis might have been turned into a feature story paired with a set of graphics. That’s usually where someone like Quealy comes in. But Quealy told me they wanted to find a more practical, active use for the analysis. “Something like this seems like a cool idea and something that, like other infographics, we publish and it’s done,” he said.

The Times is trying to strengthen its position in the business of data and analysis. After losing out on the bidding war for Nate Silver this past summer, the paper recently announced it was launching a new in-house startup “at the nexus of data and news.”

Quealy said the 4th Down Bot isn’t part of any formal larger strategy — it’s more of an experiment. Still, the ethos behind it seems to fit with the direction the Times wants to move towards; Quealy said the goal of the Bot is to “make something that is inherently useful.” Burke added, “Information technology has let us do the analytics on your laptop that you used to need a timeshare on a university super computer to do.”

While the output of the 4th Down Bot lives on NYTimes.com, it’s real use could shine through on Twitter, where it can spread quickly along with other social-heavy gameday updates like scores, snark, injury reports, and fantasy updates. Burke said he thinks the bot can catch on because of the intense interest that surrounds the NFL and the abundance of ways to experience games. Having near-instant fourth down analysis fits in well with a world that has become accustomed to NFL RedZone and constantly updated fantasy football stats.

Both Burke and Quealy know the bot is bound to cause a fair amount of debate over its calls. In a way, the 4th Down Bot shares rhetorical DNA with sports radio hosts or columnists who delight in second guessing a coach. Of course, the bot’s relying on data while the other two are often relying on their “gut.” Still, Burke said the thing to keep in mind is that the bot’s analysis is not definitive in any way. “The bot and the model are starting points for analysis,” he said. “It’s not meant to be the final arbiter of coaching decisions.” We’ll see if that line of argument holds up when some Times-reading NFL owner starts hearing about how many times his man on the sideline has screwed up.