The NFL's Math Doesn't Add Up

When it comes to making money, the National Football League is as analytical and math-savvy as a Wall Street investment bank—crunching the numbers, running the models, thinking long, and always maximizing profit while minimizing costs.

When it comes to its on-field rules, however, the same NFL can be as gut-trusting and risk-adverse as a typical head coach: well aware that the math says to go for it on fourth down, but not about to take a shot because, dagnumbit, punting just feels right.

The latest case in point? Last month, the league approved a one-season change that moves the line of scrimmage for extra points back to the 15-yard line, resulting in a 33-yard point after attempt.

The move is intended to make games more exciting, with longer PATs producing more misses, blocks and opportunities for defenses to recover said blocks and score one-point safeties.

Problem is, the NFL doesn't seem to have considered the underlying numbers: As Benjamin Morris of FiveThirtyEight.com has pointed out, though, kickers made 96.7 percent of 33-yard field goals in 2014—barely any less "automatic" than the current 99-plus percent PAT rate. Moreover, Morris' model indicates the success rate of 33-yard field goals taken from the center of the field—where PATs take place—should rise to 98 percent over the decade.

In other words, the league is changing a venerable on-field rule in order to juice the sport and keep viewers from racing to the fridge during PATs, even though the available data suggests that the rule change in question won't actually work, at least not in a statistically significant way.

Turns out this is a pattern.

Too often, the NFL's competition committee has eschewed readily-available football data, instead relying on Good Old-Fashioned Football Man Horse Sense. It's an approach that repeatedly produces unsatisfying half-measures, failing to solve existing problems while introducing new ones.

Go back to 2009. After the NFC Championship Game ended with Brett Favre watching helplessly from the sidelines, the competition committee decided to ensure overtime didn't give such a heavy advantage to the coin-flip winner, or allow 60 minutes of neck-and-neck football to be decided by a couple of special-teams plays.

The solution they came up with didn't fix the longstanding issues with overtime, as outlined by Advanced Football Analytics' Brian Burke, but did introduce a raft of complexities and what-if scenarios that broadcasters now have to explain with bullet points and on-screen text.

Before the NFL made changes, per Burke's analysis, the coin flip winner went on to win the game about 60 percent of the time. The complex set of overtime changes the NFL adopted addressed the specific scenario—the two-first-downs-and-personal-foul game winning drive—without fixing the problem. Per Burke's calculations, overtime coin-flip winners still hold a 56 percent likelihood of winning.

If the competition committee had consulted Burke's Win Probability model, it would have known that the coin-flip advantage could be nullified by simply starting overtime's opening possession on the 20-yard line, a more elegant and effective fix.

A similar thing happened in 2011, when the competition committee accepted a proposal to move kickoffs up to the 35-yard line to cut down on high-velocity impacts—but vetoed a provision that would move touchbacks out to the opposing 25. The result was painfully predictable: Per Scott Kaczmar of Football Outsiders, the league-wide touchback rate quadrupled from 11.3 percent to 45.5 percent.

Touchbacks: Feel the excitement!

"The [NFL has] at their disposal any number of stats," says Burke. This much is true. Teams pay well for insights from the Elias Sports Bureau, Stats, Inc. and others. The success rate of PATs from the two-yard line, and field goals from various distances, are easily looked up. In fact, when the competition committee makes or considers changes, it often cites historical numbers.

So what gives?

"I don't think they take that next step," Burke says, referring to building and using statistical models to understand probabilities and outcomes.

Here's what some basic modeling says about the recent PAT change: while it isn't going to produce a rash of extra misses and blocks, it does figure to eliminate the only traditionally exciting outcome of a PAT try. A fake.

Maybe only the unconventional Oregon Ducks would run a speed-option with a holder and kicker, but not even Philadelphia Eagles head coach Chip Kelly would try it from 15 yards out. Per NFL head of officiating Dean Blandino, the old Doug Flutie drop-kick can't be attempted from the two-point conversion formation either—meaning as soon as a team lines up, you know what's going to happen.

Weirder still, the supposedly concussion-obsessed NFL may have made the game a little less brain-safe. How so? "I feel bad for all my linemen," Buffalo Bills kicker Dan Carpenter told SiriusXM radio. "Being on field goal protection is probably the worst job in football. I know that and all my linemen know that." With a much more realistic chance at blocking the kick, plus the possibility of scoring, kick-block units will suddenly be going all-out on every PAT.

"For a sport that was trying to cut back on collisions," Carpenter said, "you're probably just going to add a few more on those situations."

If you're wondering how the NFL could discount mathematics to detrimental effect, the answer may lie in a separate classroom: psychology. Baltimore Ravens guard and Penn State mathematics research adjunct John Urschel wrote at The Players' Tribune that football coaches tend to be conservative, reluctant to change tried-and-seemingly-true strategies even when the numbers suggest otherwise.

New England Patriots head coach Bill Belichick has never left an advantage unexplored, and few coaches play the percentages—the real percentages, as backed up by data—as well as he does. In an infamous 2009 game against the Indianapolis Colts, he chose to go for it on a late-game 4th-and-2 in his own territory. As Burke wrote for The New York Times, math modeling indicated that his choice maximized the Patriots' chance of winning—and that it's "pretty hard to come up with" numbers that would favor punting.

Yet Belichick was excoriated for his "arrogance," and widely blamed for losing the game. If there's a coach on Earth who's earned the right to make decisions like these without question, it's the four-time Super Bowl winner. Yet for making an optimally aggressive call, millions of armchair quarterbacks criticized him.

Lesser coaches, in different circumstances, could have been fired. NFL head coaches are, above all else, desperate to hold on to one of just 32 available positions in their industry. Similarly, league owners are desperate to keep their mind-bogglingly profitable cash cow mooing.

The result is a competition committee that consistently errs on the side of perceived caution, even if that also means erring on the side of, well, error.

Aaron Schatz, editor-in-chief of the stats-heavy football analysis site Football Outsiders, says he doesn't think the owners on the committee are nearly as forward-thinking as their executives—many of whom are quickly warming up to the advantages of statistical analysis.

"It's generally in the nature of the NFL," Schatz said, "to try and make small rule changes, not large ones, because they don't want to overcorrect."

Maybe so. But as the new PAT rules illustrate, the league's conservative, minimalist, let's-try-this-and-see-what-happens approach can be willfully blinkered. When the NFL messes with its own rules, it should be eager to consider every possible angle, and every possible impact. It should be clamoring for the services of statistical modelers like Burke and Schatz, who can analyze probabilities and find optimal outcomes. Instead, the league is making a half-baked change to its core product.

"They basically were like, 'Hey, let's make a slightly longer field goal!'," Schatz says, which doesn't solve a problem, and likely will foster new ones.

Like NFL commissioner Roger Goodell, you could say that the league's approach to math has become so predictable, it's practically automatic.