Quarterbacks are Overvalued

NFL quarterbacks are being paid wild amounts of money. Seemingly every new quarterback that comes onto the free-agent market is paid a more exorbitant sum than the one that came before him. QBs have traditionally been paid handsomely, especially the good ones, but recently we have seen extreme payouts outside of the elite class of players who play this position. The likes of Jimmy Garroppolo, who had started only five career games prior to signing with the 49ers, and Kirk Cousins, whose career winning percentage coming into 2018 was below .500, signed deals worth just south of $30 million per year. Garoppolo’s contract was the richest in the history of the NFL, but it was quickly stripped of that title when Cousins signed his deal. Then, after signing for $30 million per year, Matt Ryan could briefly claim to be the richest until Aaron Rodgers signed a contract for $33.5 million. It is obvious from observation that your team won’t do well without a decent quarterback, but could Kirk Cousins really be worth this large of an investment? In other words, why is there such intense, persistent upward pressure on quarterback salaries, and is it justified?

Kevin Clark of The Ringer nicely summarized the quarterback craze in an August article, saying “Quarterback salaries are a succinct way to teach someone that life is not fair. There is no good reason why Eli Manning should make more money than Odell Beckham Jr., but he does because he’s a quarterback.” Passers are blatantly valued above any other position on the field. There are 15-20 different positions (depending on how you group them) per NFL roster but, on average, quarterbacks suck up about 10% of their team’s salary cap. ‘Team over individual’ is preached as dogma in football, but quarterback contracts suggest that in fact one person on a football team is much more important to winning games than anyone else.

Despite the common teaching of ‘team above everything’ in football, the fact that quarterbacks, especially good ones, command the highest salaries of any position makes intuitive sense to football fans. A perfect case study for this phenomenon is the 2018 Pittsburgh Steelers. Thanks to a season-long holdout, due, ironically, to the comparatively low salaries that running backs are paid, the team has played this season without running back Le’Veon Bell, an elite player at his position. Yet Bell’s absence has not wrought dire consequences upon the Steelers; they are, in fact, currently 6-2-1 and sit atop the AFC North standings. Second-string running back James Conner has fared more than adequately while filling in, sitting third on the league leaderboard for rushing yards through nine weeks with 771. It seems very unlikely that the Steelers would be doing similarly well if they were without starting quarterback Ben Roethlisberger, a comparatively worse player at his position than Bell, and had back-up Josh Dobbs at the helm.

This situation teaches two principles, both key to understanding why quarterbacks are paid so much. The first is that passing the ball is really important. In one sense, this is obvious: every year, teams with poor quarterback play do not win many games. Statistics shows us that passing is even more important than many fans probably think. Somewhat recent findings in the football analytics community prove that passing is exponentially more efficient, and correlates much more significantly with winning than running the ball. That more and more teams are catching on to this fact is a central reason for the sharp upward trend in QB salary.

The 2018 Steelers also teach us a lesson about basic economics: the scarcity of a resource is instrumental in determining its market price. The NFL labor market certainly adheres to this principle, and the simple fact that there is not an abundance of quality NFL quarterbacks is one of the main reasons why teams are willing to pay so much to get one. The Steelers had a backup running back who could step in and do close to the same job that Le’Veon Bell would have, but the same is almost certainly not true at quarterback. Although anecdotal, the perception that decent quarterbacks are a scarce resource is prevalent across the league, and has a lot to do with why the position is paid so highly.

Despite there being some logic to paying quarterbacks big money, it is still seems questionable to make Kirk Cousins or Jimmy Garoppolo the highest paid player in NFL history. To investigate whether these guys are worth the money they got, it would be useful to come up with a measure of positional value. This endeavour is tricky, though. Evaluating individual performance in team sports is difficult because luck is often a central determinant in the outcome of a play. In sports, we are all chiefly concerned with the outcomes of games and the standard ways in which we grade players is a product of that.  In baseball, a batter ripping a line-drive directly at the shortstop yields the exact same outcome as a pop up to the catcher. When reviewing the box score after the game it would appear that the first and second player fared the same in their respective at bats, but in reality, the first player performed much better.

Sports analysts, especially in the MLB and NBA, have moved away from outcome-based metrics, developing more and more advanced statistics which attempt to isolate the things players do that can be systematically explained. The NFL is beginning to move in this direction, but it faces a larger challenge for developing advanced stats compared to other sports. The issue of accounting for context is more extreme in football because it is the ultimate team sport. The outcome of any single play in a game is the result of many parts working together. When a quarterback completes a pass to his receiver, it could mean one of a hundred different things. Perhaps the QB made a really good pass, maybe the defender just missed getting a hand on the ball, maybe the quarterback’s left tackle made a great block to save a sack, maybe all of these things happened at once. Not all of these possibilities mean that the quarterback did well, but for each possibility it appears that he did in the box score.

The good people at Pro Football Focus have taken an exciting approach to this problem by developing a rigorous grading system which evaluates players on a play-by-play rubric. PFF studies every play of an NFL player’s season and decides how well he did on each one. This decision is not made based on the outcome of the play, but whether or not a player did his job on a given snap. If a running back is supposed to pass block on a play and picks up a blitzing middle-linebacker making a beeline for the quarterback, he is rewarded with a positive grade. Likewise, if he whiffs on the block, he is penalized regardless of whether or not the linebacker winds up getting a sack. These play-by-play grades are then double and triple-checked by other members of the PFF team, and a player’s season-long grade is a rolling average of how he performed on every snap. Grades are on a 1-100 scale and mean the same thing across positions, making for easy comparisons between players who do not play the same role on a team. PFF’s process provides a stringent evaluation of every NFL player and is the best attempt to extract individual performance in football to date.

We can leverage PFF grades to develop a simple model which takes offensive production to be a function of the average grade for a team at each offensive position. In other words, the success of a team’s offense can be simply explained by how well they grade at each offensive position. Running a regression using this framework will spit out weights for each position, which can serve as a rough measure of positional value. What is the best way to measure offensive production, though? This question seems straightforward at first, but it is not. Points or yards per game may come to mind immediately, but using these stats to measure offense is analogous to using touchdowns scored to measure how good a player is: neither accounts for context.

A better measure is a relatively new statistic developed by Football Outsiders called Defense-Adjusted Value Over Average (DVOA). DVOA accounts for the circumstances under which a given play was made. An offense picking up five yards on 3rd-and-10 is much different than doing so on 3rd-and-4, but both of these scenarios result in +5 to the offensive yards from scrimmage tally. DVOA weights five yard pick-ups on 3rd-and-10 and 3rd-and-4 differently, counting the latter as more significant. This process amounts to a measurement of efficiency, or in other words, how good a team is at achieving objectives throughout the game. By including context, DVOA provides a much more useful measure for determining how well a team has performed compared to simply counting up the yards or points a team has scored.

Using PFF data and team offensive DVOA numbers from 2018, we can gain a rough idea of positional value in NFL offenses. Regressing offensive DVOA on PFF grades for four position groups (quarterback, wide receiver/tight end, running back, and offensive line), and considering only players who played 100+ snaps through week 8 yields the following equation:

The number corresponding to each position should be interpreted as follows: an increase of 1 in PFF grade at the quarterback position implies a 0.6% increase in team ODVOA and so on. The regression has an R-squared value of 0.6, a decently high number which says PFF grades by offensive position explain a fair amount of the variation in ODVOA across the NFL.

The observed coefficients directly contradict the way NFL teams distribute their money across offensive positions:

We see that cap percentage by position and positional value have nearly the opposite order. One is tempted to draw the immediate conclusion that quarterbacks are wildly overpaid and the Vikings are morons for making Kirk Cousins the highest paid player in NFL history for a period of time, but we still don’t have the full story.

What about the issue of scarcity? To pass judgment on the amounts that teams pay players, we must look not only at the demand side of the market, which is determined by how valuable players are perceived to be, but also the supply side. Simply looking at the number of players with quality grades at a given position provides a rough idea of scarcity. QB, WR/TE, and OL, the three most highly paid offensive positions have the following number of players with a PFF grade over 70 through week 8:

The number of “quality” players is then divided by the total number of starters in the league at that position, in order to provide a measure of scarcity. What qualifies a player to be a starter is a tricky question, because the amount of players who get significant snaps varies significantly from team to team. For simplicity’s sake, I take teams to have one starting quarterback, three wide receiver/tight ends, and five offensive linemen.

The study presented is very simple. A much more nuanced project would be necessary to make specific claims about positional value and the scarcity of players at different positions. Still, this study is a valuable starting point. While rough, these results show that there is real reason to believe that the steep rise in quarterback salary could be a bubble. While the huge difference between the percentage of the salary cap pie that goes to the offensive line and the tiny one that running backs take is partially explained by the apparent scarcity at the two positions, there is an unjustified discrepancy between the amount passers and the people they pass to are paid. The positional value weight for receivers and tight ends is significantly higher than that of the quarterbacks, and the difference in how the two positions are paid is not explained by scarcity. Perhaps teams’ perceptions about the importance of quarterback play in the passing game are overblown. Mainstream thinking says receivers don’t matter if you don’t have a good quarterback, but the evidence certainly says otherwise. Contrarian thinking is critical to finding market inefficiencies, and it seems that there is good reason to keep exploring the relative importance of passers and pass catchers in NFL offenses.