We Discovered the Best Way to Judge a QB in the Postseason

The discourse around how many judge quarterbacks, specifically in the postseason, tends to be extreme. Some believe that if a quarterback’s team wins then it is automatically because of him and that he is somehow a winner. Meanwhile, others believe that the quarterback is replaceable and that other quarterbacks could have had the same team postseason success if put in another one’s situation.

This article aims to show whether quarterbacks in the postseason had any impact on their team’s success. Specifically, it is meant to show how much credit we should be giving a quarterback for his team’s success in individual games. The goal is to find a nuanced approach towards crediting a quarterback by judging them on whether they underperformed relative to their circumstances, met the expectations of an average quarterback, or exceeded the expectations of what an average quarterback would have done given the situation.

The way this article determines is by the simple approach of analyzing how many the quarterback’s defenses gave up in each postseason game. This approach originated from a Bleacher Report article from many years ago that analyze what the average quarterback’s win probability is based on how many points the defense gives up in a game and compares it with the specific quarterback’s team scored to see if the quarterback underperformed, performed at an average win-loss level, or exceeded the average.

The Bleacher Report article referenced a Pro-Football Reference article that not only was the source of the original calculations but also had the average quarterback’s win probability based on the points the defense gives up. The numbers are below.

A quarterback should win 93.5% of games when the defense allows 0-10 points.

A quarterback should win 76.1% of games when the defense allows 11-15 points.

A quarterback should win 52.6% of games when the defense allows 16-20 points.

A quarterback should win 38.8% of games when the defense allows 21-25 points.

A quarterback should win 17.7% of games when the defense allows 26-33 points.

A quarterback should win 3.6% of games when the defense allows 34+ points.

The calculation basically works by taking the sample size of games a quarterback started and categorizing those games in segments based on the bracket of points their defense allowed. An example would be Tony Romo having 2 games in which his defense gave up 21-25 points in the postseason. One would take the 38.8% (the win percentage the average quarterback would have under those circumstances) and multiply it by 2 (the game sample) to get .776. Afterwards one would subtract .776 from the sample size of 2 to get 1.2224. This means the average quarterback would be .776 – 1.224 (.78 – 1.22 if you want to round it) compared to Tony Romo’s record of 0-2 in that spot.

For this article, one would do this same calculation for other games the specific quarterback played in the postseason in the other brackets of defensive points allowed (0-10, 11-15, 16-20, 26-33, and 34+). Once that is done, the subsequent total numbers for the average quarterback wins and losses are added up. The average quarterback’s win is subtracted from the specific quarterback’s wins in the postseason and the same is done for losses. The differential in wins will tell us if the quarterback technically exceeded, only met the average expectations, or underperformed.

This is important because it prevents quarterbacks’ impact on winning from just being about over praising one for winning in games where the defense is dominant and over criticizing one for losing games where the defense was terrible. However, it does create a standard for giving quarterbacks credit if they are able to perform over what the average quarterback would have done such as a quarterback winning games where their defense gave up 26-33 points at a higher level than the average quarterback would have done.

Like with the original Bleacher Report article, this article will acknowledge that determining a quarterback’s impact on winning based on how it compares to the defensive support doesn’t tell the whole story in a vacuum. It does not account for the overall offensive support, coaching, special teams, injuries, and for how a quarterback could lose a game because his teammates made mistakes late or the defense failed to get a stop late. Also, the amount of points a defense gives up could be skewed by the offense turning it over or by the defense giving up garbage points. The amount of points a team scored might be skewed by a defensive score, special team’s score, or even a big run by a running back rather than it coming from the quarterback leading all of those scoring drives.

However, using how much a quarterback wins when compared to the defensive support over a large sample size still speaks to the general sense of how the quarterback is impacting the team’s ability to win. Relating it to the defensive support merely helps to show us what the average floor for wins would have been for that team. An example is how team B would have gone 9-7 with an average team (a result of how the average quarterback would perform based on the points the defense was allowing in the sample of games) but the quarterback they had was a +4 which led to the team finishing 13-3. Another example is how team A would have finished with a record of 6-10 but the quarterback was a +1 so the team finished 7-9.

If one follows sports discourse, they will hear pundits and some fans make arguments to discredit a specific quarterback’s contribution to his team’s postseason run(s) by saying “any QB could have won with that group” or “he is great but other top quarterbacks would have one with that team that season”. On the flip side, some would try to apply a lot of credit to that quarterback’s contributions by saying “it was one of the best QB postseasons runs” or “you’re not giving him enough credit for what he did. If most other quarterbacks were in that spot, they would have lost the divisional round game and not even make the Super Bowl”. This method is meant to see how many quarterbacks have been in the postseason in regards to their team’s win-loss record relative to the points the defense gave up as a way to show how much credit they should get based on what the average QB would have done in that spot.

Below is a table of many notable quarterbacks over the last 15 years that have played at least 6 postseason games. The original Bleacher Report article came out in 2009 so this one is able to show the updated performances in the postseason of many quarterbacks like Tom Brady, Peyton Manning, Drew Brees, Aaron Rodgers, and more. It will also show some of performances of recent quarterbacks such as Patrick Mahomes, Dak Prescott, and Josh Allen. However, it must be noted that the recent quarterbacks have not finished their careers so things could change when a larger sample size emerges in the following years. The projected record (what the average quarterback would have done) is rounded to the nearest tenth while the differential is rounded to the nearest hundredth.

Takeaways

Tom Brady, who is widely considered the greatest quarterback of all time, has the largest differential by a wide margin in terms of the wins his teams had with him starting compared to the projected win-loss record that the average quarterback would have had in those games based on the defensive support (points allowed). This holds up when comparing Brady to other notable quarterbacks in the past that were not on this table. Like with a lot of the quarterbacks that play a lot in the postseason, the average quarterback that played in the playoff games Brady started in would have had a losing record (22-26). One of the statistics that stuck out to explain this result with the context of the postseason win differential is how Tom Brady was 5-1 in games where his defense gave up 21-25 points, 10-3 in games where his defense gave up 16-20 points, and at least 8-7 when the defense gave up 26-33 points. His 8-7 record when defenses give up 26-33 points is the highest win percentage of all of the other quarterbacks in this table. Some quarterbacks were close in win percentage in that category, albeit in a smaller sample size, such as Flacco and Stafford finishing with a 1-1 record. Brees was 3-4 while Rodgers was 1-2.

This table can also help to explain why Mahomes is considered to be the one that has the best chance to catch Brady as the GOAT in the future. He has the second highest differential in the postseason so far. Obviously, he is still a relatively young player and time will tell if he can continue to lead teams to the postseason and win at a level above what the average quarterback would do in his circumstances moving forward in order to catch Brady in this differential gap (and subsequent Super Bowl ring gap). Mahomes is undefeated when the defense gives up 0-10 points, 11-15 points, 16-20 points, and 21-25 points. He is only 1-2 when the defense gives up 26-33 points but is impressively 2-1 when the defense gives up 34+ points in a postseason game. None of the quarterbacks have a win percentage as good as that when the defense gives up 34+ points.

Another interesting takeaway stems from Ben Roethlisberger’s postseason record differential. It is the third highest in general. He was undefeated in games where his defense up 0-10 points, 11-15 points, and 16-20 points. He was specifically without blemish in his 7-0 record when the defense gave up 16-20 points and was at least 3-1 when the defense gave up 21-25 compared to the win percentage of 38 that an average quarterback would have within that latter scoring bracket. One of the wins in the 21-25 points allowed bracket was the 2008 Super Bowl against the Cardinals in which he led a game winning drive by throwing a touchdown to wide receiver Santonio Holmes on a very difficult pass right after Holmes had dropped a touchdown pass from him on the play before. However, Big Ben was 0-4 in games where his defense gave up 26-33 points and 0-5 in games where the defense gave up 34+ points. His high differential seems to stem from how his teams basically won every postseason game when the defense played great or at least was okay. However, he was able to lead them to victory when the defense played bad or terrible (which happened more often in his later years) which contributes to why his point differential isn’t as high as Mahomes, let alone Brady.

Drew Brees and Aaron Rodgers performed roughly 3 games better than what an average quarterback would have performed in their playoff circumstances, respectively. Brees was 3-4 in games where his defense gave up 26-33 points and also 0-3 when they gave up 34+ points. When adding context, he had some tough breaks on close losses in which his offenses scored a lot. The only thing to critique is how he was only 1-1 when the defense gave up 16-20 points which is basically what the average quarterback would have done. He is 2-1 when the defense gives up 21-25 points which is fine because it is higher than the average quarterback but not as high a win percentage as Mahomes going 5-0 in such games.

Rodgers was 0-5 when the defense gave up 34+ points. Obviously, it is rare for even great quarterbacks to win under those circumstances. He is undefeated when 16-20 points is allowed which is good and is 4-1 when 21-25 points is allowed is very good compared to the averages. However, his record of only 2-1 when the defense allowed 11-15 is a missed opportunity to go undefeated in that category. The one loss was the shocking 13-10 loss to the 49ers in the 2021 NFC Divisional Round game. Average quarterbacks win a game in that scoring bracket 75% of the time. Rodgers was 1-3 in games where the defense gave up 26-33 points. The win percentage of 25 is higher than the average win percentage of 17 but not having a slightly higher above average win percentage is also a missed opportunity when compared to the win percentages of Brady in the defensive scoring bracket. Those key scoring brackets prevented Rodgers from having an even higher differential than he already had.

When analyzing Peyton Manning, his record of 1-3 when his defense gave up 34+ isn’t bad given the circumstance and is well above the average quarterback’s success in that spot. The one win was the 2006 AFC Championship matchup against the Patriots. His 1-2 record when the defense gives up 26-33 points isn’t bad since it is technically above the averages. However, the statistic that does not favor him is his 1-5 record when his defense gives up 21-25 points. This is a win percentage of 16% which is significantly lower than the 38% win that the average quarterback gets in those spots. He performs fine when the defense gives up lower than 21. The underperformance in the 21-25 points allowed range is a big factor in why his postseason record differential is barely over a game and is significantly lowered when compared to other contemporary great quarterbacks.

Despite the differences in the two players’ perceptions when it comes to being a winner, Russell Wilson and Matthew Stafford virtually have the same postseason record differential. This stems from how Russell Wilson is 0-5 when the defense allows 26-33 points and 0-1 when it is 34+ while only being 1-1 when 21-25 points are allowed. Wilson barely won his playoff games at a level above what an average quarterback would have done under his circumstances. When adding context, one could argue that Wilson’s would have won a game in the 26-33 defensive scoring bracket if his defense had stopped Matt Ryan late in the 2012 NFC Divisional Round game after Wilson led a comeback. Even though this is true, one of his victories in the 0-10 defensive scoring bracket was only the case because of an inexplicable missed field goal from short yardage by the Vikings kicker in the final seconds of the game. This occurred in the 2015 NFC Wild Card round matchup. As a result, both outcomes even out. This is part of why these findings are more useful with a large sample size of postseason games.

Meanwhile, Stafford managed to go 1-1 in games where his defense allowed 26-33 points which is noticeably higher than the average quarterback, albeit with a small sample size. However, his 0-2 record when the defense allows 21-25 points is no better but technically worse than the average quarterback. This is likely why he is also barely a game over the average quarterback in postseason record differential.

Joe Burrow has a reputation for being a big time playoff winner compared to Josh Allen but Josh Allen actually has a slightly higher postseason record differential than him. As shown in the table, the average quarterback in Burrow’s postseason situation would have had a record at or slightly above .500 while likely being 2 games below .500 in Allen’s postseason situation. Allen’s record could have been higher if things had gone differently defensively during the 13 seconds AFC Divisional Round loss to the Chiefs in the 2021 playoffs. Nonetheless, he is 1-2 when the defense gives up 26-33 points and 0-2 when they give up 34+ games.

Burrow is 3-0 in games where his defense gives up 16-20 points which is higher than what the average quarterback would have done. However, he is 1-2 in games where his defense gives up 21-25 points. As previously stated, the average quarterback wins those games at 38%. Burrow winning those games at roughly 33% is technically lower than what the average quarterback would have done. Had Burrow gone at least 2-1 in those spots (going to Chase earlier on 4th and 1 for a deep ball vs. the Rams in the Super Bowl or leading a game winning drive vs. the Chiefs in 2022), then his postseason record differential would be even higher.

Matt Ryan only has a marginally positive postseason record differential over the average quarterback. He was 0-2 in games where his defense gave up 34+. One of those losses was the Super Bowl loss in 2016 which was unfortunate. However, going 1-1 in games where his defense only gave up 11-15 points is underperforming compared to the average quarterback that would have won in those spots 75% of the time. That one loss was the 2017 NFC Divisional Round Playoffs vs. the Eagles.

Romo’s 0-2 record in games where his defense gives up 21-25 points is an underachievement compared to quarterback averages in those spots. One could defend one of those two losses (the 2006 Wild Card matchup vs. the Seahawks) by arguing that he should not have held the snap during the game winning field goal attempt since he was a starter during the field goal attempt. In the 2007 NFC Divisional Round game vs. the Giants, one of his receivers slowed down on a route that could have likely resulted in a game winning touchdown prior to the 4th down interception. However, going 0-2 is still underperforming so the argument could be made that he needed to find a way to properly hold the snap or convert on 4th down to win at least one of those games to be above average in those spots. Another option would have been to pull off an impressive and difficult victory in the two games he had where his defenses gave up 26-33 points and 34+ respectively. In one of those games, the 2009 Divisional Round game vs. the Vikings, his offense only scored 3 points.

Philip Rivers’ 1-3 record in games where the defense allows 21-25 points is below average compared to the win percentage of the average quarterback in that spot. He did not make up for it by having a win under more difficult circumstances as he was 0-1 when his defense gave up 26-33 points and 0-2 when the defense gave up 34+ points. When it comes to some of the other quarterbacks with negative differentials, one to take note of is Dak Prescott. He has not had good defensive support but Dak going 0-1 in the 16-20 points allowed bracket is an underperformance that hurts his record differential. Also, he did not underperform in the other higher scoring brackets but opportunities to improve his record differential would have been to go 2-0 instead of 1-1 in the 21-25 points allowed bracket or flip a win from the 26-33 and 34+ points allowed bracket. Ryan Tannehill underperforming in the postseason can be mostly attributed to his 0-2 record when the defense allows 16-20 points. This record is below what the average quarterback would have done in that spot. Underperforming in that scoring bracket is also what hurt Andy Dalton as well as not winning at least one difficult game in the 26-33 points allowed scoring bracket given the amount he had.

Of all of the quarterbacks on this list, Lamar Jackson sticks out a lot because he has the worst postseason record differential. He has significantly underperformed relative to the points his defenses are allowing in postseason games on average. The average quarterback in his overall postseason situation would have had a winning record compared to his 2-4 postseason record. He is 0-1 when his defense gives up 21-25 and 0-1 when they give up 26-33 points. Those are not impressive but would not necessarily be terrible given the small sample sizes as long as he was undefeated in games where the defense gave up 20 points or less. However, this isn’t the case as he is 0-2 when the defense gives up 16-20 points. The average quarterback wins those types of games at a 52% rate and would likely go 1-1 in that spot.

When comparing Jackson to the quarterbacks in this list and to the quarterbacks in the past from the Bleacher Report article, it can be argued that Lamar Jackson so far is one of the worst postseason quarterbacks ever in terms of how little he wins relative to the defensive support he is getting in the playoffs and the amount of games he has played. To be fair to Lamar, he along with some of the younger quarterbacks on this list still have more football to play. Lamar can still improve this with better postseason success relative to his defensive support but it remains to be seen if it will happen.

Other than Lamar Jackson and sort of Joe Burrow, the three quarterback Joe Flacco, Jimmy Garoppolo, and Eli Manning stick out because they are the only quarterbacks on this list in which the average quarterback is projected to have had a winning record in the postseason games played if they were to have been on the teams those two were on. This means that the three were mostly in very advantageous situations in their postseasons based on the points their defenses were allowing for the most part which is something many of the other quarterbacks on this list in the last 20 years did not have the luxury of having. These three quarterbacks were considered average at best throughout their careers.

Eli Manning was undefeated in any game that his defense gave up 20 or less which contributed to him at least having a positive postseason record differential by over 1 game. However, he was 0-3 when they gave up 21-25 points which is an underachievement. He never won a game when they ever gave up more than 20 points in the postseason. Flacco was 1-1 when his defenses gave up 16-20 which is not better than the average quarterback and ended up 0-2 when they gave up 21-25 points which is an underachievement. Ironically, his 1-1 record when they give up 26-33 points and 1-2 when they give up 34+ are impressive as they were both noticeably above the average. Garoppolo basically performed at what the average quarterback would have done.

Conclusion

The whole point of this article is to use a simple but telling method to assess how much credit we should give quarterbacks for their team’s postseason success. Football is a team sport and there are a lot of variables at play but we also know that the quarterback position can be the most impactful position on the field. However, we also know that defensive performances can sometimes impact the degree of difficulty for quarterbacks when it comes how they are when they are playing with a lead vs. trailing or playing on offensively driven teams with limited defensive talent vs. playing on defensive team with limited offensive talent that can impact what is expected of the quarterback on some games.

When trying to use this method to assess how great a quarterback is playing in the postseason, one can analyze whether they are performing any better than the average quarterback and by how much better compared to what other quarterbacks relative to those averages. When doing this, one can find interesting differences in the success rate based on the defensive scoring bracket to see if quarterbacks are having more or less success based on each bracket. Having large sample sizes over a quarterback’s career can also shed light if one’s success or failures holds up over a long time. If it doesn’t then one could look at further context themselves to see if other factors or variables played a role, or if the quarterback simply improved or declined in later years.

Some quarterback get too much credit for their team’s postseason success, some get too much blame for their postseason failures because they might be overperforming compared to what an average one would do, some great quarterbacks overperform but do not overperform as much as other ones do, other quarterbacks underachieve compared to average quarterbacks, and others are currently receiving the appropriate credit or blame. One can make inferences about the amount of credit these quarterbacks should be getting for their postseason success on the data provided and also give room for more further opportunities with the ones that are still playing.