We Discovered the Best Way to Judge a QB in the Regular Season Part 1

Recently, an article based on a quarterback’s postseason success was posted here. Some of the many points stated in that article will be stated here as well. Some people like to call any quarterback with a winning record “a winner” even if it is a small sample size of games or from a season-worth sample size. Others nitpick those they have biases against by stating that an average quarterback would have won the same amount of games. Some pundits and fans criticize some quarterbacks for their team’s losing record, even when the quarterback has a defense that consistently gives up a lot of points each season. Meanwhile, some always defend such a quarterback in that situation and make it sound like they couldn’t have done more or that they would be winning at the same level as a peer if they had that peer’s defensive support.

This article is meant to address this by looking at the points per game a quarterback’s team gives up in each game and categorizing each game in a season into brackets of points allowed (1-10, 11-15, 16-20, 21-25, 26-33, and 34+). An analysis was done to see the quarterbacks’ record in games relative to what the average quarterback’s win probability would have been. Doing this lets one know whether a quarterback’s win-loss record in a given season (and for a career as a whole) is significantly better, worse, or roughly the same as what the average quarterback’s record would have been under those circumstances.

The concept of taking such an approach originated from a Bleacher Report article years ago. The Bleacher Report article referenced a Pro-Football Reference article that not only was the source of the original calculations, but also had the average quarterback’s win probability based on the points the defense gives up. The numbers are below.

A quarterback should win 93.5% of games when the defense allows 0-10 points.

A quarterback should win 76.1% of games when the defense allows 11-15 points.

A quarterback should win 52.6% of games when the defense allows 16-20 points.

A quarterback should win 38.8% of games when the defense allows 21-25 points.

A quarterback should win 17.7% of games when the defense allows 26-33 points.

A quarterback should win 3.6% of games when the defense allows 34+ points.

The way this works is that when analyzing a quarterback for a season (ex. QB A for the 2020 season), you take the games he started and categorize them in scoring brackets. In the 21-25 point allowed group, QB A’s team had 3 games in which they gave up a score in that section. One multiplies an average quarterback’s win percentage of 38.8% with 3 (the number of games in that category) to get 1.164 (the win record).1.164 is subtracted from the number 3 (games played in the category) to get the loss record of 1.836. The average quarterback would have been 1.164-1.836. The same math is done for the other scoring bracket. The wins and losses are added up from the actual quarterback’s record (specifically, the winning record). The difference in QB A’s wins from the calculated wins the average quarterback would have gotten is the differential that determines whether the quarterback overperformed, underperformed, or met the average performances of what an average quarterback would have done.

The goal of this is to assess how much credit a quarterback should get for his team’s win-loss record. For example, not all 7-9 records are equal. A quarterback going 7-9 in his 16 starts performing 3 games above average is really good compared to one that went 7-9, but the average would have gone 7-9.

An important aspect to remember is that assessing a quarterback’s contributions does not tell the full story since there are many variables going on in each game. An example is how a quarterback could have led his team to a game-winning field goal, but the kicker missed it, so it cost the quarterback the opportunity to have a better record and a better record differential. Also, it doesn’t account for injuries that could have made it harder for a quarterback to help his team score more points if the injuries were on the offensive side of the ball. Furthermore, the quarterback could have led a go-ahead scoring drive, but his defense gives up points late.

Conversely, a team’s points allowed could be skewed by offensive turnovers such as a pick-6 or special teams. This is why this approach works over a large sample size, whether it is in the context of a 16-17 game season, a collection of multiple seasons, or a long career spanning over a decade. A quarterback may have a game where his defense gave up a late score in a shootout but also gets credited for a low scoring win when the defense makes a stop.

The longer the sample size, the better assessment one can make for determining a quarterback’s true value in wins he adds over a replacement level-average quarterback in those spots. The longer the quarterback plays, the more likely he will play in different circumstances and with different types of teams (defensive teams that keep the score low and offensive teams that have bad defenses). This is important because the charts below will show fluctuations in the record differentials of these quarterbacks, suggesting that their performance over a replacement level quarterback can be impacted by other supporting variables such as the evolution of the game offensively and defensively, opponents having tape on them, the player improving or declining, the sustainability of the quarterback’s style of play, their floor and ceiling, any changes in coaching, the quarterback’s own durability, the durability of his teammates, and especially the level of offensive help.

This article shows the record differential of many quarterbacks that were the primary starter for 4+ seasons in the 21st century. Based on the charts below relative to each quarterback’s actual performance and accolades (when accounting for context), the highest record differentials (over 4.5 games to 7+ games) tend to be seasons in which the quarterback had an MVP or all-pro caliber type of season. The exceptions were when a quarterback would have such a high number that wasn’t considered MVP/all-pro caliber but was instead a very good season that involved many clutch game winning drives and also a historic/offensive player of the year caliber performance from one of their skill position players (ex. 2022 Kirk Cousins and 2020 Ryan Tannehill). Usually, when a quarterback has an MVP season, they still have a very good offensive supporting situation when looking at their season beyond defensive support charts.

Generally speaking, the higher the record differential is, the better the quarterback is playing. Some exceptions are when a quarterback has a pretty solid record differential (2-4) but played only decently and instead had many game winning drives (or benefited from some in-game luck) in games where the defense gave up points in the amount that the average quarterback would have been less likely to do under those circumstances (2004 Ben Roethlisberger). Conversely, the closer to the average or even a below average record differential correlates with the quarterback playing average to below average. Some exceptions tend to be when the quarterback played well but was not clutch and or got some bad breaks in the games (ex., 2010 Aaron Rodgers and 2020 Deshaun Watson).

Once in a while, a quarterback might have a season where they led a lot of game-winning drives, such as having up to 4 but ended with a record differential under 1.5, which isn’t too impressive. In those cases, it meant that many of those game-winning drives helped the quarterback simply win games the average quarterback would have anyway, maybe even without needing a game-winning drive. Those game-winning drives don’t carry the same weight as a game drive that helped a quarterback win a game that an average quarterback would not have won and thus significantly increased their record differential.

An example is a quarterback leading a game-winning drive which now makes him 4-0 when the defense gives up 16–20 points compared to how an average quarterback would have been only 2-2 statistically in those circumstances. Conversely, a quarterback, once in a while, might have a year in which their record differential is a very high number, such as 6, but he only led 1 or 2 game-winning drives. In this case, he was likely consistently dominant while playing with leads and was not often put in spots to win a shootout late or to come back after a slow start in a rare up-and-down game. Both these cases with the game winning drives are not common as one will see in the charts for quarterbacks in this series.

Over a long period of time, the data for many quarterbacks show that a quarterback being able to have good record differentials is tied to them generally playing well relative to their circumstances.

Again, determining the true value a specific quarterback is giving or not giving in their team’s win totals over the average quarterback works when analyzing over a large sample size. One will see below that some quarterbacks had their record differential fluctuate or were steadfast but then noticeably declined or ascended during stretches of their careers. This means their ability to add more wins to a team, play at an average level, or even not meet the average can be helped or hurt by other variables, likely due to their offensive supporting casts. However, the extent of the quarterback’s overall impact (including the ceiling or floor) is still unique to each quarterback, which is ultimately shown by their record differentials, respectively.

Doing all of this is important because just looking at someone’s career regular season win-loss record does not really tell us how much of the winning is unique to this specific quarterback. We do not know how many wins were manageable for the average quarterback to have gotten under those same circumstances. We also don’t know if many of those losses by a quarterback with an unimpressive win-loss record were going to also be the case for some other “winning” quarterback had he faced similar circumstances in terms of the level of defensive support. As a result, this is meant to indicate how much value a quarterback is truly giving to a team over what a replacement quarterback would have given.

Essentially, this is meant to determine how much added wins a quarterback is giving to his team relative to the defensive support he is getting (and how it impacts the team’s record if an average quarterback is starting in the same situation). One interesting finding one will notice when looking at the charts is how there are a few seasons where a quarterback’s season statistics might not be as underwhelming as they look because their record differential actually speaks more favorably towards their contributions to the team. Conversely, a quarterback’s nice statistics might not be as great as they look as they might have included stat padding and or unclutch performances that their record differential could shed more light on.

When listing the record differential for these quarterbacks, the seasons that they did not play in (being on the bench, injured, etc.) are included because availability is factored into one’s ability to even have the opportunity to win for their team. Furthermore, the first few years that a quarterback spent sitting on the bench needs to be charted because the quarterback that is forced to play early, especially on a bad team, could potentially get punished when comparing the two if the quarterback doesn’t have a good record differential compared to the one that didn’t even play. We do not know how the quarterback sitting on the bench would have performed if forced to play earlier. News about offensive coordinator changes is stated when there is a defensive or special teams-minded head coach, not an offensive minded head coach, since the offensive head coach is likely running the same system even with changes to the offensive coordinators.

When reading the charts in this article and in the following ones in this series, there are few things to know:

• “gw” – game winning – the amount of game winning drives are only mentioned if is noticeably high (4+) or noticeably low(1 or 0).
• “OC” – offensive coordinator
• When there is information on a player retiring or leaving (via free agency, trade, cut, etc.) in a certain season, it means that they left at the end of the previous one.
• When information is listed about a player replacing a player at the same position, a coordinator replacing a coordinator, or a head coach replacing a head coach, it means it happened heading into the season. – if such a move was made during the season then it would stated as such.

The following are the record differentials for each season and for the careers for Tom Brady, Peyton Manning, Drew Brees, and Aaron Rodgers.

Tom Brady

Peyton Manning

Drew Brees

Aaron Rodgers

Takeaway

Tom Brady playing over 20 seasons makes it very impressive that he has won over 3 games above average over the duration of his career. It was rare that he even had a record differential under 2, and only once (his final season at age 45) did he even have a season that was basically considered average. He’s had some high peak seasons (5+ wins above average), but also a lot of regular great seasons (a little over 4 games above average). Brady was able to deal with an evolving offensive situation that features offensive coordinators coming and going after a few years as well as skill position players and those on the offensive lines.

Furthermore, he was able to provide these record differentials even in seasons where he didn’t have more than one standout player in offense, with other issues going on in that season such as in 2013. He was able to produce very good to great record differentials for the first 5 years as a full-time starter on offensive situations that ranged from mediocre to at least solid. In 2006, he had a really bad wide receiver situation after his top targets left in free agency during the offseason. Despite this, he at least basically had 2 victories above average that season. In some of these aforementioned seasons, among others, Brady would be able to maintain this type of productivity even when the sole or one of a few key players on the offense would be out for long stretches. The only time Brady was only around average to barely above average when given an offensive situation that bad or average at best was when he was in his forties (2019 and -2022). His one barely above average season that did not occur when he was in his forties was in 2009, in which he had a good offensive situation but was coming off an ACL injury.

Throughout his career, Brady sparingly had what would be considered strong run game support during the regular season. Corey Dillon in 2004 was a rare example of such help. Before Brady went to the Buccaneers, he did not get to play with elite receivers outside of slot receiver Wes Welker and a couple of seasons with Randy Moss. He has mostly had offensive line support that would be considered decent or solid other than a few bad seasons by the offensive line unit. He has had some offensive line players make at least one pro bowl or all-pro. Logan Mankins sticks out as being the one offensive lineman for Brady that made multiple pro bowls and all-pros during Brady’s time. 2007 was his best offensive line support for Brady as he had multiple pro bowlers/all-pro that one season. Brady was able to stay consistent even with changes in offensive coordinators over the years in New England and when he had a new offensive coaching system in Tampa Bay.

When it comes to defensive support, Brady had a hand full of season that was really good (the average quarterback would have 4 games above average), while the rest ranged from solid (the average quarterback goes roughly .500) to sub-mediocre/not that good (the average quarterback would have gone 7-9). He has never had defensive support that would be considered extremely bad (the average going 4-12 with the team). As a result, he never had to deal with having a season once in a while in which his record differential could have been as high as 4, but the 4 extra wins he contributed to over an average quarterback only amount to the team going 8-8 and potentially missing the playoffs because the average quarterback would have gone 4-12 with that team even if the team had a good offensive situation.

To Brady’s credit, averaging 3-4 wins above average (with potentially more peaks if the team is built more towards offense in this scenario) greatly increases a team’s chances of making the playoffs almost every year because adding that number of wins to a bad defensive situation (average quarterback going 6-10 with the team) will still produce a winning record (such as 9-7 or 10-6 in this case) that can often be enough to still get into the playoffs. It would take extremely bad and rare circumstances, such as a team somehow always going 4-12 or worse if an average quarterback was playing for every single season of one’s career, for such a record differential to potentially go in vain. Even if he had a handful of really bad defensive support seasons that would have made it difficult to make the playoffs those seasons, he still ended up playing over 20 seasons, so it means that he would still be in a position to make the playoffs the vast majority of the time due to his average record differential being impactful on the other seasons in which the defensive support was at least not really bad.

Also, Brady’s career record differential not only being as high as it is but, more importantly, it being the result of him having consistent record differentials such as being mostly between 3 and 4.5 rather than being extreme volatilities (multiple seasons with differentials at 1 or less to pair with a few more high peaks) basically means that he is able to add enough wins to get his teams to the playoffs basically every year. This is why he had basically two decades of making the playoffs every year as a full-time player after 2002, when he was able to take full advantage of not having an extremely bad defensive support.

Being able to produce three wins at least above average, even on teams that would have gone 7-9 with an average quarterback, usually results in enough wins to make the postseason. In Brady’s case, the years where he had his worst defensive support happened to years where he had some of his record differentials (5+ wins above average). This was likely due to some of those teams being more slanted toward offense (such as 2011). Having such record differentials further increases the likelihood of making the playoffs in those types of seasons. Many really good quarterbacks that average 1-2.5 games won above average would have been susceptible to missing the playoffs a number of times in the years where the defensive support was sub-mediocre or even just solid (the average quarterback being .500) because an extra win or two might not be enough in those seasons relative to the defensive support.

Brady’s ability to mostly maintain a pretty good record differential even when his offensive situations are less than ideal at times during his 20s and 30s made it easier to allocate resources towards creating a team with strong defensive support. Making a team with strong defensive support creates a relative high win floor, so if Brady adds a few more wins to what the average quarterback would have won without always needing a great offensive situation, then it basically means a team will be in a position to be a contender many times.

Peyton Manning has the highest regular-season career record differential of the modern era and amongst his peers. He averages a little over three and a half games won above average. During that time, Peyton has had some years with bad defensive support, some years with solid defensive support, and some years with really good defensive support. His first four years have some solid ones sandwiched between really bad ones (the average would have gone roughly 4–12 in 1998 and 2001). For the next decade, mostly with defensive head coach Dungy, he had defensive support that was solid (the average quarterback would have gone around .500) and even good at times, with the exception of a couple of years. In his four years with the Broncos, he had two bad defensive support seasons sandwiched between solid ones. Regardless of the level of defensive support, he never had a season averaging under a game and a half outside of his rookie season.

The one constant for Manning was that he had excellent and complete offensive support (stars on the offensive line and all skill position groups). In his time with the Colts, a lot of pro bowl/all-pro level offensive teammates were mostly durable, stayed there with him for a while, and were productive for a relatively long time. When looking at other quarterbacks’ career charts, one will notice that their own key offensive players generally did not have such longevity on the same team. He had the same offensive coordinator in Tom Moore for basically his whole duration with the Colts, which spanned over a decade, which is a luxury. Other than maybe his rookie season, he did not have what would be considered an even mediocre offensive supporting cast.

To Manning’s credit, his run with the Broncos was great in terms of record differential, especially in 2013 with it being over 7 games above average.  He did this without Tom Moore being there, and it was with a new offensive supporting situation. The offensive coordinators he had in Denver (McCoy and Gase) did not have much success without him and have not elevated other quarterbacks. His tight end Julius Thomas was not productive when leaving the Broncos. His great 2013 season occurred with his left tackle Clady not being available for the whole season. This shows that on some level, Manning is helping to elevate the talent and the coaching around him as much as they are helping him. Manning’s consistency with his style of play has played a big role in constantly having good regular season differentials (2+) and numerous high peak seasons which have led to playoff appearances almost every year he has been in the league.

Along with the other three quarterbacks on this page, Drew Brees has one of the highest career record differentials of the modern era for pro football. This stems from how he has consistently averaged close to 3 games above average for most of his career. This mostly happened with the Saints, but when he was with the Chargers, he barely won a game above average. Continued development and playing under offensively minded coach Sean Payton likely helped. However, an often overlooked variable that plays the most important role in Brees’ improvement in winning above average was in the amount of quality interior offensive linemen (centers and guards) he played with. Having excellent interior linemen to not only help step into throws as a pocket passer but to better overcome issues with seeing over the middle to height was very important for Brees. It isn’t a guarantee that a quarterback would have a team(s) for most of his career that would have success finding elite interior linemen for long stretches and be able to effectively replace them with other interior linemen of similar production.

During Brees’ time wtih the Saints, he has played with multiple pro bowl and all-pro level interior linemen and even tackles. Whenever some left, the Saints were able to find another player of a similar quality. As a result, Brees only had one season during his 15 years on the Saints where he finished with a record differential of under a game and a half. Additionally, within that timespan, he had a couple of peak seasons of record differentials that were 5+. The one season he had in the Chargers finishing over 2 games won above average was in 2004, in which the team overhauled the interior line and specifically signed productive guard Mike Goff. The interior line on the Saints always had at least one player that was not only good, but all-pro caliber. Brees has had defensive support that would alternate every 2-3 years between being solid (the average quarterback would have gone around .500) or noticeably bad (the average quarterback would 3 games under .500 and sometimes even as much as 5+ games under 500). He has almost always had a good offensive situation.

Rodgers has had a lot of high peak seasons (5+ games won above average), most of which led to MVPs. He’s also had many good record differential seasons (2-3+ wins above average). Part of why Rodger’s career differential average isn’t higher is because he’s had multiple seasons when he did not play even up to half the season. As a result, it limited opportunities for him to have more than one record differential. An example is how if a quarterback only played 4 games and was 4-0 but the average quarterback would have gone 3-1, then despite how well the quarterback performed, the highest his record differential could be is 1. Had the number of game been more, perhaps he could have had a better chance to widen the gap.

Other factors that impacted Rodgers’ record differential was in how he didn’t play his first three seasons. Not playing prevented him from having any positive record differentials. Those three seasons have to be part of the record differential calculations because other quarterbacks play early in their career with the chance to either have a positive record differential or a negative one, and it is still added. Not playing shouldn’t leave one off the hook. The Packers had Brett Favre flirting with retirement and struggling for multiple seasons in the mid 2000s, but it took until 2008 for them to finally move on from Favre and go with Rodgers. Did they feel as though Rodgers wasn’t ready to play in 2006 or 2007 since he was still working on changing his throwing motion? Had he played, what would his record differential have been like? Playing a year or two earlier would have given him the opportunity to not only add to his career record differential but to start on another contending team, especially in 2007. Rodgers not playing prevents him from adding winning value to his teams. Another interesting finding with Rodgers is that his record differentials basically ended up being average in his first three seasons as a full time starter (2008-2010) despite having solid offensive help.

Rodgers missed the playoffs in 2008 with his record differential being below what the average quarterback would have won that year, albeit not necessarily due to terrible play but a lack of many game-winning drives. 2010 and 2011 were seasons where he made the playoffs because his defensive support was strong enough that barely averaging a game above average those 2 seasons would be enough to get in. The first three seasons as a starter contrasts with the rest of his career, since he mostly averaged high record differentials.

Rodgers has mostly had defensive support that was solid in some seasons (the average quarterback being around .500 in his spot) except for couple of underwhelming defensive support seasons. Rodgers has a few seasons where he basically average after 2010 when his offensive situation wasn’t great, but what has separated him is how high and often he has peaked when the situation is excellent.

Click here to read part 2 for more quarterback’s record differentials.