How many games should your NFL team have won this season? Everyone knows a lucky bounce here and a bad call there can have a significant impact on the win-loss bottom line. Hard core fans of Sports Analytics would recognize this factor as the driver behind Pythagorean Win Totals, a statistic derived to measure true performance. Today, we are going to look to see if we can beat Pythagorean Win Totals as a predictor for how many games a team won in a certain season. IE, how many games should your team have won.
Spoiler: we can make a better predictor, but in a way that makes us re-evaluate our understanding of Pythagorean Win Totals.
If you simply want to know how many games your team should have won, you can go straight to our Spotfire Template. But, for Football Outsiders fans or those more interested in what makes up wins and losses, read on.
Pythagorean Win Totals
For those unfamiliar with the concept of Pythagorean Win Totals, don’t get too bogged down on the Pythagorean part. Just think of it as a formula for approximating true wins based solely on points for and against a team that was created for baseball by Bill James, then adapted for basketball by Daryl Morey, then finally adapted for football by Football Outsiders.
The Pythagorean Win Total (or expectation) represents how many games a team should have won without those lucky or bad bounces in close games. So, the logic follows that the Pythagorean total should be more predictive of future wins than actual wins. A team will play more like its Pythagorean Win total than real life total in future games and even the next year. A thirteen Pythagorean Win team with 10 real wins is more likely to win the Super Bowl than a team with those figures reversed.
With that in mind, I thought, let’s find a better way to evaluate how many games a team should have won, or what its true performance has been. If we more accurately predict how many games a team should have won, then we might have a better predictor for the future.
I will not delve too deep into the modeling process because I think the results are even more interesting than the process. But, I will say it was not difficult to improve quickly on Pythagorean Wins in terms of predicting how many games a team won.
To do so, I used seasons for all teams dating back to 2000, excluding this season. I split them into an 80/20 train/test set. I then used the team’s win percentage as a target variables and used Linear Regression (backwards selection). Based on various Adjusted R Squared thresholds, I created six, eight, and 14 variable models, with the following predictors:
Six Variable Model: Points For, Points Against, Passing Yards, Total Yards allowed, Passing Yards Allowed, Rushing Yards Per Attempt Allowed.
Eight Variable Model: Points For, Points Against, Net Yards Per Attempt, Passing Yards, Rushing Yards Per Attempt, Passing Yards Allowed, Rushing Yards Per Attempt Allowed, Total Yards allowed.
14 Variable Model: Points For, Points Against, Total Yards, Net Yards Per Attempt, Passing Yards, Total 1st Downs, Rushing Yards Per Attempt, Total Yards Allowed, Defensive Plays, Passing Yards Allowed, Rushing TDs allowed, Rushing Yards Per Attempt Allowed, Defensive Expected Points.
First, I want to acknowledge that several of these predictor variables are multicollinear, ie correlated, which causes model instability. For example, Passing Yards and Total Yards are both included. As this exploration is a proof of concept, I considered these models more like naïve models and did not fine tune to avoid multicollinearity. A subsequent post will come out addressing these considerations.
So, with that caveat, let’s look at how the models did predicting wins (or more precisely, winning percentage).
|RMSE||In Sample||Out of Sample|
|6 Variable LR Model||.07504||.07357|
|8 Variable LR Model||.07343||.07481|
|14 Variable LR Model||.07174||.07338|
We see that the Linear Regression Models outperformed the Pythagorean Expectation both In Sample (train set) and Out of Sample (test set) since a lower RMSE is better. We feel pretty comfortable saying that our models do a better job of stating how many games a team SHOULD HAVE won based on the peripheral stats.
That’s fantastic. Remember, the original premise is that since Pythagorean Expected Wins better measured true performance, thus the number was more predictive of wins in future seasons than a previous season’s wins and losses. So, since our models did a better job of predicting how many games a team won, will projecting those win percentages also do better at predicting next year than doing the same with the Pythagorean Expectation?
For each season, I compared our expected win percentage for this season with the actual wins for the season after. To be clear, I am not applying the model to the following season, since presumably that data is unknown. Instead, we will follow the same process as Pythagorean Win Projections and use our Win Expectations from Year 1 (stat peripherals known) for our Year 2 projections. Basically, is the Win percentage from our model for Year 1 going to be closer to the actual Year 2 results than the Pythagorean Expectation?
|RMSE||Year 1 Test Set||Year 2|
|6 Variable LR Model||.07357||.21489|
|8 Variable LR Model||.07481||.21587|
|14 Variable LR Model||.07338||.21741|
The short answer is no. The Year 1 Test Set figures are the Out of Sample results from the previous table. Again, the regression models more accurately predicted the win percentage in the test set than Pythagorean Expectation. However, when we project those percentages as a proxy for the next season’s percentage, Pythagorean Expectation overtakes our models.
Why did this happen? First, the multicollinearity discussed likely led to some model instability. Second, the Pythagorean Model is a generalized model and was not “trained” on our training data. Therefore, it likely generalized to a different season better while the Regression models might have been prone to overfitting. Most importantly, the regression models were built to predict wins for the current season, not the season after.
We were working with the assumption that this year’s “true performance” is predictive, but maybe that’s the wrong way to think about it. Maybe “generalized performance” would be a better way to think of Pythagorean Win Expectation.
I am interested in people’s thoughts on the reasons behind this reversal, so please feel free to leave them in the comments.
Still, I feel we have found a more accurate way to say how many games a team *should have* won in a season. And, I haven’t given up on finding a better predictive model for next year’s wins. The models just won’t be the same, which is part of the beauty of the generalized Pythagorean Expected Win model.
A follow up blog post will be forthcoming in which I will:
–clean up multicollinearity issues
–use next year’s win total as my target variable in training a second series of models
–see how all models do in predicting the next game and/or playoff results.
Until then, how many games should your team have won this year? Find out here.
Jason is a Data Scientist at Ruths.ai with a master’s degree in Predictive Analytics and Data Science from Northwestern University. He has experience with a multitude of machine learning techniques such as Random Forest, Neural Nets, and Support Vector Machines. With a previous Master’s in Creative Writing, Jason is a fervent believer in the Oxford comma.