Any avid college football fan, particularly one who follows one team closely over the course of a season, knows that teams of 18-22 years old kids/men are fickle. The 2005 TCU team starts the season by going into Norman and handing the Sooners the second of just 5 losses they've had at home in the Bob Stoops era. Then the next week they lose to a 5-6 SMU team. Last year's TCU team was all over the map as well, losing badly to a mediocre Iowa State team but whipping a solid Baylor team.
In many cases, such imprecision from our college football teams is rationalized away with excuses like "coach didn't get them motivated" or "trap game" or "getting caught at the Indian casino playing poker with a table of hookers shows that Johnny wasn't ready to play."
Any one or combination of such excuses might be relevant, but the reality is that college football teams, even the best ones with the most disciplined and senior-laden rosters, are extremely inconsistent.
So as we enter the first week of the 2013 college football season and we look over the Frog's schedule and tick off the wins and losses, let's review the 2012 season and look at just how confident we should be when we predict that W in the win column for the SMU game.
DUSHEE, guide the way
As we've discussed before, Point Differential compares how Team A does against Team B relative to how all of Team B's other opponents have done against them. The Point Differential (PD) tells us that if Team A beats Team B by 10 more points than the average team on Team B's schedule beat them by, then if Team A is consistent, they should be pretty close to 10 points better than the average opponent against every other team on their schedule.
Let's take the 2012 TCU team as an example. TCU's average PD for the year was 3.6, meaning that TCU was, on average, 3.62 points better against their opponents than the average team their opponents played. In turn, here were the final PD's for all of TCU's opponents on the year:
Kan Uva SMU | ISU Bay Ttech | OkSt WVU KSU | Tex OU MichSt
-15.8 -9.6 1.3 | 1.5 9.3 3.8 | 14.3 0.9 18.6 | 8.3 13.9 7.1
Table 1. Season average PD's for TCU's opponents in 2012
So if TCU (and their opponents) had been perfectly consistent in their play, we would have expected the outcome, or margin of victory, for each of those games to have been roughly TCU's PD minus their opponent's PD. So Table 2, we compare the "expected" outcome to the actual outcome:
Opp Exp. | Act. Diff
Kan 18.4 | 14 -4.4
UVa 13.2 | 20 6.8
SMU 2.3 | 8 5.7
ISU 2.1 | -14 -16.1
Bay -5.7 | 28 33.7
TTech -0.2 | -3 -2.8
OkSt -10.7 | -22 -11.3
WVU 2.7 | 1 -1.7
KSU -15 | -13 -2
Tex -4.7 | 7 12.4
OU -10.3 | -7 3.3
MichSt -3.5 | -1 2.5
Table 2. Based on Point Differential the expected outcome for each of TCU's games last year compared to the actual outcome.
Based on this, we would surmise that TCU's worst game of the season was the Iowa State game where an expected 2 point win was in reality a 14 point loss. TCU did 16.1 points worse in that game than the rest of the season indicated they should have done. They followed that game the next week with the game in which they most "out-kicked their coverage" against Baylor. Had both teams performed, on average, as they performed for the season, we should have expected Baylor to have beaten TCU by 6 points. Instead TCU beat Baylor by 4 TDs.
From this perspective, the games in which TCU (and their opponents) performed most like their "average" selves were the West Virginia, Kansas State, Tech, and Michigan State games. The Baylor, ISU, Texas and Oklahoma State games were the games most unlike our average performance.
Even the Best Are Inconsistent
Despite TCU's youth, upheaval, and conference inexperience, TCU was the 44th (out of 124) most consistent team in college football based on standard deviation of PD (13.9 points). By that metric, the most consistent team in college football last season was Troy with a standard deviation of 7.7. Assuming that their performance looks like a normal distribution (i.e., a bell curve) then it is 32% likely (1/e) that in any one game they are at least 7.7 points better or worse than their average PD would predict. And that is the most consistent team in college football. There was a roughly 1-in-3 chance that TCU's play in a given week was TWO TOUCHDOWNS or more off of their "average" performance.
Alabama, DUSHEE's (and everybody else's) best team, was the 13th most consistent team in the country, with a standard deviation of 10.1 points. If we repeat the exercise that we did for TCU in Table 2 for Alabama, we get the following:
Opp OppPD | Exp Act | Diff
Mich 11.5 | 19.6 27 | 7.4
WKU -2.8 | 33.9 35 | 1.1
Ark -4.4 | 35.5 52 | 16.5
FAU -8.4 | 39.5 33 | -6.5
Miss 6.7 | 24.4 19 | -5.4
Mizz 1.9 | 29.2 32 | 2.8
Tenn 0.5 | 30.6 31 | 0.4
MissSt 3.9 | 27.2 31 | 3.8
LSU 15.3 | 15.8 4 | -11.8
A&M 26.3 | 4.8 -5 | -9.8
Aub -8.7 | 39.8 49 | 9.2
Uga 18.7 | 12.4 4 | -8.4
ND 17.9 | 13.2 28 | 14.8
Table 3. Alabama's expected and actual performance
Besides the week 3 annihilation of Arkansas, Alabama's most "uncharacteristic" performance was the MNC game against ND. On average, we should have expected Alabama to have beaten ND by two TDs rather than 4. But again, there was a 1-in-3 chance that Alabama's performance could swing at least 20 points on a given night last season. On that night, it swung up two TDs.
Understanding this, you begin to see why going undefeated is such a difficult thing to do. Even the best teams in college football will have a game or two where they underperform by a touchdown or more. And if those games come against an opponent whose average performance is only a touchdown worse, or who happens to overperform that week, that team loses, even if it is, statistically, the better team. Alabama was, statistically, 5 points better than A&M. Play that game 100 times and Alabama probably wins 60**. But on that particular day they lost by 5.
** Monte Carlo simulations using an adjusted PD estimate that Alabama would win 55-60% against A&M -- perhaps we'll discuss such simulation techniques on a future post.
Selling Oceanfront Property in Kentucky
The most inconsistent team in college football in 2012? Kentucky with a standard deviation in PD of 27.1 points.
Opp OppPD| Exp Act | Diff
L'ville 4.8 | -19.2 -18 | 1.2
KentSt 4.5 | -18.9 33 | 51.9
WKU -2.8 | -11.6 -1 | 10.6
Fla 18.9 | -33.3 -38 | -4.7
SoCar 15.8 | -30.2 -21 | 9.2
MissSt 3.9 | -18.3 -13 | 5.3
Ark -4.4 | -10 -42 | -32
Uga 18.7 | -33.1 -5 | 28.1
Mizz 1.9 | -16.3 -23 | -6.7
Vandy 5.1 | -19.5 -40 | -20.5
Tenn 0.5 | -14.9 -20 | -5.1
Table 4. Kentucky's roller coaster season.
Kentucky "should have" lost to Kent State by 19. They beat the Golden Flash by 33. Kentucky "should have" lost to Arkansas by 10. Instead they lost by 42. Vanderbilt treated them similarly. Georgia "should have" beaten Kentucky by 33 but only beat them by 5. If you bet on Kentucky during the 2012 season, you were a fool.
In a strikingly odd statistical anomaly, of the 10 most inconsistent teams in college football last year, seven were on TCU's schedule including six from the exceedingly inconsistent Big 12:
Oklahoma St. 23.7
Texas Tech 23.1
West Virginia 21.5
Table 5: The 10 most inconsistent college football teams of 2012. TCU's opponents are bold.
So as maddening as TCU's inconsistency may have felt for fans last year, the Frogs were in reality one of the more consistent teams in their conference. Which is damning with faint praise.
So some may accuse me of writing all of this as a hedge against my performance in college pick-em contests. But I assure you my motives are purely analytical. That said, if I do poorly, come to this post to see my excuse. The rest of you suckers just got lucky ...