College football strength of schedule rankings and how to make them much more useful

1. I hate strength of schedule arguments, partly because of the contradiction. Let me explain.

In my work with advanced stats — the S&P+ ratings and so forth — I field certain categories of insult pretty frequently. Here are two of the most common:

1. “Watch the games, nerd!” The insinuation: if you’d get your nose out of the spreadsheets, dork, you’d see that Team A is clearly better than Team B.
2. “Team A ain’t played nobody!” The insinuation: How could Team A possibly be any good when they haven’t played a team that clears someone’s arbitrary bar of goodness?

“You saw what happened in the games I watched, right?” the first one says. The second says to watch one particular game and ignore the others.

2. Football’s going to have a sample size issue no matter what, so we should milk meaning out of every game we’ve got.

Arguing about strength of schedule the wrong way means limiting the sample even further and acting like we can’t learn from every play of every game. We can. Honest.

Strength of schedule is at the heart of virtually every college football argument between October and January. Hell, it’s a point of debate for every college sport. The schedules are too varied and not comprehensive enough. That the power conferences are moving toward three-game non-conference schedules in football lessens national connectivity even more.

At the pro level, there are strong and weak divisions, but schedules are infinitely more balanced. In the MLB, everybody plays everybody in their league at least a few times, with some cross-league games. In the NBA and NHL, everybody plays everybody at least once.

With 130 FBS teams and 12 games, that simply isn’t an option for college football. So we play who we play, and we yell about who our rivals ain’t played.

3. How strength of schedule determines the national champion, sort of

Arguments are one thing, but the national title is more directly affected by strength of schedule than that of any other major sport. It’s baked into the Playoff selection process.

When circumstances at the margins indicate that teams are comparable, then the following criteria must be considered:

* Championships won

* Strength of schedule

* Head-to-head competition (if it occurred)

* Comparative outcomes of common opponents (without incenting margin of victory)

We believe that a committee of experts properly instructed (based on beliefs that the regular season is unique and must be preserved; and that championships won on the field and strength of schedule are important values that much be incorporated into the selection process) has very strong support throughout the college football community.

It is decreed that the committee takes strength of schedule into account, but it intentionally doesn’t say how. It frowns on advanced analytics and gives no alternative guidance. So the committee goes with things like “wins over top-25 teams” and “assuring there’s no way in hell a team from a Group of 5 conference will ever get in.”

4. By now, though, some are figuring out how strength of schedule is taken into account.

ESPN’s stats team has created both forward- and backward-looking measures to assess the difference between the “best” teams and those determined most deserving of a title shot. As it turns out, the backward view — or judging your team by whom you’ve beaten, not by how good you are — is pretty effective at mirroring committee action.

Despite the committee’s mantra of selecting the “four best teams in the country,” it appears that in the first two years of playoff selection, the committee favored team accomplishment over team strength. So if you are trying to predict what the committee will do, take a look at strength of record, because seven of eight teams to make the playoff ranked in the top four of that metric before playoff selection. Then FPI can be used to predict which teams will ultimately come out on top.

The committee insists it is looking for the “best” team. It is not. Kirby Hocutt, chairman of the CFP committee as of 2016, has conflated “best” and “most deserving” on a number of occasions. An example:

Q: Are you looking for the four best teams or the four most-deserving teams? Is there a difference?

A: You have to take into account the entire season. The season doesn’t start in October. Everybody has 12 regular-season opportunities, and the committee is watching. At the end of the year, we want to make sure we have the four very best teams over the course of the entire season.

They do not. And that’s fine, I guess.

5. Two major problems: a one-point win is not a 30-point win, and you don’t have to wait ‘til someone plays a good team to start learning.

Take these two old articles as an example.

Here I am in Oct. 2013, writing about how that year’s Florida State had long been elite, despite a weaker schedule.

”Are they tested?” is just a box you check. While you can find examples of teams that look great against awful teams, then stumble when punched in the mouth for the first time, you can usually glean just as much from how a team dominates bad competition as from how it plays against really good teams. Picking Team A simply because it is more tested than Team B is usually a recipe for making bad picks.

The early-season stats suggested that, despite not playing a top team yet, Florida State was pretty incredible. The Seminoles went out and left no doubt on Saturday night in Clemson.

Here I am a year later writing the opposite, about how 2014 FSU was probably doomed despite wins over good teams.

The best teams, the most likely championship teams, are the ones that handle their business early and put games out of reach before luck, chance, fumbles, and offensive pass interference calls can impact the outcome.

According to the F/+ rankings, the Seminoles have been just barely good enough to survive No. 9 Clemson at home (without Winston), No. 15 Louisville on the road, and No. 19 Notre Dame at home. They survived No. 44 Oklahoma State on a neutral field, and they pulled away from No. 53 NC State in the fourth quarter. They pummeled No. 76 Syracuse and got around to doing the same to No. 89 Wake Forest.

They have solidified that they should be ranked around seventh to 12th in these ratings. Without sustained improvement, they will in no way be a favorite to beat two top-four teams in the Playoff.

The 2013 team that hadn’t played nobody, but that was destroying its opponents, went on to win the national title.

The 2014 team that was winning, but not impressing the numbers, eked out a Playoff bid and then got embarrassed. (To its credit, the CFP committee did dock FSU a bit for its lackluster performance.)

These two FSU teams are examples for this simple truism:

6. You can learn something from every game, if you try.

That’s the point of using advanced stats in the first place, be it S&P+ or any other. You set the baseline depending on the opponent(s) at hand, and you compare the actual output to that adjusted expectation. It fills in what your eyes are missing. (And with 800-plus college football games in a season, your eyes are always missing something.)

Your record does matter. Even as an advanced stats loyalist, I’m not going to call for a three-loss team to get a CFP spot, even if said team was unlucky in every loss and ranks first in S&P+. Wins and losses aren’t particularly predictive in and of themselves, but they still have to mean something. Even the best team shouldn’t get in, if it’s not high on the most-deserving list.

7. So what if we actually tried to combine the two? What if we used a “best” measure to begin approximating what “most deserving” truly is?

What if we took the Strength of Record idea and added an extra level of team quality to it? Reintroducing Résumé S&P+, which made its debut here in 2017.

Below are each FBS team’s rankings in three categories:

• S&P+, an overall team efficiency rating system you can read more about here. It can be used to predict wins and losses going forward.
• Strength of Schedule (SOS), which amounts to how well the average top-five team (according to S&P+) would fare, in terms of win percentage, against your schedule. The lower the number, the harder the schedule. Why top-five? Because we’re using the Playoff’s four-team field as a target.
• Résumé S&P+, which looks at a team’s season scoring margin and compares it to what the average top-five team’s scoring margin would likely be against the schedule at hand. If the number is positive (and for most, it won’t be), that means said team is faring better then the typical top-five team. Instead of advanced stats or win probability info, I’m adhering to actual margins.

8. We will begin using this to measure 2018 teams in a few weeks, but to use last year’s top Résumé S&P+ team as an example, let’s look at Alabama.

The Crimson Tide finished second in overall S&P+. Their schedule strength ranked 34th at the end of the 2017 regular season; if the average top-five team played Alabama’s opponents a countless number of times, it would win about 79 percent of those games. The Tide won 92 percent of their games, with an average scoring margin about 12.5 points per game higher than said average top-five team.

Compare that to the schedule of someone like Auburn. The average top-five team would have had a win percentage of only about 73 percent against the Tigers’ slate. Those tiny differences can add up; Auburn was a legitimate top-10 team with a top-10 résumé but had three losses.

Then there’s UCF. The Knights’ strength of schedule ranked just 103rd overall, and they were punished severely for it by the CFP committee. But their average scoring margin was about 3.8 points ahead of how the average top-five team would be expected to perform. That’s a Résumé S&P+ rating within 0.6 points of Oklahoma.

(This is a wide table, because it has to be; if it’s not showing well on your phone, consider taking a look on another device later.)

2017 Résumé S&P+, end of regular season

Team	Record	S&P+	Rk	SOS	Rk	Resume S&P+	Rk
Team	Record	S&P+	Rk	SOS	Rk	Resume S&P+	Rk
Alabama	11-1	20.4	2	79.3%	34	12.4	1
Penn State	10-2	17.5	5	80.8%	57	9.6	2
Ohio State	11-2	21.0	1	77.4%	13	7.8	3
Georgia	12-1	18.4	3	78.5%	27	6.8	4
Clemson	12-1	16.8	7	79.5%	38	6.8	5
Wisconsin	12-1	17.1	6	79.3%	36	5.0	6
Auburn	10-3	14.9	10	73.1%	2	4.6	7
Washington	10-2	17.7	4	83.3%	87	4.4	8
Oklahoma	12-1	16.1	8	78.2%	24	4.4	9
Central Florida	12-0	15.8	9	85.0%	103	3.8	10
Notre Dame	9-3	11.2	14	76.2%	7	0.6	11
Oklahoma State	9-3	13.1	11	82.0%	72	-1.1	12
TCU	10-3	11.2	13	79.6%	40	-1.2	13
Mississippi State	8-4	7.6	26	77.1%	11	-3.1	14
Florida Atlantic	10-3	12.1	12	84.4%	96	-3.5	15
Virginia Tech	9-3	8.0	24	83.7%	89	-3.6	16
Stanford	9-4	7.2	30	77.8%	18	-4.3	17
Iowa	7-5	4.0	47	74.5%	4	-4.4	18
Memphis	10-2	9.5	18	83.0%	85	-4.5	19
Miami-FL	10-2	9.5	19	79.5%	39	-6.1	20
Louisville	8-4	9.9	17	82.6%	80	-6.4	21
Boise State	10-3	7.3	28	81.3%	66	-6.4	22
LSU	9-3	9.3	20	79.7%	42	-6.9	23
Northwestern	9-3	5.2	43	81.0%	58	-7.2	24
Michigan	8-4	9.0	21	77.2%	12	-7.2	25
Wake Forest	7-5	6.7	36	78.2%	25	-7.4	26
South Florida	9-2	10.7	15	88.1%	126	-7.5	27
Toledo	11-2	10.0	16	86.9%	121	-7.5	28
USC	11-2	8.9	22	80.6%	52	-7.6	29
Ohio	8-4	7.0	33	86.7%	117	-7.9	30
Iowa State	7-5	4.9	44	80.5%	51	-8.1	31
Oregon	7-5	5.7	40	81.3%	65	-8.5	32
Fresno State	9-4	7.7	25	81.2%	64	-8.7	33
Texas	6-6	2.9	54	79.6%	41	-9.0	34
Troy	10-2	6.2	37	86.8%	119	-9.1	35
San Diego State	10-2	6.0	38	86.4%	114	-9.2	36
NC State	8-4	4.9	45	79.1%	31	-9.2	37
Washington State	9-3	7.2	31	80.2%	48	-9.7	38
Arkansas State	7-4	8.6	23	89.6%	130	-9.9	39
Boston College	7-5	0.7	67	77.0%	9	-10.0	40
Purdue	6-6	5.4	41	78.6%	28	-10.1	41
Florida State	6-6	3.2	51	77.0%	10	-10.3	42
Appalachian State	8-4	6.9	35	86.9%	120	-10.5	43
Utah	6-6	6.0	39	80.5%	50	-10.9	44
Missouri	7-5	7.2	32	82.9%	84	-10.9	45
Arizona	7-5	3.8	49	83.6%	88	-11.0	46
Army	8-3	2.6	59	85.1%	105	-11.1	47
Kansas State	7-5	0.9	65	81.0%	59	-11.4	48
Northern Illinois	8-4	4.0	48	86.4%	113	-11.4	49
Michigan State	9-3	7.3	27	76.9%	8	-11.4	50
Marshall	7-5	2.6	58	84.2%	94	-11.9	51
Duke	6-6	0.4	70	82.2%	73	-12.1	52
SMU	7-5	2.4	60	80.7%	53	-12.7	53
West Virginia	7-5	2.3	62	80.1%	47	-12.9	54
South Carolina	8-4	0.7	66	80.4%	49	-13.3	55
Texas Tech	6-6	2.6	57	79.2%	32	-13.4	56
Georgia Tech	5-6	0.3	72	77.7%	17	-13.5	57
Houston	7-4	7.2	29	84.0%	92	-13.5	58
Texas A&M	7-5	1.3	64	79.2%	33	-13.6	59
Navy	6-5	-0.6	78	78.3%	26	-13.7	60
Arizona State	7-5	-0.6	79	78.1%	22	-13.8	61
Indiana	5-7	3.4	50	77.5%	14	-14.4	62
Western Michigan	6-6	0.0	74	84.7%	98	-14.8	63
Colorado State	7-5	5.4	42	85.8%	110	-15.1	64
Wyoming	7-5	2.4	61	85.2%	106	-15.7	65
Utah State	6-6	2.8	55	84.2%	93	-15.8	66
Southern Miss	8-4	6.9	34	89.2%	129	-16.0	67
UTSA	6-5	3.0	52	88.0%	125	-16.1	68
Eastern Michigan	5-7	0.7	68	84.7%	99	-16.3	69
Buffalo	6-6	-0.1	76	85.5%	108	-16.7	70
North Texas	9-4	2.7	56	82.9%	82	-16.9	71
California	5-7	-2.7	88	79.9%	44	-17.0	72
Louisiana Tech	6-6	-0.9	80	83.8%	90	-17.2	73
UCLA	6-6	0.0	75	78.0%	20	-17.3	74
Central Michigan	8-4	0.1	73	85.6%	109	-17.6	75
Minnesota	5-7	-4.9	96	81.1%	62	-17.7	76
Ole Miss	6-6	4.8	46	80.0%	45	-18.0	77
Kentucky	7-5	-4.0	94	79.0%	30	-18.3	78
UAB	8-4	1.9	63	89.1%	128	-18.8	79
Tulane	5-7	-2.9	90	81.3%	67	-18.8	80
Air Force	5-7	-5.6	100	82.9%	83	-18.9	81
Colorado	5-7	-3.3	91	81.1%	61	-19.0	82
Pittsburgh	5-7	-1.6	82	79.8%	43	-19.1	83
Syracuse	4-8	-2.8	89	77.6%	16	-19.1	84
Florida	4-7	-2.4	87	78.2%	23	-19.5	85
Virginia	6-6	-0.1	77	81.8%	71	-20.0	86
Akron	7-6	-7.4	110	80.7%	54	-20.4	87
Middle Tennessee	6-6	0.4	71	87.0%	122	-20.6	88
Florida International	8-4	-1.7	83	84.3%	95	-20.9	89
Temple	6-6	-2.3	86	82.3%	76	-21.0	90
Miami-OH	5-7	2.9	53	86.7%	116	-21.2	91
Arkansas	4-8	-3.3	92	75.6%	5	-21.5	92
New Mexico State	6-6	0.6	69	87.1%	123	-22.1	93
North Carolina	3-9	-3.6	93	81.1%	60	-22.2	94
Massachusetts	4-8	-2.2	85	87.3%	124	-22.5	95
Nebraska	4-8	-5.9	102	74.0%	3	-23.2	96
Vanderbilt	5-7	-1.4	81	79.4%	37	-23.3	97
Nevada	3-9	-7.3	108	82.2%	74	-23.9	98
Western Kentucky	6-6	-1.9	84	88.6%	127	-23.9	99
UNLV	5-7	-4.6	95	85.0%	104	-23.9	100
Tennessee	4-8	-7.0	107	77.5%	15	-24.0	101
UL-Monroe	4-8	-6.6	105	81.5%	68	-24.1	102
Tulsa	2-10	-10.0	116	80.1%	46	-24.1	103
Idaho	4-8	-5.4	98	86.3%	112	-25.2	104
Rutgers	4-8	-7.5	111	76.1%	6	-25.5	105
Maryland	4-8	-8.8	114	72.6%	1	-25.5	106
BYU	4-9	-5.5	99	81.7%	69	-26.1	107
South Alabama	4-8	-5.0	97	84.6%	97	-26.6	108
Baylor	1-11	-6.6	106	78.6%	29	-26.7	109
Georgia State	6-5	-5.7	101	86.4%	115	-27.7	110
Georgia Southern	2-10	-11.6	120	82.6%	79	-29.0	111
New Mexico	3-9	-6.3	104	83.1%	86	-29.1	112
Cincinnati	4-8	-5.9	103	82.2%	75	-29.4	113
Bowling Green	2-10	-7.3	109	82.3%	77	-30.0	114
Coastal Carolina	3-9	-8.1	112	86.2%	111	-30.4	115
Old Dominion	5-7	-9.8	115	84.9%	101	-30.4	116
Illinois	2-10	-11.0	118	77.8%	19	-31.0	117
Hawaii	3-9	-11.8	121	85.5%	107	-31.3	118
Connecticut	3-9	-11.4	119	81.8%	70	-32.2	119
UL-Lafayette	5-7	-8.8	113	86.8%	118	-32.8	120
Texas State	2-10	-10.2	117	84.9%	102	-35.6	121
Oregon State	1-11	-14.9	126	79.3%	35	-35.8	122
Charlotte	1-11	-14.6	124	83.9%	91	-37.0	123
East Carolina	3-9	-13.9	123	80.7%	55	-37.0	124
Rice	1-11	-15.0	128	82.9%	81	-37.1	125
Kent State	2-10	-14.9	127	81.2%	63	-39.6	126
Kansas	1-11	-14.7	125	78.0%	21	-39.8	127
UTEP	0-12	-18.4	130	80.8%	56	-41.0	128
Ball State	2-10	-13.3	122	84.8%	100	-41.7	129
San Jose State	2-11	-16.2	129	82.4%	78	-43.2	130

Eleven teams had positive Résumé S&P+ ratings. That’s more than one would expect, if you’re comparing performances to average top-five teams. But the average top-five team last season was not nearly as good as it had been in previous years.

9. By the way, to those who want to moan about incentivizing running up the score in this measure (since we’re using scoring margins), suck it up.

Margin of victory is infinitely more informative than “did they win?” It just is. Besides, it’s odd to suddenly care about hurt feelings when I’m pretty sure telling half of FBS they don’t have a shot at the national title, no matter how well they play, is more hurtful to those feelings than winning 59-0 instead of 49-0.

10. Best versus most deserving. How you’ve played versus who you’ve played.

Maybe there’s a way to tie together these worlds after all.

Let's block ads! (Why?)

https://www.sbnation.com/college-football/2018/8/29/17795292/college-football-strength-of-schedule-rankings-2018-sos