The Importance of Quality of Competition
Hockey talk has come a long way in the past few years. It wasn't long ago that the notion of context of ice time, especially quality of competition, was considered to be a nonsense promoted by wacky Oiler fans. Even the pure math folks on the Internet thought it was folly. MC79 would routinely visit a Flames fan message board and argue the point. There was much mocking and derision going both ways.
It's not that way now, though. This is thanks largely to hockey play-by-play man Jim Hughson. Not only does he frequently talk about the match-ups on the ice, he gets them right. And nobody has done more to promote the notion on the Internet than Gabe Desjardins, this through the QUALCOMP numbers at his terrific hockey statistics site.
I thought it was time to look at just how much of an impact QUALCOMP had on a player's results. To do that I'm going to split the 09/10 NHL season in half, and look at change of results for the players from one half to the next, and compare that to their change in QUALCOMP.
Desjardins' QUALCOMP is similar to what is shown in this post of a few weeks ago. The head to head EV shifts table from that post is replaced with head to head 5v5 ice time, and that scoring chance +/- list is replaced with on-ice/off-ice +/- rate. That's his measure of player value. Plus Gabe uses only one iteration. Then he subtracts the result from the original and voila ... QUALCOMP.
It would be a bitch to gather all the head to head and on-ice/off-ice data. But once you had done that, calculating QUALCOMP is just three or four lines of code in most programming languages. Simple stuff, really.
The quality of competition metric I will use here is a bit different. Gabe's QUALCOMP only compares players to their teammates, I need a global number. Also, Gabe doesn't given season and quarterly splits for his data. And I'll need both here.
For those that care; I use Fenwick Numbers for the player value (our best proxy for player scoring chance +/-) and total shots at net by either team, in lieu of head to head ice time. Zone start is also factored into the opponent's value. I also run several iterations, in this case about 30. This is because good players tend to play a lot against good players in this league. So, for example, you run the numbers for Zetterberg and realize that he played a bunch against the other team's best outchancing players. So he's even better than his Fenwick suggest. Everyone who played against him deserves larger props. you bump his value a smidgen and run the numbers again. Same for everyone else in the league. Rinse and repeat until the results stabilize.
I chose guys that played regularly in both halves of the season. That's 412 players, about 14 per team. Obviously far fewer for teams like the Oilers, who were devastated by injury and illness this past season.
Comparing the first half of the season o the second half.
The relationship of change in Vic's Qualcomp to change in Fenwick ratio:
Pearson's r = .37
variance in player results change: 13,186
Just to keep our lives simple, we'll pretend that the data is distributed normally, so r² * total variance ~= variance attributable to Vic's Qualcomp.
variance in player results change attributable to Vic's Qualcomp: 1,853
Now Pearson's correlation doesn't mean a whole lot unless we understand the role of chance in the game.
Some of that change in players' Fenwick% is down to chance alone. We see that in the games, and it's explicit if you've been following any of the scoring chance recorders on the web.
If we had access to a million parallel universes the element of luck (chance variation) would evaporate, and Pearson's r would wander up to about .71 and stabilize. But we don't.
We do have access to the first quarter of the season and the last quarter of the season, though. That works out to about half the average ice time per player, and the element of chance would be double in that sample.
So we run that:
Pearson's r = .20
variance in player results change: 23,024
Subtraction yields the variance component attributable to luck in the first study and he non-luck element:
luck component variance: 9,837
non-luck component variance: 3,349
THE FINAL TALLY:
Vic's Qualcomp explains 55% of the change in player's Fenwick results that is unaccounted for by luck.
It would be larger if my model were better. For starters, scorer bias and score effects in the games could be accounted for.
On a team by team basis, Vic's qualcomp has a strong correlation to Desjardin's QUALCOMP, usually around r = .8. And these measures are built from completely different bricks. They are both built with reason, that's the only thing they have in common.
My methodology, crude and simple as it is, did weed out most of the injury effects, they are obviously huge for some players as well. Linemate quality is obviously going to account for a big chunk of the remainder, I think, though that covaries negatively with quality of competition (when you get better linemates, more often than not you're also going to be playing against better opposition).
Bottom line: If you ignore quality of competition, you do so at your own peril.
Next season I'll show the change in Vic's Qualcomp over season halves. The top 100 players in qualcomp change will, collectively, see their Fenwick results (or scoring chance results if we have them) move the same direction as their quality of competition. Nothing can stop it. If you can find a denier who likes to wager, prop them 3 to 1 odds that most of that Top 100 will see their results go the way of their Vic's Qualcomp. Or Desjardins' QUALCOMP for that matter.
The odds on that wager are about 400 trillion to one in your favour. It's unlosable. Props any higher than 3 to 1 will just scare off the punter, though.