NHL vs. CHL

When doing a bit of research for my last post about Canadian world junior team goalies, I took a quick look into CHL stats across the three major junior hockey leagues. For me, it was a bit of a reminder that the NHL results may not necessarily be representative of all levels of hockey.

Take, for example, shots against vs. save percentage, a common theme in this space. There is a vocal group that repeatedly maintains that those two things are positively related, and that as shots against go up save percentage necessarily also goes up. This is supposedly either because shots against the run of play are more dangerous, or because it is easier for a goalie to concentrate when he faces more shots. There have been some seasons at the NHL level where the data seem to suggest this is a possible relationship, including this season so far where James Mirtle calculated a 0.49 correlation coefficient between shots against per game and save percentage.

However, if these arguments are correct they should be generally true for all levels of hockey, including junior. Here are the correlation coefficients between shots against per game and save percentage for the past 3 seasons:

Ontario Hockey League:
2005-06: -.269
2006-07: +.078
2007-08: -.278

Western Hockey League:
2005-06: -.219
2006-07: -.347
2007-08: -.214

Quebec Major Junior Hockey League:
2005-06: -.112
2006-07: -.180
2007-08: +.266

Not much support for the "more chances = better save %" theory. The Western Hockey League is the league that tends to produce the best defensive teams with the strictest defensive systems, and that league has the strongest relationship between save percentage and shots against. The numbers indicate that playing on a strong defensive team in junior hockey probably helps your save percentage. This suggests that goalie prospects who had a big advantage in juniors because of their defensive teammates (like Justin Pogge, Leland Irving, Jeff Glass and Tyson Sexsmith) might turn out more like Kelly Guard than Carey Price.

Tyler Dellow posted recently on save percentage with certain players on the ice, and concluded that it appears to be mostly driven by randomness. Players with unusually high or low save percentages will regress to the mean as the season goes on. His conclusions make sense for the NHL level, but I would like to see similar numbers for junior hockey players, to see if the dominant players at that level had an effect on their own team's save percentage. It would also be interesting to see the numbers from an era of the NHL with a different level of competitive balance (the 1970s, for instance). In the 1970s, the save percentage-to-shots-against relationship was very similar to the junior results above, i.e. a weak negative correlation. This relationship inspired hockey analysts Klein and Reif to come up with the goalie perseverance rating, a rating that penalized goalies on low-shot teams and gave bonus points to goalies who faced more rubber, in their influential Hockey Compendium (published in 1986). Their weighting system did not stand the test of time, however, as hockey's competitive landscape changed (see the comments to this post for a discussion of how Klein and Reif's results no longer hold in the current NHL).

I think that bad hockey teams generally give up high shot quality against, and good hockey teams generally give up low shot quality against. It is only in a league with a good competitive balance (such as the NHL) that other factors come into play and affect the result, for example because the outshooting team takes longer shots or more of their shots come from their third- or fourth-line players, or possibly the players are playing a specific offensive or defensive system, or some other similar reason(s).

My overall point is that I'm not sure whether all of the statistics-based conclusions that are being made these days represent essential hockey truths, or whether we are merely collecting evidence of the high degree of parity in today's NHL. It might be wise now and then to test out conclusions with results from a different league or era to see if the findings still hold. Unfortunately, the lack of data available does not always make this possible.