The Shot Quality Effect, Part 2

I did some further calculations based on the responses to my last post, so I thought I'd just throw them in here for the record and in case somebody is interested that didn't read the comments to the last one.

It was pointed out that goalies who face fewer shots have results that are more subject to variability, so it would be more correct to adjust for sample size. Vic Ferrari suggested using an adjustment factor of sqrt(1000/shots faced) to normalize all seasons to a 1,000 shot level. I included that adjustment, and I also normalized all years based on league average to remove any seasonal effects.

As an example, if the goalie had a .925 ES SV% last year, and a .930 ES SV% this year, and he faced 500 shots, then he is 2.5 goals better this year than last year. Multiply that by sqrt(1000/500), and we get 3.5 goals, which expresses the result on the scale of a 1,000 shot season, taking into account the significance of the information provided by that sample size.

All that was left to do was sort based on whether the goalie was playing for the same team or a different team. Here are the results (I included the same minimum games played cutoffs as before, 10 GP means that the goalie must have played at least 10 games in both seasons for them to count):

0 GP: 12.0 same team, 12.6 different team
5 GP: 10.4 same team, 11.5 different team
10 GP: 10.3 same team, 12.1 different team
20 GP: 10.0 same team, 11.9 different team
30 GP: 10.0 same team, 10.9 different team
40 GP: 9.8 same team, 11.6 different team

These results suggest a team effect of about 1-2 goals per 1,000 shots, or .001-.002 in save percentage. My look at the average save percentage differences had similar results with a gap of about .002-.003. These results suggest that, for the most part, even strength shot quality is pretty consistent in the NHL. I still expect there to be a few teams who fall a bit outside of the regular curve, but it appears to be pretty clear that even-strength save percentage is a very important statistic for goalie evaluation.

Note, too, that this suggests that the typical variation in goalie performance is 10 goals per 1,000 shots, or .010 in save percentage, even for goalies playing on the same team. Quite often "bad years" or "good years" are simply because of chance. That is something that we need to keep in mind as fans before we make the mistake of calling somebody good or bad after watching them a few times or, as tends to happen at this time of year, based on results from a single seven game series.