I am tracking shots based on ESPN's shot charts for the playoffs, but I have also been regularly checking out CBS Sportsline's shot charts because they add another interesting piece of information: Where the shot was targeted on net. For each shot they record one of five target zones: High glove, high blocker, low glove, low blocker, or 5-hole.
I haven't compiled any numbers other than for this year's playoffs, but Hockey Numbers did a post a while back that gives the CBS data broken down by shot location for the 2006-07 season. There seemed to be a lot of shots missing in that sample as the average save percentage was much lower than the official NHL numbers, but the results still conclusively prove what all hockey players know: That shooters are much more likely to score if they shoot high.
It doesn't seem to matter much where a shot is targeted on net from right to left (e.g. high glove or high blocker are about the same), the key difference is whether the shot is high or low. There are of course some goalies who are better on the blocker side or on the glove side, and all goalies probably have slightly less success on five-hole shots than shots that are low to either side, but those differences are small compared to the difference between top shelf and along the ice.
The 2006-07 data has a save percentage difference of .054 between high shots and low shots. This year in the playoffs, there has been a .942 save percentage on low shots and an .872 save percentage on high shots, for an even more extreme gap of .070. Those differences make a pretty good case that shot height should be included in measurements of shot quality.
As is unfortunately often the case with real-time stats, however, the CBS reporting system is pretty suspect. They seem to have fixed the earlier problem of missing shots, but there are large variances in high shot frequency from rink to rink. Here are the stats by series:
BOS vs. MTL: .917 low, .857 high, 27% high shots
WSH vs. NYR: .946 low, .868 high, 19% high shots
NJD vs. CAR: .961 low, .907 high, 41% high shots
PIT vs. PHI: .961 low, .800 high, 14% high shots
SJS vs. ANA: .926 low, .933 high, 19% high shots
DET vs. CBJ: .919 low, .840 high, 20% high shots
VAN vs. STL: .961 low, .875 high, 16% high shots
CHI vs. CGY: .928 low, .845 high, 35% high shots
We would expect high shots to be correlated with space on the ice. The more time and space a shooter has, the more likely he is going to be able to shoot high. I wouldn't be surprised that a tight-checking series like Vancouver/St. Louis might have a below-average amount of high shots. However, even in that particular series the number of actual high shots is almost certainly understated. In game 4 in St. Louis every single one of Roberto Luongo's 49 shots against were recorded as being low shots, which is extraordinarily unlikely and seems to be merely a case of an indifferent scorekeeper. I observed several other similar games where all or nearly all of the shots were booked as low shots.
The scorers in New Jersey and Carolina appear to be the opposite, booking too many shots as high. The high shot rate in that series is almost double that of all the other series combined, and the save percentage against high shots is much higher than average. The Calgary series also has an abnormal ratio, but the save percentage on high shots is just .845. Either the shooters are really managing to go upstairs that often, or else Kiprusoff and Khabibulin are doing a very poor job of handling high shots.
In most series both goalies have faced a pretty similar number of high shots. One series stood out by having a large gap between the teams, Detroit's 4 game sweep of Columbus. Based on shot distances and shot locations, it looks like Columbus allowed easier shot quality against than Detroit. Over half (54%) of the shots against Steve Mason came from the point or were perimeter shots, while the same areas accounted for just 37% of the shots against Chris Osgood. As a result, Detroit's outshooting advantage is counterbalanced by a longer than average shot distance. I have Detroit with an expected goals figure just 0.3 ahead of Columbus for the series. That is similar to the shot quality results at Hockey Numbers (Detroit +0.5).
When we consider where the shots were targeted, it becomes a different story. The rate of high shots against Steve Mason was twice as high as the rate against Chris Osgood (26% to 13%). If we adjust only based on the average save percentages for low and high shots, that means we would expect Osgood to have the easier job with a .933 expected save percentage compared to .924 for Mason. If we recalculate the expected goals based on those save percentages, Detroit would have been expected to score 3.4 more goals than Columbus over the 4 game series.
What seems possible is that even though Columbus' shooters were getting into good shooting locations, they were mostly shooting under pressure from defenders. In contrast, Detroit was setting up more open shots, which allowed their shooters to snipe up high against Mason. I must confess I wasn't able to catch any of the games of that series, however, so if you did follow that series then feel free to comment on whether you believe the shot quality figures (both based on ice location and target location) seem correct.
If we adjust for both where the shots were coming from and where they were targeted, based on playoff averages so far, I estimate that Mason had a .933 expected save percentage while Osgood was at .922.
We can use those figures to conclude that the much-maligned Chris Osgood did surprisingly well in round 1, but it wasn't a very good playoff debut for Steve Mason. The stats suggest that the likely Calder Trophy winner had the worst overall performance of any goalie in the playoffs, although at least Jose Theodore ranks below him on a per-game basis.
I think it is pretty evident that shot quality would be improved if the target location of the shot was accurately tracked. By combining that information with where the shot was coming from on the ice, it should be possible to get a more accurate scoring probability. Unfortunately CBS Sportsline's tracking system seems to too untrustworthy to be useful at the moment. To evaluate all goalies on a level playing field it is necessary to standardize the reporting to remove or at least drastically reduce rink reporting bias.