Shots Against

This is just a short post on a little idea I had a while back and have been meaning to get around to doing. It's on the topic of the relationship between save percentage and shots against. One way to put a team in the best possible context is to look at other teams in the league that are similar. For example, if it is easier to play goalie on a team that allows a lot of shots against then it is likely that other teams with a similar shot profile would have the same advantage.

I looked at the situational save percentage numbers for the five teams that were closest in shots against per 60 minutes to the New Jersey Devils and the Florida Panthers from 2003-04 to 2008-09. New Jersey and Florida are two teams that are often brought up when various theories are advanced about whether the number of shots per game a goalie faces has an effect on their save percentage. I also reran the numbers using a few possible shot bias/goalie shot effect adjustments (adding one shot and two shots to New Jersey's totals and subtracting one from Florida's).

The result? There doesn't seem to be any significant effects from different levels of shots. If either of the teams are outliers then they also stand out against teams with similar shot prevention.

League average: .918 EV SV%, .866 PK SV%, .912 PP SV%

Team	Adjustment	EV SV%	PK SV%	PP SV%
New Jersey	0	.918	.868	.906
New Jersey	+1	.916	.865	.918
New Jersey	+2	.919	.865	.915
Florida	0	.916	.867	.913
Florida	-1	.916	.865	.911

Three Stars

I bash the media a lot in this space, but sometimes it is interesting to see what they have to say. For Puck Prospectus I put together a look at three star selections for goalies.

I just quickly looked up the same numbers last year for Steve Mason, because I'm interested by how seasonal statistics like GAA and shutouts can have a huge impact on perceptions. Fans and media personnel who watch the games can be unimpressed by a goaltender night after night over long stretches of the season, but come April the number of wins and shutouts in the goalie's stat line might lead them to quite a different conclusion. If you watch a goalie make 18 easy saves against the Carolina Hurricanes on a Tuesday night in November, you might think that was a pretty meaningless shutout. Months later, when your memories of that game have faded and you are trying to decide which goalies to put on your Vezina ballot, that shutout might make the difference between that goalie or someone else.

Mason's three star voting numbers in 2008-09 fell right in line with his save percentages. He was lights-out in December (.950 save %, named one of the game's three stars 8 times in 11 starts), but was mediocre for the rest of the season (.905 save %, named a game star in just 10 out of 41 starts). Over the entire season Mason was named a star of the game in 34% of his starts, which is pretty good but not anything unusual since the average for goalies is about 30% of the time.

For three and half months Mason's play was subpar and those watching certainly weren't being amazed by what they saw. Yet because of his terrific work earlier in the year, every time they looked at a stat sheet they saw a pretty low GAA, a decent win total and most notably a lot of shutouts. The result was a widespread perception that Mason carried the Blue Jackets into the playoffs, earning him a flat-out ridiculous 4th place finish in Hart Trophy voting.

High and Low Shot Games

I looked at low shot games in one of my Puck Prospectus articles from earlier this year, and I wanted to look at those games in a bit more detail. I took a sample of this season's games with the most and the least shots taken by one of the teams. For the most shots, I took the 40 highest totals, while for the least shots I took the top 38 (to avoid ties that would have extended the sample well past 40).

Here are some of the numbers:

Shooting Percentage by Period:

1st period: High Shot 4.3%, Low Shot 14.6%
2nd period: High Shot 5.8%, Low Shot 8.7%
3rd period: High Shot 7.6%, Low Shot 10.4%

These numbers really show how a team's shooting percentage (or, conversely, the opposing goalie's save percentage) impacts the number of shots in the game. A high save percentage tends to cause higher shots against, while a lower save percentage tends to cause lower shots against.

I did not remove empty net goals. If I had done so, I'd guess that both the high and low shot teams would have had third period numbers that were more similar to their second period results.

The low shot teams shot the lights out in the first period, and then probably scored at about the the league average rate for the next two periods (excluding likely empty-netters). This indicates fairly strongly that the direction of causation runs from the percentages to the totals, rather than the other way around. The high shot teams scored at a below-average rate throughout the game, suggesting that very high shots against numbers do tend to go hand in hand with a high save percentage (although it could still be that an outstanding goalie performance is usually required for a team to take that many shots).

If the game score had a strong effect on the numbers, then that should show up in a breakdown of shots against by period:

Percentage of Shots by Period:

1st period: High Shot 31.6%, Low Shot 32.6%
2nd period: High Shot 32.7%, Low Shot 38.4%
3rd period: High Shot 32.5%, Low Shot 27.4%
Overtime: High Shot 3.2%, Low Shot 1.6%

The third period is when we would expect to see the strongest playing to the score effects. The third period is when the least shots were taken in low shot games. This again supports the theory that low shot games are usually a result of the scoreline and the percentages early in the game.

In contrast teams with high shot totals did not show any playing to the score effect, at least on the offensive side.

I also thought to look at the shots broken down by game situation, to see whether power play or shorthanded situations might have had a significant impact.

Shooting Percentage by Game Situation:

5 on 5: High Shot 5.1%, Low Shot 10.6%
5 on 4: High Shot 7.2%, Low Shot 12.8%
4 on 4: High Shot 7.8%, Low Shot 10.0%
4 on 5: High Shot 7.7%, Low Shot 16.7%

Percentage of Shots by Game Situation:

5 on 5: High Shot 73.1%, Low Shot 76.2%
5 on 4: High Shot 19.0%, Low Shot 14.9%
4 on 4: High Shot 4.1%, Low Shot 4.8%
4 on 5: High Shot 2.1%, Low Shot 2.9%

The low shot teams had a higher shooting percentage at all four game situations. In my view this supports the theory that shooting percentage has a strong impact on shots against. I suspect there may be some shot quality effect at even strength related to shots against. For example, the team that is taking more shots will likely be also taking more from the less dangerous scoring areas, and more of their shots will come from third/fourth liners and defencemen. However, it seems unlikely to me that a goalie would perform worse while shorthanded or on the power play simply because of a low overall number of shots against in the game.

I had a theory that one of the reasons that goalies often put up low save percentages when not facing very many shots was that they faced a higher percentage of power play shots. It turns out that the exact opposite was true, at least in 2009-10. One of the reasons that those teams took few shots in the first place was that they did not tend to have very many power play opportunities. The high shot teams had a situational distribution that is very similar to the overall average.

It seems apparent that save percentage has an impact on shots against. Teams that posted high shot totals tended to have very low shooting percentages early in the game, while the opposite is true for teams with low shot totals. There does appear to be some relationship between high shots against and a higher save percentage, although it is still somewhat uncertain how much that has to do with a potential shot quality effect and how much the shots against are caused by the high save percentage.

A very high shots against total tends to come in games where one team is significantly better than the other and where the better team has some incentive to keep playing for 60 minutes (i.e. the score is close or the team being outshot has the lead). That is usually only possible when the goalie on the weaker team is having a good day. After all, if the better team gets 3 or 4 goals and builds a comfortable lead then they usually shut it down somewhat and don't end up hitting the 40+ shot range.

It may be possible that goalies on weak teams get more chances to pad their stats with 40+ shots against games where a lot of the shots are of relatively low quality. It is possible that this counterbalances the likelihood that bad teams would tend to give up higher numbers of scoring chances against. I think more analysis needs to be done to determine whether there is an overall effect. All studies I've seen that look at seasonal averages for all teams show no relationship between save percentage and shots against, so I don't doubt that for most teams it all comes out even in the wash, but it remains to be seen whether there are unusual effects on outlying teams such as the Panthers or the Devils.

Shutouts and Shots Against

Martin Brodeur is the all-time shutout leader in professional hockey, having surpassed both Terry Sawchuk's NHL record and George Hainsworth's professional record earlier this season. With his blanking of Florida last night Brodeur now sits at 108 for his career.

Holding the opposition off the scoresheet 108 times (and counting) is a terrific achievement. The vast, vast majority of hockey goalies who have ever played the game would not have achieved that mark even if placed in exactly the same situation with exactly the same opportunities, and most of them would likely not have come close.

However, it possible that there are a few who would have not only come close, but perhaps even surpassed Brodeur's mark. We'll never know for sure, because everyone is affected by situational factors and it's impossible to completely simulate any "what-if" scenario, but as with any record or achievement it is important to properly establish the context.

Shutouts depend on two variables: The quality of a goalie, and the quality of the team's defensive play. Defensive play can be split into two sections, the quantity of shots against and the quality of shots against. Evaluating shot quality is still a bit of a tricky issue but shot quantity is a simple matter of record, with the small caveat that there is good reason to believe that the counts are not completely consistent from rink to rink.

The goalie reference site HockeyGoalies.org has game-by-game breakdowns for every NHL goalie from 1985-86 to 2008-09. I chose to look at the period from 1993-94 to 2008-09, which encompasses essentially the entire career of Martin Brodeur. I chose for my sample all the goalies who rank on the current top 10 list for active leaders in career games played or career shutouts. To that group I added Hasek, Roy, Belfour, Joseph and Kolzig to make up a sample of 17 top-class netminders.

I looked at every game in which any of the goalies played at least 56 minutes and got the decision (win, loss, tie, or OT/SO loss). Here is a chart of the shutout frequency for the group. On the X-axis is the average number of shots per period, rounded off to the nearest one (e.g. 8 shots against per period means 23-25 shots against per game). The reason for grouping the shots in this manner was to increase the sample size for each data point on the chart. On the Y-axis is the percentages of games that ended in a shutout. The relationship between shots against and shutouts is very obvious. Facing fewer shots against helps a goalie record significantly more shutouts.



Having established that, we can move on to the individual breakdowns. Let's begin with my favourite comparison, Dominik Hasek vs. Martin Brodeur. Here are the number of times that each of them has faced a specific number of shots per period, and how many shutouts they have recorded in those chances.

2 shots/pd: Hasek 0/0, Brodeur 0/0
3 shots/pd: Hasek 0/1, Brodeur 1/2
4 shots/pd: Hasek 2/4, Brodeur 1/9
5 shots/pd: Hasek 5/19, Brodeur 7/37
6 shots/pd: Hasek 5/46, Brodeur 12/92
7 shots/pd: Hasek 13/69, Brodeur 19/156
8 shots/pd: Hasek 13/84, Brodeur 18/192
9 shots/pd: Hasek 7/85, Brodeur 23/171
10 shots/pd: Hasek 13/113, Brodeur 8/112
11 shots/pd: Hasek 12/89, Brodeur 5/72
12 shots/pd: Hasek 7/67, Brodeur 5/49
13 shots/pd: Hasek 2/30, Brodeur 2/30
14 shots/pd: Hasek 1/16, Brodeur 0/9
15 shots/pd: Hasek 0/10, Brodeur 0/2
16 shots/pd: Hasek 0/3, Brodeur 0/2
17 shots/pd: Hasek 0/3, Brodeur 0/0
18 shots/pd: Hasek 0/0, Brodeur 0/0
19 shots/pd: Hasek 0/1, Brodeur 0/0

On a percentage basis, Hasek has the better shutout percentage at 7 different shot levels while Brodeur has the edge 4 times. What is more noticeable is that the two goalies faced very different shot distributions. Brodeur's curve is centered around 8 while Hasek's chart peaks at 10. By multiplying the probabilities we can estimate that a typical goalie in the sample would be expected to record 53 shutouts facing Hasek's shots and 88 shutouts facing Brodeur's. That means on a relative basis Hasek was 51% better than the rest of the group compared to 15% for Brodeur.

The difference is further revealed when we take Hasek's rates and apply them to Brodeur's games, and vice versa. Hasek with Brodeur's shots against would be expected to record a whopping 125 shutouts. Brodeur with Hasek's shots against would be expected to end up with just 60.

To be fair to Brodeur, it isn't usually flattering to any goalie to be compared against the Dominator. Furthermore, there is some evidence that Brodeur prevents shots and that his home rink may have been a bit cheap in recording shots against. Let's give him the benefit of the doubt, and shift his rates by 1 shot/period (i.e. crediting him for 3 extra shots against per game). I'd say that's very probably overstating the effect, but I'll be conservative. That lowers the expected shutout numbers to 76, meaning that Brodeur outperforms by 33%. That's a solid mark, but still short of Hasek.

To make up the full difference, we would have to assume that Hasek was the opposite of Brodeur, that he creates an extra shot against per period. There is nothing at all to suggest an effect anywhere close to that large, but if Hasek's rates are shifted by 1 shot/period in the opposite direction we get the following:

Hasek: 80 shutouts, 62 expected, +29%
Brodeur: 101 shutouts, 76 expected, +33%

Even with an edge of up to six shots per game, Brodeur barely beats out Hasek. That's not taking into account the number of power play shots against and the quality of the defences in front of each goalie. Brodeur is the all-time shutout leader, but I think it's fair to say that there was at least one guy who was still clearly better at shutting out the other team.

Sample size is a potential issue with this analysis. For example, Roberto Luongo only had 3 games where he faced an average of 5 shots against per period or less, and he did not record a shutout in any of them. However that is far too few games to tell whether Luongo has any unusual performance patterns against that level of shots against.

It may be better to raise the sample size by consolidating the shot levels even further. The average shutout frequencies created some natural pairings, since as it turns out 6/7, 8/9, 10/11, and 12/13 all have shutout percentages within 1% of each other. I'll also include 4/5 as a grouping, since shutouts occurred 21% of the time with 4 shots per period and 17% of the time with 5. I excluded everything below 4 or above 13 because those events were so infrequent and therefore likely not useful for predictions. Here are the shutout results compared to expected for all the goalies in the sample, for that particular shot range only, ranked in order of performance above average:

Hasek: 79 SO, 51 exp, +55%
Luongo: 45 SO, 32 exp, +41%
Nabokov: 48 SO, 39 exp, +23%
Brodeur: 100 SO, 88 exp, +14%
Lalime: 35 SO, 32 exp, +9%
Roy: 46 SO, 43 exp, +7%
Belfour: 62 SO, 59 exp, +5%
Giguere: 32 SO, 31 exp, +3%
Kiprusoff: 30 SO, 29 exp, +3%
Turco: 36 SO, 38 exp, -5%
Vokoun: 29 SO, 32 exp, -9%
Joseph: 47 SO, 54 exp, -13%
Osgood: 48 SO, 57 exp, -16%
Khabibulin: 38 SO, 47 exp, -19%
Theodore: 27 SO, 34 exp, -21%
Roloson: 23 SO, 31 exp, -26%
Kolzig: 35 SO, 51 exp, -31%

"Average" as defined here really represents something well north of the true league average, since the only guys in this sample are top-flight goalies. That just makes Hasek and Luongo's numbers even more impressive. Factor in shot prevention and/or scorer bias and goalies like Belfour and Brodeur move up a bit as well. It's possible Brodeur should be in third place instead of Nabokov, all things considered. I think it's fair to say that Brodeur has demonstrated a very good shutout ability, and part of the reason that he doesn't have as many high-shot shutouts as other goalies is that his teammates were much less likely to give up that many shots against.

That's not too bad of a list in terms of the order, but it does seem that there are some team effects behind the numbers. Some teams are probably more likely to create easy shutouts for their goalies. Compare, say, Patrick Lalime's numbers against Curtis Joseph's. In any event, this reinforces earlier work I've done on this topic that show Hasek and Luongo were terrific at recording shutouts. Keep in mind however that while shutouts contain some information they are still a bit of an arbitrary stat. There is certainly a lot more to goaltending than shutouts.

Puck Prospectus: The Best Goalie in 2009

Here's a link to my Puck Prospectus article on the best goalie in the calendar year of 2009. Spoiler alert: It's the guy I said wasn't worth his contract in this post from the summer of 2008. Whoops. Actually, in my defence, I said that he had to improve on his play to be able to justify his paycheque, which I think was probably a true statement at the time, and Miller certainly appears to have done that over the last 12 months.

I just wanted to throw in a few comments about the Sabres and shot quality here, since they are relevant to Miller's performance. The debate continues over the significance of shot quality measures and as such I think it's a topic that is worth investigating. I'd say that the immediate post-lockout Sabres are one of the most interesting case studies for shot quality analysis, because there seems to be a discrepancy between what one would intuitively expect and what the shot quality metrics claim. The shot data says the Sabres have tended to allow easier than average shots against, but with the offensive style that Buffalo played up until this season I think the general perception was that Miller was often being hung out to dry.

I wonder if a higher percentage of shots and goals against came on the rush for the Sabres in those seasons. The team was certainly shooting the lights out themselves, which indicates that they were creating some pretty good chances, and several of their players reached scoring levels that they hadn't reached before and haven't matched since. That suggests some kind of team effect, and it seems reasonable to me that a high-event team would see the percentages go up at both ends of the ice, just like a conservative low-event team would likely make the job easier for their own goalie in exchange for creating less offence at the good end of the rink.

I also suspect that Buffalo didn't play much to the score defensively, especially at home, and kept pushing for goals. From 2005-06 to 2007-08, the Sabres scored 3.67 goals per game at home and allowed 2.84 against. On the road, they scored 3.04 and allowed 2.89. That's a 21% scoring jump at home with almost an identical defensive record. Compare that to an average team, which scores 11% more at home and allows 11% fewer goals against. In 2006-07 the Sabres won the President's Trophy while allowing the fourth-most third period goals against of any team in the league. To put that into context, the last two President's Trophy winners both allowed the fewest third period goals against in the league. At the same time, the Sabres were also shooting the lights out late in the game, as from 2005-06 to 2007-08, Buffalo's lowest rank in third period goals scored was 4th.

This year Buffalo is tied for third in fewest goals against in the third period. Buffalo's third period goal distribution (+41/-29) is in fact identical to that of the New Jersey Devils. Without knowing the shot totals it is tough to say how much the goaltending has to do with that, but that's quite uncharacteristic for post-lockout Buffalo. Over the past three seasons the most similar team to the Sabres in third period scoring has been the Carolina Hurricanes, not exactly a team that anyone would confuse with the Devils. The Sabres are also 17-0-0 when leading after two periods in 2009-10, which is quite an improvement given that closing out games is something that the team has been pretty mediocre at over the last three seasons.

Perhaps there is a shot quality effect that isn't being accounted for in terms of rush chances. Time and space affects shot quality, both in terms of allowing the shooter to make a better shot as well as giving him more available options for the defensive team and goalie to worry about and try to defend against. We know that power play shots are more likely to go in than even strength shots from the exact same spot on the ice. That is likely because the shooter has more time and space to make their shot, and that the power play team can use quick puck movement to create a more difficult scoring chance for the goalie. Those same factors are why odd-man rushes are also dangerous scoring chances.

It wouldn't be possible to identify every rush chance from the play-by-play records, but I wonder if goalies see their save percentages drop if they face a shot against within, say, 8 seconds of a shot at the other end of the rink (adjusted for scoring location, of course)?

I'd say that the statistical evidence suggests that Miller has improved his game lately from where he was a few seasons ago. He looks better to my eye this year as well. It remains possible that there were some team factors that exaggerated his apparent recent jump from good to great. I would welcome the input of any Sabres fans if they have any additional insights on the post-lockout Sabres' play, shot quality, and its possible impact on Ryan Miller.

Olympic Goaltending

The provisional Olympic rosters have been announced, and here are the goaltenders for each country (listed in alphabetical order):

Canada: Martin Brodeur, Marc-Andre Fleury, Roberto Luongo
Russia: Ilya Bryzgalov, Evgeni Nabokov, Semyon Varlamov
Sweden: Jonas Gustavsson, Stefan Liv, Henrik Lundqvist
United States: Ryan Miller, Jon Quick, Tim Thomas
Finland: Niklas Backstrom, Miikka Kiprusoff, Antero Niittymaki
Czech Republic: Ondrej Pavelec, Jakub Stepanek, Tomas Vokoun
Slovakia: Peter Budaj, Jaroslav Halak, Rostislav Stana
Switzerland: Martin Gerber, Jonas Hiller, Tobias Stephan
Germany: Dennis Endras, Thomas Greiss, Dimitri Patzold
Norway: Pål Grotnes, Andre Lysenstøen, Ruben Smith
Latvia: Edgars Masalskis, Ervins Mustukovs, Sergejs Naumovs
Belarus: Vitali Koval, Maxim Malyutin, Andrei Mezin

No real surprises among any of the top countries, at least among the guys who are actually going to be on the ice in Vancouver. Canada's top three picks were pretty obvious. Finland had some options with the top-level goalies they have available (e.g. Pekka Rinne, Tuukka Rask), but going with a tandem of Kiprusoff/Backstrom wasn't too surprising. Sweden could have picked one of their other experienced NHL goalies as their #2 or #3, someone like Johan Hedberg, but I expect Lundqvist will play nearly every game and certainly every game of consequence. Similarly it could be argued that Craig Anderson has been better than Jon Quick, but it really doesn't matter to anyone other than those two guys since the Americans' #3 goalie will be stuck behind last year's Vezina winner and this year's Vezina frontrunner.

Assuming Halak and Hiller get the majority of the starts for Slovakia and Switzerland, all of the top 8 countries have above-average NHL puckstoppers. I looked at the save percentages for the expected starters or starting tandems over the last three seasons plus this one so far (save percentages are adjusted for game situation, using factors of 80% for EV SV% and 20% for PK SV%):

1. Jonas Hiller, SUI .921
2. Niklas Backstrom, FIN .921
3. Roberto Luongo, CAN .921
4. Tomas Vokoun, CZE .920
5. Martin Brodeur, CAN .920
6. Tim Thomas, USA .919
7. Henrik Lundqvist, SWE .917
8. Jaroslav Halak, SVK .917
9. Evgeni Nabokov, RUS .914
10. Ryan Miller, USA .914
11. Miikka Kiprusoff, FIN .913
12. Ilya Bryzgalov, RUS .913

However, the bottom four goalies on that list have all been very good so far this year. In fact, if we rank the same goalies by their performance this year, we get an almost inverted list:

1. Ryan Miller, USA .932
2. Evgeni Nabokov, RUS .928
3. Ilya Bryzgalov, RUS .928
4. Miikka Kiprusoff, FIN .928
5. Jaroslav Halak, SVK .924
6. Henrik Lundqvist, SWE .921
7. Tomas Vokoun, CZE .921
8. Roberto Luongo, CAN .919
9. Tim Thomas, USA .917
10. Martin Brodeur, CAN .917
11. Jonas Hiller, SUI .914
12. Niklas Backstrom, FIN .910

Combining both metrics by taking the average of the two, to get a mix of a goalie's track record and their current level of play, we see how close the goaltending probably is (numbers are for the highest-ranked goalie only from each country):

1. USA .923
2. Russia .921
3. Czech Republic .921
4. Slovakia .921
5. Finland .921
6. Canada .920
7. Sweden .919
8. Switzerland .918

I don't necessarily agree with that exact order, I'm especially skeptical as to whether the Russian goalies can maintain their hot starts, but I think the general point is fairly clear that none of the top countries can expect to have much of a goaltending edge in Vancouver. The U.S. team might have a slight advantage if Ryan Miller can maintain his outstanding play so far this season, but over a short tournament that's still probably an expected difference of a goal or two saved compared to everyone else. Hockey games are usually decided by the skaters anyway, but that should be even more true at these Olympics where everyone has a more than capable backstop. It is certainly possible that some goalies will get hot or cold and end up deciding a key game in February, but at the moment it's little more than a guess as to which goalies that might be, if any. To borrow a quote from the movie The Incredibles: "When everyone's super, no one will be."

(Just a little postscript for Canadian fans, Marc-Andre Fleury's numbers are .911 over the past 3+ seasons and .903 this year. He may impress the type of crowd that ranks goalies entirely based on playoff wins and team success, but I certainly hope Canada doesn't entertain any thoughts of actually putting Fleury into a game.)