Thursday, January 21, 2010

Shutouts and Shots Against

Martin Brodeur is the all-time shutout leader in professional hockey, having surpassed both Terry Sawchuk's NHL record and George Hainsworth's professional record earlier this season. With his blanking of Florida last night Brodeur now sits at 108 for his career.

Holding the opposition off the scoresheet 108 times (and counting) is a terrific achievement. The vast, vast majority of hockey goalies who have ever played the game would not have achieved that mark even if placed in exactly the same situation with exactly the same opportunities, and most of them would likely not have come close.

However, it possible that there are a few who would have not only come close, but perhaps even surpassed Brodeur's mark. We'll never know for sure, because everyone is affected by situational factors and it's impossible to completely simulate any "what-if" scenario, but as with any record or achievement it is important to properly establish the context.

Shutouts depend on two variables: The quality of a goalie, and the quality of the team's defensive play. Defensive play can be split into two sections, the quantity of shots against and the quality of shots against. Evaluating shot quality is still a bit of a tricky issue but shot quantity is a simple matter of record, with the small caveat that there is good reason to believe that the counts are not completely consistent from rink to rink.

The goalie reference site has game-by-game breakdowns for every NHL goalie from 1985-86 to 2008-09. I chose to look at the period from 1993-94 to 2008-09, which encompasses essentially the entire career of Martin Brodeur. I chose for my sample all the goalies who rank on the current top 10 list for active leaders in career games played or career shutouts. To that group I added Hasek, Roy, Belfour, Joseph and Kolzig to make up a sample of 17 top-class netminders.

I looked at every game in which any of the goalies played at least 56 minutes and got the decision (win, loss, tie, or OT/SO loss). Here is a chart of the shutout frequency for the group. On the X-axis is the average number of shots per period, rounded off to the nearest one (e.g. 8 shots against per period means 23-25 shots against per game). The reason for grouping the shots in this manner was to increase the sample size for each data point on the chart. On the Y-axis is the percentages of games that ended in a shutout. The relationship between shots against and shutouts is very obvious. Facing fewer shots against helps a goalie record significantly more shutouts.

Having established that, we can move on to the individual breakdowns. Let's begin with my favourite comparison, Dominik Hasek vs. Martin Brodeur. Here are the number of times that each of them has faced a specific number of shots per period, and how many shutouts they have recorded in those chances.

2 shots/pd: Hasek 0/0, Brodeur 0/0
3 shots/pd: Hasek 0/1, Brodeur 1/2
4 shots/pd: Hasek 2/4, Brodeur 1/9
5 shots/pd: Hasek 5/19, Brodeur 7/37
6 shots/pd: Hasek 5/46, Brodeur 12/92
7 shots/pd: Hasek 13/69, Brodeur 19/156
8 shots/pd: Hasek 13/84, Brodeur 18/192
9 shots/pd: Hasek 7/85, Brodeur 23/171
10 shots/pd: Hasek 13/113, Brodeur 8/112
11 shots/pd: Hasek 12/89, Brodeur 5/72
12 shots/pd: Hasek 7/67, Brodeur 5/49
13 shots/pd: Hasek 2/30, Brodeur 2/30
14 shots/pd: Hasek 1/16, Brodeur 0/9
15 shots/pd: Hasek 0/10, Brodeur 0/2
16 shots/pd: Hasek 0/3, Brodeur 0/2
17 shots/pd: Hasek 0/3, Brodeur 0/0
18 shots/pd: Hasek 0/0, Brodeur 0/0
19 shots/pd: Hasek 0/1, Brodeur 0/0

On a percentage basis, Hasek has the better shutout percentage at 7 different shot levels while Brodeur has the edge 4 times. What is more noticeable is that the two goalies faced very different shot distributions. Brodeur's curve is centered around 8 while Hasek's chart peaks at 10. By multiplying the probabilities we can estimate that a typical goalie in the sample would be expected to record 53 shutouts facing Hasek's shots and 88 shutouts facing Brodeur's. That means on a relative basis Hasek was 51% better than the rest of the group compared to 15% for Brodeur.

The difference is further revealed when we take Hasek's rates and apply them to Brodeur's games, and vice versa. Hasek with Brodeur's shots against would be expected to record a whopping 125 shutouts. Brodeur with Hasek's shots against would be expected to end up with just 60.

To be fair to Brodeur, it isn't usually flattering to any goalie to be compared against the Dominator. Furthermore, there is some evidence that Brodeur prevents shots and that his home rink may have been a bit cheap in recording shots against. Let's give him the benefit of the doubt, and shift his rates by 1 shot/period (i.e. crediting him for 3 extra shots against per game). I'd say that's very probably overstating the effect, but I'll be conservative. That lowers the expected shutout numbers to 76, meaning that Brodeur outperforms by 33%. That's a solid mark, but still short of Hasek.

To make up the full difference, we would have to assume that Hasek was the opposite of Brodeur, that he creates an extra shot against per period. There is nothing at all to suggest an effect anywhere close to that large, but if Hasek's rates are shifted by 1 shot/period in the opposite direction we get the following:

Hasek: 80 shutouts, 62 expected, +29%
Brodeur: 101 shutouts, 76 expected, +33%

Even with an edge of up to six shots per game, Brodeur barely beats out Hasek. That's not taking into account the number of power play shots against and the quality of the defences in front of each goalie. Brodeur is the all-time shutout leader, but I think it's fair to say that there was at least one guy who was still clearly better at shutting out the other team.

Sample size is a potential issue with this analysis. For example, Roberto Luongo only had 3 games where he faced an average of 5 shots against per period or less, and he did not record a shutout in any of them. However that is far too few games to tell whether Luongo has any unusual performance patterns against that level of shots against.

It may be better to raise the sample size by consolidating the shot levels even further. The average shutout frequencies created some natural pairings, since as it turns out 6/7, 8/9, 10/11, and 12/13 all have shutout percentages within 1% of each other. I'll also include 4/5 as a grouping, since shutouts occurred 21% of the time with 4 shots per period and 17% of the time with 5. I excluded everything below 4 or above 13 because those events were so infrequent and therefore likely not useful for predictions. Here are the shutout results compared to expected for all the goalies in the sample, for that particular shot range only, ranked in order of performance above average:

Hasek: 79 SO, 51 exp, +55%
Luongo: 45 SO, 32 exp, +41%
Nabokov: 48 SO, 39 exp, +23%
Brodeur: 100 SO, 88 exp, +14%
Lalime: 35 SO, 32 exp, +9%
Roy: 46 SO, 43 exp, +7%
Belfour: 62 SO, 59 exp, +5%
Giguere: 32 SO, 31 exp, +3%
Kiprusoff: 30 SO, 29 exp, +3%
Turco: 36 SO, 38 exp, -5%
Vokoun: 29 SO, 32 exp, -9%
Joseph: 47 SO, 54 exp, -13%
Osgood: 48 SO, 57 exp, -16%
Khabibulin: 38 SO, 47 exp, -19%
Theodore: 27 SO, 34 exp, -21%
Roloson: 23 SO, 31 exp, -26%
Kolzig: 35 SO, 51 exp, -31%

"Average" as defined here really represents something well north of the true league average, since the only guys in this sample are top-flight goalies. That just makes Hasek and Luongo's numbers even more impressive. Factor in shot prevention and/or scorer bias and goalies like Belfour and Brodeur move up a bit as well. It's possible Brodeur should be in third place instead of Nabokov, all things considered. I think it's fair to say that Brodeur has demonstrated a very good shutout ability, and part of the reason that he doesn't have as many high-shot shutouts as other goalies is that his teammates were much less likely to give up that many shots against.

That's not too bad of a list in terms of the order, but it does seem that there are some team effects behind the numbers. Some teams are probably more likely to create easy shutouts for their goalies. Compare, say, Patrick Lalime's numbers against Curtis Joseph's. In any event, this reinforces earlier work I've done on this topic that show Hasek and Luongo were terrific at recording shutouts. Keep in mind however that while shutouts contain some information they are still a bit of an arbitrary stat. There is certainly a lot more to goaltending than shutouts.


Navin Vaswani (@eyebleaf) said...

Excellent post.

Bruce said...

Good stuff, CG. Too bad this one slipped off the front page so fast.

Let's give him the benefit of the doubt, and shift his rates by 1 shot/period (i.e. crediting him for 3 extra shots against per game).

I had the exact same thought while reading the initial table. Seems pretty reasonable.

I wouldn't necessarily shift Hasek the other way, he was a pretty aggressive puckhandler if not a real great one. My one question about him concerns the shot counting during the Buffalo portion of his career. I do NOT, however, question his greatness.

I think it's fair to say that Brodeur has demonstrated a very good shutout ability, and part of the reason that he doesn't have as many high-shot shutouts as other goalies is that he and his teammates were much less likely to give up that many shots against.

With that slight amendment I entirely agree with this statement.

Bruce said...

Btw, if I'm reading your data correctly, that recent game in MSG was the first time in Brodeur's entire career that he's faced 50+ shots in a game, let alone record a shutout under such circumstance. Pretty surprising considering how many games he's played and how many leads he's protected.

Not that I believe the MSG shot counter for a New York minute, but that was close to a record breaking performance. To the best of my knowledge the biggest save totals in regular season shutouts are as follows:

53: Craig Anderson
52: Curtis Joseph
51: Terry Sawchuk, Martin Brodeur

Given that Brodeur also made 4 (uncredited) saves in the shootout without a blemish, in one loose sense he's set a new mark. Lundqvist finally cracked on the 104th shot of the game.

I'm tempted to pull the video from the archives and give it a watch, to see how wide open the game actually was. Always have loved those goaltending duels.

Unknown said...

Since there is no way of quantify the quality of shots faced by any of the goalies, this approach is ok but doesn't really mean much.

Anonymous said...

Your a fucking dumb ass hasek sucks fucking dick broduer is the all time leader bitch so get fucked

Anonymous said...

You're a moron, shots against doesn't mean shit because Marty never even let people into the zone. AN ENTIRE RULE WAS CREATED BECAUSE MARTY WAS TOO GOOD. The defense played well because his stick skills were so good that the Devils basically had a 3rd defenseman. Your stats are completely irrelevant he has the most wins and most shutouts.