Saturday, December 31, 2011

Good Enough to Win

One thing that is often discussed about goalies by hockey fans is whether they are "good enough" to win a Stanley Cup. This is usually based on things like gut feel and extrapolating huge amounts of significance from small sample sizes; I have rarely seen anyone try to quantify what good enough to win a Cup actually means.

First of all, winning is obviously a team result. The talent and luck necessary for a goalie to win a Cup on the Detroit Red Wings is a lot different than what is required on the New York Islanders. The historical record shows that only strong regular season teams win Cups, so we need to look at the kind of numbers goalies put up on those teams. That said, hot goaltending does have an impact. At some point even a good team is going to run against other strong opponents in their quest for the Cup, and in those games superior goaltending can become a tiebreaking factor (e.g. the 2011 Cup Finals).

Here are the save percentages for each Cup-winning goalie in the save percentage era, adjusted to league average and normalized to the current average save level (.911):

1984: Grant Fuhr, .933
1985: Grant Fuhr, .925
1986: Patrick Roy, .946
1987: Grant Fuhr, .932
1988: Grant Fuhr, 913
1989: Mike Vernon, .930
1990: Bill Ranford, .934
1991: Tom Barrasso, .937
1992: Tom Barrasso, .926
1993: Patrick Roy, .945
1994: Mike Richter, .933
1995: Martin Brodeur, .934
1996: Patrick Roy, .931
1997: Mike Vernon, .932
1998: Chris Osgood, .923
1999: Ed Belfour, .933
2000: Martin Brodeur, .933
2001: Patrick Roy, .940
2002: Dominik Hasek, .923
2003: Martin Brodeur, .936
2004: Nikolai Khabibulin, .933
2006: Cam Ward, .928
2007: J.S. Giguere, .927
2008: Chris Osgood, .932
2009: Marc-Andre Fleury, .911
2010: Antti Niemi, .910
2011: Tim Thomas, .939

Those numbers are quite consistent. Twenty out of 27 goalies had an adjusted save percentage between .923 and .937. The average was .930 on 582 shots against. The only goalies to win the Cup with average save numbers were Fuhr in 1988, Fleury in 2009 and Niemi in 2010. Patrick Roy was the only Cup winning goalie to record a number of .940 or better, which remarkably he managed to do in each of his three Conn Smythe winning performances.

The overall averages for Cup Finalists were very similar, with an average of .927 on 572 SA. The distribution was different however. Eight of 27 had an adjusted save percentage of .939 or better, which reflects the fact that there have been a number of weaker teams that needed strong goaltending just to make it to the Final. Eight other Finalists had adjusted save percentages of .917 or worse. Some of those simply benefitted from strong teammates, while others were mostly good for three rounds and then saw their numbers nosedive as results starting going against them during the Finals.

This analysis gives the rough historical rule of thumb that to win a Stanley Cup, you need a goalie capable of putting up a .930 save percentage over 600 shots. That level of statistical performance doesn't guarantee a Cup, a number of goalies have played at an even higher level than that only to see their teams fall short at the final hurdle, but it makes a ring very possible if other variables (scoring, defence, injuries, opposition, etc.) also happen to break right.

Because of variance, nearly any goalie who makes it to the NHL could put up a .930 on 600 shots with enough luck, although it is not very likely to happen for a replacement level goaltender. According to the binomial probability function, a .900 talent goaltender would have a 0.7% chance of putting up that target number on any given 600 shot stretch. That means that even if they saw 600 shots against in every playoff season, they would still need to play in 99 of them to have a greater than 50% chance of going over .930 in one of them. Unless they happen to be astronomically lucky or find themselves playing for a complete powerhouse, I think it is fair to say that in general a .900 talent goalie is not good enough to win a Stanley Cup.

For a .905 goalie, the odds increase to 1.9%, still not even once every 20 times which is the usual cutoff point to determine statistical significance. The point where the probability moves above 5% is when the goalie's talent is .911, which is right about league average. The chances are much greater for one of the league's elite goalies. A .920 talent goalie would play at .930 or better over 600 shots about 20% of the time.

That shows that a top goalie on a contending team would have a reasonably good chance to win a Cup, but that it is still far from a sure thing. Goalies have short careers and often only a short window of opportunity to compete in the playoffs with a true contender. That is why it is not surprising that some elite goalies who conclusively proved their talent over hundreds of regular season games still never managed a deep playoff run. It was not because they weren't good enough to win a Cup, it was because they didn't get the breaks that are required to win a championship.

The idea that most starting goalies in the league are capable of .930 over 600 shots is also backed up by the actual statistical record. There are only seven teams who do not currently employ a goaltender that has put together at least one stretch of consecutive games with a save percentage of .930 or better on at least 600 shots against since 2009-10. Even among those seven, five of them have goalies who very narrowly missed the cutoff:

Reimer, Toronto: .929 on 622 SA
Theodore, Florida: .929 on 608 SA
Lehtonen, Dallas: .929 on 581 SA
Varlamov, Colorado: .931 on 563 SA
Brodeur, New Jersey: .929 on 603 SA

Some might quibble with the relatively unproven Reimer or Varlamov, or even the aging Brodeur, but I'll count all five as being close enough. That leaves just Columbus and Tampa Bay. Steve Mason's best stretch since '09-10 was .923 on 571 shots, while Roloson's best in the last two and a bit regular seasons was .926 on the same 571 SA. However, these numbers are from the regular season only; if playoff numbers count then from March 29 to May 19 Roloson had a .931 on 623 shots against.

Despite his hot spring last year, I think Roloson's age makes it unlikely that he will regain that form, and Mason's recent track record is simply not very good at all. I think it is probably fair to say that the Blue Jackets and Lightning may not currently have goaltending that is good enough to win a Cup. Other than that, every other team has a goaltender that has shown they can play at a high enough level for a long enough period of time that while they may not be a good bet to provide that level of goaltending during a playoff run, they would at least have an outside chance at winning a Cup if they were fortunate enough to be a member of a contending team.

This discussion should make it quite clear that evaluating a goalie's ability to win a Cup based on their past history is a woefully inadequate method. Nearly every starting goalie in the league is good enough to win a Cup in the right situation, and for a typical playoff team any netminder that is average or better should probably be considered good enough to win if they can just get on a hot streak at the right time. However, in any individual playoff season the odds would still be very much against them, even for an elite goalie on a contending team.

The only caveat to this analysis is that it seems likely that the threshold for a Cup winning goalie has dropped even further since the lockout. As Copper 'n Blue pointed out recently, there haven't exactly been huge gains in parity even with the salary cap system. As a result, goaltending performance has become less decisive than in previous seasons. Four out of six post-lockout goalies have been below .930, with two of them well below that mark. The numbers have dropped for Cup Finalists as well, who have averaged .923 since 2006. I'd estimate that the threshold is probably closer to .925 than .930 in today's NHL, which would leave only Columbus as a team without a goalie with the proven ability to perform above that level.

5 comments:

ILR said...

Could you give some examples of very high/low adjusted save percentages on runner up teams? I'm interested to know at what level the goaltenders of some classic underdog finalists such as Anaheim 2003 or Buffalo 1999 needed to play at to advance that far.

Also, what about Roy's 1989 playoff run? How close did he get to a fourth .940 mark?

Anonymous said...

Something I've been thinking about: if you are a flawed team trying your best to win a Stanley Cup, is it statistically better to be a good team with bad goaltending (like the '06 Oil or '09 Wings) or a mediocre team with good goal prevention ('06 Flames or '08 Mighty Ducks)?

The Contrarian Goaltender said...

Stanley Cup Finalists:

High: Patrick Roy 1989 (.941), John Vanbiesbrouck 1996 (.941), Olaf Kolzig 1998 (.944), Dominik Hasek 1999 (.941), Arturs Irbe 2002 (.940), J.S. Giguere 2003 (.947).

Pretty much the usual suspects there, with Irbe maybe sneaking in a bit under the radar.

Low: Mike Vernon 1995 (.900), Garth Snow 1997 (.899), Martin Brodeur 2001 (.906)

The Contrarian Goaltender said...

I'd take a good team with poor goaltending over a mediocre team with good goal prevention. See this article for why a team's ability to generate offence is crucial to playoff success.

Hostpph said...

oh I recall that game, it was so good because my team won and I watched at my best friend's place and we enjoyed the victory a lot