Goaltending is All About Winning

OK, I'm finally caving in to the commentors and anonymous angry emailers. Let's look at the wins. That's what a lot of people say about goalies - stats are nice, sure, but does he win? I am always amused by the insinuation that goalies aren't actually helping their teams by making a lot of saves, but in a sense I do agree with this point because winning is indeed the point of the game. The reason that wins are not very good for evaluation purposes is not that they are unimportant, it is simply that there are too many other factors involved. OK then, so let's try to remove some of those factors and look at wins from a bit of a different perspective, and that will maybe allow us to more clearly see who did the best job of winning games.

I like to look at backup goalie stats to try to develop a team context and as a point of comparison for the starting goalie. Now I'm going to take it one step further, to compare how the backups did on other teams compared to how they did when playing with a particular starting goalie. In that way, we can identify the team effect in terms of winning, and then adjust the goalie's record to take that into account.

Here's an example: a goalie has a .600 winning percentage, and his backup goalies are at .550. Let's say that when they played on other teams, the goalie's teammates had a combined winning rate of .450. That means they were probably below average goalies (assuming there is a fairly large sample size of minutes played). We can take that into account by normalizing the backup numbers to a league average of .500 (.550*.500/.450), and come up with an adjusted backup goalie win percentage of .611. This means that the goalie was actually probably slightly worse than his backups in terms of winning games, and the main reason he outplayed them was because they were bad goalies, not that he was anything special. This also suggests that the main reason for his high win rate was not great goaltending, but a very good team.

One of the problems of this type of study is that you run into era effects. Some goalies played most of their careers in the 1980s, and so if I am comparing their career results against just the seasons when they played with certain other goalies in the mid- to late-'90s, it skews the results. This does not matter so much for wins, of course, because a win in 1975 is the same as a win in 1999 (although shootouts do obviously affect the post-lockout period). However, when I did the number-crunching I was looking at a few other statistics as well that were more impacted by era, so I decided to limit the starter/backup comparisons to only results that took place during the study goalie's career. For example, when evaluating Hasek based on how much he won with Fuhr compared to without Fuhr, I only took Fuhr's seasons from 1991-92 to 1999-00 since they were concurrent with Hasek, and ignored the prior years.

The only other thing to adjust for here is when the winning percentage of the backups is compiled, it is important to adjust for minutes played with the study goalie. Otherwise you could have misleading results, for example a goalie that only spent part of a season as a teammate of the study goalie and had a long career elsewhere would affect the aggregate of the "other teams" numbers much more than he would the "same team" numbers. By weighting based on minutes played, we can avoid this issue.

So who is the clutchest, winningest goalie? I looked at a group of the top starters of the '90s and '00s, and came up the following top 10 list (with quite a surprise at the top):

1. Arturs Irbe, +0.125
2. Dominik Hasek, +0.123
3. Roberto Luongo, +0.118
4. J.S. Giguere, +0.098
5. Marty Turco, +0.068
6. Ed Belfour, +0.067
7. Evgeni Nabokov, +0.057
8. Curtis Joseph, +0.049
9. Ron Hextall, +0.029
10. Mike Richter, +0.019

Hasek and Luongo are no real surprise, but if you really want a clutch winner, go with Arturs Irbe, a guy you can always count on to make the key saves at key times in the game! The results do suggest that it is easier for a goalie to make an impact on mediocre teams, where a few extra saves can turn losses into wins (or vice versa). On a good team, the team will often win regardless of how well the goalie plays, so it is harder to have the same marginal impact.

I do think Arturs Irbe is a very underrated goalie though. He played almost his entire career on bad teams (the average increase in GAA when his teammates played with Irbe compared to when they didn't was 0.70, and their average winning percentage was .344 with Irbe, .480 without). All things considered, he did very well.

There were two large omissions on the list, of course, Martin Brodeur and Patrick Roy. I'll get back to Brodeur in a second, but the biggest surprise of my little study was that Patrick Roy was probably a lot more of a team creation that I ever realized. Roy's winning percentage was .069 better than his backups, but his backup goalies had just a .400 winning percentage everywhere else. This results in an adjusted winning percentage that was .068 worse than his backups, which was actually the lowest result of any goalie I looked at.

To get a better picture of Roy, I split his results into the Montreal and Colorado samples, which also roughly coincide with high-scoring and low-scoring periods in the NHL. In both cases, Roy's stats were impressive, but his backups did far better when they were his teammates than when they weren't. The team effects, calculated by finding the difference between what the goalies did when playing with Roy and when not playing with him were larger than any other goalie I looked at, both in Montreal (+0.241 win %, -1.16 GAA, +.026 save %) and Colorado (+0.080 win %, -0.62 GAA, +0.018 save %). Roy's Montreal playing partners Brian Hayward, Steve Penney, Doug Soetaert, Jean-Claude Bergeron, and Rolie Melanson were generally awful in the minutes they played on teams other than Montreal.

On a raw basis compared to his backups, Roy was roughly the equivalent of someone like Belfour, Turco or Giguere, but after factoring in the skill level of his playing partners Roy was way worse. I don't particularly trust the Montreal numbers; for example, Hayward's numbers outside of Montreal were bad because Winnipeg was an awful team while Steve Penney was a bit like Jim Carey in that he flamed out early in his career. The evidence is that Montreal was certainly a great place to play goalie, but I'm not sure it is fair to say that Roy was an underachiever. In Colorado, Roy was +.045 in adjusted winning percentage over backups, which would have ranked him a little bit below Curtis Joseph in 9th on the above list.

Rankings like these are going to be more relative than absolute - if the stats are close than it means goalies are in a similar range, not necessarily that one is clearly better than the other. But I think it can be reasonably concluded that Patrick Roy had a very large advantage because of the teams he played on.

I expected Martin Brodeur would rank a little higher, but he came out just slightly above average at +.008, which is not too far behind Richter. Brodeur does have a smaller teammate sample size than usual, so his backup results could be more subject to randomness/luck, and Chris Terreri makes the results look a bit worse as well since Terreri played nearly all of his non-New Jersey career on bad teams, primarily expansion-era San Jose (without Terreri included, Brodeur would rank about even with Hextall). The team effects I always keep harping on showed up yet again: When your backup goalies put up a .522 winning percentage, 2.56 GAA and .902 save percentage playing with you and a weighted average of .419, 3.08, .891 when they aren't, you can be reasonably certain that the skaters in front of you are doing something right.

Conclusion: Wins are a team stat. If you want to judge goalies on wins, you have to consider the very heavy impact of team context, and the results will probably be quite different than expected. My analysis indicates that Arturs Irbe and Roberto Luongo are two of the best goalies in recent years in terms of winning games for their teams. Their career losing records are because of weak teams, not their lack of skill or lack of clutch play. As per usual with goaltending evaluations, Dominik Hasek also ranked near the very top.

At the other end of the scale, goalies like Chris Osgood and Mike Vernon were very average in terms of winning games, and even goalies like Patrick Roy and Martin Brodeur were not particularly outstanding given the records and quality of their backups. Nevertheless, fans will of course remember them all for their wins and team successes.

Shots Against Leaders

The Top 10 in fewest shots against per 60 minutes played (career):

1. Marty Turco, 24.8
2. Roman Turek, 24.8
3. Martin Brodeur, 25.4
4. Roman Cechmanek, 25.6
5. Chris Osgood, 26.1
6. Jamie McLennan, 26.2
7. Evgeni Nabokov, 26.6
8. Ed Belfour, 26.7
9. Patrick Lalime, 26.7
10. Tommy Salo, 26.8

Obviously there are era effects on a list like this. All of these goalies played in the late 1990s, when shots were low. Ed Belfour would probably be in the top 5 if all the numbers were adjusted to league average. Mike Vernon ended up 12th with 26.9, but he played over 10,000 minutes in the 1980s so he would also probably be right near the top in league-adjusted totals. There are a few other goalies who we have partial shot data for and who posted low shots against totals, such as Pat Riggin.

There are a few good goalies on the list, and a few not so good ones. There are a few good puckhandlers on the list, and a few not so good ones. What is driving shot totals, is it puckhandling? Rebound control? Goalie quality? If any of those are factors, they look to be mostly lost in the noise of team effects. Except for Tommy Salo, everybody on the list played most of their careers on outstanding defensive teams. If there is an effect from puckhandling or similar "soft" goaltending skills, it appears to be a small one.

Vezina Trophy 2008

Congratulations to Martin Brodeur on winning the 2008 Vezina Trophy. For the first time in his career, I believe that he was deserving of the award. It was a bit of a wide open race with Luongo having an off year. No goalie was able to particularly distinguish himself in save percentage, as 10 netminders ended up finishing between .920 and league leader Dan Ellis' .924. Both Nabokov and Lundqvist had the type of seasons we have traditionally associated with Brodeur - winning lots of games and posting slightly above average save percentages while facing few shots per game on good teams. Those are not the type of seasons that are generally deserving of a Vezina Trophy, as they tend to be a greater reflection of team defensive strength than outstanding goaltending play. Of the 3 goalies who were nominated I would definitely select Brodeur. Brodeur had a high save percentage on a team that was solid but not outstanding defensively, and likely ended up finishing fairly high in shot-quality neutral save percentage as well. I think the best goalie was either Brodeur or Tomas Vokoun, who had a strong year (.919) while facing the most shots in the league on a weak team in Florida.

Overall, Brodeur faced more shots than usual and especially more power play shots as the Devils were not as disciplined under Brent Sutter (only Kiprusoff faced more shots on the penalty kill this year than Brodeur). It has been primarily Brodeur's play on the penalty kill that is responsible for his improvement in recent years, and that was shown again this year with a .903 save percentage when a man down, 4th best among starting goalies. At even-strength, Brodeur was just 12th among starters. His .932 was actually behind rivals like Leclaire, Luongo and Hasek, and it was penalty killing that ended up making the difference.

Can Brodeur keep it up next year having turned 36? Based on standard career curves, we have to expect some decline as he continues to age, and it is doubtful that his team support will be any better next year. Penalty kill save percentages tend to be more variable than even-strength ones, so it is probable that Brodeur will regress to the mean somewhat in that area next season. Another potential warning sign is that according to Hockey Numbers, Brodeur led the league in save percentage against difficult shots (>20% chance of being a goal) with 74%. He was just 21st against average quality shots, which tend to be the most consistent category for goalies and the best one for ranking. Brodeur did exhibit a similar breakdown last year (and subjectively to me has exhibited that tendency throughout his career of making a great save and then letting in a stoppable one), so he probably has some ability to keep making the toughest stops, but will he retain that ability as he gets closer to 40?

Despite this year's setbacks, Roberto Luongo is waiting in the wings to take over the popular mantle of the best goalie in the league (in the eyes of many he already has), and is likely to return to form next season. Especially if Vancouver's new GM can surround him with talent Luongo is more than capable of having an Hasek-type elite season. If anyone can hold off the pursuit of time you would imagine it could be Brodeur, the NHL's goalie iron man, but his age plus likely worsening team factors plus a prime Luongo indicate that there is a fairly good chance this Vezina trophy will end up being Brodeur's last.

The Masses Weigh in on Goalies

And it looks like I may have to keep spreading my message for quite some time. According to a current TSN poll asking visitors to rank the top 3 goalies of this generation (Hasek, Roy, Brodeur), 54% picked Roy first while 38% picked Brodeur. Just 8% picked Hasek. Not only that, but a full 75% of respondents ranked Hasek 3rd out of the 3. The results are similar in American polls: Voters on NBC Sports have the all-time goalie rankings going 1. Roy, 2. Sawchuk, 3. Brodeur, 4. Hasek, 5. Dryden, while at ESPN it goes 1. Roy, 2. Brodeur, 3. Sawchuk, 4. Hasek, 5. Plante.

A lot of European fans on Internet hockey message boards often throw out accusations of "Canadian bias" or "North American bias". Most of the times these claims are absurd, but I think it may not be too far from the truth in this case. The reason is that many of the arguments against Hasek have nothing to do with his performance, but attack his durability, his flakiness, or an alleged lack of character and reliability. Quite frankly, the reaction towards Hasek from many hockey fans can be described as bitterness. I'm not going to dispute that he was a jerk at times, but I just don't see how that has anything to do with how good he was. Patrick Roy was a jerk as well who bailed on his team in Montreal as well as his country in the 2002 Olympics, but that doesn't seem to stick to him like more minor transgressions seem to stick to Hasek.

I think for many fans the Olympics have had a significant subconscious impact in the way they view the careers of Brodeur and Hasek: love for Brodeur because he won in Salt Lake, dislike of Hasek because he foiled Canada in Nagano. If Hasek had done what he did in 1998 playing on Team Canada, I somehow doubt that he would be seen as taking a clear backseat to the two other guys that he outplayed and owned in terms of individual awards during his career.

There are arguments that can be made for ranking Patrick Roy ahead of Dominik Hasek, primarily based on Roy's excellent playoff career. I don't see them as particularly convincing arguments, but if you place a heavy emphasis on playoff play and career longevity then there is at least a rationale to preferring Roy to Hasek. But I will never understand how people rank Marty ahead of the Dominator. That is like rating Ron Francis ahead of Mario Lemieux. You would probably have been laughed at for taking Brodeur over Hasek in 1999, but for some reason almost a decade later everything has flipped around. As always when evaluating goalies, it should be the goalie's performance that matters not the performance of the rest of their team, and that leads to Hasek-Roy-Brodeur as the only reasonable outcome.

Farewell to the Best Goalie Ever

Dominik Hasek has officially retired from the NHL, most probably for good this time.

As posted many times on this site, I consider Dominik Hasek to be the greatest goalie ever. He won 6 Vezinas and 2 Hart Trophies, and led the league in save percentage for 6 years in a row, even despite competing head-to-head against Patrick Roy. Hasek played with some decent backup goalies in his career (like Fuhr, Roloson, Biron, and Shields), and his numbers still blew them all away. No goalie has even come close to piling up the individual hardware Hasek has without playing on a dynasty or a very dominant team.

All his accomplishments came despite coming over late to the NHL (and not even getting a chance to play right away in Chicago). If Hasek had played another 5-6 full seasons in the NHL, there would probably be no doubt about his all-time ranking.

Here are a couple of my all-time favourite statistics that show how good the Dominator was at his peak, taken from this post comparing Hasek and Brodeur:

From 1993-94 to 2001-02, Dominik Hasek faced 1,060 more shots than Martin Brodeur, and gave up 135 fewer goals.

Looking at only age 29-34 seasons, Hasek faced 1,494 more shots than Martin Brodeur, and still gave up 41 fewer goals.

And here's one last one (St. Louis fans might want to turn away): Between 1995-96 and 2000-01, St. Louis had a strong defence led by Pronger and MacInnis. Buffalo's top pair was Zhitnik/Smehlik. In those six seasons, Buffalo allowed 2,781 more shots against than St. Louis. Yet 40 more goals were scored against the Blues than against the Sabres.

I repeatedly preach the mantra that goalies are usually only as good as the team in front of them. Dominik Hasek is certainly the biggest exception to that rule that I have ever seen. Goalies can't do everything on their own but an elite goalie can have a very large impact, something that is particularly obvious in the career of Dominik Hasek, the best there's ever been.

No to HHOF for Chris Osaverage

The Detroit Red Wings are the 2008 Stanley Cup champions, providing yet more evidence that you do not need an elite goaltender or elite goaltending to win the Stanley Cup. The Wings followed in the long tradition of great teams winning Lord Stanley's mug. In NHL history, only 2 teams that finished the regular season outside of the top 6 in the league have won the championship, and both of those teams (the '91 Penguins and '95 Devils) won another Cup within the next 5 years with a similar cast of players, indicating that they were not flukes.

I've already addressed Chris Osgood's play in these playoffs - he was as good as he needed to be, given the huge advantage of playing on the Red Wings. But there were several players on the Wings who had a lot more to do with their victory than Osgood, led by Conn Smythe Trophy winner Henrik Zetterberg and captain Nik Lidstrom.

But in the course of the Wings' playoff run, a strange idea has been floated around among broadcasters, bloggers and message boarders, the notion that Chris Osgood is a Hall of Fame goalie. See here or here for examples. Some of it is just Red Wings fans being homers, of course, but evaluating Osgood once again gets into the basic question of the importance of team success for a goalie.

I particularly like this Barry Melrose quote from the second link above:

'"Marty Brodeur (of New Jersey) probably saw less shots than Ozzie's seen, with the Devils in their prime, and yet everyone thought he's such a great goaltender,'' Melrose said. "So I don't think Ozzie gets enough respect. He doesn't have to be great. He has to make key saves at key times. He always does that.'''

There are of course two ways to look at that comparison: either Osgood doesn't get enough respect, or the other guy gets too much of it, and I don't have to tell you which side I would take in that debate. And I would love to know when the "non-key times" of the game are when goalies can allow goals against without it having any impact on his team. Osgood "always" makes the key save at the key time in the game? Did Melrose miss game 5 of this series? Does anyone actually believe these ridiculous cliches?

I could write a big long summary of Osgood's save percentages and performance statistics and try to evaluate his team contexts in a quantitative fashion, but instead I'm just going to simply compare what his team did with Osgood in net compared to when he was on the bench:

Winning percentage, Chris Osgood (career): .631

Winning percentage, Chris Osgood's teammates: .639

Osgood has been rotating between the starter and backup roles for most of his career, so the teammates he played with range from outright backup types to future Hall of Famers. Sometimes he was the starter playing against the top opponents, and sometimes he was being sheltered as the backup for somebody else. Because of this, I don't think quality of teammates or opponents is an excuse for Osgood vs. his teammates. Osgood "won" a lot of games, as his fans love to point out, but so did everyone else who played goal for Detroit in the '90s and '00s. At the end of the day, through his long career, his team won slightly more often without him than with him, and that defines him as what he is - an average goalie who played on mostly great teams.

Chris Osgood and his teammates have accomplished a lot during his career, something is becoming apparent to many now that Osgood has spent some time in the spotlight of the Stanley Cup Final. But there could be 10-15 guys of his generation who are as good as or better than he was. That is why team success is mostly meaningless, since the main difference between Chris Osgood and someone like Arturs Irbe is not talent or actual performance, but merely Nicklas Lidstrom and Steve Yzerman.

Esa Tikkanen, John Tonelli, Jean-Guy Talbot, Ross Lonsberry, and Rejean Houle aren't in the Hall of Fame, and you would probably get laughed at for even mentioning them as candidates. I doubt there will be many people championing Jamie Langenbrunner, Adam Foote, or Slava Kozlov either when they become eligible. But the Cheevers/Osgood-type goalie keeps not only getting mentioned, but sometimes even ends up getting voted in. Role players on dynasties or great teams should not be Hall of Famers, whether they are scorers, checkers, or goalies. In an ideal world, only truly dominant individuals should receive the honour, with team success far down the list of factors that determines who gets in and who is left out.