Goaltending is All About Winning

OK, I'm finally caving in to the commentors and anonymous angry emailers. Let's look at the wins. That's what a lot of people say about goalies - stats are nice, sure, but does he win? I am always amused by the insinuation that goalies aren't actually helping their teams by making a lot of saves, but in a sense I do agree with this point because winning is indeed the point of the game. The reason that wins are not very good for evaluation purposes is not that they are unimportant, it is simply that there are too many other factors involved. OK then, so let's try to remove some of those factors and look at wins from a bit of a different perspective, and that will maybe allow us to more clearly see who did the best job of winning games.

I like to look at backup goalie stats to try to develop a team context and as a point of comparison for the starting goalie. Now I'm going to take it one step further, to compare how the backups did on other teams compared to how they did when playing with a particular starting goalie. In that way, we can identify the team effect in terms of winning, and then adjust the goalie's record to take that into account.

Here's an example: a goalie has a .600 winning percentage, and his backup goalies are at .550. Let's say that when they played on other teams, the goalie's teammates had a combined winning rate of .450. That means they were probably below average goalies (assuming there is a fairly large sample size of minutes played). We can take that into account by normalizing the backup numbers to a league average of .500 (.550*.500/.450), and come up with an adjusted backup goalie win percentage of .611. This means that the goalie was actually probably slightly worse than his backups in terms of winning games, and the main reason he outplayed them was because they were bad goalies, not that he was anything special. This also suggests that the main reason for his high win rate was not great goaltending, but a very good team.

One of the problems of this type of study is that you run into era effects. Some goalies played most of their careers in the 1980s, and so if I am comparing their career results against just the seasons when they played with certain other goalies in the mid- to late-'90s, it skews the results. This does not matter so much for wins, of course, because a win in 1975 is the same as a win in 1999 (although shootouts do obviously affect the post-lockout period). However, when I did the number-crunching I was looking at a few other statistics as well that were more impacted by era, so I decided to limit the starter/backup comparisons to only results that took place during the study goalie's career. For example, when evaluating Hasek based on how much he won with Fuhr compared to without Fuhr, I only took Fuhr's seasons from 1991-92 to 1999-00 since they were concurrent with Hasek, and ignored the prior years.

The only other thing to adjust for here is when the winning percentage of the backups is compiled, it is important to adjust for minutes played with the study goalie. Otherwise you could have misleading results, for example a goalie that only spent part of a season as a teammate of the study goalie and had a long career elsewhere would affect the aggregate of the "other teams" numbers much more than he would the "same team" numbers. By weighting based on minutes played, we can avoid this issue.

So who is the clutchest, winningest goalie? I looked at a group of the top starters of the '90s and '00s, and came up the following top 10 list (with quite a surprise at the top):

1. Arturs Irbe, +0.125
2. Dominik Hasek, +0.123
3. Roberto Luongo, +0.118
4. J.S. Giguere, +0.098
5. Marty Turco, +0.068
6. Ed Belfour, +0.067
7. Evgeni Nabokov, +0.057
8. Curtis Joseph, +0.049
9. Ron Hextall, +0.029
10. Mike Richter, +0.019

Hasek and Luongo are no real surprise, but if you really want a clutch winner, go with Arturs Irbe, a guy you can always count on to make the key saves at key times in the game! The results do suggest that it is easier for a goalie to make an impact on mediocre teams, where a few extra saves can turn losses into wins (or vice versa). On a good team, the team will often win regardless of how well the goalie plays, so it is harder to have the same marginal impact.

I do think Arturs Irbe is a very underrated goalie though. He played almost his entire career on bad teams (the average increase in GAA when his teammates played with Irbe compared to when they didn't was 0.70, and their average winning percentage was .344 with Irbe, .480 without). All things considered, he did very well.

There were two large omissions on the list, of course, Martin Brodeur and Patrick Roy. I'll get back to Brodeur in a second, but the biggest surprise of my little study was that Patrick Roy was probably a lot more of a team creation that I ever realized. Roy's winning percentage was .069 better than his backups, but his backup goalies had just a .400 winning percentage everywhere else. This results in an adjusted winning percentage that was .068 worse than his backups, which was actually the lowest result of any goalie I looked at.

To get a better picture of Roy, I split his results into the Montreal and Colorado samples, which also roughly coincide with high-scoring and low-scoring periods in the NHL. In both cases, Roy's stats were impressive, but his backups did far better when they were his teammates than when they weren't. The team effects, calculated by finding the difference between what the goalies did when playing with Roy and when not playing with him were larger than any other goalie I looked at, both in Montreal (+0.241 win %, -1.16 GAA, +.026 save %) and Colorado (+0.080 win %, -0.62 GAA, +0.018 save %). Roy's Montreal playing partners Brian Hayward, Steve Penney, Doug Soetaert, Jean-Claude Bergeron, and Rolie Melanson were generally awful in the minutes they played on teams other than Montreal.

On a raw basis compared to his backups, Roy was roughly the equivalent of someone like Belfour, Turco or Giguere, but after factoring in the skill level of his playing partners Roy was way worse. I don't particularly trust the Montreal numbers; for example, Hayward's numbers outside of Montreal were bad because Winnipeg was an awful team while Steve Penney was a bit like Jim Carey in that he flamed out early in his career. The evidence is that Montreal was certainly a great place to play goalie, but I'm not sure it is fair to say that Roy was an underachiever. In Colorado, Roy was +.045 in adjusted winning percentage over backups, which would have ranked him a little bit below Curtis Joseph in 9th on the above list.

Rankings like these are going to be more relative than absolute - if the stats are close than it means goalies are in a similar range, not necessarily that one is clearly better than the other. But I think it can be reasonably concluded that Patrick Roy had a very large advantage because of the teams he played on.

I expected Martin Brodeur would rank a little higher, but he came out just slightly above average at +.008, which is not too far behind Richter. Brodeur does have a smaller teammate sample size than usual, so his backup results could be more subject to randomness/luck, and Chris Terreri makes the results look a bit worse as well since Terreri played nearly all of his non-New Jersey career on bad teams, primarily expansion-era San Jose (without Terreri included, Brodeur would rank about even with Hextall). The team effects I always keep harping on showed up yet again: When your backup goalies put up a .522 winning percentage, 2.56 GAA and .902 save percentage playing with you and a weighted average of .419, 3.08, .891 when they aren't, you can be reasonably certain that the skaters in front of you are doing something right.

Conclusion: Wins are a team stat. If you want to judge goalies on wins, you have to consider the very heavy impact of team context, and the results will probably be quite different than expected. My analysis indicates that Arturs Irbe and Roberto Luongo are two of the best goalies in recent years in terms of winning games for their teams. Their career losing records are because of weak teams, not their lack of skill or lack of clutch play. As per usual with goaltending evaluations, Dominik Hasek also ranked near the very top.

At the other end of the scale, goalies like Chris Osgood and Mike Vernon were very average in terms of winning games, and even goalies like Patrick Roy and Martin Brodeur were not particularly outstanding given the records and quality of their backups. Nevertheless, fans will of course remember them all for their wins and team successes.