It Doesn't Matter How Good His Teammates Were, He Still Had To Make the Saves

"Hey, I just back from the new shopping mall. Gotta give it up for our mayor, that guy is the best ribbon-cutter in the town."

"He is? How could you possibly go about trying to determine that?"

"Well, I was there, and he was the guy in charge of opening the mall, and it got opened. Can't argue with results."

"OK, but that's kind of a simple thing, isn't it? I mean, any public figure can give a short speech, work a pair of scissors and pose for a few photos. The mayor is just the guy that gets assigned to show up and do it, that doesn't mean he is the best."

"Sure it does, you still have to cut the ribbon. Doesn't matter how easy you think the job is, it still needs to get done. You don't see anybody else cutting the ribbon, do you?"

"No, but that's because it is the mayor's job, not because other people couldn't do it."

"Who cares about other people? It's his job and he did it, that's all that matters, that's why he deserves the credit."

"It is? There were mayors before him that did it just as well and there will be mayors after him that will do exactly the same thing. Why does it matter who actually does it? You don't think that guy that narrowly lost the election two years ago could have done it?"

"I don't want to talk about what-ifs, just what actually happened. Yesterday the mall wasn't open, and today it is. I don't know why you hate the guy so much, why you won't give him any credit, you're probably just jealous. I was there when they opened the new library too, sure he may have flubbed a line or two and dropped the scissors twice but he showed his mental toughness by bouncing right back and when the big moment came he got the job done."

"I'm not arguing that he didn't get the job done. I'm saying that his job was fairly easy, and he did not make much of a difference in doing it. The mall and the library are both just as open right now as they would have been if another politician was wielding the ceremonial scissors. Giving credit for an easily replaceable effort makes little sense at all, you're just being overly impressed by the privileges of the guy's position."

"Obviously you don't understand anything about politics. Since that guy became mayor, he has cut the ribbons at 3 grand openings, while all the other losing candidates in that election have combined for zero. How can you argue with those stats?"

"Again, I'm not disputing that those accomplishments happened. I'm disputing their value. Of course our mayor did the job, but he did it no better than somebody with similar qualifications would have done it. The difference between him and those other guys with respect to grand openings was entirely a matter of opportunity, not a matter of skill. Why is that something to be celebrated? I'm completely baffled here."

"Say whatever you want, he can't hear you because of all of the freshly cut ribbons in his ears. You geeks never give that guy any credit, he's the most underrated mayor ever. Haters gonna hate, I guess. We're done here, I'll catch you later."

The Two Sentence Argument against Chris Osgood

"It's not easy to win in this league, otherwise everybody would be doing it. I don't know how many goalies have played here in the last 15 years that I've been here, but I'm still here and I'm still wanted. That's what matters most and accounts for more than anything else. I'm a winner. That's all I do."
Chris Osgood

On the occasion of Chris Osgood's 400th win in the National Hockey League, I feel compelled to throw up this brief post. Every goalie needs to be evaluated in the context of their team environment. That's especially crucial for those who want to rely on statistics like wins and Cups in their analysis. My best attempt to concisely summarize Osgood's team environment is the following:

Chris Osgood's career record in playoff series where his team had a 20+ point regular season advantage over their opponents:

7 wins, 1 loss

Chris Osgood's career record in playoff series where his team had an advantage of less than 20 points over their opponents:

8 wins, 8 losses

(You might want to pull out a third sentence as well, either to establish context or in case that other guy whips out an anecdote about Montreal beating Washington or some other rare event: Over the course of Osgood's career, teams with a 20+ point advantage are 33-7 in playoff series).

When the Detroit Red Wings were way better than their playoff opponents, they nearly always heavily outplayed and outshot them. It's not too surprising that the stronger team with a big edge in scoring chances usually won.

On the other hand, when his teammates weren't far better than the opposition then it was pretty much a coin flip for Osgood's teams in the playoffs. When two even teams meet, you'd expect them both to win about half of the time. If one of them has a great, game-changing clutch performer then you'd expect his team to win more often, yet that wasn't the case. Throughout Osgood's playoff career in Detroit the Red Wings were mostly the higher-ranked team even in the closer matchups, yet with home ice advantage and a "winner" in net they still lost as often as they won.

The context is further displayed by the results of the other goalies who played in Detroit. Not only was Osgood often relegated to a backup role come playoff time, but overall the Red Wings really didn't suffer with other goalies in net. Here's a third line that really should be added to the two above to paint a complete picture:

Chris Osgood's teams' career record in playoff series where Osgood was sitting on the bench as the backup goalie:

9 wins, 4 losses

Expressing records like this can help drive home the point that certain goalies who won lots of jewellery did not, in fact, outperform others in the playoffs who won none. They merely played for much stronger teams, and when they did not have that advantage on their side then they were not big enough difference-makers to lift their team results above average.

Chris Osgood deserves credit for the determination and competitiveness that have allowed him to beat the odds and carve out a long and lucrative big-league career. Career milestones are a fitting time to show him that appreciation, but any attempt to portray him as one of the best of his peers is misinformed, career win totals notwithstanding.

Follow the Money

A lot of people put stock in what NHL general managers have to say about different players and goalies around the league. It makes sense that these talent evaluators should have a better than average ability to rate hockey players, the problem is that few are going to go on record as being critical of anybody, and sometimes it's difficult to tell if a quote is a bland nicety or a real endorsement. Furthermore, nobody knows a player better than his own team's management group, but in almost all cases they are going to support the player in any kind of a public setting. Vezina voting gives us some indication, but there remains a difference between rating the best goalie of a particular season and the best goalie in general.

When you want to know what people really think, and not just what they say they think, it is usually better to focus on actions than words. That's why I think there is some value in looking at the size of the paycheques of goalies around the league. As with any numerical ranking, it's often good to bring in some outside knowledge to inform the rankings (insane longterm deals, home-town discounts, ELC/RFA/UFA years, salary inflation, to name but a few things that have an impact), but at the end of the day nobody pays anybody millions of dollars a year without expecting significant performance in return. And on the flip side, a general manager can go on for as long as he wants about how the goalie on his winning team is underrated and mentally tough and makes the big saves and is a great teammate, but when he refuses to open the purse strings come free agency time that is a stronger indication that his objective viewpoint is something altogether different.

Using Hockey Zone Plus and Cap Geek, I was able to put together career salary histories for most of the experienced starting goalies since 1990, and then applied a factor to adjust for the league average goalie salary in each season.

Here are the total adjusted earnings figures:

1. Patrick Roy, $76,550,000
2. Dominik Hasek, $55,510,000
3. Ed Belfour, $54,840,000
4. Martin Brodeur, $51,000,000
5. Mike Richter, $49,830,000
6. Curtis Joseph, $49,000,000
7. Tom Barrasso, $36,720,000
8. Nik Khabibulin, $35,340,000
9. Olaf Kolzig, $33,200,000
10. Sean Burke, $32,560,000
11. Jose Theodore, $27,410,000
12. Roberto Luongo, $27,280,000
13. J.S. Giguere, $25,430,000
14. Felix Potvin, $24,520,000
15. Kirk McLean, $24,400,000
16. Bill Ranford, $23,450,000
17. Chris Osgood, $22,940,000
18. Evgeni Nabokov, $21,320,000
19. Marty Turco, $20,920,000
20. Miikka Kiprusoff, $20,350,000

Contract timing and the adjustment factors make it unlikely that this list would come out with a perfect ranking, but it still comes out surprisingly well, especially when you look at tiers rather than specific rankings. The top four are correct, although admittedly very few people would put them in that order. Mike Richter shows one of the flaws of this method, that a goalie who was entrenched for a long time on a large market team with a spend-happy GM is likely to get paid disproportionately to everyone else. It's not too hard to make the argument that Joseph's repeated free agency paydays from several different franchises are more impressive than Richter cashing in on the '94 Stanley Cup run.

Barrasso, Khabibulin, Kolzig and Burke were a solid group of goalies that make up the next tier. Jose Theodore is the surprising leader of the the younger group of goalies, although this is the last year he will be ranked ahead of Roberto Luongo. Luongo projects to end up in Joseph/Richter territory by the end of his lifetime contract in Vancouver, and quite possibly higher depending on your assumptions of salary inflation.

The other interesting guy to point out is #17 on the list. If Chris Osgood was really the #2 goalie of the last decade, as some claim, then he needs to immediately sue his agent for gross malpractice. I think the reality is that, in the vote done with their owners' dollars, the league's general managers just weren't all that impressed by Osgood.

Here's a measure of peak effectiveness, each goalie's best 5 consecutive years in adjusted salary:

1. Dominik Hasek, $29,650,000
2. Patrick Roy, $29,271,000
3. Curtis Joseph, $25,297,000
4. Mike Richter, $23,424,000
5. Ed Belfour, $21,884,000
6. Nik Khabibulin, $21,726,000
7. Martin Brodeur, $21,103,000
8. Olaf Kolzig, $20,926,000
9. Roberto Luongo, $20,651,000
10. Tom Barrasso, $20,328,000
11. Jose Theodore, $18,719,000
12. Marty Turco, $16,580,000
13. Miikka Kiprusoff, $16,201,000
14. J.S. Giguere, $15,915,000
15. Kirk McLean, $15,888,000
16. Evgeni Nabokov, $15,793,000
17. Bill Ranford, $14,604,000
18. Henrik Lundqvist, $14,484,000
19. Tomas Vokoun, $13,705,000
20. Sean Burke, $13,536,000

This time Hasek beats out Roy for the top spot, with the two of them both well clear of the field. I think it's safe to say that Nikolai Khabibulin has had some pretty good representation over the years, while Brodeur likely left a good chunk of change on the table because he preferred to play in New Jersey. A few current goalies are slowly creeping up this list, in a couple of years Henrik Lundqvist will be just outside of the top 10.

Just one caveat, the adjustment is likely over-correcting for post-lockout goalies. With a maximum salary level, with more goaltending talent around the league and with restricted free agents getting paid big dollars earlier than ever before, there is no longer the huge salary disparity between the top veterans and the newcomers that was seen in the late '90s.

GMs and Goalie Stats

The 1994-95 NHL regular season was very unusual. A lengthy lockout shortened the schedule to just 48 games per team, and to make the scheduling work every team only played against opponents within their own conference. The condensed schedule also gave teams fewer off-days, meaning they had less of an opportunity to catch up with the goings-on around the rest of the league.

The short season made it tough for all awards voters to identify the league's best, but it had to have been especially difficult for the league's general managers required to vote on the 1995 Vezina. Many of them would not have seen much at all of the other conference, and likely would only have seen the best goalies in their own division 3-4 times. Based on this I expect that 1994-95 was probably the season where GMs were most likely to rely on statistics when filling out their award ballots.

If that's true, then looking at the voting results should give some insight on how general managers rate goalies based on their performance numbers (the full Vezina ranking can be found here). How did they fare? In my opinion, not very well. Most of them voted for Dominik Hasek, but that shouldn't have been a very difficult choice at all given that Hasek was the defending Vezina winner and led the league in GAA, shutouts and of course save percentage, where he crushed the field by .013. The other rankings were less clear, and some of the choices left something to be desired.

In particular, there were three voters that seem to have completely failed the test, the trio of league decision-makers who rated Mike Vernon as the best goalie in the league.

Compare the stats:

Vernon: 19-6-4, 2.52, .893, 1 SO
Hasek: 19-14-7, 2.11, .930, 5 SO

I really hope those were Western Conference GMs who just saw Vernon really good against their own teams and didn't feel comfortable ranking Eastern goalies that they hadn't seen play. Even in that event they still made a very poor decision, but it would have been completely embarrassing for them if they actually looked at everyone's numbers and decided that Vernon had the best season based on wins and losses.

Detroit had a terrific team as usual that season, with a defensive unit led by Nicklas Lidstrom, Paul Coffey, Slava Fetisov and Vladimir Konstantinov, plus their usual strong group of forwards. Backup Chris Osgood actually had better numbers than Vernon (14-5-0, 2.26, .917). On the other hand, when Hasek wasn't in net Buffalo's goaltending numbers were 3-5-0, 3.86, .864.

Any given shot was 67% more likely to go in against Vernon than against Hasek. Vernon allowed 3.21 goals per 30 shots against compared to Hasek's 2.10. If you were to swap Vernon's goal support in Detroit with Buffalo's goalscoring during Hasek's first 30 games of the season (excluding one short relief appearance), it would have had a dramatic effect on the records of both goalies. Vernon would have had a losing record at 11-15-4, while Hasek would have improved to a spectacular 22-3-5. And that's based on raw goals against without even taking into account the fact that Vernon faced nearly 7 fewer shots against per game.

It wasn't even remotely close, that's the point I'm trying to make.

Looking at the Vezina votes overall, they ended up being split around a number of goalies, which was perhaps to be expected given the peculiar circumstances. In addition to Hasek and Vernon five other goalies received at least one first place vote, only two of whom ended up in the top 10 in save percentage, and 15 goalies ended up with at least one vote. No other season has ever seen more different goalies get named on Vezina ballots than 1994-95.

If you remove Hasek from the sample, since he was so obviously the best goalie that even a voter who never saw him play and was just going by traditional numbers like GAA or shutouts should still have been able to figure out that the Dominator was the most deserving, here are the correlations between the Vezina voting and other statistics among all other goalies with 20+ games played that season:

GAA: -.591
Win Percentage: .589
Wins: .536
Shots/60: -.534
Shutouts: .455
Save Percentage: .290

That makes it seem pretty apparent that most early '90s NHL general managers rated goalies based on wins and the strength of the defence in front of them. That's scary stuff, and is yet another reminder that we should be cautious when using historical Vezina voting results to rank goalies.

The Swedes Have It Right

Do you want to know who is leading the Swedish elite league in goaltender wins this season? I don't know, and as far as I can tell there's no way to find out on their official website. They simply omit wins as a stat category.

If you want to sort Swedish goalies by statistics, you can only sort them by save percentage, GAA, total shots against and total saves against. That's it. They apparently don't seem to care that they aren't providing us with the crucial information that lets us know whether league-leader Victor Fasth is a "martyr goaltender" pointlessly racking up numbers on a bad team, or whether #2 ranked Alexander Salak really knows how to make the big saves in the clutch. How are us North Americans supposed to make snap judgments about these goalies' characters and their ability to perform under pressure when they are leaving out the crucial information of how good their teammates are?

It's quite possible there is some other way to find out which Swedish goalie has the most wins, but I just like the fact that they obviously don't consider that to be of any importance, given that they don't even bother to show the numbers.

Same thing with the International Ice Hockey Federation. Go to their website, and you won't find win/loss records either. Not only that, but like the other European sites their default sort is by save percentage (so much so that the link you have to click on to see goalie numbers is called "Goalkeepers (SVS%)", as shown here at the site for the 2010 Olympics). In contrast most North American sites will rank by GAA first, and even the NHL's own stats summary page for goalies ranks them by wins.

The only reason anybody gives any weight at all to goalie wins is because of the long tradition of tracking them and the related myths told by generations of hockey people and broadcasters. Europe probably doesn't have the same tradition, or else perhaps somewhere along the line somebody decided to speak up and point out how stupid it was to track team results at the individual level. Either way, good for them. Given that they don't care about wins and losses, how do you think Swedish and Finnish coaches are evaluating their young goalie prospects? No doubt they are using almost exclusively save percentage, and with the recent European goalie invasion it's pretty hard to argue with the results.