Goaltending Parity

I was flipping around Hockey Reference the other day, looking at the results from the 1995-96 NHL season. That was a strange year in many ways. The still-terrible expansion franchises in Ottawa and San Jose were both doing their part to skew the standings. In the West, Detroit cleaned up, winning 62 games to set a new league record, while the Wings' bitter rivals and eventual Cup champions Colorado Avalanache were the league's second-best regular season team, leading to a rare situation where 10 out of 13 Western Conference teams finished below .500. In the East it was the exact opposite situation, with nine teams finishing at 86 points or better, including the defending Stanley Cup champions from New Jersey who missed the playoffs despite a record that would have ranked them fourth in the West.

It was a unique year for goaltending as well, particularly as many of the big stars had off-seasons or down years. Patrick Roy got traded by Montreal, Ed Belfour had an off-year and was in the process of losing his starting job in Chicago, while Dominik Hasek and Martin Brodeur both played well but missed the playoffs. All that combined to allow a 22-year old sophomore named Jim Carey to walk off with the Vezina Trophy, all of the voters completely unaware that he would have only three seasons remaining in his professional career.

The league was still full of the old guard of standup goaltenders, many of whom were past their prime or struggling to keep up with the changing game. The result was a huge spread in the save percentage numbers among starting goalies, all the way from Hasek at the top with .920 down to Don Beaupre at .872.

The large gap in results was likely influenced by a higher level of shot quality differences across teams than we see today, particularly for goaltenders representing the Sens or the Sharks. However, even within teams there was a broad range of performance numbers, suggesting that goaltending was a real difference-maker back then. Going through team by team, it is impossible to avoid noticing that the starters almost always had much better win/loss records than the backups.

Compiling the numbers league-wide demonstrates this point (I just took the goalie with the most games played that season for each team to represent their "starter"):

Starters: 611-512-156, .539

Backups: 318-417-118, .442

The totals can be skewed a bit by some team's starters playing more games than others, but even if you take the average of each team's starter and backups you get .536 and .436, a full .100 increase in winning percentage with a team's most-used netminder in the game.

Only five out of 26 teams had a better win/loss record with their backup goalie(s) in the game. Only three more teams had their backups post a win percentage that was even within .050 of their starter.

Let's compare that to 2010-11:

Starters: 838-605-186, .572

Backups: 392-328-111, .539

That gap is much closer, even more so when the averages are taken for each team (starters .564, backups .547). Thirteen out of 30 teams had a better winning percentage when their top goalie didn't get the decision, and eight more had a difference of less than .050 between their starter and backups.

These results strongly confirm what analysts all over the place have been pointing out regarding today's goalies, that there is far more depth at the position today than in prior decades. The two big factors in the increased level of talent was the technical revolution sweeping the game and the increasing influx of European goaltenders.

In 1995-96, only 7 out of 78 goalies in the league were European (I don't count Olaf Kolzig as a European product, he grew up in Canada and played all his minor hockey there). They combined to play a total of 247 games.

By last season, there were 29 Europeans among the league's 87 goaltenders, meaning the percentage of Europeans rose from 9% to 33% in just 15 years. The European goaltenders also combined to play over four times as many games (1077) as they did in 1995-96.

Based on this evidence, it is perhaps unsurprising that there appears to have been a stronger correlation between goalie talent and championships won in the mid-to-late 1990s than in the post-lockout era, where the best goalies have mostly struggled to achieve much team success. Today, it's simply much harder to stand out from the pack.

The Significance of Two Vezinas

As everyone knows, strange things can happen to a single goalie over the course of a single season. Jose Theodore can win the Hart, Jim Carey can win the Vezina, Andrew Raycroft can win the Calder. There are a lot of goalies who had one great season mixed in with a nondescript or average career. Seen in retrospect, that year seems to be most likely founded on a lot of luck and perhaps aided by teammates, or else perhaps came at a point in time where the rest of the league was not yet aware of and able to exploit that netminder's particular weaknesses and tendencies. In a few cases, it is likely that the surprising goalies were legitimately performing at a high level for a brief peak, before later falling off to a lower standard of play as as result of injuries, age, or some other factor.

But two great seasons, that's a different story.

Those who are interested in the Hall of Fame debate often look at comparables, trying to determine if a player with a specific profile has company already in the Hall of Fame. For example, if all players who finished top-10 in scoring X number of times are already inducted, then it seems reasonable to view that as support for any player who achieved that same number of top finishes.

For goalies, there is a very simple Hall of Fame cutoff that so far works with 100% success: Every goalie with 2 or more First Team All-Star selections is a Hall of Famer.

That is not to say that every goalie in the Hall of Fame was voted at least twice as the game's best goalie. Several of them only achieved that honour once, and Gerry Cheevers never did it at all. But everyone with two is in, and that brings us to Tim Thomas.

Tim Thomas had one of the most impressive goalie seasons ever last year, especially when the playoffs are taken into account. Including the postseason, Thomas played in 82 games and stopped 93.9% of the shots against him. His even strength save percentage over that stretch was simply off the charts at .948. That's a level that nobody has come close to since Dominik Hasek was in his prime. Assuming no shot quality or scorer bias effects, Thomas was about 45 goals better than a league average (.913) goalie during the regular season, and another 23 goals better in the playoffs. Thomas faced 33 shots per game in the playoffs and still ranked #1 in GAA. In short, he not only had video game numbers, but he was absolutely dominant at the most important time of the year. In my opinion, Thomas should have won the Hart Trophy.

Does that mean Tim Thomas is a Hall of Famer? With a Cup and a Conn Smythe to go with his two Vezinas, his trophy case is already worthy of the Hall, but longevity really hurts him in any such discussion. Thomas was already 31 years old when he first won an NHL starting job, and at the age of 37 he only has 319 career regular season games played. It remains to be seen how many campaigns are left for a goalie who thrives on his athleticism, but if Thomas can keep his game at a high level for another three or four seasons, he would at least be approaching the numbers that would make it seem like much less of a long shot (500 career games, 50 career shutouts, a career save percentage in the .920 range). At least it wouldn't if the Hall is open to rewarding dominance, rather than just counting longevity and career compiling. With his current 90th place ranking on the career wins list, Thomas isn't likely to end up among the all-time leaders in any of the counting categories.

It will be interesting to watch the conclusion of Thomas' career, to see whether he is the modern-day Johnny Bower or if he merely has a short but meteoric prime. Either way, he will be an interesting test case as a Hall of Fame candidate a decade or so down the road.

Pre-Lockout Chris Osgood Was Not Clutch

"I always loved the fact that when we were tied or the games were close in the last 10 minutes, I'd shut the door and we'd win the game," he said.

"I knew how I did my job on a great team." (Chris Osgood)

I figure that after a long silence in this space, it would only be fitting to get back into it with one of my favourite topics: the overratedness of Chris Osgood.

Actually, to be honest, I wish I didn't have to make posts like these. The recently-retired Osgood should be remembered as a guy who overcame all kinds of obstacles and worked hard to outlast a ton of other goalies who may have had more natural talent. I thought this was a terrific read that showed Osgood's dedication in rebuilding his game to incorporate modern techniques. I get why Detroit fans loved their scrappy netminder, it's great that fan bases identify with blue collar guys who give it everything they have out on the ice.

But, unfortunately, most people still can't separate individual play from team success. In their eyes, 400 wins and 3 Cups make you a Hall of Famer, no further analysis required. They portray Osgood as something that he simply never was, and that's not fair. Ergo, as long as there are specious and silly arguments being thrown out in his favour by people with influence within the hockey community, then I'm going to keep making posts to set the record straight. Sorry, Ozzy, it's nothing personal, I just believe that credit should go where credit is due.

One of the points I have repeatedly tried to make regarding Chris Osgood is that even if you think he was a supreme clutch performer in the 2008 and 2009 playoff runs, that should still not have any impact at all on how you rate his playoff performances from earlier in his career. Many fans seem to have a tendency to revise their evaluations of a player based on their late-career performance, and that makes no sense.

I think Osgood got a lot of help in 2008 and a lot of favourable bounces in 2009, but I will still readily concede that it is much, much more supportable to assert that Ozzy was clutch in those two seasons than it is to claim that Osgood was clutch in the playoffs from 1994-2004.

It would in fact be far, far easier to make the case that Osgood was a spectacular choker in his early career than it would be to argue that he made the big saves when his team needed them most.

Here's the data to support that statement. I looked at Chris Osgood's playoff numbers in the third period based on the game score from 1994 to 2004 (source: Hockey Summary Project). Without play-by-play records to separate out the shots by score, I chose to measure Osgood's GAA in each situation:

Trailing by 2+: 1.02
Trailing by 1: 2.10
Score tied: 2.98
Leading by 1: 2.53
Leading by 2+: 1.88
Overtime: 3.18

The most high-leverage situations with the highest loss in win probability from allowing a goal against are when a team is tied or leading by one goal late in the game. It's hard to miss the observation that these precise situations are the ones where the other team was most likely to score on Osgood. Coincidentally, his goals against numbers dropped in situations where the penalty of a goal against was the lowest. That is not the expected profile of a goalie who was giving up goals when it didn't matter and slamming the door when the game was on the line.

Grouping the numbers into just two groups, the most high-leverage situations (tie game in third & OT and preserving a late one-goal lead) and then everything else, you get these numbers:

OT/tied/up by 1: 2.81

All other situations: 1.71

Of course his teammates playing to the score would have had an impact on those numbers, but did the Red Wings allow over 60% more shots against in the most pressure-packed situations? There's simply no way that was the case, which means that Osgood's individual numbers definitely dropped as the penalty for a goal against rose.

Assuming the shots were distributed evenly regardless of score, Osgood would have had an .881 save percentage with the score tied or his team leading by one, compared to a .924 save percentage the rest of the time. The one situation where it is possible to fully separate out Osgood's save percentage is overtime, where he let in 6 goals on 46 shots for a wholly unimpressive .870 save percentage.

In an attempt to better account for score effects I estimated the shot frequency for each score by taking the average shots in only third periods with more than 15 minutes played with that particular score, and then used those averages to adjust Osgood's expected shots based on his minutes played. The result was that Osgood's numbers got even worse in the most clutch situations, falling to .880, while his save percentage rose to .929 with his team either trailing or leading by 2 or more goals. Even if you want to go so far as to ignore that attempt and simply assume that the Wings allowed shots against at a 20% higher rate in the high leverage situations, the save percentage split would still be .901/.915.

All this is despite the fact that save percentages are higher on average for goalies in the lead than they are for goalies who are trailing, because trailing teams tend to put as many pucks on the net as possible. For example, in this post I show some playoff split numbers for five real elite goalies, who combined to put up a .930 save percentage in third periods that they entered leading by one, compared to a .918 save percentage in third periods they began with a one goal deficit. If you need further convincing, a recent Hockey Analysis post gives even strength numbers broken down by score that show how save percentages rise for the team in the lead.

I also recently developed an additional measure of a goalie's clutch play using the Hockey Summary Project box scores. It is an estimated game-tied save percentage, calculated by noting how much of each period was spent with the score tied, pro-rating the shots for each team during that period by that amount of time, and then noting how many tiebreaking goals were scored by each team. After compiling those figures for each playoff game, a save percentage can be calculated to estimate a goalie's save rate with the score deadlocked. Because of score effects it is not likely to be exact, but it should provide a reasonable estimate. Another benefit is that this measurement covers the entire game, rather than just the third period and OT.

From 1994 to 2004, Chris Osgood's estimated save percentage with the score tied was .890, which is right in line with the estimates from his GAA. That is a substantial drop from his overall pre-lockout playoff save percentage of .910, implying a .922 save percentage in situations where one team (usually his own) held a lead. In addition, it was estimated that only 28% of Osgood's shots against came with the score tied. The main reason that Osgood's teams usually won was that they heavily outshot the opposition in close games (estimated ratio of 1.25 to 1 with the score tied).

Small sample sizes are always a concern when looking at playoff stats, and even more so when the sample is broken down into smaller chunks based on game score. The entire third period and OT sample covers just 681 shots, and the estimated game-tied shots are even lower at 608, which does leave room for the possibility that Osgood was simply unlucky. The process of putting these numbers together is also based on tedious compiling, which raises at least the possibility of an error although I checked the numbers where I could.

At the very least, however, we should be able to claim that there is no evidence to suggest that Osgood improved his play when the pressure rose. On the contrary, the statistical record is very clear that the more desperately the other team needed to score, the more likely they were to slip one past Chris Osgood.

In 2008 and 2009, Osgood's numbers vastly improved in the same situations:

OT/tied/up by 1: 1.53

All other situations: 1.21

Yet again, however, Osgood's GAA was higher when the game was on the line and lower when the outcome was less in doubt, although the split is not as extreme as the one above. I'm not saying that to be critical of Osgood's performance, merely to argue against the claim he selectively raised his game in certain spots. His estimated game-tied save percentage was .933, which is slightly higher than his .928 overall, which could indicate that he was slightly better when the score was close. However, there remains little reason to suggest that Osgood made a significantly greater contribution to winning than his overall numbers indicate.

It remains possible to make a clutch argument for 2008 and 2009 based on the way Osgood's numbers improved from the regular season to the playoffs. I don't buy that it was a conscious thing that Osgood decided to just make himself play well once the puck dropped in the postseason, but the subjective and objective evidence does certainly support the claim that he played better from April to June than from October to March. Maybe Osgood learned how to be clutch, maybe he went on a hot streak, maybe he was just playing behind a dominant defensive team. Either way I don't think that is enough to make up for all the "big goals" Osgood allowed over the rest of his career.

I've stated before that I'm always skeptical of how subjective observers rate the clutch play of an athlete because they let other factors enter the picture, often unknowingly. This appears to be another example of that exact error. Detroit Red Wings fans, Osgood's teammates and even apparently Osgood himself all want to believe in the idea that their team's long-time netminder was clutch, that he made the key saves for the team, that his average numbers are misleading because he always came up big with the game on the line. The problem is that the evidence suggests it was probably just a misperception caused by selective memory and attributing things to Osgood that were more than likely primarily caused by other players on the team. If anything, Osgood appears to have been the opposite of clutch through the vast majority of his playoff career.

What's Going on Out West?

So far in the Stanley Cup playoffs, there has been a pretty stark contrast in goalie numbers between the two Conferences.

Eastern Conference: 2.04 GAA, .931 save %
Western Conference: 3.26 GAA, .893 save %

If you rank the starters in order of save percentage, here's the breakdown based on their conference:

1. East
2. East
3. East
4. East
5. East
6. East
7. West
8. East
9. West
10. East
11. West
12. West
13. West
14. West
15. West
16. West

Even more surprisingly, the top 3 goalies out West in save percentage thus far are Corey Crawford, Ray Emery and Jimmy Howard. The big name puckstoppers (Luongo, Bryzgalov, Rinne, Quick, Niemi) have been absolutely lit up to the tune of a combined 3.77/.880.

I'm not sure if this can be attributed to poor goalie play or a more wide-open style of play or just standard variance over a 1,000 shot sample. There have been some softies, but there have been also been a lot of shooters making their shots as well.

The argument can be made that the Eastern goalies as a group are slightly better, but I don't really see the gap as being all the large. During the regular season, the Eastern playoff starters combined for a .9209 save percentage while the Western starters were at .9205.

It seems inevitable that the tide will eventually turn for the Western netminders (and defensive units), but it has certainly made for an unusually high-scoring first round of the playoffs.

Goalie Point Shares

Hockey Reference introduced a new point shares metric recently. Is it a valuable stat that sheds more light on historical goalie performance?

Let's look at the all-time top 20 list:

1. Patrick Roy
2. Martin Brodeur
3. Terry Sawchuk
4. Tony Esposito
5. Glenn Hall
6. Jacques Plante
7. Curtis Joseph
8. Ed Belfour
9. Dominik Hasek
10. John Vanbiesbrouck
11. Roberto Luongo
12. Rogie Vachon
13. Sean Burke
14. Harry Lumley
15. Gump Worsley
16. Tomas Vokoun
17. Grant Fuhr
18. Tom Barrasso
19. Bernie Parent
20. Billy Smith

I conclude pretty confidently that the above list doesn't come close to passing the common sense test, as it deviates significantly from virtually every other ranking method out there. There's lots that I could nitpick, but I'll just point out that I don't think I've ever seen any other career list, statistical or subjective, that had Joseph, Belfour and Hasek ranked in that order.

It looks like the percentage of credit given to the goalies for team results is too high, and longevity has an excessive impact on the rankings. I probably rate Sean Burke higher than most, especially for his work in the second half of his career, but he still shouldn't be anywhere close to 13th all-time. In fact, given that Burke ranks 12th in career games played, he's a pretty good test case to see whether you are evaluating a goalie's actual performance or just giving them points for showing up for work. I can't help but conclude that Goalie Point Shares does far too much of the latter.

If you're looking for a one stat representation of a goalie's career, I'd still recommend sticking to era-adjusted save percentage or GVT.

Longevity

Here are some career games played numbers, counting all games played in the regular season and playoffs in the NHL, all games played in a major European professional league and all games played in senior international competitions:

1. Dominik Hasek: 1,358
2. Martin Brodeur: 1,325
3. Patrick Roy: 1,282
4. Ed Belfour: 1,153
5. Curtis Joseph: 1,092

Brodeur and Roy have been rightly praised for their excellent longevity, but it's the guy at the top of the list that might surprise some NHL fans.

Hasek was the youngest professional hockey player ever as a 16 year old in the Czech League, and he led the KHL in shutouts this season three decades later at the age of 46. To put that into perspective, Gordie Howe's professional career spanned 34 years, just 4 more than Hasek's to date.

Hasek ranks 5th all-time among NHL players in career GVT, even though just 63% of his professional games were played in the NHL. Give him credit for those European games (the vast majority of which were played while Hasek was good enough to play in the NHL but prevented from doing so by communism), and there's probably a good argument to rank him no worse than 3rd behind only Howe and Wayne Gretzky for the highest total value career ever.

Get 'Em While They're Hot

Through December 31, Antti Niemi had a career save percentage of .907 on his first 2,197 shots faced in the NHL, including his playoff run with Chicago. That's not enough to even cover league average, which has been north of .910 for the last couple of seasons. Given that Niemi is an undrafted free agent who has never stood out from the pack in any professional league has played in, that doesn't seem to indicate much of a future as an NHL starting goalie.

In the 2011 calendar year, though, things have turned around for the Finnish netminder. Niemi has stopped 670 of 716 shots for a .936 save percentage since January 1. The result is that the Sharks recently signed him to a 4 year, $15.2 million deal.

Chicago wasn't willing to pay Niemi $2.75 million for just one season, yet San Jose apparently thinks he's worth a million bucks more than that for each of the next four years. This is also after the Sharks signed Niemi for $2 million during the offseason, and after Niemi struggled early on. Two bad months + two good months = roughly two times the paycheque? Both the term and the money make that a pretty baffling move from San Jose's perspective. Perhaps they're so desperate to finally win something in the playoffs that they're willing to overpay anybody with a Stanley Cup ring on his finger.

To quote Gabe Desjardins in a comment written as a follow-up to his terrific post on a goalie's first 15 NHL games:

"The main point here is that you need 2000+ shots to make a decision about a goalie…And that to get a 2000-shot tryout, you’re best off hitting a performance high in your first few hundred shots…"

Niemi did it backwards. He got his 2000 shot tryout, and he couldn't even manage to be league average despite playing on strong teams the entire time. Then he hit a performance high in his next 700 shots and cashed in. The result is that he'll have a higher cap hit next season than either Pekka Rinne or Kari Lehtonen.

Rinne, Lehtonen and Niemi are all Finnish starting goalies, but I'd still maintain that one of them is not like the others. Whether it was a result of falling in love with his recent form or overrating his playoff team success, San Jose appears to have made a pretty bad bet with the Niemi contract.