Goaltending Parity

I was flipping around Hockey Reference the other day, looking at the results from the 1995-96 NHL season. That was a strange year in many ways. The still-terrible expansion franchises in Ottawa and San Jose were both doing their part to skew the standings. In the West, Detroit cleaned up, winning 62 games to set a new league record, while the Wings' bitter rivals and eventual Cup champions Colorado Avalanache were the league's second-best regular season team, leading to a rare situation where 10 out of 13 Western Conference teams finished below .500. In the East it was the exact opposite situation, with nine teams finishing at 86 points or better, including the defending Stanley Cup champions from New Jersey who missed the playoffs despite a record that would have ranked them fourth in the West.

It was a unique year for goaltending as well, particularly as many of the big stars had off-seasons or down years. Patrick Roy got traded by Montreal, Ed Belfour had an off-year and was in the process of losing his starting job in Chicago, while Dominik Hasek and Martin Brodeur both played well but missed the playoffs. All that combined to allow a 22-year old sophomore named Jim Carey to walk off with the Vezina Trophy, all of the voters completely unaware that he would have only three seasons remaining in his professional career.

The league was still full of the old guard of standup goaltenders, many of whom were past their prime or struggling to keep up with the changing game. The result was a huge spread in the save percentage numbers among starting goalies, all the way from Hasek at the top with .920 down to Don Beaupre at .872.

The large gap in results was likely influenced by a higher level of shot quality differences across teams than we see today, particularly for goaltenders representing the Sens or the Sharks. However, even within teams there was a broad range of performance numbers, suggesting that goaltending was a real difference-maker back then. Going through team by team, it is impossible to avoid noticing that the starters almost always had much better win/loss records than the backups.

Compiling the numbers league-wide demonstrates this point (I just took the goalie with the most games played that season for each team to represent their "starter"):

Starters: 611-512-156, .539

Backups: 318-417-118, .442

The totals can be skewed a bit by some team's starters playing more games than others, but even if you take the average of each team's starter and backups you get .536 and .436, a full .100 increase in winning percentage with a team's most-used netminder in the game.

Only five out of 26 teams had a better win/loss record with their backup goalie(s) in the game. Only three more teams had their backups post a win percentage that was even within .050 of their starter.

Let's compare that to 2010-11:

Starters: 838-605-186, .572

Backups: 392-328-111, .539

That gap is much closer, even more so when the averages are taken for each team (starters .564, backups .547). Thirteen out of 30 teams had a better winning percentage when their top goalie didn't get the decision, and eight more had a difference of less than .050 between their starter and backups.

These results strongly confirm what analysts all over the place have been pointing out regarding today's goalies, that there is far more depth at the position today than in prior decades. The two big factors in the increased level of talent was the technical revolution sweeping the game and the increasing influx of European goaltenders.

In 1995-96, only 7 out of 78 goalies in the league were European (I don't count Olaf Kolzig as a European product, he grew up in Canada and played all his minor hockey there). They combined to play a total of 247 games.

By last season, there were 29 Europeans among the league's 87 goaltenders, meaning the percentage of Europeans rose from 9% to 33% in just 15 years. The European goaltenders also combined to play over four times as many games (1077) as they did in 1995-96.

Based on this evidence, it is perhaps unsurprising that there appears to have been a stronger correlation between goalie talent and championships won in the mid-to-late 1990s than in the post-lockout era, where the best goalies have mostly struggled to achieve much team success. Today, it's simply much harder to stand out from the pack.

The Significance of Two Vezinas

As everyone knows, strange things can happen to a single goalie over the course of a single season. Jose Theodore can win the Hart, Jim Carey can win the Vezina, Andrew Raycroft can win the Calder. There are a lot of goalies who had one great season mixed in with a nondescript or average career. Seen in retrospect, that year seems to be most likely founded on a lot of luck and perhaps aided by teammates, or else perhaps came at a point in time where the rest of the league was not yet aware of and able to exploit that netminder's particular weaknesses and tendencies. In a few cases, it is likely that the surprising goalies were legitimately performing at a high level for a brief peak, before later falling off to a lower standard of play as as result of injuries, age, or some other factor.

But two great seasons, that's a different story.

Those who are interested in the Hall of Fame debate often look at comparables, trying to determine if a player with a specific profile has company already in the Hall of Fame. For example, if all players who finished top-10 in scoring X number of times are already inducted, then it seems reasonable to view that as support for any player who achieved that same number of top finishes.

For goalies, there is a very simple Hall of Fame cutoff that so far works with 100% success: Every goalie with 2 or more First Team All-Star selections is a Hall of Famer.

That is not to say that every goalie in the Hall of Fame was voted at least twice as the game's best goalie. Several of them only achieved that honour once, and Gerry Cheevers never did it at all. But everyone with two is in, and that brings us to Tim Thomas.

Tim Thomas had one of the most impressive goalie seasons ever last year, especially when the playoffs are taken into account. Including the postseason, Thomas played in 82 games and stopped 93.9% of the shots against him. His even strength save percentage over that stretch was simply off the charts at .948. That's a level that nobody has come close to since Dominik Hasek was in his prime. Assuming no shot quality or scorer bias effects, Thomas was about 45 goals better than a league average (.913) goalie during the regular season, and another 23 goals better in the playoffs. Thomas faced 33 shots per game in the playoffs and still ranked #1 in GAA. In short, he not only had video game numbers, but he was absolutely dominant at the most important time of the year. In my opinion, Thomas should have won the Hart Trophy.

Does that mean Tim Thomas is a Hall of Famer? With a Cup and a Conn Smythe to go with his two Vezinas, his trophy case is already worthy of the Hall, but longevity really hurts him in any such discussion. Thomas was already 31 years old when he first won an NHL starting job, and at the age of 37 he only has 319 career regular season games played. It remains to be seen how many campaigns are left for a goalie who thrives on his athleticism, but if Thomas can keep his game at a high level for another three or four seasons, he would at least be approaching the numbers that would make it seem like much less of a long shot (500 career games, 50 career shutouts, a career save percentage in the .920 range). At least it wouldn't if the Hall is open to rewarding dominance, rather than just counting longevity and career compiling. With his current 90th place ranking on the career wins list, Thomas isn't likely to end up among the all-time leaders in any of the counting categories.

It will be interesting to watch the conclusion of Thomas' career, to see whether he is the modern-day Johnny Bower or if he merely has a short but meteoric prime. Either way, he will be an interesting test case as a Hall of Fame candidate a decade or so down the road.

Pre-Lockout Chris Osgood Was Not Clutch

"I always loved the fact that when we were tied or the games were close in the last 10 minutes, I'd shut the door and we'd win the game," he said.

"I knew how I did my job on a great team." (Chris Osgood)

I figure that after a long silence in this space, it would only be fitting to get back into it with one of my favourite topics: the overratedness of Chris Osgood.

Actually, to be honest, I wish I didn't have to make posts like these. The recently-retired Osgood should be remembered as a guy who overcame all kinds of obstacles and worked hard to outlast a ton of other goalies who may have had more natural talent. I thought this was a terrific read that showed Osgood's dedication in rebuilding his game to incorporate modern techniques. I get why Detroit fans loved their scrappy netminder, it's great that fan bases identify with blue collar guys who give it everything they have out on the ice.

But, unfortunately, most people still can't separate individual play from team success. In their eyes, 400 wins and 3 Cups make you a Hall of Famer, no further analysis required. They portray Osgood as something that he simply never was, and that's not fair. Ergo, as long as there are specious and silly arguments being thrown out in his favour by people with influence within the hockey community, then I'm going to keep making posts to set the record straight. Sorry, Ozzy, it's nothing personal, I just believe that credit should go where credit is due.

One of the points I have repeatedly tried to make regarding Chris Osgood is that even if you think he was a supreme clutch performer in the 2008 and 2009 playoff runs, that should still not have any impact at all on how you rate his playoff performances from earlier in his career. Many fans seem to have a tendency to revise their evaluations of a player based on their late-career performance, and that makes no sense.

I think Osgood got a lot of help in 2008 and a lot of favourable bounces in 2009, but I will still readily concede that it is much, much more supportable to assert that Ozzy was clutch in those two seasons than it is to claim that Osgood was clutch in the playoffs from 1994-2004.

It would in fact be far, far easier to make the case that Osgood was a spectacular choker in his early career than it would be to argue that he made the big saves when his team needed them most.

Here's the data to support that statement. I looked at Chris Osgood's playoff numbers in the third period based on the game score from 1994 to 2004 (source: Hockey Summary Project). Without play-by-play records to separate out the shots by score, I chose to measure Osgood's GAA in each situation:

Trailing by 2+: 1.02
Trailing by 1: 2.10
Score tied: 2.98
Leading by 1: 2.53
Leading by 2+: 1.88
Overtime: 3.18

The most high-leverage situations with the highest loss in win probability from allowing a goal against are when a team is tied or leading by one goal late in the game. It's hard to miss the observation that these precise situations are the ones where the other team was most likely to score on Osgood. Coincidentally, his goals against numbers dropped in situations where the penalty of a goal against was the lowest. That is not the expected profile of a goalie who was giving up goals when it didn't matter and slamming the door when the game was on the line.

Grouping the numbers into just two groups, the most high-leverage situations (tie game in third & OT and preserving a late one-goal lead) and then everything else, you get these numbers:

OT/tied/up by 1: 2.81

All other situations: 1.71

Of course his teammates playing to the score would have had an impact on those numbers, but did the Red Wings allow over 60% more shots against in the most pressure-packed situations? There's simply no way that was the case, which means that Osgood's individual numbers definitely dropped as the penalty for a goal against rose.

Assuming the shots were distributed evenly regardless of score, Osgood would have had an .881 save percentage with the score tied or his team leading by one, compared to a .924 save percentage the rest of the time. The one situation where it is possible to fully separate out Osgood's save percentage is overtime, where he let in 6 goals on 46 shots for a wholly unimpressive .870 save percentage.

In an attempt to better account for score effects I estimated the shot frequency for each score by taking the average shots in only third periods with more than 15 minutes played with that particular score, and then used those averages to adjust Osgood's expected shots based on his minutes played. The result was that Osgood's numbers got even worse in the most clutch situations, falling to .880, while his save percentage rose to .929 with his team either trailing or leading by 2 or more goals. Even if you want to go so far as to ignore that attempt and simply assume that the Wings allowed shots against at a 20% higher rate in the high leverage situations, the save percentage split would still be .901/.915.

All this is despite the fact that save percentages are higher on average for goalies in the lead than they are for goalies who are trailing, because trailing teams tend to put as many pucks on the net as possible. For example, in this post I show some playoff split numbers for five real elite goalies, who combined to put up a .930 save percentage in third periods that they entered leading by one, compared to a .918 save percentage in third periods they began with a one goal deficit. If you need further convincing, a recent Hockey Analysis post gives even strength numbers broken down by score that show how save percentages rise for the team in the lead.

I also recently developed an additional measure of a goalie's clutch play using the Hockey Summary Project box scores. It is an estimated game-tied save percentage, calculated by noting how much of each period was spent with the score tied, pro-rating the shots for each team during that period by that amount of time, and then noting how many tiebreaking goals were scored by each team. After compiling those figures for each playoff game, a save percentage can be calculated to estimate a goalie's save rate with the score deadlocked. Because of score effects it is not likely to be exact, but it should provide a reasonable estimate. Another benefit is that this measurement covers the entire game, rather than just the third period and OT.

From 1994 to 2004, Chris Osgood's estimated save percentage with the score tied was .890, which is right in line with the estimates from his GAA. That is a substantial drop from his overall pre-lockout playoff save percentage of .910, implying a .922 save percentage in situations where one team (usually his own) held a lead. In addition, it was estimated that only 28% of Osgood's shots against came with the score tied. The main reason that Osgood's teams usually won was that they heavily outshot the opposition in close games (estimated ratio of 1.25 to 1 with the score tied).

Small sample sizes are always a concern when looking at playoff stats, and even more so when the sample is broken down into smaller chunks based on game score. The entire third period and OT sample covers just 681 shots, and the estimated game-tied shots are even lower at 608, which does leave room for the possibility that Osgood was simply unlucky. The process of putting these numbers together is also based on tedious compiling, which raises at least the possibility of an error although I checked the numbers where I could.

At the very least, however, we should be able to claim that there is no evidence to suggest that Osgood improved his play when the pressure rose. On the contrary, the statistical record is very clear that the more desperately the other team needed to score, the more likely they were to slip one past Chris Osgood.

In 2008 and 2009, Osgood's numbers vastly improved in the same situations:

OT/tied/up by 1: 1.53

All other situations: 1.21

Yet again, however, Osgood's GAA was higher when the game was on the line and lower when the outcome was less in doubt, although the split is not as extreme as the one above. I'm not saying that to be critical of Osgood's performance, merely to argue against the claim he selectively raised his game in certain spots. His estimated game-tied save percentage was .933, which is slightly higher than his .928 overall, which could indicate that he was slightly better when the score was close. However, there remains little reason to suggest that Osgood made a significantly greater contribution to winning than his overall numbers indicate.

It remains possible to make a clutch argument for 2008 and 2009 based on the way Osgood's numbers improved from the regular season to the playoffs. I don't buy that it was a conscious thing that Osgood decided to just make himself play well once the puck dropped in the postseason, but the subjective and objective evidence does certainly support the claim that he played better from April to June than from October to March. Maybe Osgood learned how to be clutch, maybe he went on a hot streak, maybe he was just playing behind a dominant defensive team. Either way I don't think that is enough to make up for all the "big goals" Osgood allowed over the rest of his career.

I've stated before that I'm always skeptical of how subjective observers rate the clutch play of an athlete because they let other factors enter the picture, often unknowingly. This appears to be another example of that exact error. Detroit Red Wings fans, Osgood's teammates and even apparently Osgood himself all want to believe in the idea that their team's long-time netminder was clutch, that he made the key saves for the team, that his average numbers are misleading because he always came up big with the game on the line. The problem is that the evidence suggests it was probably just a misperception caused by selective memory and attributing things to Osgood that were more than likely primarily caused by other players on the team. If anything, Osgood appears to have been the opposite of clutch through the vast majority of his playoff career.