Good Enough to Win

One thing that is often discussed about goalies by hockey fans is whether they are "good enough" to win a Stanley Cup. This is usually based on things like gut feel and extrapolating huge amounts of significance from small sample sizes; I have rarely seen anyone try to quantify what good enough to win a Cup actually means.

First of all, winning is obviously a team result. The talent and luck necessary for a goalie to win a Cup on the Detroit Red Wings is a lot different than what is required on the New York Islanders. The historical record shows that only strong regular season teams win Cups, so we need to look at the kind of numbers goalies put up on those teams. That said, hot goaltending does have an impact. At some point even a good team is going to run against other strong opponents in their quest for the Cup, and in those games superior goaltending can become a tiebreaking factor (e.g. the 2011 Cup Finals).

Here are the save percentages for each Cup-winning goalie in the save percentage era, adjusted to league average and normalized to the current average save level (.911):

1984: Grant Fuhr, .933

1985: Grant Fuhr, .925

1986: Patrick Roy, .946

1987: Grant Fuhr, .932

1988: Grant Fuhr, 913

1989: Mike Vernon, .930

1990: Bill Ranford, .934

1991: Tom Barrasso, .937

1992: Tom Barrasso, .926

1993: Patrick Roy, .945

1994: Mike Richter, .933

1995: Martin Brodeur, .934

1996: Patrick Roy, .931

1997: Mike Vernon, .932

1998: Chris Osgood, .923

1999: Ed Belfour, .933

2000: Martin Brodeur, .933

2001: Patrick Roy, .940

2002: Dominik Hasek, .923

2003: Martin Brodeur, .936

2004: Nikolai Khabibulin, .933

2006: Cam Ward, .928

2007: J.S. Giguere, .927

2008: Chris Osgood, .932

2009: Marc-Andre Fleury, .911

2010: Antti Niemi, .910

2011: Tim Thomas, .939

Those numbers are quite consistent. Twenty out of 27 goalies had an adjusted save percentage between .923 and .937. The average was .930 on 582 shots against. The only goalies to win the Cup with average save numbers were Fuhr in 1988, Fleury in 2009 and Niemi in 2010. Patrick Roy was the only Cup winning goalie to record a number of .940 or better, which remarkably he managed to do in each of his three Conn Smythe winning performances.

The overall averages for Cup Finalists were very similar, with an average of .927 on 572 SA. The distribution was different however. Eight of 27 had an adjusted save percentage of .939 or better, which reflects the fact that there have been a number of weaker teams that needed strong goaltending just to make it to the Final. Eight other Finalists had adjusted save percentages of .917 or worse. Some of those simply benefitted from strong teammates, while others were mostly good for three rounds and then saw their numbers nosedive as results starting going against them during the Finals.

This analysis gives the rough historical rule of thumb that to win a Stanley Cup, you need a goalie capable of putting up a .930 save percentage over 600 shots. That level of statistical performance doesn't guarantee a Cup, a number of goalies have played at an even higher level than that only to see their teams fall short at the final hurdle, but it makes a ring very possible if other variables (scoring, defence, injuries, opposition, etc.) also happen to break right.

Because of variance, nearly any goalie who makes it to the NHL could put up a .930 on 600 shots with enough luck, although it is not very likely to happen for a replacement level goaltender. According to the binomial probability function, a .900 talent goaltender would have a 0.7% chance of putting up that target number on any given 600 shot stretch. That means that even if they saw 600 shots against in every playoff season, they would still need to play in 99 of them to have a greater than 50% chance of going over .930 in one of them. Unless they happen to be astronomically lucky or find themselves playing for a complete powerhouse, I think it is fair to say that in general a .900 talent goalie is not good enough to win a Stanley Cup.

For a .905 goalie, the odds increase to 1.9%, still not even once every 20 times which is the usual cutoff point to determine statistical significance. The point where the probability moves above 5% is when the goalie's talent is .911, which is right about league average. The chances are much greater for one of the league's elite goalies. A .920 talent goalie would play at .930 or better over 600 shots about 20% of the time.

That shows that a top goalie on a contending team would have a reasonably good chance to win a Cup, but that it is still far from a sure thing. Goalies have short careers and often only a short window of opportunity to compete in the playoffs with a true contender. That is why it is not surprising that some elite goalies who conclusively proved their talent over hundreds of regular season games still never managed a deep playoff run. It was not because they weren't good enough to win a Cup, it was because they didn't get the breaks that are required to win a championship.

The idea that most starting goalies in the league are capable of .930 over 600 shots is also backed up by the actual statistical record. There are only seven teams who do not currently employ a goaltender that has put together at least one stretch of consecutive games with a save percentage of .930 or better on at least 600 shots against since 2009-10. Even among those seven, five of them have goalies who very narrowly missed the cutoff:

Reimer, Toronto: .929 on 622 SA

Theodore, Florida: .929 on 608 SA

Lehtonen, Dallas: .929 on 581 SA

Varlamov, Colorado: .931 on 563 SA

Brodeur, New Jersey: .929 on 603 SA

Some might quibble with the relatively unproven Reimer or Varlamov, or even the aging Brodeur, but I'll count all five as being close enough. That leaves just Columbus and Tampa Bay. Steve Mason's best stretch since '09-10 was .923 on 571 shots, while Roloson's best in the last two and a bit regular seasons was .926 on the same 571 SA. However, these numbers are from the regular season only; if playoff numbers count then from March 29 to May 19 Roloson had a .931 on 623 shots against.

Despite his hot spring last year, I think Roloson's age makes it unlikely that he will regain that form, and Mason's recent track record is simply not very good at all. I think it is probably fair to say that the Blue Jackets and Lightning may not currently have goaltending that is good enough to win a Cup. Other than that, every other team has a goaltender that has shown they can play at a high enough level for a long enough period of time that while they may not be a good bet to provide that level of goaltending during a playoff run, they would at least have an outside chance at winning a Cup if they were fortunate enough to be a member of a contending team.

This discussion should make it quite clear that evaluating a goalie's ability to win a Cup based on their past history is a woefully inadequate method. Nearly every starting goalie in the league is good enough to win a Cup in the right situation, and for a typical playoff team any netminder that is average or better should probably be considered good enough to win if they can just get on a hot streak at the right time. However, in any individual playoff season the odds would still be very much against them, even for an elite goalie on a contending team.

The only caveat to this analysis is that it seems likely that the threshold for a Cup winning goalie has dropped even further since the lockout. As Copper 'n Blue pointed out recently, there haven't exactly been huge gains in parity even with the salary cap system. As a result, goaltending performance has become less decisive than in previous seasons. Four out of six post-lockout goalies have been below .930, with two of them well below that mark. The numbers have dropped for Cup Finalists as well, who have averaged .923 since 2006. I'd estimate that the threshold is probably closer to .925 than .930 in today's NHL, which would leave only Columbus as a team without a goalie with the proven ability to perform above that level.

The Value of a Quick Start

I was one of a number of hockey observers who thought that by this point in time Jonathan Bernier would have mounted a serious challenge for the Los Angeles' Kings starting job, if not taken it over outright from Jonathan Quick. However, there hasn't been much of a goalie controversy at all, and there is no doubt that the 3rd round pick from Connecticut is the currently the clear #1 ahead of the highly-touted first-rounder from Quebec.

Is Quick a better goalie than Bernier? At the moment that seems obvious, based on their play over the last season and a bit and the way the team has distributed the playing time between them. However, it should be noted that Quick has a significant advantage in the battle between the two young goalies, having been born two years earlier. At the moment, Quick is 25 and Bernier is 23. For the sake of comparison, Jaroslav Halak was 25 and Carey Price was two months away from turning 23 when Montreal made the controversial move to trade their 2010 playoff hero away at the end of the postseason and bet on their younger goaltender. Quick is considerably more experienced, with an extra 180 regular season and playoff games in the NHL under his belt. That raises the reasonable question of whether Quick is the better goalie, or whether he is merely the more developed talent for the moment.

In addition to being a higher draft pick, Bernier had also had a better minor league career. If you compare the two of them by age, Bernier's progression was well ahead of Quick's through his early twenties.

At 19, Quick was still in high school, although he put up very good numbers. Bernier was recognized as one of the best goaltenders in the CHL, made the Kings out of camp and ended up with a brief cup of coffee in the NHL (4 games), before joining Manchester at the end of his junior season and starting 3 out of the team's 4 games in the AHL playoffs. Bernier also made Canada's under-20 national team for the world junior championships, while Quick was not selected to Team USA.

For his age 20 season, Quick went to UMass, where he would spend two seasons. At 20, Bernier was already a solid pro, posting a .914 save percentage in 54 games played as an AHL starter. The next year Bernier was even better with a spectacular .937 save percentage in 74 regular season and playoff games, earning the AHL's top goaltender award.

At the age of 22, Bernier joined the NHL seemingly for good as the Kings' backup goalie. He did fairly well as a backup (.913 in 25 games), although he didn't exactly take the league by storm as he had in the AHL. Compare that to Quick, who turned pro for his age 22 season which he split between the ECHL (38 GP, .905) and AHL (19 GP, .922), plus a trio of brief appearances in the NHL.

To summarize:

19: Quick in high school, Bernier AHL playoff starter
20: Quick in NCAA, Bernier AHL starting goalie
21: Quick in NCAA, Bernier AHL goaltender of the year
22: Quick ECHL starter/AHL backup, Bernier NHL backup

Up to that point in their careers, Bernier's development was clearly surpassing Quick's. However, things turned around in their age 23 seasons. Quick started the year in the AHL, but was called up to Los Angeles in December. When he arrived he made the most of his chance, playing very well early on in stopping 94.6% of the shots against and recording two shutouts in his first six starts. With Erik Ersberg and Jason LaBarbera both playing poorly, Quick ended up quickly taking over the Kings' starting job. Over the remaining 37 games he would play in that season, Quick's save percentage was .909, almost exactly league average (.908).

In his 23 year old season, Bernier also managed to record pretty average numbers in the NHL (.913 in 25 games), but he started very slow, losing 5 of his first 7 starts with a mere .889 save percentage. Bernier's early season struggles were magnified by the fact that Quick did very well out of the gate in the same 2010-11 season (7-1-0, 1.84, .936 in October of 2010), further solidifying his claim to the starting job.

This season Quick yet again began red-hot, going 6-1-2, 1.52, .947 in October, before quite naturally tailing off a bit since November 1. That was the third time in the last four NHL seasons that Quick was almost unbeatable in the first month he played, leading to some fairly extreme splits for his young career: so far:

Quick since 2008-09:
First month played: 23-8-4, 2.02, .928
Rest of season: 89-74-12, 2.72, .911

Compare that to Bernier, who for the second year in a row is off to a slow start.

First 7 games in 2010-11: 2-5-0, 3.30, .889
Remainder of 2010-11: 9-3-3, 2.13, .923
First 7 games in 2011-12: 2-4-0, 3.10, .883

Games at the start of the season often take on extra significance because they help establish a team's pattern of distributing starts between their goaltenders and the impact on a goalie's seasonal statistics is more noticeable. Look at how many fans around the league were either pronouncing their team's #1 as a Vezina candidate in November because of a strong early run of form or were panicking because their team's goalie took a while to discover their usual game. A hot or cold start takes longer to average out, whereas a slump in February or March has much less of an effect on a goalie's seasonal numbers to date because they may already have 40-50 games played by that point in the season.

The first few games can also be quite critical in terms of establishing a reputation for a young goalie trying to crack the NHL. It took all of five Jon Quick starts in 2008 before Los Angeles traded Jason LaBarbera to Vancouver and pretty much anointed Quick the starter. If those early starts had included a couple of blowout losses, then the team may very well have decided that LaBarbera wasn't actually that bad after all while Quick was in need of more seasoning in the minors. There is no doubt that Quick's hot streaks were almost perfectly timed to advance his career. The only way he could have timed them any better was if he managed to get on a real roll in the postseason.

I think there is a fair chance that Quick and Bernier are not that far apart in true talent, even disregarding the age gap between them. In the AHL, Quick put up a .923 on 1033 shots, while Bernier recorded a .928 on 3937 shots. Minor league success does not always translate to the big leagues, but it is at least evidence that the two were in a similar ballpark, even though Bernier was doing it at a younger age.

In the NHL, Quick's career regular season save percentage is .914 compared to Bernier's .906. However, if we exclude Bernier's four games at age 19, where he quite naturally struggled like the majority of teenagers do in the NHL at that age, and we add playoff numbers to increase the sample size, suddenly the career NHL save percentages for the two goalies converge quite a bit:

Quick: .913 on 6026 SA
Bernier: .911 on 925 SA

Quick's sample size is much larger, which means we are far more confident in our estimate of his true talent level. In addition, given that the majority of goalies are below average, it is safest to assume that a young goalie who hasn't proven much in the NHL is below average until he has faced a few thousand shots against. Bernier's pre-NHL track record makes it somewhat more likely that he is a real talent, but it is best to remain conservative at this point. Nevertheless, I will be very interested to revisit these numbers after the end of this season to see if they remain similar.

One thing that could be argued in Quick's favour is that his numbers have been trending upward over the last three seasons. However, I'm not sure how much to credit him for that given that his improvement has come almost entirely on the penalty kill.

2008-09: .926 EV, .869 PK
2009-10: .919 EV, .853 PK
2010-11: .921 EV, .903 PK
2011-12: .928 EV, .922 PK

Quick is a better goaltender today than he was in 2008-09, but he has also probably been very lucky on his last 435 shots against while shorthanded as .910 is an unsustainable PK SV%. For comparison's sake, Bernier's career numbers are .916 at EV and .890 on the PK (again, excluding his 19-year old season). Bernier may also have gotten a bit of luck on special teams, or perhaps the Kings' penalty kill has been unusually strong as of late, although his PK sample size is very small.

Looking at the progression and career numbers of the two young Kings goaltenders, it is hard not to wonder about the size of the impact of hot starts as well as Quick's opportunity to compete for the starting job in 2008-09 while there was not already an established starter, when Quick's two year head start in terms of age really turned into an advantage. Bernier is currently on pace to end his age 23 season with a career total of 49 games played. Jonathan Quick got into 116 of them in his age 23 and 24 seasons while competing mostly against Erik Ersberg, who was probably never more than a replacement level goaltender. Even if he starts playing better than Quick right now, Bernier will probably never get anywhere near that much playing time over this season and next.

If the crease didn't open up for Quick and he didn't have hot starts to begin nearly every season, it is certain that he would have played in fewer games, potentially many fewer games. The Kings brought Bernier along slowly, and it is possible that his development may have stalled somewhat given his infrequent usage as a backup goalie this season and last, whereas Quick went from an AHLer to an NHL starter in one season, and as a result his extra games played in the show at age 23 and 24 were likely very significant in helping him close the early career development gap against his teammate. That said, it is also possible that Bernier was never going to fulfill his earlier promise anyway. Even highly rated draft picks sometimes don't pan out.

This comparison reminds me of an article I wrote a while back which discusses the impact of opportunity on another Kings goalie, Jamie Storr. Storr was sort of the anti-Quick in that he was never able to put together a good run when it seemed like the starting job was available to be won, even though his overall save numbers were actually pretty good.

It's always hard to separate talent and luck early in a young goalie's career, when we really don't know enough about them to properly assess their true ability at the NHL level. Right now Quick is solidly ahead of Bernier, but has that been because of talent or opportunity? It may never be possible to figure that out with any degree of confidence, but the years ahead will give some additional information that will help make a more accurate estimate. For now, I certainly wouldn't rule out the possibility that Bernier is either as good Quick right now or that he will still eventually turn out to be better when both are in their primes. Los Angeles may be facing a Halak/Price type of decision at the end of next season when both goaltenders become free agents, and it will be interesting to what choice they will end up making.

What Does a Clutch Goaltender Look Like?

One of the most common goaltending cliches is, "It's not how many saves you make, it's when you make them." Colour commentators tell us this over and over again in their rush to credit goalies for wins or blame them for losses, and many hockey fans agree with them. Not every save is the same, they say. Saves that keep a team in the game or allow them to hold on to the lead are more important, and what happens when the game is already decided really doesn't matter so much.

Following that logic, a clutch goalie will make a higher percentage of saves when the score is close, giving their team a "chance to win". As such, we would expect them to have much better numbers in close games than in garbage time.

After the 2011 postseason, I don't think the people saying this kind of stuff really believe what they are saying. Either that or they simply don't recognize it when they see it. The reason for that claim is simply this: Strictly based on the above definition of clutch play (a goalie who performs better when the score is close than when it isn't), I doubt there are many goalies who have ever had more "clutch" numbers over the course of a single playoffs than Roberto Luongo in 2011. Yet, needless to say, Luongo didn't exactly get a whole lot of praise as someone who "made the key saves at the key times."

First, let's look at Luongo's stats at the series level:

Series tied: 7 wins, 1 loss, 0.97 GAA, .968 save %

VAN up 1: 6 wins, 4 losses, 2.56 GAA, .909 save %

VAN up 2+: 2 wins, 5 losses, 4.59 GAA, .868 save %

The Canucks never trailed in any series throughout the playoffs, and a big reason for that was their ridiculous goal prevention record when a series was tied. Luongo was particularly sharp in opening games, posting 3 shutouts in 4 tries.

On the other hand, when the Canucks went well out ahead of their opponents in the series they repeatedly suffered meltdowns. That's not particularly admirable, but the same logic that rates some saves are more key than others could be applied to rate some games as more important than others. And, if you do that, then any reasonable definition of those "key games" would have to include games where the series was tied, since a team that always wins when the series is tied could never be eliminated.

For the sake of comparison, here are Tim Thomas' numbers:

Series tied: 7-4, 1.39, .955

BOS up 1: 1-3, 3.11, .909

BOS up 2+: 2-0, 1.00, .967

BOS down 1: 4-2, 2.84, .921

BOS down 2+: 2-0, 1.50, .961

With the series tied or their teams up one game, Luongo's numbers were better than Thomas'. The main difference was that Thomas was great at closing out the Flyers series and he was also great in the two games where the Bruins faced 0-2 deficits.

Let's move on to the game level. Here are Luongo's numbers broken down by game score:

Lead by 5: 0 saves, 1 shot, .000 save %

Lead by 4: 0 saves, 1 shot, .000 save %

Lead by 3: 18 saves, 19 shots, .947 save %

Lead by 2: 56 saves, 60 shots, .933 save %

Lead by 1: 154 saves, 164 shots, .939 save %

Tie game: 257 saves, 276 shots, .931 save %
Down by 1: 84 saves, 92 shots, .913 save %

Down by 2: 24 saves, 32 shots, .750 save %

Down by 3: 41 saves, 45 shots, .911 save %

Down by 4: 14 saves, 17 shots, .824 save %

Down by 5: 2 saves, 3 shots, .667 save %

Down by 6: 0 saves, 1 shot, .000 save %

What's obvious from that chart? If the game was close, Luongo made a high percentage of the saves. But if it was a blowout, then everything went in. Down by 2 or more goals, Luongo had an .827 save percentage on 98 shots, which is almost unbelievably bad. In garbage time, Luongo made very few saves at all - with either team leading by 4 or more goals, he stopped just 16 out of 23 shots for a mere .696 save percentage. If I was a member of Vancouver's coaching staff, I'd be somewhat concerned by that number and what it could indicate about Luongo's compete level. That said, if it is only "when you make the saves" that counts, then his performance in blowout games should be considered meaningless anyway.

Copper 'N Blue also tracked scoring chances for all of Vancouver's playoff games. Because it is possible for goals to be allowed on non-scoring chances (which Luongo did four times in the Nashville series and six times in the playoffs overall), I calculated scoring chance save percentage as (scoring chances against minus goals against) divided by scoring chances against.

Lead by 5: 2 chances, 1 goal, .500 save %

Lead by 4: 1 chance, 1 goal, .000 save %

Lead by 3: 5 chances, 1 goal, .800 save %

Lead by 2: 29 chances, 4 goals, .862 save %

Lead by 1: 76 chances, 10 goals, .868 save %

Tie game: 127 chances, 19 goals, .850 save %

Down by 1: 50 chances, 8 goals, .840 save %

Down by 2: 19 chances, 8 goals, .579 save %

Down by 3: 26 chances, 4 goals, .846 save %

Down by 4: 5 chances, 3 goals, .400 save %

Down by 5: 2 chances, 1 goal, .500 save %

Down by 6: 1 chance, 1 goal, .000 save %

The numbers are pretty similar, although Luongo's numbers with the lead come down slightly relative to his trailing numbers, likely because of score effects. With the Canucks leading, the opposition took 2.2 shots for every scoring chance recorded. When Vancouver trailed, the ratio dropped to 1.8. Still, the overall conclusion is the same: Luongo's problems were not with the game close or Vancouver in the lead, but rather with the Canucks trailing by more than one goal. And in blowout games he apparently wasn't even trying (7 goals against on 11 scoring chances with either team leading by 4 or more).

Not surprisingly, Vancouver did a lot better in close games than in blowouts. Excluding empty net goals, they were 12-4 in games decided by one goal, 2-0 in games decided by two goals, and 1-6 in games decided by three goals or more.

Having said all that, let's compare Luongo's save percentages by score to those of his Stanley Cup goaltending counterpart, Tim Thomas:

Score within 1 goal: Luongo .931, Thomas .940

Up by 2 or 3 goals: Luongo .937, Thomas .944

Down 2 or 3 goals: Luongo .844, Thomas .957

Luongo was knocked for his ability to hang on to leads, especially during the Nashville series after giving up a couple of late tying goals, but his numbers while leading actually weren't too far off what Thomas was putting up for Boston. With the score close, Thomas again had an edge but not nearly as much as one would think based on their overall numbers. By far the biggest difference in performance between the two goalies was that Luongo was blown out repeatedly and did far worse when his team fell behind by two goals or more.

Did Luongo's awful play in blowouts have much of an effect on Vancouver's win/loss record? Maybe, maybe not. It would certainly be possible to argue that the Canucks would have been unlikely to come back in those situations anyway, especially with how anemic their offence was at times during the playoffs. On the other hand, Boston came back twice from two goal deficits, once again Montreal and once against Tampa Bay. If Thomas hadn't made the saves in those situations, the Bruins would quite possibly have never have made it to the Finals.

To me, the implications of these numbers are pretty clear. On one hand, you can choose to cling to the "key saves at key times", "the only thing that matters is clutch play" logic and praise Roberto Luongo for elevating his game in key situations with the score close in last year's Stanley Cup playoffs. Alternately, you could consider that most of the game is a "key situation", realize that effort and desire don't correlate perfectly with success, and understand that most goalies have the mindset to keep the puck out of the net at all times. That would cause you to reject the "key saves" logic, and instead rate goalies based on their overall performance. In that case, you would be perfectly entitled to knock Luongo's playoff performance, particularly relative to that of Tim Thomas.

That's assuming people are being logically consistent, however. In reality, there isn't an awful lot of logic involved in either hockey fandom or hockey reportage. When most people say "clutch goaltending", they usually simply mean "good goaltending". If not that, then they mean "goaltending behind excellent teammates". Another tactic is to move the goalposts, narrowing the sample size to focus on a few particular games (e.g. game sevens or Stanley Cup Finals games) as the only ones that qualify for the "clutch" distinction. I disagree with that logic since a team that didn't win in the earlier rounds would not have made it to the Finals at all, and similarly without winning three out of the first six no team would still be alive to participate in a game seven.

I've been in the camp that focuses on overall performance all along, so I'm not defending Luongo here. I think that throughout the 2011 playoffs, factoring in both his good and his bad games, Luongo was pretty ordinary. I do think that his play was better in close games, and I'm quite confident that if there was a hockey metric similar to baseball's win probability added, Luongo would rate much better according to that than he would based on overall save percentage. Whether that is a significant observation depends on your perspective on clutch goalies.

At this point I think it's also impossible not to accept the theory that Luongo has a higher-than-normal propensity to melt down when things turn against him. I used to agree with Tyler Dellow that it was essentially media narratives driving the perception, but the repeated blowouts involving Luongo and the above save percentage splits by score make that position tough to hold. I would still argue that Luongo's overall career playoff performance is reasonable relative to his true talent level, but has he ever had some forgettable single game performances on the way.

Dave Bolland and the Percentages

It's fun early in the season to click around on Behind the Net and look at the huge variation in on-ice shooting percentage and on-ice save percentage for hockey players around the league. For example, Detroit's Drew Miller has a .988 save percentage behind him this season. Matt D'Agostini is at .980, benefitting from Brian Elliott's completely unexpected early season star turn in St. Louis. In San Jose, Martin Havlat and Michal Handzus are at .966 and .964. At the other end of the scale, Ottawa's goalies have been absolutely ventilated (.795) with rookie Stephane DaCosta on the ice, explaining his team-worst -9 rating. Steve Downie, Marty Reasoner, Craig Adams and Matthew Lombardi are other regulars with on-ice save percentages still below .850.

There can be huge differences even between teammates playing in front of the same defencemen and goaltenders. Da Costa's teammate Jesse Winchester has a .940 save percentage behind him, meaning a shot has been 3.4 times more likely to go in the net with DaCosta on the ice than with Winchester. There's obviously no way a forward could have anything even close to that much of an impact on shot quality. It's a similar story in Detroit, where nobody scores with Drew Miller, Cory Emmerton (.955), Darren Helm (.955), or Tomas Holmstrom (.960) on the ice, but perennial Selke nominees Henrik Zetterberg (.891) and Pavel Datsyuk (.904) get lit up. Some small part of that is likely related to quality of competition, but with those kind of ranges and samples it is mostly luck.

The same type of thing can happen over a postseason as well, given that a typical Stanley Cup winner plays 22-26 games, and when it does it becomes more widely noticed and emphasized because the whole hockey world is paying attention. An example of a player who has been absolutely rocking the percentages in the playoffs lately is Chicago's Dave Bolland.

A huge part of both the 'Hawks Cup run in 2010 and their almost-comeback against the Canucks in their first round series in 2011 was the ability of the team's checking line to outscore strong opposition. Bolland attracted a lot of attention for his work, and a good portion of those accolades are indeed deserved. Bolland is a key member of the Blackhawks because of his ability to play tough minutes and has a big impact on Chicago's team depth when he is in the lineup. However, over the last three playoff series he and his linemates were outscoring at a completely unsustainable rate.

Copper 'n Blue counted scoring chances for the Blackhawks' series against San Jose and Philadelphia in 2010, as well as the first round matchup against Vancouver last season. With Bolland on the ice in the 14 games he played over that stretch, Chicago created 49 scoring chances and gave up 69 against at even strength. Given the difficulty of his minutes in terms of opposition and defensive zone starts, that's really not too bad for Bolland and his linemates. The curious part is that at the same time Bolland's plus/minus somehow managed to end up at +13.

Excluding shorthanded and empty net goals, Chicago outscored their opponents 16-5 with Bolland on the ice at even strength. That means the team scored in over 1 of 3 recorded chances (34.8%), while the scoring chance save percentage behind him has been an improbable .922. Depending on who is counting them, typical rates are 1 goal scored per 6 or 7 scoring chances. If you add up those two numbers, you get a way-off-the-charts scoring chance PDO number of 1270.

I often write about how much luck there is in the playoffs, and how randomness plays a big factor in what kind of labels get applied to players. When you have a player who is both playing well and getting lucky it becomes very tough for the opposition to overcome, and perhaps equally tough for sportswriters to avoid hyping up the impact of that particular player. Focusing strictly on results in terms of goals for and against, it is hard to deny that Bolland had a huge impact on the last three series for the Blackhawks. However, the underlying numbers show that it would be virtually impossible for that level of outscoring to continue, as the percentages needed to be extremely skewed in Bolland's favour for the numbers to come out the way they did. I wouldn't bet on his line outscoring Joe Thornton or the Sedins again in their next postseason matchup.

This season, Bolland's percentages have been trending in the exact opposite direction, which explains why he currently sits at -5. Despite facing the highest QualComp on the team and starting in his own zone 33.9% of the time, Bolland manages to almost break even in terms of Corsi and the Hawks have actually outshot their opponents 124-112 with him on the ice at 5 on 5. All that indicates that Bolland is playing extremely well, but unfortunately his PDO so far is just 943, mainly because the save percentage behind him is just .868, and as a result he has been outscored 17-10. In the long run it typically all evens out.

Pekka Rinne Comparables

Pekka Rinne's 7-year, $49 million extension from the Nashville Predators has been one of the biggest goaltending stories of this season. Before coming to terms on that huge deal, which was announced on his 29th birthday, Rinne had a total of just 188 NHL games played in his young career through the age of 28. There's no denying that Rinne had a great season last year and has been playing well for the most part in 2011-12, but that is still not a huge sample size to make a seven year bet on a goalie's future, particularly with the increased downside risk associated with a big money guaranteed contract in a capped league.

To help project how Rinne might be expected to perform through the entirety of that contract, I looked for other goalies since 1990 with a similar amount of NHL experience at the same age (150-200 games played through their age 28 or age 29 season), and selected only goalies who were starters for the last two or three of those seasons (i.e. goalies who had already peaked and had already become backups or washed out of the league were not included). To adjust for different scoring contexts, I also ran the goals over average numbers (calculated by subtracting league save percentage from each goalie's save percentage and multiplying by the number of shots). This was the career progression of those goalies by age:

25: 346 GP, .906, 30.2 GOA

26: 484 GP, .903, -18.3 GOA

27: 615 GP, .911, 74.3 GOA

28: 579 GP, .909, 40.2 GOA

29: 610 GP, .909, 53.2 GOA

30: 555 GP, .908, 28.6 GOA

31: 474 GP, .905, -20.6 GOA

32: 379 GP, .912, 39.0 GOA

33: 375 GP, .908, -9.3 GOA

34: 298 GP, .905, -35.6 GOA

35: 88 GP, .897, -21.8 GOA

36: 74 GP, .904, 0.1 GOA

Part of the reason for the steep decline in games played is that some of these goalies are still active and have not yet reached their age 36 season. However, the save percentage numbers show how the performance declined in the aggregate, especially from the age of 33 and onwards.

Of the goalies in the sample, four of them were particularly comparable to Rinne in that they had well above average save percentage performance in the three seasons prior to the cutoff. Those four goalies were Marty Turco, Miikka Kiprusoff, Guy Hebert, and Mike Dunham. Here are the same numbers for that quartet:

25: 100 GP, .916, 25.6 GOA

26: 104 GP, .901,-8.4 GOA

27: 108 GP, .918, 76.6 GOA

28: 84 GP, .913, 32.7 GOA

29: 87 GP, .915, 67.1 GOA

30: 120 GP, .913, 63.4 GOA

31: 124 GP, .904, -25.4 GOA

32: 96 GP, .912, 19.2 GOA

33: 132 GP, .908, 3.0 GOA

34: 136 GP, .904, -28.8 GOA

35: 35 GP, .897, -12.5 GOA

Through the age of 30, the four were elite at stopping the puck, worth an average of 2-3 wins each per season compared to an average goaltender. Once they got on the wrong side of 30, however, things went south pretty quickly. From age 31 to 35, they had a combined save percentage of .906, which was a whopping 44.5 goals below league average. Miikka Kiprusoff is still hanging around in Calgary, making $5.8 million per season and seemingly skating by without taking any blame for his declining performance, but the teams employing the other three goalies are no doubt very grateful that they didn't sign any of those goalies to long-term contracts.

That is why the Predators are taking on quite a bit of risk in signing Rinne to a seven year deal. Rinne may be an elite goalie today, but that doesn't guarantee that he will be one at age 32 or 35. Turco and Kiprusoff were once elite as well, both having led the league in save percentage and having been voted a postseason All-Star, and yet that didn't stop them from ending up ranked last and third-last respectively on the list of post-lockout save percentages for goalies with at least 200 games played.

Very few goalies are great throughout their careers. Those that are consistently elite typically break in early, usually taking on significant NHL playing time by the age of 22 or 23. Then again, that is not a hard and fast rule, as sometimes there are late bloomers like Tim Thomas who like Rinne came from relative nowhere to put together a pretty good initial three year run as an NHL starter after the lockout at the age of 31-33 (.914 save percentage). Then, instead of seeing his play decline like many others, Thomas only got better from there, winning the Vezina in two of the next three seasons.

Being from Europe and having less exposure to North American scouts may have contributed to Rinne waiting until his mid-twenties before winning a starting job. Rinne is not the first European netminder in a similar situation; Jonas Hiller had exactly the same number of games played through his age 28 season as Rinne. Hiller has been a quality netminder so far, but it remains to be seen how well he can sustain that level. He has also struggled in the early going this year after an extended absence due to vertigo, which has to be somewhat of a concern. Then again, even in a worst case scenario, Hiller has three years left on his contract at $4.5 million, meaning that Anaheim would have much less of a problem than Nashville if their starting goalie is unable to play up to his usual level.

Looking back six years to 2005-06, only 11 out of 30 starting goaltenders still hold down a #1 role today (and that's counting Brodeur, Khabibulin and Roloson as starters, all of whom may or may not still be in that position at season's end depending on how well they are able to stem the tide of old age). Just six of those 11 are still with the same team (Brodeur, Kiprusoff, Lundqvist, Fleury, Miller, Thomas). Competition for NHL starting jobs is fierce, and the odds are generally against a goalie holding his position for an extended period of time.

In the salary cap era, flexibility is important, and tying up a lot of money in a non-performing asset can be crippling to a team's chances of success. The evidence suggests that paying a goaltender $49 million for 7 years, particularly when that goalie wasn't an NHL starter until the age of 26, is generally not a good bet. Driving Play had it right when they concluded that Rinne would essentially have to surpass all historical comparables to outperform his contract. It is possible that Rinne stays healthy, happy and productive in Nashville, but there are simply so many things that can go wrong that the Predators will probably end up regretting their decision.

Fun With Small Sample Sizes

I couldn't help but find this amusing, even though it covers just two handfuls of games:

Chris Osgood, career against the Detroit Red Wings:

1 win, 7 losses, 2 ties, 3.35 GAA, .879 save percentage

The Florida Outlier

In honour of Ed Belfour's Hall of Fame induction, I thought to take a closer look at one of the weirdest single-season statistical anomalies that I can think of since the lockout.

The Florida Panthers have routinely been one of the worst teams in allowing shots against. Over the past seven seasons the team has had consistently good goaltending and consistently bad shot prevention. There was, however, one very large exception. See if you can pick it out:

2003-04: .924 save %, 34.0 SA/60
2005-06: .913 save %, 34.3 SA/60
2006-07: .899 save %, 28.9 SA/60
2007-08: .922 save %, 33.2 SA/60
2008-09: .925 save %, 34.3 SA/60
2009-10: .920 save %, 33.6 SA/60

2010-11: .918 save %, 31.5 SA/60

That raises the intriguing question: Did the lower shot totals in 2006-07 cause the goalies to have a correspondingly lower save percentage? Or was it the lower save percentage in the first place that caused the other team to take fewer shots against? Or were there some other factors at play?

The Panthers' starting goalie that year was Belfour, in the final season of his illustration career. Throughout his career, Eddie the Eagle typically faced fewer shots against than the other goalies that he has played with. Therefore, it seems quite likely that at least some portion of the effect is likely because of him. However, given the numbers through the rest of his career that suggest he prevents about one shot against per game compared to an average goalie, I don't think there is any chance at all that a 41-year old Belfour was able to prevent 4-5 shots per game entirely through his own efforts.

Here are the home/road splits for each goalie for 2006-07 along with the two seasons before and after:

Luongo, '04: 2.55, .926, 34.6 at home, 2.28, .935, 35.3 on road

Luongo, '06: 2.67, .920, 33.3 at home, 3.28, .909, 36.1 on road

Belfour, '07: 2.75, .901, 27.8 at home, 2.80, .903, 28.8 on road

Vokoun, '08: 2.55, .921, 32.2 at home, 2.85, .916, 33.9 on road

Vokoun, '09: 2.21, .933, 32.9 at home, 2.92, .915, 34.3 on road

From those splits it doesn't look like Florida changed their official scorer, as shots went down significantly in all venues.

Looking at the numbers for the backup goalies shows that there was a lot more to it than any individual discrepancies between Belfour and Luongo or Vokoun:

Steve Shields, 2003-04:

Starts: 3-6-1, 3.67, .874, 29.1 SA/60

Non-starts: 0-0-0, 2.37, .904, 24.6 SA/60

Jamie McLennan, 2005-06:
Starts: 2-3-2, 3.71, .895, 35.3 SA/60

Non-starts: 0-1-0, 1.43, .941, 24.4 SA/60

Alex Auld, 2006-07:
Starts: 7-13-5, 3.28, .889, 29.6 SA/60

Non-starts: 0-0-0, 4.55, .857, 31.9 SA/60

Craig Anderson, 2006-07:

Starts: 1-1-1, 2.54, .924, 33.3 SA/60

Non-starts: 0-0-0, 0.00, 1.000, 23.6 SA/60

Craig Anderson, 2007-08:
Starts: 7-5-1, 2.23, .936, 35.0 SA/60

Non-starts: 1-1-0, 2.33, .924, 30.8 SA/60

Craig Anderson, 2008-09:
Starts: 15-7-3, 2.75, .924, 36.0 SA/60

Non-starts: 0-0-2, 2.27, .931, 32.8 SA/60

The special teams numbers don't give much of a hint either, given that Florida had a below-average rate of power play opportunities and a higher-than-average rate of power plays against. Those two factors would typically result in more shots against, rather than fewer. The only major difference in the special teams numbers for '06-07 was the number of shots against per PK:

2003-04: 374 PPOA, 571 SA, 1.53 SA/PP

2005-06: 514 PPOA, 744 SA, 1.45 SA/PP

2006-07: 443 PPOA, 530 SA, 1.20 SA/PP

2007-08: 374 PPOA, 532 SA, 1.42 SA/PP

2008-09: 311 PPOA, 513 SA, 1.65 SA/PP

(PPOA=Power play opportunities against)

Jacques Martin was the coach from 2005-06 to 2007-08, which makes it even more surprising that the numbers would change so suddenly.

Team shots for:

2003-04: 2273

2005-06: 2724

2006-07: 2730

2007-08: 2549

2008-09: 2412

Given that '05-06 was skewed by a high rate of power plays, this suggests that the Panthers were a much better territorial team in '06-07.

Florida had similar personnel in '06-07 compared to '05-06. Jay Bouwmeester was the #1 defenceman and Olli Jokinen was the #1 forward. Nathan Horton (21) and Stephen Weiss (23) both probably took large steps forward in 2006-07, and may have had some impact on turning results around, although the two of them remained in Panther uniforms until 2009-10 while Florida's results dropped back to their usual levels. The defence replaced Sean Hill, Lukas Krajicek and Joel Kwiatkowski with Ruslan Salei, Bryan Allen and Steve Montador, which is probably an upgrade. Still, it remains curious that there was such a huge difference in results. If a personnel change was responsible for the sudden shot drop, why did it reappear again the following season?

Breaking down shots against by period from the Hockey Summary Project, the biggest drop in shots against from '05-06 to '06-07 came in the first period:

First Period: -196 (-20%)

Second Period: -104 (-11%)

Third Period: -113 (-13%)

Overtime: 0 (0%)

I looked at the leading and trailing numbers, and 2006-07 does not appear to be an outlier in that regard. The Panthers may have played a more low-event game early on, suppressing shots against in the first period. However, there was clearly a significant shot prevention effect throughout the entire 60 minutes.

The last thing I thought to look at was blocked shots:

2003-04: 463 home, 549 away, 1012 total

2005-06: 403 home, 454 away, 857 total

2006-07: 501 home, 520 away, 1021 total

2007-08: 410 home, 570 away, 980 total

2008-09: 485 home, 620 away, 1105 total

2009-10: 584 home, 819 away, 1403 total

2010-11: 507 home, 618 away, 1125 total

This may explain some of the effect, perhaps reflecting a strategic shift by the Panthers' defence to adjust for the team's weaker goaltending. There is still a large amount left unexplained, however, as combined blocked shots and recorded shots against still show a drop of about 300 during 2006-07 compared to the seasons before and after.

For what it's worth, given that shot quality measures have tended to be unreliable and not predictive year-to-year (see Gabe Desjardins' summary rant on the subject), Alan Ryder estimated the Panthers' shot quality at 0.957 in 2005-06, 1.014 in 2006-07, and 1.008 in 2007-08 in his annual NHL reviews. Numbers below one indicate easier than average shots against while numbers above one indicate more difficult than average shots. That would support the hypothesis that the Panthers were able to block or prevent more long-range shots in '06-07 compared to '05-06. However, the shot quality estimate was similar in '07-08 compared to '06-07, despite the large increase in shots against.

I still don't entirely know what to make of the Florida's team defensive performance in 2006-07, it looks like a confluence of factors was responsible for the one-year dip in the team's shots allowed. It makes sense that the Panthers would have played more conservatively that year because of the goaltending change, leading to more blocked shots. The metrics suggest that team improved in terms of puck possession and spent more time at the other end of the rink, which helped cut down shots against. Switching goaltenders from Roberto Luongo to Ed Belfour may also have had a shots against impact of 1-2 shots per game. Other contributing factors could have been improved penalty killing as well as facing fewer opposing power plays against as players adjusted to the new post-lockout rules.

Yet while that may account for the changes from '05-06 to '06-07, it still doesn't explain why the Panthers dropped back to their usual awful shot prevention level during '07-08 and following years. It looks like coach Martin and his skaters put everything together for one season, but then for whatever reason were unable to recreate that success. Unfortunately for them, the one season where they managed to put together a pretty strong team effort (.898 win threshold, 13th best in the league) was also the one season where the team's goaltending was poor, and the result was yet another non-playoff appearance for the Florida Panthers.

Why Mike Richter Might Be Overrated

Mike Richter is a goalie that is remembered pretty fondly by many. That shouldn't be too surprising, as he played a long career on a big market team, he won a Cup, and he represented the U.S.A. admirably in a number of international tournaments. However, looking at his career numbers and especially his Vezina and All-Star voting record makes it pretty clear that Richter was not a member of the goaltending elite. Pretty much the only thing that was elite was his paycheque; in his entire career nobody ever gave Richter a first place vote for the postseason All-Star teams.

I recently realized, however, that Richter may not even have been as good as his numbers suggest. The reason is that there is some evidence that Richter may have benefited from a generous home scorer during the prime of his career in New York. Madison Square Garden has long been known as a rink that produces abnormal statistics for things like shot distance. During the mid-1990s, they might have been recording some screwy numbers when it comes to total shots as well.

For seven consecutive seasons from 1991-92 to 1997-98, Mike Richter's backup goalies faced a higher rate of shots against at home than on the road. Richter faced a higher rate himself in five out of the seven seasons and narrowly missed the two other times, finishing 0.8 lower in '91-92 and 0.5 lower in '97-98. Having a higher rate of shots against at home in any season is relatively rare, given that teams typically play better at home. When it happens seven years in a row, it is a major outlier.

Richter ('92-'98):

Home: 99-60-24, 2.71, .911, 30.5 SA/60

Road: 71-66-13, 3.01, .896, 29.1 SA/60

NYR backup goalies ('92 to '98):

Home: 45-25-18, 2.81, .910, 31.2 SA/60

Road: 47-60-14, 3.01, .894, 28.4 SA/60

The home/road GAA splits are quite normal. The backups had a very skewed record at home vs. on the road, which implies that the Rangers played a lot better in front of them at home. Given that, one would not expect shots against to go up by nearly 3 per game. The only effect that could somewhat account for that would be score effects. The Rangers playing to the score might explain why the backups had more of a differential between home and road than Richter did, but Richter himself had a more typical home/road split yet still had a higher shots against rate at home. Thus it seems that all Rangers goalies were getting extra credit for saves at MSG.

It is at least possible that the Rangers played a very high event game at home, although if that were true it would be expected that the goalie's home save percentages would have dropped or stayed the same rather than rose substantially compared to their numbers on the road.

Richter's numbers at home and on the road were almost identical to those of his backups. For a several of those seasons that was nothing to be ashamed of, as quality veteran John Vanbiesbrouck was Richter's playing partner, but for the rest of it the Rangers had a fairly undistinguished collection of backups, led by Glenn Healy. Healy's numbers cratered once he left the Rangers to play on the Leafs, which probably had a lot to do with age, but may have also had something to do with artificially inflated home numbers.

League average over the period was roughly .898. Richter's overall save percentage checked in at .905, suggesting that he was a pretty valuable goalie, worth nearly two wins above average to his team per season. However, the numbers show that nearly all of his excess value was coming based on the performance he recorded on home ice.

If we assume that in reality Richter faced the same rate of shots against at home and on the road and that the difference was due to generous scorekeeping, his home save percentage would drop to .907 and his overall save percentage for the period would fall to .902. If Richter actually faced one fewer shot against per 60 minutes at home, his numbers would fall even further to .904 at home and .900 overall, a result that would leave him about 20 goals above an average goalie. That's still pretty good, but it would have a pretty dramatic impact on Richter's career numbers. It would cost him about half of his career value in terms of goals above average, causing him to plunge well out of the "decent starters" range on this list.

On the road, Mike Richter was almost exactly an average goalie, based on his save percentage numbers. At home, his numbers were up among the best in the league. Given that the numbers of his backups followed the same pattern, it seems unreasonable to conclude that this was due to anything related to Richter himself. Furthermore, it seems unlikely that the Rangers would have been successfully suppressing shot quality against at least 10% better than an average team while playing at home yet for some reason choosing not to do the same thing on the road. Perhaps there were some team effects, but on the whole it seems like the best explanation is probably that there was some degree of shot padding at Madison Square Garden in the mid-1990s which boosted Mike Richter's statistics.

The Razor-Thin Edge of a Winning Reputation

If you ask hockey fans who is the best goalie in the league, you'll probably usually hear a handful of names listed before Cam Ward enters the conversation. But ask instead which goalie they would want if they were playing in game 7 of the Cup Finals, and don't be surprised if you'll hear Ward mentioned in the first 2 or 3 names.

Why is that? Simply because Cam Ward has a reputation as a winner. He won a Stanley Cup, he has a Conn Smythe, he's only lost one playoff series in his career. And if you get into an argument with a Hurricanes fan about their goaltender, it usually doesn't take long at all for them to break out Cam Ward's career record in game sevens: 4 wins, 0 losses.

That is an impressive statistic, to be sure, but like any win/loss record it is dependent on Ward's teammates and opponents. The reality is that Cam Ward was very, very close indeed to being 0-2 in his career in game sevens.

In his first career game 7, Ward allowed Jochen Hecht to bank in a shot from behind the net with 5 seconds left in the second period to give Buffalo a 2-1 lead. In the entire 2006 playoffs, teams leading after two periods were 60-9, but Ward's Carolina teammates scored three times in the third to bail out their goaltender and advance to the Cup Final, where the 'Canes again won in seven games.

In 2009, Carolina went to game seven in the first round against New Jersey, and again found themselves trailing in the third period. This time the outlook was even bleaker, with New Jersey leading 3-2 at home with less than 90 seconds remaining before Jussi Jokinen and Eric Staal combined to stun the Devils and snatch victory from the jaws of defeat. Excluding that game, teams were 59-2 in the 2009 playoffs when leading after two periods, which shows just how miraculous the 'Canes comeback was.

If not for two terrific comebacks, Cam Ward would be 0-2 in game sevens and nobody would have the impression that he was a particularly clutch goalie. That is not to say that Ward had no impact at all in those comebacks; clearly another goal or two against would have made much more difficult for his team to pull off the improbable. Nevertheless, it is an entirely reasonable counterfactual to suggest that with a bit less help Ward could easily have had not just no Cup and no Smythe, but not even a single game seven victory to his credit.

The game seven argument is also fairly meaningless because it represents such a small sample of Ward's career. Win/loss record aside, it is correct to say that Ward has performed well in game sevens (1.85 GAA and .932 save percentage), but we don't have to go very far to find examples of important games where Ward didn't manage to get the job done. Twice he has played the final game of the season in "win and you're in" situations, and both times he gave up four goals in a Hurricanes loss that eliminated them from playoff contention. Ward also gave up five goals in a world championship final loss on Canadian ice in Quebec City in 2008. These performances show that his performances in must-win games are far more variable than the "4-0 in game sevens" narrative implies.

Cam Ward is a very good goalie, and his recent performance has been trending solidly upward (.919 save percentage over the past three seasons). Is he unusually clutch in pressure situations? I think we still have to simply wait and see.

It's Not Only About the Stats

One of the goalies off to a red hot start this season is Kari Lehtonen (8-1-0, 1.75, .947). Moving to Dallas has rejuvenated Lehtonen's career, and while he obviously won't maintain those lofty numbers for an entire season, the Finnish goalie had a decent year last year and is still only 27 years old. Few people have questioned his talent, but Lehtonen may finally have figured out how to combine that with the hard work and professionalism needed to perform as one of the league's better netminders.

Lehtonen can be used as an example both of the power of statistics and of how other factors can be important beyond the numbers. I touted him for a while prior to his trade to Dallas as a talented NHL goalie who was stuck in a bad situation in Atlanta, based primarily on the strong even strength save percentage results he put up in the early part of his career. However, at the same time observers were pretty much unanimous that Lehtonen was not properly utilizing his talent through a lack of preparation and repeatedly showing up overweight and out of shape, which led to a lot of his injury problems. This year all reports are that Lehtonen has finally put in the off-ice work needed to get into great shape. There is a noticeable difference between what he looks like now and what he looked like when he was playing in Atlanta.

Off-ice training and mental preparation and hours spent on the practice rink working on technique drills are all very important for a top professional goaltender. However, typically goalies who make it to the top levels of the sport have developed the training and work habits they need, especially since for most of them it is a fairly long road to the get to the NHL and if they weren't putting in that time then they would have washed out well before they made it to the show. If everyone is working hard, then it doesn't become much of an advantage for anyone, and it becomes much less likely that a goalie who has been training hard year-round for a number of years will suddenly make a huge leap forward primarily based on those off-ice factors. On the other hand, someone who is very talented but doesn't take the steps to maximize his talent would be a candidate to see his performance improve if he is able to finally put everything together, which may be the case for someone like former second overall pick Kari Lehtonen.

There's always good reason to be skeptical about claims justifying early season success. This is the time of the year where dozens of articles are written by reporters claiming that a good offseason of training is fully responsible for a player's 30% shooting percentage through 10 games, and is the reason why that player is going to hit the 40 or 50 goal mark for the first time in their careers. Needless to say, those players always regress significantly by season's end, as luck was almost certainly a bigger factor than anything that happened in a weight room or on a practice rink. On the other hand, we shouldn't completely dismiss the human factors either. I'm still far from convinced that Kari Lehtonen will end up in Vezina contention, but it will be interesting to see how long he can sustain his sizzling start. At the very least there appears to be a good chance that he is headed for a career year in 2011-12.

The Ecological Fallacy

I found a scholarly name for the tendency to rate goalies on winning teams as better than goalies on losing teams. It's the ecological inference fallacy. You can read all about it on Wikipedia, but in short it is the assumption that all members of a certain group share the same characteristics of the entire group. To quote Wikipedia:

If a particular sports team is described as performing poorly, it would be fallacious to conclude that each player on that team performs poorly. Because the performance of the team depends on each player, one excellent player and two terrible players may average out to three poor players. This does not diminish the excellence of the one player.

Nor does it boost the performance of an average player who happened to have great teammates. Avoid the ecological inference fallacy and give credit where it is due, based on an individual's contribution to the team effort regardless of the final result.

To see this type of thinking in action, go read one of the post-game report cards that fans put up after their team plays. In many cases, when a team wins there are As and Bs across the board. Yet when the team loses, everyone gets Cs and Ds.

Check out, for example, Vancouver blog Nucks Misconduct's report cards from last year's Cup Finals. In Vancouver wins, the average score for Vancouver fourth-liners was 9.1. When the Canucks lost, the average score for fourth-liners was 7.0. Can the Canucks' losses be blamed on a line that barely played and had little impact overall on the series? Of course not. It just so happens that when Roberto Luongo was making saves and the other forward lines were scoring then Victor Oreskovich, Tanner Glass, Jeff Tambellini et al looked better by association. In contrast when Luongo got shelled and the Sedins were shut down, the same guys playing their usual 6 or 7 crash-and-bang minutes without a goal for or against ended up getting hung with the same mediocre grade as the stars who were actually driving the bus. That's the ecological fallacy in action.

I'd say this logical error explains quite a few of the most common mistakes made in rating goalies. Add in the base rate fallacy that causes people to exaggerate their praise or criticism for a goalie's performance by not properly factoring in the play of a typical replacement ("Without Lundqvist, the Rangers would have lost at least 10-1!"), plus the fundamental attribution error which makes people lean towards personality-based explanations for team successes or failures ("Carey Price's teams will never win in the playoffs because he lacks mental toughness"), and finally availability bias ("I don't remember any of Mike Liut's career except for that Canada Cup Final where he let in 8 goals, but that game proves he was an awful clutch performer"), and you've probably covered 95% of the rest of them as well.

Desire and Success

(I wrote the rough draft for this short post at the end of last season but never ran it, and was recently reminded of it while watching the Winnipeg Jets get beaten 5-1 in their home opener by the Montreal Canadiens. That was not entirely the same situation as the one described here, given that Montreal was certainly looking to add two points just like every other team this early in the season, but one still would have thought that the Jets players would have that extra motivation to kick off a new era of NHL hockey in Winnipeg with some success. Nevertheless, they still came up four goals short.)

There are abundant cliches in sports that attempt to relate winning to effort level. How many times have you heard an announcer say something like "they just wanted it more" in an attempt to explain why one team emerged victorious while the other team did not?

I've posted before about effort-based explanations being largely ridiculous at the professional level given the stakes involved, but there are some situations where there is in fact a clear imbalance in incentives between two teams, such as late in the season where one team is already out and the other is facing a must-win game. What happens in that case, does the team that wants it more always win?

During the last weekend of the 2010-11 regular season, three teams (Carolina, Chicago and Dallas) all controlled their own destinies and all only needed to win their final game to clinch a playoff berth (the Hawks actually only needed to get to OT). None of their opponents had anything to play for, as all three of them were either eliminated or could not change their playoff seeding. Carolina and Chicago were playing in front of their home fans, while Dallas got a non-playoff opponent in the Minnesota Wild. In addition, Detroit was the only one of the three opponents that went with their starting goalie. In every case, the situation looked very favourable for the team that needed to win to get in, especially if "wanting it more" is a good predictor of success in the NHL.

Those three teams combined to go 0-3. Every playoff home date is worth millions to their franchises and earning the opportunity to compete for a Stanley Cup has huge intangible benefits to NHLers yet all three teams squandered their chance. The Chicago Blackhawks did manage to qualify for the postseason, but only because they got lucky when Dallas also failed to seal the deal.

The statistical case for the heavy role of luck in hockey has been well-made, but there remains a resistance for many traditionally-minded hockey fans to accept numbers-based conclusions. That's why sometimes it is good to use other types of arguments (I particularly like the game-charting ones, like this one, for example, because they can't be simply dismissed out of hand by the people who have an ingrained anti-stats outlook). I'd submit that the fact that a team can have a skill advantage and home ice advantage and a starting goalie facing an opposing backup and a huge advantage in incentives and yet can still lose the game is a simple yet powerful observation that supports the heavy role of luck in the sport of hockey.

Goalie Effects on the Jack Adams Award

Last week Hockey Prospectus asked me to predict the 2011-12 award winners as part of its ongoing season preview. I usually just throw out names for that kind of thing because it's a pure guess anyway, and to be honest I'm not at all sure that I know much more than the next guy about who is going to be this year's best rookie or best defenceman or best coach.

Anyway, for Jack Adams I filled in Guy Boucher's name after not much more than a few seconds of thought about it. I figured he did a pretty good job last year and attracted lots of attention during Tampa's run to the Conference Finals, and if the Lightning finish with another 100+ point season maybe people would write his name down on their ballots.

Not too long afterwards, there was an insightful comment on Coppernblue by dkball7: "DeBoer should be everyone's pre-season pick for Jack Adams. As long as the team's PDO regresses to 100%, he will look like a genius."

In hindsight, it makes sense that I should not have picked a coach on a team that performed well the year before, like Boucher's Lightning, but should instead have picked one on a team that underachieved, like DeBoer's Devils. The Coach of the Year often goes to a candidate on a team that massively improved relative to the year before, which often is caused in large part by a significant swing in percentages from one year to the next.

I looked at the teams for each of the Jack Adams award winners since 1990, and compared each team's performance during the year where their coach picked up his hardware to the team's performance during the year prior and the year following. Here are the percentages, with everything adjusting to an average level of 10.0% shooting/.900 save percentage:

Prior to Jack Adams Year: 10.2% shooting, .897 goaltending, 99.9 PDO

During Jack Adams Year: 10.7% shooting, .909 goaltending, 101.6 PDO

Following Jack Adams Year: 10.4% shooting, .905 goaltending, 100.9 PDO

Only one out of 21 of the teams that produced a Jack Adams award winner posted a below-average save percentage during that season. In contrast, during the prior year, 13 out of those 21 teams had a below-average save rate. During the year after, the goaltending still remained strong for the most part, with just three teams dropping back to below-average save numbers.

The following year numbers imply that either the teams had slightly above average shooting and goaltending talent as a whole, or the award-winning coaches themselves combined for a positive effect on the team's numbers.

I think the goaltending numbers in the year after are more likely to reflect goalie talent than the coach's system of play. There were a number of top goalies represented (e.g. Hasek, Brodeur, Luongo, Thomas). Overall, the goalies had an average career save percentage of .906 over a period where the league average save percentage was .902. Considering that some of their careers stretched back further than 1989-90, when the league average was even lower, it seems reasonable that the combined goalies were about .005 better than league average, although it should be at least noted that some of the coaches in the sample have been known to affect shot quality, particularly guys like Burns and Lemaire, which may account for a small part of the above-average result.

It is interesting that many of the teams had the same goalie during the Jack Adams year as they did the season before. Eleven of 21 teams had the same starting goalie, with all of them playing a relatively similar number of games as well. Here are the save percentage numbers for the season prior, during and after, split out by whether the team had the same netminder as the year before (numbers adjusted again to league average with a baseline of .900):

Same goalie: .901, .909, .907

New goalie: .893, .909, .904

The largest improvements came for teams that brought in new goalies, obviously, but a good goalie coming off of an average season can also have a big impact in improving a team's fortunes and getting his head coach some extra attention.

The numbers do suggest that a lot of things simply went right for coaches during their winning years, but I certainly don't want to imply they had no effect at all. There is, for example, the shots for and against evidence, which shows that the teams also had a substantial improvement in their underlying possession metrics during the Jack Adams winning years. Teams playing for a Jack Adams winning coach were also more disciplined than average, as well as more disciplined compared to the year before.

Prior to Jack Adams Year: 1.005 SF/SA ratio, 362 PPOA

During Jack Adams Year: 1.064 SF/SA ratio, 340 PPOA

Following Jack Adams Year: 1.044 SF/SA ratio, 350 PPOA

Goaltending and shooting luck do not determine everything, but a lot of what the best coaches do is difficult to judge and rate, especially from a distance. For that reason, exernal factors can often come into play. As the old hockey saying goes, "Show me a good coach and I'll show you a great goaltender."

In summary, if you want to maximize your chance of being named the NHL's best coach, you should try to get a job on a team that either had awful goaltending the season before and made a move to address that weakness, or where a good goaltender had a down year. Either one of those scenarios would give a coach the best chance to see his team's percentages swing around in a hurry, leading to a significant improvement in the standings. That will in turn cause many people to think there must have been some coaching magic at work, and if you're lucky the awards recognition will soon follow.

To make a better Jack Adams prediction, we should apply this logic to this year's teams, and find a team with good goaltending that had weak goaltending last year and is likely to improve in the standings. The Flyers and Caps brought in new top-flight goaltenders, but both actually had pretty good save percentages last season. Several other teams have also improved in net but are still expected by most to finish near the bottom of the league and as a result are unlikely to produce a Jack Adams winner (Islanders with Nabokov, Senators with Anderson, Avalanche with Varlamov).

There are five teams that had subpar goaltending last season as well as overall PDOs below 100 that could be primed to do better in 2011-12:

New Jersey: 7.3 SH%, .906 Sv%

Toronto: 9.0 SH%, .907 Sv%

Tampa: 9.3 SH%, .903 Sv%

St. Louis: 9.5%, .902 Sv%

Columbus: 8.4%, .900 Sv%

Guy Boucher's Lightning show up on this list, suggesting that my random intuition may not have made that bad of a pick after all. There has to be some concern, however, for the fact that starter Dwayne Roloson is turning 42 next week. That said, he should still be better than the combo of Mike Smith and Dan Ellis (.894 last season), and Mathieu Garon will also provide improved backup goaltending. The Lightning may see their shooting regress slightly, but if they can duplicate last year's outshooting results and if Roloson can hang together to give them better goaltending then they will be definite challengers in the East. That could put Boucher in the conversation for Coach of the Year, but I'm not sure if it would be enough, especially if there is someone else out there who oversaw a much larger improvement in terms of wins and losses.

Age is also a concern for Martin Brodeur, but nobody is expecting a .903 again, and the New Jersey shooters are virtually guaranteed to improve (7.3% is a major outlier for a team shooting percentage). Jaroslav Halak also would be a good candidate for a bounceback year in St. Louis. If James Reimer is the real deal with .920 talent in the NHL he could certainly win Ron Wilson a trophy this year, but that still doesn't look to me like a good bet. As long as Columbus is going to continue to bet all their chips on Steve Mason I'm not sure I'll be expecting above-average goaltending in Columbus, although there likely could be some improvement there, perhaps even enough to get into the playoffs depending on luck and how well the rest of the team plays.

After considering this evidence, I think Peter DeBoer is the best pick for the 2012 Jack Adams, with all signs pointing to the Devils coming back strongly this season. There is always the chance, of course, that a team loses a star player and keeps on trucking, like Pittsburgh did in Dan Bylsma's award-winning campaign, or that one of the league's best teams has a spectacular year and cleans up at awards time, or that a team with a lot of new additions like Buffalo really comes together and climbs the standings. At the end of the day, the most likely winner is probably the coach who saw the largest improvement from the year before, and for this season the team with the best chance to improve is almost certainly the New Jersey Devils.

Why Doesn't Carolina Get Better Backup Goalies?

As I pointed out in my last post, the depth of talent among the league's goaltenders has improved substantially in the NHL over the last 15 seasons. Quality talent has never been more readily and cheaply available than now. At the same time, the salary cap has increased parity across teams, resulting in close playoff races nearly every season in both conferences. The simple conclusion to make based on this fact is that no team should accept awful backup goaltending. It doesn't cost much more to get average goaltending than it does to get replacement level goaltending, and bubble teams that are content to let a washed-up veteran or an over-his-head youngster play backup minutes are jeopardizing their playoff chances in doing so.

There has been one NHL franchise in particular that has seemed to not understand this principle, having been repeatedly burned by weak backup goaltending. That team is the Carolina Hurricanes.

Last year, Carolina finished two points behind the Rangers for 8th in the East, despite a terrific season by Cam Ward (37-26-10, 2.56, .923). Ward actually had a better win/loss record than Rangers starter Henrik Lundqvist (36-27-5), but the decisive difference that sent the Blueshirts to the playoffs at the expense of the 'Canes was what happened when neither #1 netminder was in the net. Solid veteran Martin Biron had a .923 save percentage and an 8-6-0 record in New York, while youngster Justin Peters was lit up in his infrequent playing time in Carolina (3-5-1, 3.98, .875). While Carolina saved money with Peters' $525K cap hit, it would have only cost them an extra $350,000 to pay a guy like Biron.

According to Capgeek, the Hurricanes had $9.5 million in salary cap room last season. Would the team's ownership have been willing to spend an extra $400K if they knew there was a good chance it would have helped the team earn the extra three standing points needed to earn millions in revenue from at least two extra playoff home dates? They would surely have agreed to that deal in a heartbeat. The Canes' management can't be entirely faulted, as Peters was a four-year minor league pro coming off a pretty good season in the AHL and he was probably at least somewhat unlucky to post numbers that terrible. On the other hand, one of the main reasons to get a good #2 option is to minimize the risk of a relying on a unknown quantity.

It was a similar story in Carolina in 2007-08. Ward wasn't quite as good back then, but much of the roster was just two years removed from winning the Cup. Despite a .904 save percentage, Ward's record was 37-25-5, easily good enough to put the 'Canes in playoff position. The problem was that backups John Grahame and Michael Leighton combined for a brutal 6-8-1, 3.58, .878, and the team was again left one win short of making the playoffs.

Backup goaltending left the 'Canes out of the playoff picture for a third time in '99-00, as the team finished an agonizing one point out after their backup goalies combined to go 3-7-1, 3.22, .883. Apparently the organizational indifference to goaltending depth was carried over from Hartford, as the Whalers had more or less the same thing happen in 1996-97 (two points out of the playoffs despite a great year from Sean Burke because the backups combined for .887 and a 10-17-5 record).

Over the last 17 seasons, the numbers are pretty staggering for the Whalers/Hurricanes franchise:

#1 goalies: .532 win %, 2.60, .911

Backups: .413 win %, 3.15, .890

Those splits aren't entirely fair because there may have been a few times when the preseason #1 goalie was supplanted by a backup (as was the case in 1997-98 with Trevor Kidd outplaying incumbent starter Sean Burke, for example). However, it is still perfectly correct to say that Carolina/Hartford has had mostly awful backup goaltending for the better part of two decades, and that has likely had a significant impact in causing the team to fall short of the playoffs on multiple occasions.

Scouting, evaluating and predicting goaltender performance is always difficult. Not every bet is going to pay off, and many organizations get decisions wrong. Take Buffalo, for example, a team that has developed and employed a number of top-quality netminders in recent years, yet still paid Patrick Lalime $2.65 million for three years of service where the Sabres went 9-26-5 in games where he got the decision. Lalime probably cost his team a playoff spot in '08-09, posting a 5-13-3 record as the Sabres fell just two points short.

There have been a number of other teams that were left outside the playoff pictures because of the performance of their backup goalies. Sometimes teams missed out because a goalie they counted on to be a starter or take on a significant workload in a platoon role simply had an awful season (e.g. '06-07 Avs, '08-09 Predators). Others simply had a few options behind their starting goalie ('09-10 Rangers, '06-07 Maple Leafs).

Most of these examples of weak backup goalies are dealing with small sample sizes, so it may not be entirely fair to blame the goalies. All the standard problems of relying on win/loss records for goalies apply, although in nearly all cases they had awful save stats as well. There may also have been other factors at work. Perhaps they weren't playing a favourable schedule, or maybe some of them just had puck luck go against them for 200-300 shots. There is the very large advantage of hindsight available to us now in pointing out some of these teams' decisions. Yet when there is a long-term trend of undeperformance, as is the case in Carolina, a reasonable criticism can certainly be advanced about the way the team handled their goalie situations.

The overall point is that while it is not smart to pay huge money for goalies, the depth of available goaltending talent means that you should never, ever have to settle for bad goaltending. If you have a hole on the roster, it really doesn't cost much more to pick up a veteran or an up-and-coming talent from Europe than it would to roll the dice on an unproven minor-leaguer in your system. For a penny-pinching playoff bubble team, it's probably well worth it to invest some decent money in goalie scouting and development or free agency to reduce the risk of having a not-ready-for-primetime backup come in and sink the season.

Over the summer Carolina signed Brian Boucher to a two-year deal worth $950,000 per year, which may be a sign that the organization is more willing to loosen up the purse strings for their backup position. Then again, the 'Canes haven't always gone with a backup like Justin Peters; over the last 17 years there have been a number of veterans who were brought in but didn't pan out. I think there is a good chance that pattern may repeat again with Boucher, a guy turning 35 in January with a post-lockout save percentage of .902. Boucher is expected to take more of the load off of Cam Ward this season, but he'll need to deliver good enough results, especially if the Hurricanes again find themselves in a dogfight in the middle of the Eastern Conference. If he does not, then the hopes of Carolina fans may yet again be dashed by their backup goaltender.

Goaltending Parity

I was flipping around Hockey Reference the other day, looking at the results from the 1995-96 NHL season. That was a strange year in many ways. The still-terrible expansion franchises in Ottawa and San Jose were both doing their part to skew the standings. In the West, Detroit cleaned up, winning 62 games to set a new league record, while the Wings' bitter rivals and eventual Cup champions Colorado Avalanache were the league's second-best regular season team, leading to a rare situation where 10 out of 13 Western Conference teams finished below .500. In the East it was the exact opposite situation, with nine teams finishing at 86 points or better, including the defending Stanley Cup champions from New Jersey who missed the playoffs despite a record that would have ranked them fourth in the West.

It was a unique year for goaltending as well, particularly as many of the big stars had off-seasons or down years. Patrick Roy got traded by Montreal, Ed Belfour had an off-year and was in the process of losing his starting job in Chicago, while Dominik Hasek and Martin Brodeur both played well but missed the playoffs. All that combined to allow a 22-year old sophomore named Jim Carey to walk off with the Vezina Trophy, all of the voters completely unaware that he would have only three seasons remaining in his professional career.

The league was still full of the old guard of standup goaltenders, many of whom were past their prime or struggling to keep up with the changing game. The result was a huge spread in the save percentage numbers among starting goalies, all the way from Hasek at the top with .920 down to Don Beaupre at .872.

The large gap in results was likely influenced by a higher level of shot quality differences across teams than we see today, particularly for goaltenders representing the Sens or the Sharks. However, even within teams there was a broad range of performance numbers, suggesting that goaltending was a real difference-maker back then. Going through team by team, it is impossible to avoid noticing that the starters almost always had much better win/loss records than the backups.

Compiling the numbers league-wide demonstrates this point (I just took the goalie with the most games played that season for each team to represent their "starter"):

Starters: 611-512-156, .539

Backups: 318-417-118, .442

The totals can be skewed a bit by some team's starters playing more games than others, but even if you take the average of each team's starter and backups you get .536 and .436, a full .100 increase in winning percentage with a team's most-used netminder in the game.

Only five out of 26 teams had a better win/loss record with their backup goalie(s) in the game. Only three more teams had their backups post a win percentage that was even within .050 of their starter.

Let's compare that to 2010-11:

Starters: 838-605-186, .572

Backups: 392-328-111, .539

That gap is much closer, even more so when the averages are taken for each team (starters .564, backups .547). Thirteen out of 30 teams had a better winning percentage when their top goalie didn't get the decision, and eight more had a difference of less than .050 between their starter and backups.

These results strongly confirm what analysts all over the place have been pointing out regarding today's goalies, that there is far more depth at the position today than in prior decades. The two big factors in the increased level of talent was the technical revolution sweeping the game and the increasing influx of European goaltenders.

In 1995-96, only 7 out of 78 goalies in the league were European (I don't count Olaf Kolzig as a European product, he grew up in Canada and played all his minor hockey there). They combined to play a total of 247 games.

By last season, there were 29 Europeans among the league's 87 goaltenders, meaning the percentage of Europeans rose from 9% to 33% in just 15 years. The European goaltenders also combined to play over four times as many games (1077) as they did in 1995-96.

Based on this evidence, it is perhaps unsurprising that there appears to have been a stronger correlation between goalie talent and championships won in the mid-to-late 1990s than in the post-lockout era, where the best goalies have mostly struggled to achieve much team success. Today, it's simply much harder to stand out from the pack.

The Significance of Two Vezinas

As everyone knows, strange things can happen to a single goalie over the course of a single season. Jose Theodore can win the Hart, Jim Carey can win the Vezina, Andrew Raycroft can win the Calder. There are a lot of goalies who had one great season mixed in with a nondescript or average career. Seen in retrospect, that year seems to be most likely founded on a lot of luck and perhaps aided by teammates, or else perhaps came at a point in time where the rest of the league was not yet aware of and able to exploit that netminder's particular weaknesses and tendencies. In a few cases, it is likely that the surprising goalies were legitimately performing at a high level for a brief peak, before later falling off to a lower standard of play as as result of injuries, age, or some other factor.

But two great seasons, that's a different story.

Those who are interested in the Hall of Fame debate often look at comparables, trying to determine if a player with a specific profile has company already in the Hall of Fame. For example, if all players who finished top-10 in scoring X number of times are already inducted, then it seems reasonable to view that as support for any player who achieved that same number of top finishes.

For goalies, there is a very simple Hall of Fame cutoff that so far works with 100% success: Every goalie with 2 or more First Team All-Star selections is a Hall of Famer.

That is not to say that every goalie in the Hall of Fame was voted at least twice as the game's best goalie. Several of them only achieved that honour once, and Gerry Cheevers never did it at all. But everyone with two is in, and that brings us to Tim Thomas.

Tim Thomas had one of the most impressive goalie seasons ever last year, especially when the playoffs are taken into account. Including the postseason, Thomas played in 82 games and stopped 93.9% of the shots against him. His even strength save percentage over that stretch was simply off the charts at .948. That's a level that nobody has come close to since Dominik Hasek was in his prime. Assuming no shot quality or scorer bias effects, Thomas was about 45 goals better than a league average (.913) goalie during the regular season, and another 23 goals better in the playoffs. Thomas faced 33 shots per game in the playoffs and still ranked #1 in GAA. In short, he not only had video game numbers, but he was absolutely dominant at the most important time of the year. In my opinion, Thomas should have won the Hart Trophy.

Does that mean Tim Thomas is a Hall of Famer? With a Cup and a Conn Smythe to go with his two Vezinas, his trophy case is already worthy of the Hall, but longevity really hurts him in any such discussion. Thomas was already 31 years old when he first won an NHL starting job, and at the age of 37 he only has 319 career regular season games played. It remains to be seen how many campaigns are left for a goalie who thrives on his athleticism, but if Thomas can keep his game at a high level for another three or four seasons, he would at least be approaching the numbers that would make it seem like much less of a long shot (500 career games, 50 career shutouts, a career save percentage in the .920 range). At least it wouldn't if the Hall is open to rewarding dominance, rather than just counting longevity and career compiling. With his current 90th place ranking on the career wins list, Thomas isn't likely to end up among the all-time leaders in any of the counting categories.

It will be interesting to watch the conclusion of Thomas' career, to see whether he is the modern-day Johnny Bower or if he merely has a short but meteoric prime. Either way, he will be an interesting test case as a Hall of Fame candidate a decade or so down the road.

Pre-Lockout Chris Osgood Was Not Clutch

"I always loved the fact that when we were tied or the games were close in the last 10 minutes, I'd shut the door and we'd win the game," he said.

"I knew how I did my job on a great team." (Chris Osgood)

I figure that after a long silence in this space, it would only be fitting to get back into it with one of my favourite topics: the overratedness of Chris Osgood.

Actually, to be honest, I wish I didn't have to make posts like these. The recently-retired Osgood should be remembered as a guy who overcame all kinds of obstacles and worked hard to outlast a ton of other goalies who may have had more natural talent. I thought this was a terrific read that showed Osgood's dedication in rebuilding his game to incorporate modern techniques. I get why Detroit fans loved their scrappy netminder, it's great that fan bases identify with blue collar guys who give it everything they have out on the ice.

But, unfortunately, most people still can't separate individual play from team success. In their eyes, 400 wins and 3 Cups make you a Hall of Famer, no further analysis required. They portray Osgood as something that he simply never was, and that's not fair. Ergo, as long as there are specious and silly arguments being thrown out in his favour by people with influence within the hockey community, then I'm going to keep making posts to set the record straight. Sorry, Ozzy, it's nothing personal, I just believe that credit should go where credit is due.

One of the points I have repeatedly tried to make regarding Chris Osgood is that even if you think he was a supreme clutch performer in the 2008 and 2009 playoff runs, that should still not have any impact at all on how you rate his playoff performances from earlier in his career. Many fans seem to have a tendency to revise their evaluations of a player based on their late-career performance, and that makes no sense.

I think Osgood got a lot of help in 2008 and a lot of favourable bounces in 2009, but I will still readily concede that it is much, much more supportable to assert that Ozzy was clutch in those two seasons than it is to claim that Osgood was clutch in the playoffs from 1994-2004.

It would in fact be far, far easier to make the case that Osgood was a spectacular choker in his early career than it would be to argue that he made the big saves when his team needed them most.

Here's the data to support that statement. I looked at Chris Osgood's playoff numbers in the third period based on the game score from 1994 to 2004 (source: Hockey Summary Project). Without play-by-play records to separate out the shots by score, I chose to measure Osgood's GAA in each situation:

Trailing by 2+: 1.02
Trailing by 1: 2.10
Score tied: 2.98
Leading by 1: 2.53
Leading by 2+: 1.88
Overtime: 3.18

The most high-leverage situations with the highest loss in win probability from allowing a goal against are when a team is tied or leading by one goal late in the game. It's hard to miss the observation that these precise situations are the ones where the other team was most likely to score on Osgood. Coincidentally, his goals against numbers dropped in situations where the penalty of a goal against was the lowest. That is not the expected profile of a goalie who was giving up goals when it didn't matter and slamming the door when the game was on the line.

Grouping the numbers into just two groups, the most high-leverage situations (tie game in third & OT and preserving a late one-goal lead) and then everything else, you get these numbers:

OT/tied/up by 1: 2.81

All other situations: 1.71

Of course his teammates playing to the score would have had an impact on those numbers, but did the Red Wings allow over 60% more shots against in the most pressure-packed situations? There's simply no way that was the case, which means that Osgood's individual numbers definitely dropped as the penalty for a goal against rose.

Assuming the shots were distributed evenly regardless of score, Osgood would have had an .881 save percentage with the score tied or his team leading by one, compared to a .924 save percentage the rest of the time. The one situation where it is possible to fully separate out Osgood's save percentage is overtime, where he let in 6 goals on 46 shots for a wholly unimpressive .870 save percentage.

In an attempt to better account for score effects I estimated the shot frequency for each score by taking the average shots in only third periods with more than 15 minutes played with that particular score, and then used those averages to adjust Osgood's expected shots based on his minutes played. The result was that Osgood's numbers got even worse in the most clutch situations, falling to .880, while his save percentage rose to .929 with his team either trailing or leading by 2 or more goals. Even if you want to go so far as to ignore that attempt and simply assume that the Wings allowed shots against at a 20% higher rate in the high leverage situations, the save percentage split would still be .901/.915.

All this is despite the fact that save percentages are higher on average for goalies in the lead than they are for goalies who are trailing, because trailing teams tend to put as many pucks on the net as possible. For example, in this post I show some playoff split numbers for five real elite goalies, who combined to put up a .930 save percentage in third periods that they entered leading by one, compared to a .918 save percentage in third periods they began with a one goal deficit. If you need further convincing, a recent Hockey Analysis post gives even strength numbers broken down by score that show how save percentages rise for the team in the lead.

I also recently developed an additional measure of a goalie's clutch play using the Hockey Summary Project box scores. It is an estimated game-tied save percentage, calculated by noting how much of each period was spent with the score tied, pro-rating the shots for each team during that period by that amount of time, and then noting how many tiebreaking goals were scored by each team. After compiling those figures for each playoff game, a save percentage can be calculated to estimate a goalie's save rate with the score deadlocked. Because of score effects it is not likely to be exact, but it should provide a reasonable estimate. Another benefit is that this measurement covers the entire game, rather than just the third period and OT.

From 1994 to 2004, Chris Osgood's estimated save percentage with the score tied was .890, which is right in line with the estimates from his GAA. That is a substantial drop from his overall pre-lockout playoff save percentage of .910, implying a .922 save percentage in situations where one team (usually his own) held a lead. In addition, it was estimated that only 28% of Osgood's shots against came with the score tied. The main reason that Osgood's teams usually won was that they heavily outshot the opposition in close games (estimated ratio of 1.25 to 1 with the score tied).

Small sample sizes are always a concern when looking at playoff stats, and even more so when the sample is broken down into smaller chunks based on game score. The entire third period and OT sample covers just 681 shots, and the estimated game-tied shots are even lower at 608, which does leave room for the possibility that Osgood was simply unlucky. The process of putting these numbers together is also based on tedious compiling, which raises at least the possibility of an error although I checked the numbers where I could.

At the very least, however, we should be able to claim that there is no evidence to suggest that Osgood improved his play when the pressure rose. On the contrary, the statistical record is very clear that the more desperately the other team needed to score, the more likely they were to slip one past Chris Osgood.

In 2008 and 2009, Osgood's numbers vastly improved in the same situations:

OT/tied/up by 1: 1.53

All other situations: 1.21

Yet again, however, Osgood's GAA was higher when the game was on the line and lower when the outcome was less in doubt, although the split is not as extreme as the one above. I'm not saying that to be critical of Osgood's performance, merely to argue against the claim he selectively raised his game in certain spots. His estimated game-tied save percentage was .933, which is slightly higher than his .928 overall, which could indicate that he was slightly better when the score was close. However, there remains little reason to suggest that Osgood made a significantly greater contribution to winning than his overall numbers indicate.

It remains possible to make a clutch argument for 2008 and 2009 based on the way Osgood's numbers improved from the regular season to the playoffs. I don't buy that it was a conscious thing that Osgood decided to just make himself play well once the puck dropped in the postseason, but the subjective and objective evidence does certainly support the claim that he played better from April to June than from October to March. Maybe Osgood learned how to be clutch, maybe he went on a hot streak, maybe he was just playing behind a dominant defensive team. Either way I don't think that is enough to make up for all the "big goals" Osgood allowed over the rest of his career.

I've stated before that I'm always skeptical of how subjective observers rate the clutch play of an athlete because they let other factors enter the picture, often unknowingly. This appears to be another example of that exact error. Detroit Red Wings fans, Osgood's teammates and even apparently Osgood himself all want to believe in the idea that their team's long-time netminder was clutch, that he made the key saves for the team, that his average numbers are misleading because he always came up big with the game on the line. The problem is that the evidence suggests it was probably just a misperception caused by selective memory and attributing things to Osgood that were more than likely primarily caused by other players on the team. If anything, Osgood appears to have been the opposite of clutch through the vast majority of his playoff career.