Saturday, August 29, 2009

Making the Big Save

Some hockey fans think playoff wins are the best way to evaluate a goalie. The other stats don't matter, they say, because it doesn't matter how many saves you make but when you make them, and the teams that win are the teams that get important saves from their goaltenders at the most important times in the game. This means that stats are just excuses for losers. If those goalies played better when the game mattered then their team would have won.

I've already dealt with some of the "big saves" argument in this post that points out that most of the game of hockey is played with the score close. However, it remains at least theoretically possible that teams and goalies could optimize their goals for and against to improve upon their win/loss record. Hockey games are won and lost by goals for and goals against, but a team that wins the first game of a playoff series 8-0 and loses the second 2-1 has the same record as their opponents, despite a much better goal differential. We know from observed results that in the long run most of those differences even out and that records can usually be predicted quite accurately by goal differential. However, the possibility does remain, no matter how rare it might be, that a goalie could "stretch" their teams' goals farther than normal and win more games by playing better in key situations.

It seems unlikely to me in today's widely-televised, big-money NHL that a goalie would be goofing off when it didn't matter and bearing down only when it did, but what about the goalies of yesteryear who were playing for fun and had to work other jobs in the offseason? Did some of them only care about winning and adjust their effort accordingly? If that was the case, then maybe they could be legitimately considered better than another goalie with equivalent stats because they did more to help their team win hockey games. Is this something that goalies actually do or have done in the NHL playoffs?

The Pythagorean expected points equation (GF^2/(GF^2+GA^2)) allows us to predict what a team's record will be given their goals for and against. If a goalie has exceptional timing in terms of allowing goals, then they will allow more goals when it doesn't matter and fewer when it does matter. Doing this should allow the team to outperform its expected win total.

Let's give an example: If a goalie plays in 20 playoff games and his team scores 50 goals and allows 50 against, then it is difficult to claim that the goalie is a clutch winner if his team's record is 10-10. That is of course exactly where we would expect the team to end up. If they instead are 15-5, then that must mean they were winning small and losing big, and either they got lucky or the skaters and/or the goalie were doing more to contribute to winning than the basic stats would suggest.

For each playoff season where the goalie played over half of his team's minutes in the playoffs, I took the total goals for and against during that playoff season, used that to project the team's record, and then compared it to their actual wins and losses. Over their entire career we can see if their teams did better or worse than expected. I wasn't able to split out individual goalie performance because game summaries aren't always available, but this shouldn't have big effect since most would have played the vast majority of their teams' minutes.

This makes for a very simple analysis, as all we need are win/loss records and goals for and against. We can thus compare goalies throughout the entire history of the league, all the way from Benedict to Brodeur, including goalies from the early years for whom we have no shot data or save percentage stats available.

Great teams tend to overperform their expected win rate, especially in the playoffs. This is mainly because teams tend to play to the score more in the postseason. If they gave it a full effort for the entire 60 minutes, a dynasty team would likely blow out their opponents by an even greater margin that might be more representative of their actual dominance, but teams that are ahead usually trade offence for defence to increase the chances of holding onto their lead. Therefore goalies who played on very strong teams would quite possibly show good results here even if their own performance was consistent across different game situations.

Just to be clear, this metric has little to do with a goalie's overall performance. A good or bad result doesn't prove anything, but only tells us where to take a closer look if we are trying to see if some goalies contributed more to winning games than their basic stats would imply. A more in-depth analysis would be required to confirm or deny the hypothesis. I'm not at all convinced that any goalies have consistently shown good timing or an unusual ability to raise their level of play in high-leverage spots. I think the ability of the rest of the team to play to the score is likely far more important than a goalie's clutch performance. Some (maybe even most) of these results might be entirely because of team factors, and some of these results might be entirely because of luck. Be aware of that as you draw your conclusions.

Here are the results with the goalies grouped into tiers, sorted by number of wins above expected:

The Clutch Performers:
Billy Smith (+12.8), Grant Fuhr (+12.8), Patrick Roy (+12.7), Ken Dryden (+11.6)

All of these guys are well known for their playoff successes. They were all probably above average in the postseason, but they all also had the benefit of playing most of their playoff careers on very strong teams.

I have already looked at some of Grant Fuhr's peak playoff seasons in Edmonton in some detail, as well as Patrick Roy's later career. I don't think the evidence suggests that either of them were making a huge impact on their team's win/loss record by making the so-called "big saves". I think the Oilers and the Avalanche were similar in that they were both high-percentage scoring teams that did not dominate on the shot clock and played to the score, which is the kind of team that might very well win more than their goal differential suggests. I would be interested to look at Roy's Montreal game results in more detail, but unfortunately most of those box scores aren't readily available at the moment.

Fuhr and Smith are goalies who are usually subjectively rated well above what the statistical record suggests they deserve. Just like Brodeur, that makes them interesting objects of study, to try to assess whether there is something extra hidden in the numbers. I'm still not entirely sure what to make of Smith, I'm always skeptical about goalies who have reputations that depend primarily on their team playoff successes. However unlike Fuhr Smith actually has a pretty strong save percentage record, although he played mostly in a diluted league and didn't outperform his backups which suggests that there was a strong team context.

The Strong Performers:
J.S. Giguere (+7.3), Jacques Plante (+6.8), Turk Broda (+5.8), Gump Worsley (+4.5), Marc-Andre Fleury (+4.0)

Giguere has a win total that is far above expected, primarily as a result of his 12-1 record in OT games. I don't know whether he is going to be able to keep repeating that close game success, but it is impossible to deny that Giguere has had some great clutch performances to this point. Broda is considered to be one of the best playoff goalies ever, so I was expecting him to do well by this measure. Worsley does surprisingly well although his outperformance is nearly entirely on the Canadiens, which suggests that his team was helping, something we should also keep in mind when evaluating his Montreal predecessor Jacques Plante.

The Solid Performers:
Ed Belfour (+3.8), Gerry Cheevers (+3.3), Mike Vernon (+2.8), Terry Sawchuk (+2.6), Tom Barrasso (+2.4), Rogie Vachon (+1.9), Cam Ward (+1.9), Dominik Hasek (+1.6), Andy Moog (+1.2), Bernie Parent (+1.0), Frank Brimsek (+0.9), Ron Hextall (+0.9), Martin Brodeur (+0.8), Evgeni Nabokov (+0.7)

Considering his teams I don't think Cheevers' playoff record is all that special. Barrasso's position is entirely from the 1991 and 1992 Cup runs (+5.8 wins in '91/'92, -3.3 wins for the rest of his career). Who was the real clutch player, Barrasso or Mario? I know what my guess would be. I don't think many observers would consider Andy Moog clutch, but he won more than expected, suggesting that his Edmonton or Boston teammates were good at playing to the score.

I was expecting Hasek, Brimsek and Parent to finish a bit higher than they did. In fact, Hasek's relatively ordinary ranking suggests to me that these results have a lot more to do with the team than the goalie. With Hasek in net the Sabres won pretty much exactly what they were projected to win, even though Hasek had an astonishing .949 combined save percentage in Buffalo in overtime and in third periods that began tied or with the teams within one goal of each other. Hasek was also almost unbeatable when Buffalo was ahead late in the game (the numbers are all in this post). It's pretty hard to get much more clutch than that, and yet if you plug Buffalo's goals for and goals against into the Pythagorean formula you get a winning percentage that is within .002 of the actual observed result.

The Average Performers:
Chris Osgood (+0.2), Mike Richter (+0.1), Roberto Luongo (+0.1), Bill Durnan (-0.1), Johnny Bower (-0.2)

All of these guys won pretty much exactly as much as the goals for and goals against predict they should have. Again that doesn't mean they didn't play well, just that there is little evidence to suggest that their teams won more games than they should have with them in net. Therefore to me it makes sense to base our evaluation of them on statistical measures of their individual performance rather than their team success. Chris Osgood may have a lot of playoff wins and Cup rings, but that doesn't mean he is an unusually clutch player. The opposite would apply for Roberto Luongo.

Below Average Performers:
Clint Benedict (-0.7), Mike Liut (-0.9), George Hainsworth (-1.1), Tiny Thompson (-1.3), John Vanbiesbrouck (-1.4), Glenn Hall (-1.8), Ed Giacomin (-1.9), Tony Esposito (-2.3), Curtis Joseph (-2.8)

Several of these goalies have reputations for playing poorly in the playoffs (especially Hall, Giacomin, Esposito, Cujo). I've looked at Joseph's results in some detail and I think that some of his underperformance is the fault of the scorers in front of him, although he has not done particularly well in some high-leverage situations such as playoff OT. Similarly, I think Glenn Hall's teammates are likely more to blame than he is. On the other hand, Giacomin and Esposito both had some weak playoff seasons on strong teams.

The Worst Clutch Goalie?
Harry Lumley (-5.5)

Lumley's teams had a winning percentage that was .073 below what was expected based on their goal differential, which suggests something was going very wrong in those tight games. Whether he was a choker or merely the most unlucky one in this group I'm not sure, but given that he played for three different franchises and was mostly on good teams it's possible Lumley had something to do with it.

(Post edited to reflect mistake with Martin Brodeur's numbers)

29 comments:

seventieslord said...

Brilliant post for so many reasons. It is a wonderful way to keep the individual empirical evidence of the goalie's performance right out of the equation, and really speak to those who say sv% is mostly meaningless if you save a lot of shots when you don't need to, and allow goals when it hurts the most.

I like how possibly the four moneyest of money goalies end up at the top, showing that this calculation really does tell us something; in other words, it tends to align with common perception.

Mike Vernon is an interesting result. His personal save percentages over the years have been mostly bad in the playoffs. But he has a reputation as a good playoff goalie. He came out positive here, and I bet the "clutch" tag really applies to him, to some degree, and possibly Cheevers as well.

A couple things:

I assume you only used Benedict's NHL stats? I would recommend you re-run his numbers using his NHA playoffs and inter-league Stanley Cup games.

Also, any reason you used only 38 goalies? I could think of at least 20 more whose results would be of interest.

Lawrence said...

I'd be interested to see what the difference is between Brodeur when he was winning in the playoffs (with perhaps a stronger team) and Brodeur today (with perhaps a weaker team).

I would also wonder when NJ won three games in the finals in '03 3-0 if you consider that a win that can be more greatly attributed to the scorers or the goalie? Three goals for was higher than average that year I believe, but three shutouts in the finals effectively is as near a guarantee for the Stanley Cup as any. I guess only less than four shutouts.

As for Cujo, Roy and Smith it doesn't surprise me in the least. Subjective opinions of goalies aren't entirely of base.

As I wrote in the other post, I firmly believe that one's regular season performance is distinct enough from one's playoff/tournament performance that the one large statistical body (reg. season performance) cannot be used to infer overall performance over the small (playoff/tournament) one. I don't think the sample size rebuttal applies if that is the entire distinct population. I'm glad you have addressed that in part in this post.

Lastly, I, personally, don't support this idea:

Some hockey fans think playoff wins are the best way to evaluate a goalie. The other stats don't matter, they say, because it doesn't matter how many saves you make but when you make them, and the teams that win are the teams that get important saves from their goaltenders at the most important times in the game.

I have and always will believe that one must use reason and systems thinking when looking at the statistical numbers. First by establishing that playoff and tournament performance is distinct from regular season in numerous ways and therefore must be considered, and considered additionally and individually.

Second that regular season performance also has a huge importance due simply to the fact of sample size and confidence in the numbers.

[5 P-O GP .939 & 15 Reg. GP with .897], is very different from:

[5 P-O GP .939 & 215 Reg. GP with .897] as it is from:

[5 P-O GP .939 & 15 Reg. GP with .939] as it is from:

[3 different 5 Game tourney's w/ 15GP .897, .891, .903 with 215 Reg. team GP with .939]

Those are either or many of: not the same goalie, not the same teams, luck (good or bad) or a goalie who doesn't deal well with pressure (amongst other possibilities).

seventieslord said...

Didn't Brodeur face something like 55 total shots throughout those three shutouts?

The Contrarian Goaltender said...

I like how possibly the four moneyest of money goalies end up at the top, showing that this calculation really does tell us something; in other words, it tends to align with common perception.

Sure, I think common perception is likely correct that the '70s Habs, '80s Islanders, '80s Oilers, and late '80s/early '90s Canadiens were the most "clutch" teams of the recent era. Now we just need to address the tricky issue of cause and effect. Did the goalies get their reputations from helping their teams be clutch, or did they get their reputations because they were lucky enough to play with clutch teammates?

I assume you only used Benedict's NHL stats? I would recommend you re-run his numbers using his NHA playoffs and inter-league Stanley Cup games.

NHL-only, yes. What's the best source for scores from playoff games in the pre-NHL era?

Any reason you used only 38 goalies? I could think of at least 20 more whose results would be of interest.

Not really, just time constraints and I didn't want the post to look like a huge numbers dump or start to get into some guys with real small sample sizes. If anyone is curious about how some additional goalies fare then just leave a comment and I can run the numbers.

The Contrarian Goaltender said...

I'd be interested to see what the difference is between Brodeur when he was winning in the playoffs (with perhaps a stronger team) and Brodeur today (with perhaps a weaker team).

I noticed I had two numbers accidentally switched around on Brodeur, so his actual total was +0.8 wins. I have edited the post to correct the mistake. Here are the split results:

1994-2003: 141 games, +3.9 wins
2004-2009: 37 games, -3.4 wins

So, if you like the guy you're probably going to call him clutch and blame his post-lockout teammates. If you hate the guy you can call him a loser and attribute everything from pre-2004 to Stevens and Niedermayer. Or you can, like me, think that he was more or less the same guy all the way through and that this entire exercise tells us more about a goalie's teammates or luck than his clutch abilities.

Lawrence said...

...or maybe Brodeur has just lost that extra 2% 'spring in his step' due to his age for example, and we don't notice it as much during the regulars because of the level of competition.

The Contrarian Goaltender said...

First by establishing that playoff and tournament performance is distinct from regular season in numerous ways and therefore must be considered, and considered additionally and individually.

As far as I'm concerned, this has not been established. What are the reasons that you believe this to be true?

Also, to Seventieslord re: Mike Vernon, I disagree that Mike Vernon was clutch in any way. Vernon played in 12 playoff seasons, and his teams only outperformed their expected winning percentage in three of them. They just happened to do it by a lot in those three seasons which were all deep playoff runs (1989, 1995, and 1997). Not coincidentally all of them were excellent teams.

I thought at the time Vernon's Smythe was a joke and I still do, to me he's always been a mediocre goalie on a strong team.

Lawrence said...

As far as I'm concerned, this has not been established. What are the reasons that you believe this to be true?

I'm not sure what you are looking for here, and as a goalie yourself I think you likely have experienced the difference yourself. Therefore, I think you could answer that question yourself, but if I were to form a list, it would look something like this:

-Duration of time
-Concentration of games over the duration
-Quality of competition (Likely a seeding process has taken place, where in most cases, other that maybe the Olympics, the overall competition is higher)
-Additional expectations (playing while injured, dehydrated, fatigued)
-Psychological effects (pressure)
-Financial effects (in the NHL playoff wins -> playoff rounds won -> additional revenue -> superior team.) Not always the case, but Phoenix underspends at the cap for a reason and a Stanley Cup win would likely increase spending on better players. Call it the 'Dynasty effect'
-Less time to 'average out' mistakes. Two poor games back to back likely means your out, or one in the Olympics. Regular season this consistency is less meaningful.

I'm sure the list could go on and on, and these effects are not isolated to a goalie of course, but with the position being especially mental as opposed to physical, the 'pressure' or demands on a goalie are very high. Some people excel in these situations, and some don't. Some goalies are 'clutch' and some aren't.

Calculating this statistically can be a never-ending abyss, but as a goalie, you know it exists.

Ultimately, the rules and objectives are the same, but the constraints are different, therefore one is not = to the other. Which is one step to explaining the Osgood's, Luongo's, Cujo's, Roy's, Dryden's, Jim Carey's and Giguere's of the world.

The interesting thing about Hasek, was that he was so good, always.

CG: Even you yourself have wrestled with this concept re: Giguere. "Unfortunately that is not a peak, that is a 21 game hot streak. In his next 21 NHL starts after losing game 7 of the '03 Cup Finals, Giguere was 5-14-2, 2.76, .910. Sometimes you get hot, and sometimes you get cold." It is a very valid point but in this case, the 21 game timing of Giguere's hotstreak was much more significant than Mason's for example. What matter's is that the hot streak DID happen in the playoffs, and the cold in the regular season.

I don't know a goalie out there who has been facing elimination and a deficit after a goal and said to themselves "I cannot allow the next goal", without some understanding of significance behind that phrase.

seventieslord said...

NHL-only, yes. What's the best source for scores from playoff games in the pre-NHL era?

Probably someone who owns The Trail of the Stanley Cup... like me. I'll get back to you on this tonight or tomorrow.

The Contrarian Goaltender said...

Lawrence: I agree with you that there are some minor structural differences between tournaments/playoffs and regular season play. But I don't think that makes for enough of a difference to treat them as separate cases for goalies.

I'm sure the list could go on and on, and these effects are not isolated to a goalie of course, but with the position being especially mental as opposed to physical, the 'pressure' or demands on a goalie are very high. Some people excel in these situations, and some don't. Some goalies are 'clutch' and some aren't.

Some people excel in those situations and some people don't, absolutely. But we're not talking about "some people". We're talking about the 30 best goalies in the world. What percentage of goalies who have made it all the way to the NHL and are good enough to be starting goalies on playoff teams do not excel in pressure situations? Do you think we are likely to find that, for example, any of the top 20 opera singers in the world have problems with stage fright or sing much worse when they perform in front of large crowds? Do the top stage actors in the world often forget their lines in the middle of their performances because it is opening night and the spotlights are shining just a bit brighter than normal?

People who are performers get selected for their ability to perform on demand and to perform under pressure. That group includes athletes. If you're a choker, your career ends in junior, or the ECHL, or in the Swiss league, or somewhere else short of the bright lights of the NHL. Why in the world would the very far end of the long tail of elite goalies not be able to perform just because the pressure is higher? If they couldn't do that, they wouldn't be a part of that group to begin with.

Go through the entire history of the NHL and find goalies who played in a large number of playoff games and have substantially worse numbers in the playoffs than during the regular season. There are not many of them, and most of the ones that do show up have some team factors that are likely at least partially responsible.

To me, the evidence suggests that for goalies there is not much difference between the regular season and the playoffs. Give a goalie enough playoff games, and you'll see about the same level of play in both environments.

Bruce said...

I thought at the time Vernon's Smythe was a joke and I still do, to me he's always been a mediocre goalie on a strong team.

I'd say a good goalie on a strong team, but you're right about that Smythe. Nobody else reached out and grabbed it, but Vernon's performance in '97 was underwhelming to say the least. That he won a Smythe and Fuhr (who was a strong candidate three or four times) or Brodeur (who was a very strong candidate with better stats than '97 Vernon in all three Devils' Cup runs) never did, speaks to the capriciousness of the award. Some years there are no real overwhelming candidates, other times there are 2 or 3.

Why in the world would the very far end of the long tail of elite goalies not be able to perform just because the pressure is higher?

Why can't Scott Hoch or Kenny Perry make a 3-foot putt at the 72nd hole of Augusta? Don't tell me they're not elite, but the occasion clearly got to both of them. It happens.

[Word verification: sessesse
... what are the odds of 8 random letters being just 2 different letters??]

The Contrarian Goaltender said...

We agree on the Conn Smythe. I also think too much weight gets put on that very situational award.

Why can't Scott Hoch or Kenny Perry make a 3-foot putt at the 72nd hole of Augusta? Don't tell me they're not elite, but the occasion clearly got to both of them. It happens.

I'm not at all saying it never happens. I'm saying I'm unconvinced that there is a significant difference in clutch skill in the population of elite athletes.

Hoch and Perry had their bad moments, absolutely, just like all of Fuhr, Smith, Dryden and Roy did at one time or another as well. Yet both those golfers and those goalies also lots of times when they came through when it mattered.

Those who support the clutch play argument tend to make it seem like in an important playoff situation one "winning" goalie will have a .950 save percentage while another "losing" goalie will be at .850, even though they have a similar level of overall ability. That's just not the case, and that's why for the most part we don't see much difference between the expected and actual win numbers.

Lawrence said...

I'm not at all saying it never happens. I'm saying I'm unconvinced that there is a significant difference in clutch skill in the population of elite athletes.

"Significant" - This we both agree on, however, I think you're overlooking the importance of the minutia of the differences. For me, this is the same logic people use to explain why Miikka Kiprusoff who was considered one of the best in the world three years ago is a sieve today. Has he just become a sieve overnight or has something else changed? However, here, we are comparing reg. season to reg. season, but the minutia of the difference speaks volumes when really the difference is 93% to 91%. That difference, however, is between the top 95th% of the league and the middle 50th%. I personally remain unconvinced, that he is suddenly a sieve.

With the playoffs, however, we regularly see a step up in performance, with the same team, same coach, different constraints:

Carreer numbers:
Roy: R -.910, P -.918
Hasek: R -.922, P -.925
Brodeur: R -.914, P -.920
Kipper: R -.912, P -.921
Giguere: R -.914, P -.925
Luongo: R -.919, P -.930
Osgood: R -.906, P -.916
Belfour: R -.906, P -.920

In fact, it is difficult to find a goalie who has worse numbers over his career in the playoffs than in the regular season. This is further magnified by the fact that all intuitive factors would suggest the opposite - increased fatigue (more games, more frequently, later in the year), increased competition (Top 16 teams only vs All 30),higher expectations (playing while injured) etc. etc.

So, it is indisputable that there is a significant statistical difference, between the two 'seasons'. Therefore, we have to ask why? Is it 'clutch' - I don't know. Is it exactly the same goalie on a 'clutch team? - I'm not sure...but I am sure that there is a difference. Therefore, one statistical body cannot be used to infer from the other - they are unique, but similar populations.

So, if we agree on that, we cannot use the 'small sample size' counter for the distinct 'playoff and tournament' pool of statistics, if these can even be lumped together, because it is our whole population of numbers.

What I enjoy about this post, and measuring 'clutch' play is it exposes the misleading data about Roberto Luongo's playoff performances (.930 sv%) as the subjective fan in Vancouver will tell you he has been terrible in the playoffs when it counts.

That subjective opinion aligns with your 'clutch' findings and my sv% analysis in 'critical games' where Luongo has posted a .930% in all playoff games but a .904% in the elimination and 3-x 'next elimination scenario.

So, on the verge of his reported 28million - 4 year - 7 million cap hit contract, the subjective will say "It's too much for a goalie who is weak when it counts" and the objective will say "yeah, I initially didn't think so with a .930%, but on closer look, it looks like you're probably correct." Bad deal for the Canucks and a great one for the now highest paid (per year) goalie in history.

Statman said...

Somewhat off-topic, but I believe one reason why playoff SV% is usually higher than regular season SV% has been that ref's have put their whistles away & so offensive chances have not been as prevalent... shots/game might not be much lower, but the clutch-&-grab evident in many playoff games (especially prior to the lockout) really impeded good scoring chances.

Although I haven't looked at it recently, average # of powerplays per game might also be lower in the playoffs.

The Contrarian Goaltender said...

You have to do more than show that everybody's numbers go up to show there is a meaningful difference between the regular season and playoffs. If everybody's numbers rise, that just suggests a more low-scoring environment. Just like, say, 2003-04 was more low-scoring than 2005-06. There are a number of possible theories to explain why playoff numbers would rise (fewer power pays, lower shot quality, greater playing to the score effects, etc.). I'm not sure exactly what is causing it, but the main difference is not goalie clutch play, I can tell you that.

"What I enjoy about this post, and measuring 'clutch' play is it exposes the misleading data about Roberto Luongo's playoff performances (.930 sv%) as the subjective fan in Vancouver will tell you he has been terrible in the playoffs when it counts."

How does it do that? It tells us the Canucks won exactly what they should have, given their goal differential. If Luongo was a sieve when the game was on the line and a brick wall when it wasn't, then we would have expected the Canucks to win a lot less than expected. Given how heavily they have been outshot, the only reason the Canucks' goal differential was even close in the first place was because of Luongo's .930.

I've watched every playoff elimination game Luongo has ever played, so I don't particularly care what subjective opinion some reactionary Vancouver fans have. Saying he has been terrible in the playoffs when it counts is completely absurd. I've been linked to recently by some Vancouver message boards and blogs because of the Luongo vs. Brodeur posts, and from what I've read there you are vastly overestimating their consensus on this matter.

So, if we agree on that, we cannot use the 'small sample size' counter for the distinct 'playoff and tournament' pool of statistics, if these can even be lumped together, because it is our whole population of numbers.

Yes, we can. You are mixing up two concepts here, the concept of something being important and the concept of something being statistically significant. Let's assume that you are right, and that playoff performance is completely distinct from regular season play, and that goalies have widely divergent skill sets in both environments. We still can't conclude anything about a goalie with 22 career playoff games played. We just don't have enough evidence to say anything about it.

That doesn't mean playoff games aren't important, or that teams don't make money by advancing deep into the playoffs, or that a goalie who played poorly shouldn't be held responsible for that. It just means we don't know for sure how good he really is, just like we don't know how good a rookie goalie is until we watch him play a couple of seasons in the league.

Did you read what I posted in the other thread about Martin Brodeur's elimination game performances in 3 year stretches? If you used his first 3 years to predict his next 3 years, you would have been wrong. If you used either of the first two three-year stretches to predict the third, you would have been wrong. As he played more games, however, his performances in that subset started to become a lot more similar. That is basic statistics. As the sample size grows, a goalie's performance gets closer and closer to his "true" level of performance.

The Contrarian Goaltender said...

I have a couple questions for you, Lawrence, if you feel like addressing them, because I'd like to get a better understanding of how you see this issue.

First, what do you think a fair salary is for goalie like Luongo?

Secondly, let's say Luongo and Brodeur are both playing in the third period of a playoff game with their teams ahead by a goal. What do you think the true level of skill is for each of them, reflecting both their abilities and their ability to play in that important situation? .910/.920? .900/.930? .850/.950?

Lawrence said...

1. 5.5-6 million for two-three years. That would be a cap hit number. Considering his first contract in Vancouver was based on predicting what he would/could do, I don't think he was in line for a raise at all. I think he was more valuable to Florida than he has been to Vancouver, and this is coming from someone living in Vancouver who is not a Canucks fan that told all of his friends that they just signed the best 'pure' goalie in the NHL when they got him from FLA.

2. Their true level of skill? I'm not sure I understand the question and the options. Luongo's skill is second to none...I would argue even not second to Hasek. I think Luongo is the most naturally skilled puckstopper I personally have watched play, so he could be .950+, it's his mind that is his downfall. I would say Brodeur is the most well-rounded Goaltender and the best overall 'player' and could easily be .950, but I would say .940 and Hasek has the best mind, he thinks that game on a whole different level, is tremendously composed and adaptable like no other goalie I have seen and that translates into his consistency and 'clutch' ability (if there is such a thing). Let's say .945. I'm not sure I know what context you are giving these numbers, so don't put too much weight into those figures I picked.

Lawrence said...

You have to do more than show that everybody's numbers go up to show there is a meaningful difference between the regular season and playoffs. If everybody's numbers rise, that just suggests a more low-scoring environment. Just like, say, 2003-04 was more low-scoring than 2005-06.

I'm only stating that there is a difference, not explaining it is weighting it. I am saying that for whatever reason, it is different enough that on body of stats cannot be used to infer the other, they both must be considered.

How does it do that? It tells us the Canucks won exactly what they should have, given their goal differential.

From the numbers I have looked at they have won big and lost big, the consistency isn't there. The losses have come when the Canucks were facing elimination...and in those games Luongo wasn't great.

I don't particularly care what subjective opinion some reactionary Vancouver fans have. Saying he has been terrible in the playoffs when it counts is completely absurd...from what I've read there you are vastly overestimating their consensus on this matter.

I've never said it was consensus. I just feel it is his most frequent criticism and my observation. All goalies have there strong points and weak points, and I think your investment in Luongo, your 'like' of him, may be putting you in a position to defend these criticisms. He has yet to prove he can win when it really matters. That's a big hurtle for a goalie who many think is the best, and is paid to be the best. Fair or not, Luongo won't be considered good enough until he wins the cup for Vancouver, and I don't think that is unreasonable to expect from a man in his position. The best in the world on a great team? What's the problem?

Ok...statistics. This is getting a bit frustrating for me.

We still can't conclude anything about a goalie with 22 career playoff games played. We just don't have enough evidence to say anything about it.

Why not? I think you are creating barriers. What if the goalie only plays 22 career games, it's impossible to say anything about it? Even with assumptions noted, or an asterix? Come on. Really? What if a population only has 22 possible data points?
What if it has a billion? Or six...like a world population? Are you going to count every one? No, you will sample and infer, correct? Well, what if you have every data point? You can't use them all and make a conclusion?
Of course you can. Tell that to every fringe goalie who plated one year in the NHL. You gonna tell a GM, nope...wait, it's small sample size, he could be the next Sawchuck. Don't think so. Again, I'm not saying one over the other. I'm just saying you can look at those numbers and make conclusions. I'm also saying the populations are different, even if it's because they are inherently lower scoring games.

How bout this: How many playoff games would Luongo have to play before we could stop saying...small sample size? 400? 1000? 1,000,000? More than the reg season games? What if a goalie plays 82 reg season games and 400 playoff games?

I may have to put up with you blatantly saying I don't know statistics, but when you say you don't care about subjective matters and simply disregard them, then I think you're allowing the 'statistics' to get in the way of reason. Unfortunately, your defining the parameters of the argument to favor your argument.

Lawrence said...

Whoops. That should say, not explaining it, or weighting it.

Lawrence said...

Oh, and before Statman comes in and jumps to some other conclusion: I am NOT saying that Roberto Luongo is better than Dominik Hasek.

Statman said...

"He has yet to prove he can win when it really matters. That's a big hurtle for a goalie who many think is the best, and is paid to be the best. Fair or not, Luongo won't be considered good enough until he wins the cup for Vancouver, and I don't think that is unreasonable to expect from a man in his position. The best in the world on a great team? What's the problem?"

Vancouver is a "great team"?

13th overall in goals scored during the 08-09 season.... 17th overall in PP%...

There are not many teams that have won the Cup with such mediocre offense, regardless of goalie quality.

Lawrence said...

Updated: and this is what you get for initially listening to the brutal Vancouver media. Luongo signs for 12 years...wow... and reportedly 64mil. 5.3/year. That's more like it. The news here was confirming the 7mil deal. Jerks.

5.3 million a year for Roberto Luongo for the next 12 years. Good on ya Vancouver. Let's just not mention that Rick guy out east, or that he needs to be playing until he is 42?.

I'm wondering if this is another Hossa deal and what Brian Burke will have to say about that.

@statman. If winning the division and finishing 3rd in the conference (7th overall) doesn't make you a great (not elite) team, I'm not sure what does. If my team wins the division...my expectations are at least the conference finals...at least. As well, three of those teams that scored more goals, didn't even make the post season. Are they great teams because they scored more than 246 goals?

Statman said...

Winning the division makes a team "great"? Not necessarily. Being 7th overall, albeit out of 30 teams? Not necessarily "great". That's only about the 75th percentile.

That's like saying anyone who plays on a Cup-winning team is a "winner". A gross oversimplification.

I commented about goals scored because that is virtually independent of goaltending. I'm sure there are years that NJ (for instance) was mediocre in offense but finished high in the standings, but NJ tended to allow very few shots & had among the least penalty killing opportunities against (both factors again virtually indepedent of goaltending, despite the claim that Brodeur might have reduced shots against by about 1 per game).

Statman said...

Not that I care one way or another about the Canucks... but I disagree with: "If my team wins the division...my expectations are at least the conference finals...at least."

Vancouver only finished 2 pts ahead of Cgy in their division... & Van was 4th overall (not 3rd) in their conference... why should being 4th (& nearly 5th) in their conference mean they should be in the final 2 of their conference?

Statman said...

correction ... NJ had among the least powerplay opportunities against...

The Contrarian Goaltender said...

Lawrence, I think I understand where you are coming from better now. In the other thread, you wrote this:

"I could care less about the sample size and predictions, I'm only interested in what happened and that is that Luongo didn't stop enough pucks."

Sure. That is true, and it is true that Luongo's save percentage in elimination games is whatever it is. But my point about sample sizes and predictions is that we have to assess whether there might be other factors that explain Luongo's performance, rather than some standard cliches like he's not mentally tough or he can't win when it really matters. If something is a result of skill, then it helps us predict the future. It only isn't good for predicting things if it is a result of luck.

A goalie does not perform the same in every game. That could be from outside factors (injury, fatigue, etc.), from team factors, or from the simple fact that athletic performance is variable and everyone has great days and off days. Everyone would like to be at their best when they are needed most, but sometimes you go out there, ready to do everything you can to win the game for the team and you just don't have it. And sometimes you go out there and you do have it, and everything the other team shoots has eyes and still ends up in the back of the net. That is why every top goalie has had bad playoff games, and has let in bad goals at key moments. They did their best to prepare and they were trying their hardest, but they didn't get it done.

When I evaluate what a goalie has done over a few games, one of the things I consider is whether it is likely that he was just unlucky, i.e. he had a few more bad days in there than normal. Because again, everybody has those days, and they don't come in some regular pattern. It is very likely that the 7-5 loss to Chicago will end up being the worst game of Luongo's playoff career, results-wise. It takes on added significance because Luongo doesn't have much of a history behind him. Nobody is raking Brodeur over the coals in similar fashion because of his game 7 against Carolina, because there is a whole career behind him (and I'm not saying that they should either). It doesn't make any sense to me to penalize somebody because they had their worst game early in their playoff career. We don't know that it will be his worst game, but given everything else we know I'd say it is quite likely that it will be.

The other disagreement we have is that you are constraining the sample size by arguing that the regular season is distinct from the playoffs and that Luongo's overall play was fine but that he didn't come up big when it mattered most. That means we are talking about a few games here and there. I am quite content to let Luongo's 566 combined regular season and playoff stand up as proof that he is a goalie who helps his teams win. If you want to focus on such a narrowly defined situation, then you need a large sample size to be able to evaluate it. Until then, it makes sense to look at the overall results and conclude that it is likely that Luongo had a few more bad days at the wrong times in 2009 than he is likely to have in 2010 and on.

So, more or less, the way I see it you are criticizing Luongo for flipping a few more tails than heads so far in his first 22 tosses of the coin. I don't see any reason to think that the coin is weighted (it didn't seem weighted in the 544 "practice runs" that he did), so I see the results as more bad luck than anything. There are lots of goalies who hit an early "run of tails" in their playoff careers, Dominik Hasek being most prominent among them, that turned out alright in the end.

By all means criticize Luongo for how he played in a game here and there if you think he underperformed. But be careful when you start to generalize from there. You can say that Luongo was weak when it counts, and you might even be able to argue that one. But you wrote a few times that Luongo is weak when it counts, and then you're on unsupported territory, in my opinion.

The Contrarian Goaltender said...

As far as how good the Canucks are:

Vancouver with Luongo: 115-64-22 (.627)
Vancouver without Luongo: 18-22-5 (.456)

Draw your own conclusions.

Lawrence said...

You can say that Luongo was weak when it counts, and you might even be able to argue that one. But you wrote a few times that Luongo is weak when it counts, and then you're on unsupported territory, in my opinion.

THIS, is entirely true. I agree wholeheartedly and it's my sloppy writing.

As for Luongo. Let's see how this contract works out, but I'm not a Luongo hater by any stretch. I just don't think we can yet make the claim he is better than Brodeur over any long period of time we analyze. IF the Canucks get him for 5.3/year and the collective bargaining agreement doesn't trample that contract when it's next negotiated. Then, the Canucks, as it stands, now have the best goalie at the best value in the league bar none. I do believe that.

The Contrarian Goaltender said...

Seventieslord was kind enough to send me the NHA and Stanley Cup series numbers for Clint Benedict, so I updated them for completeness' sake.

For his entire career, Benedict came out at +1.7 wins in 52 games, which would put him in the "solid performers" group. His results were a bit skewed by the 1915 finals where his team was destroyed by Vancouver and thus made his overall goal differential look worse, but even without that he would have still been 1.1 wins above average.