Wednesday, December 14, 2011

What Does a Clutch Goaltender Look Like?

One of the most common goaltending cliches is, "It's not how many saves you make, it's when you make them." Colour commentators tell us this over and over again in their rush to credit goalies for wins or blame them for losses, and many hockey fans agree with them. Not every save is the same, they say. Saves that keep a team in the game or allow them to hold on to the lead are more important, and what happens when the game is already decided really doesn't matter so much.

Following that logic, a clutch goalie will make a higher percentage of saves when the score is close, giving their team a "chance to win". As such, we would expect them to have much better numbers in close games than in garbage time.

After the 2011 postseason, I don't think the people saying this kind of stuff really believe what they are saying. Either that or they simply don't recognize it when they see it. The reason for that claim is simply this: Strictly based on the above definition of clutch play (a goalie who performs better when the score is close than when it isn't), I doubt there are many goalies who have ever had more "clutch" numbers over the course of a single playoffs than Roberto Luongo in 2011. Yet, needless to say, Luongo didn't exactly get a whole lot of praise as someone who "made the key saves at the key times."

First, let's look at Luongo's stats at the series level:

Series tied: 7 wins, 1 loss, 0.97 GAA, .968 save %
VAN up 1: 6 wins, 4 losses, 2.56 GAA, .909 save %
VAN up 2+: 2 wins, 5 losses, 4.59 GAA, .868 save %

The Canucks never trailed in any series throughout the playoffs, and a big reason for that was their ridiculous goal prevention record when a series was tied. Luongo was particularly sharp in opening games, posting 3 shutouts in 4 tries.

On the other hand, when the Canucks went well out ahead of their opponents in the series they repeatedly suffered meltdowns. That's not particularly admirable, but the same logic that rates some saves are more key than others could be applied to rate some games as more important than others. And, if you do that, then any reasonable definition of those "key games" would have to include games where the series was tied, since a team that always wins when the series is tied could never be eliminated.

For the sake of comparison, here are Tim Thomas' numbers:

Series tied: 7-4, 1.39, .955
BOS up 1: 1-3, 3.11, .909
BOS up 2+: 2-0, 1.00, .967
BOS down 1: 4-2, 2.84, .921
BOS down 2+: 2-0, 1.50, .961

With the series tied or their teams up one game, Luongo's numbers were better than Thomas'. The main difference was that Thomas was great at closing out the Flyers series and he was also great in the two games where the Bruins faced 0-2 deficits.

Let's move on to the game level. Here are Luongo's numbers broken down by game score:

Lead by 5: 0 saves, 1 shot, .000 save %
Lead by 4: 0 saves, 1 shot, .000 save %
Lead by 3: 18 saves, 19 shots, .947 save %
Lead by 2: 56 saves, 60 shots, .933 save %
Lead by 1: 154 saves, 164 shots, .939 save %
Tie game: 257 saves, 276 shots, .931 save %
Down by 1: 84 saves, 92 shots, .913 save %
Down by 2: 24 saves, 32 shots, .750 save %
Down by 3: 41 saves, 45 shots, .911 save %
Down by 4: 14 saves, 17 shots, .824 save %
Down by 5: 2 saves, 3 shots, .667 save %
Down by 6: 0 saves, 1 shot, .000 save %

What's obvious from that chart? If the game was close, Luongo made a high percentage of the saves. But if it was a blowout, then everything went in. Down by 2 or more goals, Luongo had an .827 save percentage on 98 shots, which is almost unbelievably bad. In garbage time, Luongo made very few saves at all - with either team leading by 4 or more goals, he stopped just 16 out of 23 shots for a mere .696 save percentage. If I was a member of Vancouver's coaching staff, I'd be somewhat concerned by that number and what it could indicate about Luongo's compete level. That said, if it is only "when you make the saves" that counts, then his performance in blowout games should be considered meaningless anyway.

Copper 'N Blue also tracked scoring chances for all of Vancouver's playoff games. Because it is possible for goals to be allowed on non-scoring chances (which Luongo did four times in the Nashville series and six times in the playoffs overall), I calculated scoring chance save percentage as (scoring chances against minus goals against) divided by scoring chances against.

Lead by 5: 2 chances, 1 goal, .500 save %
Lead by 4: 1 chance, 1 goal, .000 save %
Lead by 3: 5 chances, 1 goal, .800 save %
Lead by 2: 29 chances, 4 goals, .862 save %
Lead by 1: 76 chances, 10 goals, .868 save %
Tie game: 127 chances, 19 goals, .850 save %
Down by 1: 50 chances, 8 goals, .840 save %
Down by 2: 19 chances, 8 goals, .579 save %
Down by 3: 26 chances, 4 goals, .846 save %
Down by 4: 5 chances, 3 goals, .400 save %
Down by 5: 2 chances, 1 goal, .500 save %
Down by 6: 1 chance, 1 goal, .000 save %

The numbers are pretty similar, although Luongo's numbers with the lead come down slightly relative to his trailing numbers, likely because of score effects. With the Canucks leading, the opposition took 2.2 shots for every scoring chance recorded. When Vancouver trailed, the ratio dropped to 1.8. Still, the overall conclusion is the same: Luongo's problems were not with the game close or Vancouver in the lead, but rather with the Canucks trailing by more than one goal. And in blowout games he apparently wasn't even trying (7 goals against on 11 scoring chances with either team leading by 4 or more).

Not surprisingly, Vancouver did a lot better in close games than in blowouts. Excluding empty net goals, they were 12-4 in games decided by one goal, 2-0 in games decided by two goals, and 1-6 in games decided by three goals or more.

Having said all that, let's compare Luongo's save percentages by score to those of his Stanley Cup goaltending counterpart, Tim Thomas:

Score within 1 goal: Luongo .931, Thomas .940
Up by 2 or 3 goals: Luongo .937, Thomas .944
Down 2 or 3 goals: Luongo .844, Thomas .957

Luongo was knocked for his ability to hang on to leads, especially during the Nashville series after giving up a couple of late tying goals, but his numbers while leading actually weren't too far off what Thomas was putting up for Boston. With the score close, Thomas again had an edge but not nearly as much as one would think based on their overall numbers. By far the biggest difference in performance between the two goalies was that Luongo was blown out repeatedly and did far worse when his team fell behind by two goals or more.

Did Luongo's awful play in blowouts have much of an effect on Vancouver's win/loss record? Maybe, maybe not. It would certainly be possible to argue that the Canucks would have been unlikely to come back in those situations anyway, especially with how anemic their offence was at times during the playoffs. On the other hand, Boston came back twice from two goal deficits, once again Montreal and once against Tampa Bay. If Thomas hadn't made the saves in those situations, the Bruins would quite possibly have never have made it to the Finals.

To me, the implications of these numbers are pretty clear. On one hand, you can choose to cling to the "key saves at key times", "the only thing that matters is clutch play" logic and praise Roberto Luongo for elevating his game in key situations with the score close in last year's Stanley Cup playoffs. Alternately, you could consider that most of the game is a "key situation", realize that effort and desire don't correlate perfectly with success, and understand that most goalies have the mindset to keep the puck out of the net at all times. That would cause you to reject the "key saves" logic, and instead rate goalies based on their overall performance. In that case, you would be perfectly entitled to knock Luongo's playoff performance, particularly relative to that of Tim Thomas.

That's assuming people are being logically consistent, however. In reality, there isn't an awful lot of logic involved in either hockey fandom or hockey reportage. When most people say "clutch goaltending", they usually simply mean "good goaltending". If not that, then they mean "goaltending behind excellent teammates". Another tactic is to move the goalposts, narrowing the sample size to focus on a few particular games (e.g. game sevens or Stanley Cup Finals games) as the only ones that qualify for the "clutch" distinction. I disagree with that logic since a team that didn't win in the earlier rounds would not have made it to the Finals at all, and similarly without winning three out of the first six no team would still be alive to participate in a game seven.

I've been in the camp that focuses on overall performance all along, so I'm not defending Luongo here. I think that throughout the 2011 playoffs, factoring in both his good and his bad games, Luongo was pretty ordinary. I do think that his play was better in close games, and I'm quite confident that if there was a hockey metric similar to baseball's win probability added, Luongo would rate much better according to that than he would based on overall save percentage. Whether that is a significant observation depends on your perspective on clutch goalies.

At this point I think it's also impossible not to accept the theory that Luongo has a higher-than-normal propensity to melt down when things turn against him. I used to agree with Tyler Dellow that it was essentially media narratives driving the perception, but the repeated blowouts involving Luongo and the above save percentage splits by score make that position tough to hold. I would still argue that Luongo's overall career playoff performance is reasonable relative to his true talent level, but has he ever had some forgettable single game performances on the way.

13 comments:

Anonymous said...

Good post, my only quibble is your final sentence about his "overall career playoff performance". I would just add that the only reason he's had any recent (as in past two seasons) playoff success at all is that he's played behind Cadillac teams that vastly outclassed their opponents for both of those years, particularly the last. He has yet to take command of a single series where the other team was equal to Vancouver.

Here's some food for thought for you to address in your next blog post. Why have so many very good goalies struggled so severely this season? Jonas Hiller, Ilya Bryzgalov, and Cam Ward, all of whom were elite minders for the past couple of seasons, have been well below average this year. You touched some on Hiller, who likely hasn't shaken off his vertigo/postconcussion syndrome, in an earlier post, but what of the other two?

In addition, two minders widely considered to be better than average and often called "clutch", Dwayne Roloson and Steve Mason, are doing unbelievably horribly.

Wanna touch any of that?

The Contrarian Goaltender said...

Anonymous: That's exactly the topic of an article I wrote for Hockey Prospectus that should be posted shortly, so I won't go into too much detail other than to say that I'd mostly bet on guys with an elite track record bouncing back, unless they happen to be 40+ years old.

Steve Mason doesn't belong in that group though, I don't think many people saw him as above average or clutch even prior to this season. I expected him to be one of the worst goalies in the league this season and so far he has played in line with that expectation.

Anonymous said...

In 2010 and 2011, Mason was at least okay--nothing great, but at least average. Why the severe dropoff in your view?

The Contrarian Goaltender said...

Mason wasn't even close to average in 2010 and 2011. A .901 save percentage is simply not good enough in a league where the average is in the .912-.913 range and starting goalies typically do even better than that. To put his numbers in perspective, here is how Mason ranked in save percentage in the league among goalies with at least 50 games played in the last 2 seasons:

2009-10: 20th out of 20
2010-11: 24th out of 25

That's why I expected him to be one of the worst starters in the league again this season, because that's what he has been for pretty much the last two and a half seasons. The fact that his play has actually declined this season compared to even that low starting point makes things just that much worse.

me said...

This is pretty fascinating stuff. I was wondering though, is there a way to remove team affects? Obviously, in a game where Vancouver is winning, there is a higher likelihood that the team is playing well, and vice versa. As you well know, the correlation will not be 1, bu it's probably close.

Two ways that immediately come to mind are: 1) look at top line/top d pairing +/- as a proxy (relying on the assumption that a team will have a better game if it's top pairing plays well) and 2) looking at the same stats for backups (presumably the team plays no worse or better in front of backups in a given game).

Agent Orange said...

Steve Mason has made more out of 2 months than anyone else I've ever seen.

"I would still argue that Luongo's overall career playoff performance is reasonable relative to his true talent level, but has he ever had some forgettable single game performances on the way."

Have you done any breakdowns of great/average/bad games by Luongo in the playoffs compared to the regular season?

From my perspective it seems like Luongo has more of those forgettable games in the playoffs but that might just be the stage/level of competition.

Of course all goalies have ups and downs and good games/bad games. But with Luongo it seems so extreme in the playoffs. When he is on he is unbeatable. But when he is bad he is so bad his team basically doesn't have a chance.

Anonymous said...

"Steve Mason has made more out of 2 months than anyone else I've ever seen."

I was never a real fan of his, but he used to be okay (CG and I might differ on what okay is, but he wasn't horrible). This year he just leapt off a cliff.

"But with Luongo it seems so extreme in the playoffs. When he is on he is unbeatable. But when he is bad he is so bad his team basically doesn't have a chance."

Ever notice that virtually all of his "on" days come against weak teams... the awful '09 Blues and offense-less '11 Preds come to mind? But he has struggled severely even with low-seeded clubs. One call going the other way in this year's Chicago series would have resulted in Luongo playing golf after the first round.

Agent Orange said...

"I was never a real fan of his, but he used to be okay (CG and I might differ on what okay is, but he wasn't horrible). This year he just leapt off a cliff."

He was very good the first couple months of 08-09. Then people got the book on him: high glove side. This was never more apparent than in the series against the Wings in the 09 playoffs.

Then I guess you could make the argument he is averageish on a terrible team? CG might be able to give you more on Columbus shot quality against but I don't really see anything that screams good.

"Ever notice that virtually all of his "on" days come against weak teams... the awful '09 Blues and offense-less '11 Preds come to mind? But he has struggled severely even with low-seeded clubs. One call going the other way in this year's Chicago series would have resulted in Luongo playing golf after the first round."

This is hyperbole even for me. I will agree he is consistently good against those teams. Against better teams he seems to split the series couple giveaway games, couple games you can't lose and then a couple games that could go either way.

The "knock" on him has been that he has never really dominated a series against an even team or better team. As CG has shown that doesn't happen often.

The Contrarian Goaltender said...

me: I think the scoring chance numbers help remove some of the team effects, and show how the Canucks did suppress shot quality against somewhat while playing with the lead. With a playoff sample we really don't have the option of comparing against backup goalies.

The Contrarian Goaltender said...

Ever notice that virtually all of his "on" days come against weak teams... the awful '09 Blues and offense-less '11 Preds come to mind?

I'd certainly disagree with that. Luongo's had plenty of great performances against good teams. Just last year he was very good in the first three games against Chicago, he made 54 saves against San Jose in the series clinching game, he shut out Boston twice in the Finals.

You might be able to make the case that Luongo has been more likely to have awful days against good teams than other goalies. But to claim that he has never been good against them is just selective memory.

Anonymous said...

I wouldn't call the 2011 Hawks a great team. Good by the standards of an eighth seed, perhaps, like the 09 Ducks or 99 Pens, but nowhere near elite. They were pretty much decimated by the cap and lost many of their key cogs like Byfuglien, and struggled all year. For the first three games of the series Vancouver pretty much had complete control and Lou didn't face too much dangerous pressure. Then they fell apart.

He was good against the Sharks, I will admit, but they still weren't quite as good as Vancouver. On the other hand I don't assign much weight to his shutouts on Boston. Vancouver totally dominated the series in shots and quality of shots/scoring chances and Luongo was not tested at all in those two shutouts.

The Contrarian Goaltender said...

I think the 2011 Hawks were one of the best teams in the league. Great outshooting numbers, strong goal differential, one of the best records in games decided by 3+ goals, great top-end talent, and a 33-18-6 record with Corey Crawford in net. The Hawks only struggled early in the year in large part because of Marty Turco and his sub-.900 save percentage.

Secondly, the fact that Vancouver went up 3-0 has made many people in hindsight seem to think those were easy wins, as you apparently do. The scoring chance numbers disagree. Luongo let in only 5 goals on 53 scoring chances in the first 3 games, which is a very impressive record and his play was a big reason why the Canucks were able to go ahead early in that series.

Finally, I agree that Luongo did not have to stand on his head in his Finals shutouts, but to say he was not tested at all is unfair. The number of chances he faced in his two shutouts wasn't that much lower than what Thomas was facing in his wins (Luongo averaged 9 scoring chances against in his shutouts, Thomas averaged 12.5 chances against in his 4 wins).

I also don't think it is correct to say that Vancouver dominated the scoring chances in the Finals. Copper 'N Blue counted the chances as 103-87 in favour of Boston.

But really, I don't see how any of this is important. Because of the sample size, the only way to evaluate playoff play is to look at the entire thing, rather than demanding only to see a certain type of performance like winning despite getting heavily outchanced by a superior opponent. Otherwise you end up getting into all kinds of subjectivity and small sample size narrative building and I don't think that adds much to the conversation, to be honest.

Hostpph.com said...

if you would have asked me that question before I read this post, I would not have known what to answer, but now I do! :D