Let's do a thought experiment where we imagine a goalie who is perfect and has the ability to stop every single shot he faces. That's because he is so athletic, or so big, or because he can read shooters' minds, or however else you want to imagine the hypothetical. His skills have only one limitation, they don't apply in shootouts because shootouts have nothing to do with the rest of the game of hockey. This goalie's skill is average in shootouts and as a result his team has the same 50% chance of winning a shootout as anybody else.
The goalie also happens to be an attention seeker and a thrill junkie, who loves pressure situations. He doesn't care at all about personal stats, only whether his team wins the game or not. His team is in a non-traditional hockey market and is struggling to sell tickets. As a result, the owner wants to have every game be as exciting as possible to boost the gate receipts.
The owner has discussed strategy with the goalie, and they have agreed that the goalie will adjust his effort to try to make every game as close as possible, but at the same time never intentionally throwing a game. They decide it is also best not to tell the team's coach of this plan, since the owner is thinking of firing him at the end of the season and doesn't want the media or the league to find out that his team is in effect shaving points.
Since he doesn't know he has a goalie with super powers, the coach does not use any all-out offensive strategies that might make sense in such a scenario (aggressive forechecking, playing with four or five forwards, using cherry-pickers, etc.). Instead he plays a typical offensive system, and the team is about average offensively.
What does the goalie do in this scenario? Obviously he can freely give up goals when his team is ahead by 2 goals or more, because he can then shut the door and preserve the one goal margin. What about if his team is leading by a single goal? If there is little time left in the game, then it is unlikely that his team will score again. A shootout brings a 50% chance of losing, so if the goalie intentionally gives up a goal late in the game when ahead by one there is an even money chance that he is costing his team a win. Earlier in the game it is more likely that the goalie's team will score again to retake the lead, but nothing is guaranteed since hockey is a low-scoring game.
If he wants his team to pick up two points in every game, the goalie can't risk giving up any lead at all. There are some situations where the gambling odds might be pretty favourable, such as playing at home against a weak opponent where his team goes up 1-0 early in the first period. But give up enough goals in those scenarios and eventually there will be a game that will remain tied and be lost in a shootout.
By similar logic, the goalie can't give up a goal when the game is tied. In fact, that is considerably more crucial than allowing a goal when his team is ahead. A tie game is at least guaranteed to earn a loser point from going to a shootout, but if the team is trailing they need a goal just to force a shootout and face the prospect of a regulation loss if they aren't able to score again.
So this goalie, even if he is really daring, will probably never allow the first goal against and will most often just wait until his team scores 2 or 3 goals before he starts letting some in. He might for fun let in a goal or two early in the game against a bad team, or let the other team tie it up at 1-1 or 2-2 early in the game if he's pretty sure his teammates can score again. Despite giving up as many goals as he can he'll still probably lead the league in GAA and have a very high save percentage.
Now let's revise the scenario to make it a little more realistic. Some goals in hockey are just unstoppable by any goalie because they come on deflections, screens, lucky bounces, great setups, or shots that are simply too fast and too perfectly placed for anyone to stop. Let's assume that on average there is one goal against per game the goalie can't do anything about. These goals occur randomly and without any regularity. Sometimes the goalie goes several games in a row without allowing any, and sometimes he lets in 2 or 3 of them in the same game. How does that change his strategy, given his objective of not trying when his team is very likely to win yet still winning every game possible?
The answer is that now the goalie will essentially have to try to stop everything unless his team is up by 2 goals or more. Even if they are ahead by 2, he could end up blowing the lead by messing around and giving up a goal for fun which is then followed immediately by another one on an unlucky bounce. Similarly, he wouldn't want to intentionally let the other team tie the game because there is always the chance they will then take the lead on an unstoppable shot.
If his team scores 0, 1 or 2 goals in the game (which happens 47% of the time on average) the goalie will likely try to stop every shot he faces. If the team scores 3 or 4 goals (which happens 38% of the time on average), the goalie might let in one or two intentionally. About 14% of the time the team will score 5+ goals, and in that case the goalie can let in some goals for fun.
Multiply out the probabilities of the team's goal support with the expected intentional goals the goalie might allow, and the average would be about one intentional goal against per game. So if a goalie was trying to win every game while simultaneously never trying to stop the puck when he didn't have to, his goals against would be about one goal higher than it would otherwise be. That is, a save percentage of close to .000 when it doesn't matter would equal a GAA increase of about one.
I think it's pretty clear that under any even remotely realistic assumptions about how much a goalie is going to be goofing off when his team has the game in hand that the GAA effect is going to be very minimal, unless the goalie is playing on some kind of dynasty that is always blowing out the opposition. And even then it's certainly no positive trait in my book that a goalie doesn't care whether or not he gets scored on, even if his team is well in front.
It remains possible that there may be slight differences in team performance because of differences in goalie success in high-leverage situations. But I'm not convinced that is either significant or based on repeatable skill. If it exists it will create a slight difference between goalies with similar performance levels, but there is simply no way that a goalie like Marc-Andre Fleury (.907 career save percentage) is doing more to help his team win than a Tomas Vokoun (.917), regardless of career win/loss records.
I think the basic problem behind this common misconception is that people consistently overestimate the level of control a goalie has over his own play and the number of goals against. Vic Ferrari writes constantly about how hockey is a game of luck, and that's really true. Yet fans have heard lots of stuff about how goaltending is such a mental game, so they think that a goalie in the proper mental state can just decide to make himself unbeatable. Either that or or they give too much credence to one of those overplayed stories where Patrick Roy or somebody stood up in the dressing room and said, "Get me one more goal, boys, because I'm not letting in another one." He probably said that almost every time they were tied after two periods in the playoffs, and needless to say his team did not always win the game.
Sometimes the shooters make their shots, that's a fundamental basic truth of goaltending. It's not always the goalie's fault that the puck hits the back of the net. Anyone who wants to be a goalie or who wants to properly evaluate goaltending needs to be aware of that simple fact, or you're evaluating the luck-soaked result instead of the skill-based process.
If there exist some goals that are unstoppable and a goalie can't predict his team's future offence, which are two basic assumptions that obviously hold in hockey, there really is very little opportunity for a goalie to mess with the scoreline without costing his team. And that is why attempts to explain away poor individual save statistics with references to a goalie's win total are just biased nonsense.
Long story short, the next time you hear an announcer say something like, "It's not how many saves you make, it's when you make them", what he's actually telling you is, "I have no idea how to separate the contribution of a goalie from the contribution of the rest of the team."
Subscribe to:
Post Comments (Atom)
20 comments:
it's certainly no positive trait in my book that a goalie doesn't care whether or not he gets scored on, even if his team is well in front.
What if the whole team tends to fall asleep in that situation? Do you put that on the goalie?
No. If the team gives up a lot of chances and goals against then that's their fault.
I'm just pointing out that one of the consequences of claiming that a goalie's stats aren't representative because he tries harder when it matters is that he must then necessarily not try as hard when it doesn't matter. If not his numbers would be great because he's playing hard the whole time. Deciding to tune down your effort level is not particularly admirable, and I don't think that's the correct attitude for a goalie, which I believe is to view it as a personal defeat every time the puck crosses the goal line regardless of score.
I'm on the edge of disagreeing with you - not because what you wrote is incorrect, but because I think that what you wrote isn't what people tend to mean by "big saves."
Obviously, the game is won or lost by how many goals are scored, so a guy who makes saves more often is more valuable. It's also obvious that the guy who makes more saves is going to have lots of "big saves" among them. To stick with your example, Tomas Vokoun has plenty of big saves, probably more than Marc-Andre Fleury. He's just stuck on a bad team where there are fewer leads for him to preserve; Fleury's offense makes him look better by giving him many more opportunities to "be clutch." If he isn't, his team is more likely than Vokoun's to get him another crack at it.
So far so good. It gets tricky when you get to psychology, however. Take two identical goalies, each with a 2.50 GAA and .915 sv%. The first, we'll call Mariano Rivera. Goalie Mo is only average when your team is behind, and doesn't do well early in games. In the third period and overtime he is a wall, and especially when protecting a lead.
Now, our second goalie: Armando Benitez. Benitez tends to do very well when his team is tied or behind, but in tight games late, he has a habit of allowing cream puff goals.
We can safely assume for the sake of our example that each keeper is trying his best at all times. These tendenceies are not the result of letting up or slacking off in certain game situations. It could just be nerves. But Mo will still be thought of as the better goalie because his team knows that he will almost always close out leads after two periods. In effect he shortens the game. Benitez' team never knows when he will screw up all their hard work late.
That's what I think is meant by "big saves" and "it's when you make them." There's nothing worse than keeping five minutes of pressure up on the opposing team, then watching them score a softie on their first counterattack. A team facing that goalie is going to be confident even when under pressure early, or trailing late. A team facing a Mo goalie is going to press. It's possible that the two types of goalies have a subtle effect on their opponents in that way; it may not be large enough to be distinguishable from luck, especially in a pro league where the difference between great and mediocre is so close already. But the team facing Mo may miss nets because they're trying for perfect, unstoppable shots instead of just hitting the net and looking for deflections and rebounds. As a result they unconsciously reinforce the skill he already demonstrates. A team facing Benitez just throws everything they can at him as the game wears on, and give themselves a better chance of catching a break and having a weird goal (or a bad goal) occur.
I suppose one way to start would be to see if keepers with clutch reputations face fewer shots in late and close games than their counterparts, or fewer shots late/close than they do otherwise, while "soft" goalies face more or higher-quality shots, especially five-on-five.
I agree that any such effect for goalies with similar overall statistics is likely to be indistinguishable from luck. Which makes it pretty much impossible to find and of little value in trying to evaluate.
As I stated in the post, I agree that it is possible that goalies with similar stats could have a slight difference in high-leverage performance that leads to a better or worse overall record. However, listen to some media types, they want to give credit to the Jon Quicks of the world entirely on the basis of their win totals, and that's when the "key saves at key times" justifications really ring hollow. Especially after a game where a goalie allowed the other team to take the lead several times or allowed the other team to come back and simply got bailed out by their team's offence.
I vividly remember an Ottawa/Boston game on the last day of the 92-93 season. It was Brad Marsh's last NHL game, he hadn't scored a goal all season and everybody was pulling for him to get one. Late in the game he took a shot on net that rookie Boston goalie John Blue stopped, then Greg Millen (who was doing commentary) said "A veteran goalie would've opened the five-hole a bit and let that in." Being that Millen is a former NHL goalie, I took that to mean that he would've intentionally let in the goal in that same situation.
So there may be instances where a goalie's effort may not be there due to game circumstances. There's many stories from the 70s about Gerry Cheevers stepping aside to avoid a hard slapshot in games where the Bruins had a big lead.
I do agree that "clutch" play is pretty much exaggerated by media and fans. But it would be an interesting study to go through game summaries and compare goalies records in various situations (i.e. leading by a goal in the third) and see if some of the guys with these clutch reputations were in fact significantly better than their peers.
I wonder if goalies were more likely to stop going 100% during the 70s and 80s. Scoring levels were higher, most teams made the playoffs, and the talent was far more concentrated in the best teams then. There would have been a lot more "meaningless" saves made.
If so, it's possible that the idea of the "big save" was once more meaningful than it is now.
I have no doubt that Cheevers and some of his peers let in a few goals intentionally here and there. I doubt they did it very often however, that's likely one of those things where the funny anecdote tends to cause people to overstate the actual effect.
If there is a material impact on Cheevers' stats I haven't found it yet. His win/loss record falls pretty much in line with what you would expect based on his goal support and GAA.
I agree that there was a lot more garbage time in the '70s and '80s. The problem is that if goals against rates rise in blowouts it's not always easy to tell whether the goalie stopped trying or if the rest of the team mailed it in.
I'm hoping at some point to use the Hockey Summary Project numbers to look at regular season third period results for Fuhr, Smith and Cheevers (the typical names that come up when people talk about goalies that knew how to win when it counted) and see if their results are highly correlated to the scoreline.
I agree that any such effect for goalies with similar overall statistics is likely to be indistinguishable from luck. Which makes it pretty much impossible to find and of little value in trying to evaluate.
I came to the same conclusion about "clutch" performances recently:
elite level athletes aren't the general population (I'm guessing the sissies get crowded out of the sample long before they reach the higher levels) and given the rarity of goals and the influence of variance on outcomes, it's more probable that "clutchness" (that is, the perceived, persistent ability to perform better than ones peers in higher stress situations) at high levels is actually more chance than a true talent. Think Fernando Pisani and the 05/06 Play-off run. There are dozens of other examples of blazing 'clutch' hot streaks that never recurred again in the players' career.
Kent, nice post. I know you've been beating the drum on this for a while. The evidence makes it pretty clear that there is very little "big goal/save" effect in the observable NHL results (at least the current NHL).
What I was trying to get at with this post is that even hypothetically speaking there isn't going to be a large effect under reasonable assumptions even if someone went out of their way to try to make their performance as "clutch" as possible. That's why I really don't understand the other side of this argument. I guess, as you wrote, people just love the narrative.
CG: You know when a defenseman dumps a puck in the zone, gets a lucky bounce and scores his first goal in 55 games. On the scoresheet the next day, without seeing the goal, how would you describe it visually?
Point shot? End to end rush? Lucky bounce? You wouldn't know would you? This is the effect of analyzing numbers after the fact with no direct reference to what happened.
Create all the hypothetical situations you will, some goalies and people in general, think better and perform better under pressure and some don't. In the NHL, the difference is likely more slim but it will still exist.
That difference in "focus" if you will can create meaningful differences in "clutch" just on the basis of mind meeting technique.
When a goalie is not as "focused" he may stand deep in his crease, drop to his knees early vs when he is not. All of these characteristics can have a direct effect on the likelyhood of a goal going in. Or the likelyhood of a save. The save at the right time, before, during or after a perceivable shift in players emotions, can be that difference.
However, you know what you'll see on the scoresheet the next day?
3-2, nothing more.
Lawrence, I don't think you have to deny all difference in clutch play or ability to agree that the mainstream hockey media gives goaltenders a lot of credit or blame for "big saves" beyond the aggregate statistics, way out of proportion to any reasonable estimate of their importance.
The worst of it is that the goaltenders with the best teammates usually get the most credit for big saves.
Overpass: Sure, I'll give you that, but to what end? Is it preferred that announcers say this:
"It's game 7 of the Stanley Cup finals and you have just witnessed one of the most clutch performances in the history of hockey."
or this:
"Well actually Jim, although ______ has made 22 saves this period, including three breakaway stops, a 3-on-1 and a glove save off a deflection, this has nothing to do with ability, or value, it's actually all randomness, probability and luck. In fact, I have no idea how to separate the contribution of a goalie from the contribution of the rest of the team, so we'll just say that was great team execution"
I think what drives the stat-heads crazy is that they cannot measure why players like Eberle are so consistently "clutch". There are aspects of sport that will always remain immeasurable - passion, interest, desire, health, mental ability, mental state etc. After all there has to be a reason why it's always him and not 5 other players each chipping in one at a time.
Additionally, even IF, there is no difference from one player to the next on their ability to perform under pressure, if luck has it's hand and the coin lands heads 39 times consecutively for a Roy, Eberle or Hasek, then there's something special in that as well.
Just because you cannot measure clutch, or focus, or desire doesn't mean it doesn't exist. How bout this, give me a measure for love...love of the game, when you're a kid on the street and tell me that is meaningless in shaping future stars careers.
And don't tell me that you're not saying it doesn't exist, you're just saying that it's so immeasurable that it's 30 parts noise and one part signal.
Cause that's just a way of saying "I still cannot quantify it, therefore, I don't like the concept, therefore I will discount it.
What percentage of NHL goals are scored by a guy shooting the puck in from center ice, ballpark? Maybe 1 in 500? One in 1,000? It's a fluke that has no big picture effect whatsoever, so why should I care? The next night maybe the goalie robs him of a sure goal, and his luck balances out. Skill wins out over luck in the long run.
I agree that a goalie who is playing well with great focus will play with more depth, challenge the shooter, and other technical things like that, which will improve his results. The thing is that everybody has those games with great focus. Even in important games. Name the biggest choker in NHL history, I don't care who you think it is, it doesn't matter. As long as he played a few seasons in the bigs then I guarantee he had some great games where he bore down and showed terrific focus and "made the big saves" and all that kind of stuff. Name the guy you think had the best focus in big games, and there will be soft goals in big spots, third-period collapses, elimination game defeats, etc. It's the variability of the position, so let's be careful about presenting it as a false dichotomy between the guys who always rise to the occasion and the guys who always crack under the pressure.
We're talking about tiny margins here, even with the assumptions most favourable to a clutch play effect, that's my point. As you said, I wouldn't be surprised if it's something like "30 parts noise and 1 part signal."
That is exactly the reason I discount this stuff, it's not because I don't like it or because I have some anti-clutch philosophy. It's because I don't want to be wrong. I don't want to predict, say, Marc-Andre Fleury will be much better in the 2010 playoffs than he has been in the 2010 regular season, unless I have better than even odds of being right.
If clutch exists, then whether I can calculate it on a spreadsheet or if there is some wise NHL observer who can accurately assess it for all goalies makes little difference to me, actually. I don't think perceptual biases and time constraints allow for the latter observer to exist in the real world, but if he did he would be just as useful as an accurate shot quality neutral save percentage model. It's the quality of the measurement that counts, not what type of measuring tool you happen to choose.
But if in the real world nobody is able to subjectively evaluate all goalies accurately and clutch effects don't appear to be significant, then I'm better off ignoring them unless I have a very large sample size because otherwise it's just as likely to be leading me in the wrong direction as in the right direction.
I wouldn't be surprised if clutch play did exist throughout prior periods of the NHL. Maybe Turk Broda really didn't care during the regular season and turned it on for the playoffs. I'd say that's entirely possible, based on a quick review of his numbers and contemporary accounts of his play. I'm not positive, but I'm certainly not ruling it out because I have an unshakable belief that all hockey players are robots or something.
The reality is that I only have a limited amount of time to spend on goalie analysis, and I'm trying to spend my time where it's most useful. In my assessment, that's not in trying to determine who the most and least clutch goalie are.
And by the way, what the announcers should say after that game 7 is "Goalie X played a great game." Period.
You don't have to extrapolate every single game into some huge assessment of an athlete's character, motivation or ability to handle stress. People have a phenomenal ability to create storylines, and to view everything in other people as a reflection of some inner drive. People will tell you that "sports don't build character, they reveal it." That has some element of truth to it, but I'd say quite often sports reveal how poorly people assess the character of others. Someone doesn't become a weak-kneed wimp because the puck took a funny bounce at an inopportune time.
If you're a goalie, sometimes the puck looks like a beach ball and sometimes it looks like a pea. Sometimes the screen shots and one-timers from the slot hit you, and sometimes they don't. All of those scenarios can happen whether it is game 1 of preseason or game 7 of the Stanley Cup Final. Hockey really isn't that complicated.
The reality is that I only have a limited amount of time to spend on goalie analysis, and I'm trying to spend my time where it's most useful. In my assessment, that's not in trying to determine who the most and least clutch goalie are.
Well then why bother writing an article that focuses on discounting "clutch" or big saves in the first place?
I'm a goalie and I do evaluate goalies, every summer for the rep teams from Pee Wee through Midget, both boys and girls, both on and off the ice. You've pointed out that:
a goalie who is playing well with great focus will play with more depth, challenge the shooter, and other technical things like that, which will improve his results
and I've seen 16 year old boys cry because they played poorly during evaluations. I've seen them shattered by two quick goals to the point that they literally CANNOT move out past the top of the crease because they have no confidence at all in their ability.
I've also seen 14 year old kids who are so cool and so calm that nothing, and I mean nothing phases them. I've probably subjectively and objectively evaluated over 200 goalies during these summer camps.
I read your blog because it's interesting for me to see how other people are evaluating the game and I learn a great deal of things from all of the posts and links around this internet-hockey-stats world.
One thing that seems consistent is the desire to discount the "human" aspects of the game, the ability for a "big save" to be made.
I agree with you that every save is a big save, and every goal a big goal, but let's be serious here there are times and places when these saves or goals are bigger, perhaps not to the pencil marks on the scoresheet, but to the players sitting on the bench. You may want to discount big saves and "clutch" performances but the players playing have not, and that's what makes it "clutch"
From Roberto Luongo to a kid in Atom hockey...the saves that makes them "feel" bigger, makes them "feel" better will translate into them being bigger (by cutting down the angles) and being better (by playing on the knifes edge of confidence/control, aggression and steady.)
Yes, all of this can happen in game one of the preseason or game 7 of the Stanley Cup Final, and that's why it's important when it does happen.
For some players out there, like Eberle, it just always happens in the game 7 scenario, where as others don't even want the puck, or opt to make the safe play.
Of course in hockey there is a chance for anything to happen, some players consistently take that chance and make something happen, and some consistently don't. No matter what word you want to subscribe to it, most people call that "Clutch" and for a goalie who can keep his toes on the edge of over-aggression, that can be the difference between a shut-out and a blow-out.
Well then why bother writing an article that focuses on discounting "clutch" or big saves in the first place?
If I never investigated it, I wouldn't be able to conclude much about its importance, would I?
I've coached kids, and I would never discount the human aspects of the game at that level. The argument is that the kids who can't perform at the tryouts or who get shattered when they give up a goal never make it to the NHL.
If there is a selection process to remove everyone who isn't clutch, then clutch skill is less valuable because it is prevalent, and there is little margin between goalies at the top levels. I think that is a fair assumption that is supported by the data. That does not mean there are no clutch performances, or that some goalies don't play great for extended periods of time in important situations. It just means that in the big picture and over large sample sizes there really is little effect.
It's a similar argument to goaltending at the Olympics coming up. All of the top teams have a top-flight goalie, which means that nobody has a significant advantage in net. The Olympics will not be decided by goaltending skill. They may very well be decided by goaltending performance, because I don't know whether Miller, Nabokov, Brodeur, Luongo, Vokoun or whoever will be hottest over the next few weeks.
Saying the Olympics won't be decided by goaltending skill doesn't mean I think goaltending is unimportant, or that I have some preconceived opinions that I'm trying to justify about goaltenders. It's simply a conclusion based on the available evidence.
And just to be clear, Jordan Eberle doesn't drive me crazy at all. If you know anything about variation you know that some very unlikely things are going to happen by chance alone. Scoring a few very important goals doesn't necessarily mean a player is more clutch than others with a similar level of overall ability, any more than someone who won the lottery twice is different from the rest of the population. I'm not saying Eberle is not clutch, he might have some incredibly clutch skill. If he does, then he'll likely prove that to us beyond all doubt in the future, that is assuming that his dominance against junior players can be extrapolated to also occur against NHLers. For now it remains perfectly possible that he is a great junior scorer who had an improbable streak of timing. Either way he's just one data point in the much larger sample of all hockey players, which is of course the sample that I'm much more interested in.
And just to be clear, Jordan Eberle doesn't drive me crazy at all. If you know anything about variation you know that some very unlikely things are going to happen by chance alone.
This is probably the most important point here.
Chance alone demands that some lucky fuckers go on late-game scoring/puck-stopping streaks, and vice versa some unlucky bastards go cold and sieve out when the game is on the line. Just going by the way that SH% and SV% behave in this universe, this has to happen.
It's just the way it is, luck is cruel. In hockey and in life.
Every so often though, there really is the one dude who can bear down when his team needs him. Thing is, luck is so huge in this aspect of the game that you might only get (let's say) 1 guy in 100 so-called clutch scorers who actually has clutch ability. And luck still has the final say anyway, the clutch guy might just get stoned by another guy with a horseshoe up his ass, and his clutch ability doesn't manifest.
I'd ask people who believe in clutch so much that they bring it up every time someone scores a goal with the game on the line: would you rather be right 1% of the time or right 99% of the time?
Or, to really drive the point home: would you rather wager your house on a bet with 1% odds or with 99% odds?
RO: Agreed completely, I figure that the uncertainty is so great my conclusions about clutch play are more likely to be wrong than right, and that's why it's not even worth worrying about it in the absence of compelling evidence.
BAHAHAHAHAHAH This whole entry is EPIC FAIL lol....thanks for the laugh pal!!!
Post a Comment