Wednesday, September 21, 2011

Pre-Lockout Chris Osgood Was Not Clutch

"I always loved the fact that when we were tied or the games were close in the last 10 minutes, I'd shut the door and we'd win the game," he said.

"I knew how I did my job on a great team." (Chris Osgood)

I figure that after a long silence in this space, it would only be fitting to get back into it with one of my favourite topics: the overratedness of Chris Osgood.

Actually, to be honest, I wish I didn't have to make posts like these. The recently-retired Osgood should be remembered as a guy who overcame all kinds of obstacles and worked hard to outlast a ton of other goalies who may have had more natural talent. I thought this was a terrific read that showed Osgood's dedication in rebuilding his game to incorporate modern techniques. I get why Detroit fans loved their scrappy netminder, it's great that fan bases identify with blue collar guys who give it everything they have out on the ice.

But, unfortunately, most people still can't separate individual play from team success. In their eyes, 400 wins and 3 Cups make you a Hall of Famer, no further analysis required. They portray Osgood as something that he simply never was, and that's not fair. Ergo, as long as there are specious and silly arguments being thrown out in his favour by people with influence within the hockey community, then I'm going to keep making posts to set the record straight. Sorry, Ozzy, it's nothing personal, I just believe that credit should go where credit is due.

One of the points I have repeatedly tried to make regarding Chris Osgood is that even if you think he was a supreme clutch performer in the 2008 and 2009 playoff runs, that should still not have any impact at all on how you rate his playoff performances from earlier in his career. Many fans seem to have a tendency to revise their evaluations of a player based on their late-career performance, and that makes no sense.

I think Osgood got a lot of help in 2008 and a lot of favourable bounces in 2009, but I will still readily concede that it is much, much more supportable to assert that Ozzy was clutch in those two seasons than it is to claim that Osgood was clutch in the playoffs from 1994-2004.

It would in fact be far, far easier to make the case that Osgood was a spectacular choker in his early career than it would be to argue that he made the big saves when his team needed them most.

Here's the data to support that statement. I looked at Chris Osgood's playoff numbers in the third period based on the game score from 1994 to 2004 (source: Hockey Summary Project). Without play-by-play records to separate out the shots by score, I chose to measure Osgood's GAA in each situation:

Trailing by 2+: 1.02
Trailing by 1: 2.10
Score tied: 2.98
Leading by 1: 2.53
Leading by 2+: 1.88
Overtime: 3.18

The most high-leverage situations with the highest loss in win probability from allowing a goal against are when a team is tied or leading by one goal late in the game. It's hard to miss the observation that these precise situations are the ones where the other team was most likely to score on Osgood. Coincidentally, his goals against numbers dropped in situations where the penalty of a goal against was the lowest. That is not the expected profile of a goalie who was giving up goals when it didn't matter and slamming the door when the game was on the line.

Grouping the numbers into just two groups, the most high-leverage situations (tie game in third & OT and preserving a late one-goal lead) and then everything else, you get these numbers:

OT/tied/up by 1: 2.81

All other situations: 1.71

Of course his teammates playing to the score would have had an impact on those numbers, but did the Red Wings allow over 60% more shots against in the most pressure-packed situations? There's simply no way that was the case, which means that Osgood's individual numbers definitely dropped as the penalty for a goal against rose.

Assuming the shots were distributed evenly regardless of score, Osgood would have had an .881 save percentage with the score tied or his team leading by one, compared to a .924 save percentage the rest of the time. The one situation where it is possible to fully separate out Osgood's save percentage is overtime, where he let in 6 goals on 46 shots for a wholly unimpressive .870 save percentage.

In an attempt to better account for score effects I estimated the shot frequency for each score by taking the average shots in only third periods with more than 15 minutes played with that particular score, and then used those averages to adjust Osgood's expected shots based on his minutes played. The result was that Osgood's numbers got even worse in the most clutch situations, falling to .880, while his save percentage rose to .929 with his team either trailing or leading by 2 or more goals. Even if you want to go so far as to ignore that attempt and simply assume that the Wings allowed shots against at a 20% higher rate in the high leverage situations, the save percentage split would still be .901/.915.

All this is despite the fact that save percentages are higher on average for goalies in the lead than they are for goalies who are trailing, because trailing teams tend to put as many pucks on the net as possible. For example, in this post I show some playoff split numbers for five real elite goalies, who combined to put up a .930 save percentage in third periods that they entered leading by one, compared to a .918 save percentage in third periods they began with a one goal deficit. If you need further convincing, a recent Hockey Analysis post gives even strength numbers broken down by score that show how save percentages rise for the team in the lead.

I also recently developed an additional measure of a goalie's clutch play using the Hockey Summary Project box scores. It is an estimated game-tied save percentage, calculated by noting how much of each period was spent with the score tied, pro-rating the shots for each team during that period by that amount of time, and then noting how many tiebreaking goals were scored by each team. After compiling those figures for each playoff game, a save percentage can be calculated to estimate a goalie's save rate with the score deadlocked. Because of score effects it is not likely to be exact, but it should provide a reasonable estimate. Another benefit is that this measurement covers the entire game, rather than just the third period and OT.

From 1994 to 2004, Chris Osgood's estimated save percentage with the score tied was .890, which is right in line with the estimates from his GAA. That is a substantial drop from his overall pre-lockout playoff save percentage of .910, implying a .922 save percentage in situations where one team (usually his own) held a lead. In addition, it was estimated that only 28% of Osgood's shots against came with the score tied. The main reason that Osgood's teams usually won was that they heavily outshot the opposition in close games (estimated ratio of 1.25 to 1 with the score tied).

Small sample sizes are always a concern when looking at playoff stats, and even more so when the sample is broken down into smaller chunks based on game score. The entire third period and OT sample covers just 681 shots, and the estimated game-tied shots are even lower at 608, which does leave room for the possibility that Osgood was simply unlucky. The process of putting these numbers together is also based on tedious compiling, which raises at least the possibility of an error although I checked the numbers where I could.

At the very least, however, we should be able to claim that there is no evidence to suggest that Osgood improved his play when the pressure rose. On the contrary, the statistical record is very clear that the more desperately the other team needed to score, the more likely they were to slip one past Chris Osgood.

In 2008 and 2009, Osgood's numbers vastly improved in the same situations:

OT/tied/up by 1: 1.53
All other situations: 1.21

Yet again, however, Osgood's GAA was higher when the game was on the line and lower when the outcome was less in doubt, although the split is not as extreme as the one above. I'm not saying that to be critical of Osgood's performance, merely to argue against the claim he selectively raised his game in certain spots. His estimated game-tied save percentage was .933, which is slightly higher than his .928 overall, which could indicate that he was slightly better when the score was close. However, there remains little reason to suggest that Osgood made a significantly greater contribution to winning than his overall numbers indicate.

It remains possible to make a clutch argument for 2008 and 2009 based on the way Osgood's numbers improved from the regular season to the playoffs. I don't buy that it was a conscious thing that Osgood decided to just make himself play well once the puck dropped in the postseason, but the subjective and objective evidence does certainly support the claim that he played better from April to June than from October to March. Maybe Osgood learned how to be clutch, maybe he went on a hot streak, maybe he was just playing behind a dominant defensive team. Either way I don't think that is enough to make up for all the "big goals" Osgood allowed over the rest of his career.

I've stated before that I'm always skeptical of how subjective observers rate the clutch play of an athlete because they let other factors enter the picture, often unknowingly. This appears to be another example of that exact error. Detroit Red Wings fans, Osgood's teammates and even apparently Osgood himself all want to believe in the idea that their team's long-time netminder was clutch, that he made the key saves for the team, that his average numbers are misleading because he always came up big with the game on the line. The problem is that the evidence suggests it was probably just a misperception caused by selective memory and attributing things to Osgood that were more than likely primarily caused by other players on the team. If anything, Osgood appears to have been the opposite of clutch through the vast majority of his playoff career.

21 comments:

Anonymous said...

While you are writing about inverse clutch playoff performances, perhaps you could do a focus on Bobby Lou. You could highlight games like this: http://www.youtube.com/watch?v=LBLbVUPOoh8&feature=related
or like this: http://www.youtube.com/watch?v=HRGpCRhzzpc&feature=related
or like this:
http://www.youtube.com/watch?v=f8TFpdtgyuY
or like this:
http://www.youtube.com/watch?v=gSWrljZQJcY&feature=related
or like this:
http://www.youtube.com/watch?v=lVRyv1PAouA&feature=related
or like this:
http://www.youtube.com/watch?v=gz9tH0z7U3k
Oh.. my bad on the last one, it was not the playoffs, just the Olympics where he almost blew Canada's chances before Crosby bailed him out.

Wow, for such a short playoff tenure, he sure does have a great non-clutch resume. Now I am no Osgood fan, but he at least won Cups despite not being clutch. Maybe you should admit your bias and start opening your posts up to other goalies besides Chris Osgood, and perhaps start with the most overrated of them all, the giant padded, big-mouthed rebound machine aka Roberto Luongo.

Robert Vollman said...

How does that compare to his regular seasons numbers?

Is it typical for goalies to let in more goals in OT, when tied, or up by 1?

Though I haven't studied it objectively, it certainly appears that the breaks/calls seem to favour the team that's trailing.

The Contrarian Goaltender said...

I haven't compiled Osgood's regular season numbers. I think I've seen it demonstrated that the trailing team is slightly more likely to score because they get more calls and both teams play to the score, but it isn't a large effect. I looked at one sample of playoff goalies that had pretty much an identical GAA in periods that they started up one compared to periods where they started out tied. When it is tied late, teams will often play more conservatively, which will slightly depress GAA, although I think it depends on the team.

Anonymous said...

Different Anonymous from the first here. I generally am on-board with your analyses but I want to know how how would classify Dwayne Roloson, who is probably in his final year and whose skill level is probably much the same as Osgood's, but lacks his Cups.

Roloson is much like Osgood; an average or somewhat above-average goaltender with great longevity that was able to achieve some pretty good years due to team success (i.e. under Minnesota's trap system) but little when not playing for a very defensive team. He had one spectacular run in 2006. Was he really "clutch" in 2006 or did he get some good bounces and timely protection from Pronger and Horcoff?

Also, Mike Vernon strikes me as being similar to Osgood, but not as good.

The Contrarian Goaltender said...

Anonymous: I agree with you on Vernon, he won because of big team advantages and had even worse numbers relative to the league than Osgood.

As for Roloson in 2006, I think he was good but I don't think there is much evidence to suggest he was unusually good in high leverage situations. He stopped 54 out of 55 shots in OT, which is terrific, but overall I estimate his game-tied save percentage to be .936, which is very good but not that far above his overall rate of .927. I didn't run the numbers, but looking through the box scores it looks like the Oilers blew a few leads as well.

The Oilers as a team did very well when the score was tied, in large part because they scored a lot with the game tied (estimated team shooting percentage of 12.3%).

Because of the small sample of one playoff season, there is rarely convincing evidence that a goalie's performance was unusually clutch. In most cases, a goalie that played well also played well at important times in the game.

Kyle A. said...

One of my friends who is a Red Wings fan commonly brings up the opening quote when talking about Osgood but more along the lines of:

"After the Red Wings take the lead, Osgood shuts the door"

I tell him from the same logic one could say:

"Osgood played poorly, and the Wings were loosing the game until the rest of the Wings scored enough to get ahead, and the game ended before the other team managed to get a tieing goal."

Agent Orange said...

CG! Its been a while. I just wanted to make one comment on this post.

"I don't buy that it was a conscious thing that Osgood decided to just make himself play well once the puck dropped in the postseason..."

I don't think when most people make a comment about a player stepping it up or being clutch they mean it as a conscious decision by the player. Have you ever followed a player who seemed to make better plays when it was all on the line?

I don't think people mean to imply the player is goofing off or not taking his job seriously when the game is not on the line and then deciding to play for real.

Its just an internal thing where they appear to take it to the next level. This opinion is usually formed when a player performs in a certain way consistently in the same situation (like Elway and his 4th quarter/OT drives).

RJ said...

Chris Osgood isn't proof. Chris Osgood is proof that no amount of stats, even an entire website dedicated to them, can make up for what your eyes can see on the ice. Osgood in 1998 was as good as he had to be, which could've easily been bad and in 2008, he was a couple Zetterberg non-goals away from the Conn Smythe.

But you don't watch hockey. Number can only tell you so much but people like you can't survive without them, so you cling to them like a security blanket. The smartest sports minds don't need statistics whatsoever.

Plenty of goalies have great teams. Not all of them ever win Cups. The Wings were good but it's not like Osgood was sitting behind the trap and facing poor shots his entire career like a certain Mr. M Brodeur.

The Contrarian Goaltender said...

Agent Orange: OK, so what am I supposed to conclude for Osgood? Evidence suggests he did not "take his game to the next level" in important situations pre-lockout. Post-lockout, maybe he did. Did he learn something? Was it luck? Was it a hot streak? How do we know?

The Contrarian Goaltender said...

RJ: What is your claim, exactly? That Osgood was "as good as he had to be" for the Red Wings to win? That Osgood won Cups? Nobody is disputing those things.

From 1998-2004, my eyes told me that Osgood was pretty much an average goalie, years before I ever dug into the advanced stats. I think the majority of hockey observers would have agreed with that perspective, it was only after his last two playoff runs that this whole "Osgood was a clutch winner his whole career" meme seemed to really take off.

The smartest sports minds don't need statistics whatsoever.

I think you'll actually find that the smartest sports minds all take advantage of every method of evaluation which is available to them to ensure the best possible results, and yes, that does include various types of statistical analysis.

Agent Orange said...

CG

My post wasn't intended as a for/against Osgood comment. Just my interpretation of what people mean by the comment.

I don't think we really have such a different opinion of Osgood.

I think our biggest sticking point is how the HoF should/does evaluate goalies. I think based on their history Osgood gets in. You're of the opinion that he shouldn't get in. I don't think we'll change each other minds.

ILR said...

You do realize that these two

* I think based on their (HoF) history Osgood gets in.
* I think that Osgood shouldn't get in.

aren't contradictory statements, right?

Agent Orange said...

ILR

Assume that was directed toward my comments.

I was a bit unclear.

I think CG and I have a similar opinion of where Osgood ranks in terms of goalies of his time. And of his ability. I personally don't buy in on clutch/unclutch.

I usually just tend to think about things in terms of what kind of expectation a player builds for himself and then if he exceeds that or not. I think early in his career Osgood's playoffs met the expectations he set for himself in the regular season in terms of overall play. Late in his career his playoffs exceeded the expectations he set for himself. I don't think he was a clutch shut-down goalie. I think for 2 playoff seasons he exceeded expectations and the result was some very good playoff goaltending.

I personally would rather have a goalie who will set a performance level during the regular season and give me the same performance in the playoffs (this opposed to a goalie who's performance drops in the playoffs). I know what I am going to get this way. If everyone on my team does this and I am a good regular season team I expect to do well in the playoffs.

I think that based on how the HoF has evaluated goalies in the past he has a resume which should get him in (400+ wins 2 cups).

CG is of the opinion that those resume points shouldn't get him into the HoF.

I disagree with him on this part.

One question for CG. If Osgood were to get in the HoF do you think he would be the worst goalie in the hall?

(I'm not saying this would make the case for him just interested in your opinion).

The Contrarian Goaltender said...

It's hard to know for sure how good some of the historical goalies were with how uneven the NHL has been through some portions of its history. If someone developed a magical system to perfectly evaluate every goalie who ever played, I wouldn't be too surprised if it turned out that someone like Gerry Cheevers, Ed Giacomin or Harry Lumley was in reality little better than average. Then again, we know with a high degree of confidence that Osgood himself was little better than average, and those other guys all had contemporary observers who rated them better than Osgood's observers have rated him. That's why I would be fairly confident in claiming that Osgood would very probably be the worst goalie in the Hall.

As for whether it will get him in, I think the modern Vezina/save percentage era has significantly changed the way goalies are evaluated, so we can't just assume that past standards still apply. Secondly, Mike Vernon has 385 wins and 2 Cups (and a Conn Smythe) and he's not in the Hall and very probably won't ever get there. You really think Osgood's 401 is that much more significant? Take away shootouts, and Osgood's edge in wins is just 391-385. In terms of accomplishments, Vernon and Osgood are pretty much the same goalie. I don't see how you induct one and not the other, which would seem to support my opinion that Osgood is not a serious Hall of Fame candidate despite what some media types are proclaiming.

Anonymous said...

I agree Osgood should not get into the Hall; however, I think he deserved the CS in 2009, and that he was truly better than Mike Vernon.

Agent Orange said...

CG, I think we have had this discussion a couple times.

"You really think Osgood's 401 is that much more significant?"

No I don't. Would my personal opinion of Osgood be changed if he won 399? Not at all. But how many people talk about the goalies in the 399 win club?

It is stupid I agree. But are the people who cast the votes the brain-trust of hockey? Doesn't seem to be that way.

I don't think we can draw any conclusion on how the HoF is evaluating goalies based on what people are developing on the internet (places like this site). We don't have any data. Patrick Roy was a no-brainer and I imagine you would argue Fuhr shouldn't be in the hall.

The Contrarian Goaltender said...

I think you're definitely right on when you say that we don't have any data. I don't think the Hall will induct Osgood, but I really don't know. We'll get a bit more of an idea when we see what they do with a guy like Curtis Joseph, but we probably won't know for sure until Osgood comes up for induction himself.

Agent Orange said...

Curtis Joseph is a way more interesting to to me than Osgood.

Cujo was frequently considered one of the top goalies in the NHL during his career but usually choose a big payday over going to a team with the best opportunity for team and personal success.

A buddy of mine who is a Blues fan were discussing what could have been for him which led me to run some numbers.

We estimated what would have been if Cujo had gone to the Wings* in 1995-96 after leaving Stl and stayed there until they parted ways after the lock-out.

We assumed all other things being the same including Cujos minutes played and save % (easy to make the argument that this would be better in Det than Edmonton/Toronto) and determine he lost roughly 60 wins and increased his career GAA by 0.11 as a result of not going to a contender. He was twice a Vezina finalist in Toronto and his numbers would have likely been better (based on our calculations) in Detroit would he have surpassed the winner for those trophies? Maybe not but the race would have at least been closer.

Do you have any numbers for the Wings goalies vs league average and Cujo vs league average for SV% between 1995-96 and 2003-04? It would be great to further refine these numbers (just the league average numbers would allow me to do this as well).

*Note we choose the Wings for this because we estimated they were the best team per goaltending (i.e. a good team who might be interested in improving goaltending) during the time (the aves had Roy for example and wouldn't be interested).

In general based on the way the HoF has selected people I think Osgood has a better resume than Cujo. I am NOT saying Osgood was better than Cujo just that he compiled more over the course of his career... or at least enough to make up for the 50 less wins (in 200 less games played). Again I'm not saying this is right/fair/the way it should be. Just what I have observed.

The Contrarian Goaltender said...

I have a post in the pipeline about Joseph's Hall of Fame case, so I'll probably address it in some more detail then, but you certainly present an interesting hypothetical about what he could have accomplished on better teams in the late '90s/early '00s. Joseph's save percentage numbers were actually pretty average in his post-St. Louis career, but with powerhouse teams in front of him he definitely would have racked up a lot more wins.

Anonymous said...

To diffrent anonymous. You cannot compare Dwayne Roloson to Chris Osgood. That's like saying Jason Blake is similar in skill to Guy Lafleur.

Cujo does not belong in the hall of fame. While being a very good goaltender he never won a vezina or a stanley cup despite being on the 03-04 red wings who finished 1st in the entire league 2nd in goals. That team had the likes of Steve Yzerman, Brenden Shanahan, Brett Hull, Nick Lidstrom, Chris Chelios, Pavel Datsyuk, Robert Lang and Derian Hatcher.

To contrarian goaltender Osgood is better the Giaciomin, Lumley and Cheevers. I dont know how you could come to that conclusion. If Osgood only won cups because of the great detroit teams he was on why did Cujo never win one? Why was Hasek as great as he was only able to start for one cup on a team that had Robert Lang, Sergei Federov and Luc Robataille on the second line? By no stretch of the imagination is Osgood a top ten goalie of all time but to say that he does not belong in the hall of fame is assinine. People also neglect to mention that he took an average islanders team to the playoffs in 2002 and weak st. louis teams in 2002 and 2003. I don't care who you are if you win 400 games and start for two cups you deserve to be in the hall. Do I think brodeur is a fraud when it comes to the best goalie of all time? Yes. Do I believe he doesn't belong in the top ten? Yes. But do i believe he shouldn't be in the hall of fame? No. Regardless of if a players accomplishments are responsible because of the players around them if said player has certain credentials he should get into the hall. He doesn't have to be the greatest of all time or even in the top ten or fifteen to make the hall of fame.

Anonymous said...

Whoops the year i meant to reference was 02-03 for cujo, so scratch hatcher and lang off the list and add lucky luc. Still a stacked team though