Friday, October 23, 2009

Are Shootouts An Indicator of Skill?

With four full seasons of shootout results in the books, we can start to make sense of the results. I wanted to check if better goalies tended to have better shootout results. I calculated the correlation between shootout save percentage and overall save percentage for every goalie who faced at least 50 shootouts against.

Correlation: 0.004

That's about as close to zero as you are going to find in a real life sample. What that suggests is that shootout performance tells you nothing at all about how good a goalie is.

This is reinforced by looking at the leaderboard. Among the goalies who have faced 50+ shootouts in their careers heading into this season, the top 3 were Johan Hedberg (.820), Mathieu Garon (.812), and Jose Theodore (.790). Henrik Lundqvist, Rick DiPietro, Kari Lehtonen and Tim Thomas were all good goalies who had good shootout results as well (all .740 or better). Roberto Luongo (.716) and Martin Brodeur (.715) were both above average, but not by a lot, while Tomas Vokoun was right about at average (.670). Goalies who were below average at stopping shootouts included J.S. Giguere (.652), Cristobal Huet (.604), Miikka Kiprusoff (.600), Ilya Bryzgalov (.597), Niklas Backstrom (.568), and Evgeni Nabokov (.568). The worst goalie was Vesa Toskala (.512).

Conclusion: Shootout skill is distinct from overall goalie skill. Shootout results don't provide much evidence of a goalie's overall abilities. All they do is measure how good a goalie is at stopping breakaways.

For this reason, I don't consider shootout performance at all when evaluating goalies. Other analysts do, and I understand the reasons for it. All things being equal it is better to have somebody who is good at stopping shootouts in the current NHL where shootout success directly leads to standings points. If I was a GM then I'd probably take it into account. For predicting future team success it also makes sense to include shootout skill in your prediction. I just see shootouts as a sideshow that is separate from the actual game, a rare game situation that has little meaning in an overall sense and that is based on a distinct skill set. My objective is to identify the best and the worst goalies at playing hockey, i.e. the game in its most common state with 4 or 5 skaters on each side. Shootouts don't give us much useful information to that end, so to me they don't matter.


seventieslord said...

Your long disclaimer at the end covered off the only objection I would have had.

Thanks for compiling these numbers, as always.

Scott Reynolds said...

I wish there was a coach out there (maybe the guy in Atlanta) who would get a backup with great shoot-out numbers and just let him play all of the shoot-outs. I know Craig MacTavish tried this "closer" thing once (it failed) but it would be interesting to see a coach try it over a season. I'd imagine Hedberg's value to his team's point totals would increase substantially if he could provide that level of performance (or close to it) off the bench. It would be particularly valuable to a team like Calgary that plays their starter almost every game have a starter that stinks at shoot-outs.