Thursday, May 27, 2010

Special Teams Play: Luck or Skill?

There is a Vezina Trophy winner hidden in the following table of even strength save percentage leaders from one of the seasons in the last decade. See if you can spot him:

1. Goalie A, .934
2. Goalie B, .931
3. Goalie C, .930
4. Goalie D, .929
5. Goalie E, .929
6. Goalie F, .929
7. Goalie G, .925

Obviously it's not the guy at the top of the list, because that would be too easy and I wouldn't be asking the question in the first place. It's not the guy at the bottom of the list either, I only included him because he is a future Hall of Famer. That leaves the goalies in slots 2 through 6, who have nearly identical save percentages. None of these goalies are backups who got lucky for a few months, they are all their team's clear #1 starter. Goalie E played in 53 games while the rest played between 58 and 67.

Let's add in their save percentages on special teams, and the % of the shots faced that came on special teams:

1. Goalie A: .934 EV, .897 ST, 22.6% on ST
2. Goalie B: .931 EV, .931 ST, 18.5% on ST
3. Goalie C: .930 EV, .889 ST, 23.6% on ST
4. Goalie D: .929 EV, .882 ST, 23.8% on ST
5. Goalie E: .929 EV, .888 ST, 20.6% on ST
6. Goalie F: .929 EV, .877 ST, 25.0% on ST
7. Goalie G: .925 EV, .884 ST, 23.5% on ST

Now I think it's pretty obvious who won the Vezina. Not only that, but he also won the Hart Trophy. Goalie B is Jose Theodore, and the year is 2001-02. The other goalies, in order, are Patrick Roy, Sean Burke, Evgeni Nabokov, J.S. Giguere, Roberto Luongo and Dominik Hasek.

Did Theodore deserve his awards that year? That all depends on how you assess his special teams performance, whether you consider it to be a result of his skill or whether it was mostly luck or random chance or the play of his team's PK unit. You can make the argument that Theodore was the most valuable goalie that year because he faced many more shots against than any of the other goalies behind Montreal's porous defence, but there's not too much evidence to suggest that he was better at even strength than the other goalies who had almost identical rates.

I think Theodore had a career year in 2001-02, but that he still was pretty fortunate on special teams. It's possible that Theodore played better in certain game situations than other goalies and therefore was actually the best that year, but I'm not entirely convinced. The other goalies listed above almost all have much better track records than Theodore. I wouldn't say that Theodore's season was one of the weaker Vezina-winning seasons, as finishing second in EV SV% is still an impressive result, but I would say it is the most "smoke and mirrors" .930+ save percentage season that I am aware of.

In fact, I'm pretty sure that early-career Jose Theodore must have been among the luckiest special teams goalies in league history. In Theodore's first four seasons he stopped 89.3% of the shots he faced on the penalty kill. Since then, his PK SV% has dipped to a mere .850.

That might be attributable at least in part to a decline in Theodore's performance or the play of the team in front of him, but his performance on the power play was luckier still. In his first five seasons in the NHL, Jose Theodore stopped 321 out of 333 shots against while his team had the man advantage. That's a .964 save percentage in a situation where the league average was .917. In the year of his Vezina glory in 2001-02, Theodore peaked with a perfect 69-for-69 on power plays.

This year's Vezina is also likely going to be won by a special teams overachiever. Here are the numbers for this year's Vezina nominees:

Ryan Miller: .928 EV, .929 ST, 19.4% ST
Ilya Bryzgalov: .927 EV, .896 ST, 22.0% ST
Martin Brodeur: .924 EV, .873 ST, 15.7% ST

Here are the same statistics for several other goalies, all of whom had EV SV% equal to or better than Miller in 2009-10:

Tomas Vokoun: .937 EV, .873 ST, 19.2% ST
Tuukka Rask: .937 EV, .906 ST, 19.1% ST
Jaroslav Halak: .933 EV, .885 ST, 18.8% ST
Jonas Hiller: .930 EV, .874 ST, 20.5% ST
Henrik Lundqvist: .929 EV, .886 ST, 18.7% ST
Evgeni Nabokov: .928 EV, .900 ST, 22.1% ST
Miikka Kiprusoff: .928 EV, .888 ST, 21.0% ST

I think it was actually a pretty tight race for the top goalie this season, which is interesting since Miller has probably had the Vezina wrapped up since Christmas. I'm very interested to see how Miller does next season. I think it's a pretty safe bet to expect a great deal of regression to the mean in his special teams totals. I'm not saying he's a Jose Theodore, but I'm also not anointing him the best goalie in the world yet either.

This is not to say that special teams play is pure randomness. It can sometimes seem that way over a short sample, for example a playoff series or two, but PK save percentages generally correlate with 5 on 5 save percentages over multi-year samples. The problem is that typically only 1 in 5 shots come on the penalty kill, which means that evaluating a goalie based on single-season penalty kill performance is roughly the equivalent of evaluating a goalie's overall performance based on 12-15 games. As these playoffs have shown, anybody can run hot or cold over a short stretch like that.

In the long run goalies should be rewarded for persistently strong performance on the penalty kill, but results should be viewed skeptically over a single season. We don't want to be awarding the Vezina every year to the goalie who was the luckiest on the penalty kill, but we also should give some credit to a goalie who was excellent while his team was shorthanded. That makes it difficult for analysis.

6 comments:

Agent Orange said...

If you were given a vote who would be your pick?

For me it would be Miller.

For a season award its all about who had the most impressive season however it was done (luck, skill, whatever).

By hook or by crook Miller stopped pucks better than anyone but Rask this season and the performance was close. To me Millers minutes give him the edge. Its difficult to predict what Rask would have done with 60+ starts but we will find out next season.

Anonymous said...

What about Raycroft's "career year"?

AWF said...

"We don't want to be awarding the Vezina every year to the goalie who was the luckiest on the penalty kill, but we also should give some credit to a goalie who was excellent while his team was shorthanded. That makes it difficult for analysis."

Doesnt this comment assume that the people voting for the Vezina are actually doing any "analysis"? ;)

The Contrarian Goaltender said...

I'd still vote for Miller, but I think it was actually fairly close between him and a half-dozen other guys.

Re: Raycroft, he actually had a strong .941 EV SV% in 2003-04. In retrospect it definitely seems like a fluke, but it wasn't because of special teams.

Security News Media said...

Hello, I apologize for contacting you in this fashion, but I think you might be interested in submitting your site to my new sports directory…at thesportszone.org

I'm assuming comments are moderated so when I click submit this post won't automatically appear on site, if it does, I again apologize.

Tom Awad said...

Good stuff CG. Here's a different way of looking at the question (which I don't know the answer to): is short-handed save percentage, over 1 season, any more or less luck driven than a 15-game stretch of EV play?

I think the answer will be that they're the same, but I've yet to see it.