Wednesday, March 7, 2007

Goalie Consistency

Consistency is often used in a vague sort of way when it comes to sports. It can become an easy solution ("We need more consistency!"), and broadcasters like to use it as a convenient prescription for a struggling player.

Consistency would seem to be a valuable attribute in a goalie. Certainly any coach would like to have a goalie that they can count on to put forth a strong, consistent effort. This is especially true on good teams, because if they know that the goalie behind them will make the routine saves they will have a very good chance to win.

Consistency is not always a good thing, however. Consistent bad play is obviously undesirable. In addition, it is better for goalies on bad teams to be less consistent and have the capability for outstanding games. A goalie who consistently plays at an average level and never steals games is useful to a league leader, but much less valuable to a bottom feeder. It is unclear whether coaches and GMs appreciate this fact, but it is part of the goalie value equation.

To measure consistency, I calculated the variance of each goalie's save percentage from each game this year. There are some lurking variables affecting this analysis, such as quality of opposition, shot quality, and the other usual things that affect save percentage, but it is at least a starting point.

Here are the most consistent starting goalies in the league:
1. Manny Fernandez, MIN
2. Olaf Kolzig, WSH
3. Chris Mason, NSH
4. Miikka Kiprusoff, CGY
5. Marc-Andre Fleury, PIT

And the least consistent:
1. Evgeni Nabokov, SJS
2. J.S. Giguere, ANA
3. Dwayne Roloson, EDM
4. Henrik Lundqvist, NYR
5. Manny Legace, STL

Some other notables: DiPietro ranks 6th, Luongo 7th, Brodeur 11th and Hasek 22nd.

Bryzgalov, Markkanen and Weekes also posted fairly high variances, which suggests that maybe team defensive play is the reason for the variance. This is true for some on the other end as well, as McLennan, Backstrom, and Thibault also have been consistent.

More study is required to find out if consistency is predictive (i.e. the same goalies are consistent year after year), and possibly to find ways to adjust for some of the other influencing factors.

1 comment:

JavaGeek said...

A more detailed explanation of the technique I will use below can be found here in the Actual Results part of "Overtime going into extra innings"

Basically you compare actual variation to expected random variation using some simple model. Shots on net are generally modeled as binomial, with variance npq. So I compared actual goaltender variance with expected variance and normalized the difference into a Z-score by dividing out the standard error (goalies with fewer games with have more variation in the variation eg. game 1: shutout, game 2: 5 goals on 20 shots). If you look at the distribution of the Z scores you can figure out if there is more or less variation than expected if neither of those are true you will reject those possibilities in favour of the fact that there is equal variation in the sample as one would expect with random variation.

Goaltenders in general had more variation than random expectation [largely due to the fact that goaltenders see different amount of shots in each game, but the binomial variation assumes shots are constant], but the standard deviation of the variations equals 1 (0.99 in 2006, 1.02 in 2007). Standard deviation of 1 suggesters that random variation can explain all of the actual variation, so there is no addition variation in regards to a "consistency skill".

If you go through individual goaltenders in each year you will find just as many goaltenders at the top of list who fall to the bottom as there are top players who stay at the top.

Long and short I'm not sure there's any statistical evidence for "consistent" goaltending.