Thursday, October 2, 2008

Save Percentage vs. League Average

One of my frequent commenters did an analysis of save percentage vs. league average recently, and I wanted to revisit that metric for the goalies of the save percentage era. This statistic has been referred by some as "Goals Saved", so I'll use that name. For each goalie, I calculated a weighted league average save percentage, based on the league average and the number of shots they faced each season. This allows us to compare their actual save percentage to league average, and we can convert it into goals by multiplying by the number of shots faced. I did this for both the regular season and the playoffs to arrive at a combined total number of goals saved compared to average.

Remember that these are unadjusted save percentages, so there are hidden team effects for some of these goalies. That is why a straight 1-2-3-4-5-etc. ranking isn't really called for here, so I divided them up into tiers, based on a few very obvious breaks. Longevity is also important in this ranking, so someone like Kiprusoff, whose rate numbers are right up with the Tier 2 guys, still drops into Tier 3 because he hasn't faced that many shots.

I think if there is one thing you need to take away from these numbers, it is the gap between the two guys at the top and everyone else. A lot of people want to bring Brodeur, Belfour, Fuhr or others into that top group, but it is really not even close. Also note, however, that Roberto Luongo is only 29 years old, and has a good chance to surpass the 300+ goals saved mark and move into the top group before his career is over.

I also threw in a few well-known guys at the end for interest's sake, because tearing down reputations is what we like to do around here.

Tier 1 (All-Time Greats):

NameSv%LgSv%G SvPl Sv%Pl LgSv%Pl G SvTot G Sv
Patrick Roy.910.895425.3.918.90592.9518.2
Dominik Hasek.922.903384.2.925.91436.1420.3

Tier 2 (Top starting goalies):

NameSv%LgSv%G SvPl Sv%Pl LgSv%Pl G SvTot G Sv
John Vanbiesbrouck.899.891197.4.915.90324.9222.3
Martin Brodeur.913.905194.0.919.91325.6219.6
Curtis Joseph.907.900184.9.917.91123.6208.5
Ed Belfour.906.900148.5.920.90949.2197.7
Roberto Luongo.919.907178.7.941.9199.4188.1



Tier 3 (Decent starters):

NameSv%LgSv%G SvPl Sv%Pl LgSv%Pl G SvTot G Sv
Kelly Hrudey.893.887122.0.891.893-5.2116.8
Mike Richter.904.89996.1.909.9058.9105.0
J.S. Giguere.915.90790.5.925.91810.3100.8
Guy Hebert.909.902100.8.913.918-1.998.9
Andy Moog.892.886114.9.890.897-18.996.0
Tom Barrasso.892.88888.4.902.9007.195.5
Tomas Vokoun.914.90789.6.922.9210.490.0
Sean Burke.902.89893.2.888.892-4.289.0
Miikka Kiprusoff.915.90674.7.925.9198.783.4
Ron Hextall.895.89081.8.897.8970.081.8



Tier 4 (Overrated):

Chris Osgood.907.90533.1.914.915-2.530.6
Grant Fuhr.887.8870.0.900.89520.120.1
Mike Vernon.890.892-39.8.896.900-14.1-53.9

19 comments:

Anonymous said...

i know admitting this would castrate pretty much all your points, but save percentage is not the end all statistic. goalies do not go out there trying to pad their save percentages, they go out there to win. do you really think that when brodeur poke checks a guy on a breakaway, or cuts off a centering pass, he's thinking to himself "shit, now my save % isnt gonna be as high as if i took that shot"? or when he redirects a shot out to the side boards in an attempt to start up a break out, he going "well if i played like hasek, i could just let it sit there, have guys wack away 3 or 4 times, and up my save %"? both roy and hasek were very sloppy in controlling the puck and managing the crease area. this is something that can not be quantified in its value through the complex statistics you try to convolude things with. my main point though, is that no single statistic can determine how great a goalie is. it has to be done by evaluating all of them, in various situations.
additionally, save % hasnt been the most important thing considered for vezina voting, which is conducted by people getting paid millions to evaluate hockey talent. i doubt you know something these guys arent aware of. now i am not completely discrediting your case because it was these gm's that picked hasek over brodeur and roy throughout the 90's. the only thing worth noting is that brodeur already has more vezina's than roy in fewer seasons, and to consider he's missed 1.5 season during the course of his career due to lockouts, his numbers could be a ton better than they already are. also if you are going to challenge brodeur's vezina's what about roy's? not to mention that brodeur has played with the same team his entire career, while both hasek and particularly roy jumped ship to better teams when the going got tough. could you imagine if brodeur played for detroit the past few season when everyone said the devils were done?

The Contrarian Goaltender said...

No, when Brodeur makes a pokecheck I think he is thinking something more along the lines of, "Good thing I didn't have to take that shot, since that guy was in a great scoring position. Given that his scoring percentage from there is likely much higher than the average shooting percentage against me, I am helping out my save percentage by intercepting the pass and avoiding the shot."

no single statistic can determine how great a goalie is. it has to be done by evaluating all of them, in various situations.

No single statistic we have at the moment, I agree, because we still aren't able to perfectly establish team context. That doesn't mean all statistics are equal, however, far from it. Save percentage is far more important than wins or shutouts, because hockey is a team game and those two stats are impacted a lot more by the rest of the team than save percentage is.

If other statistics like wins and GAA are adjusted for team and league context, then I have no problem using them, and have in fact used both of them in the past. I am planning to do a similar table to the one above with GAA vs. average, wins vs. average, and shutouts vs. average, so those who like those other stats can take them into consideration as well.

Anonymous said...

i was wondering what your take on henrik lundqvist is? i mean out of any goalie in the past decade, the guy they have already dubbed "king" in new york seems to to be an even more glaring example of an overrated, team product. using almost every measure of goaltender evaluation you have used here, is there any evidence to really say this guy is that good? considering he had 10 shutouts last year, i know there were a few were he faced less than 20 shots, and i beleive his average shots/game in the shutouts was like 22 or 23 shots.

The Contrarian Goaltender said...

You are right that Lundqvist plays in a favourable team environment. But just because someone plays on a strong defensive team does not automatically mean they are not a good goalie. I think Lundqvist's 2007-08 season was very much overrated because of the defence in front of him, but I think there is substantial evidence that he is a quality starting goalie.

If we look at the measure used above, Lundqvist's career save percentage is .917, and his weighted league average is .905, meaning he is 62.8 goals above average already in his short career. He is almost already in with the tier 3 guys, most of who played long careers.

Secondly, we can compare him to his backup goalies. Lundqvist has a .622 win%, 2.27 GAA, and .917 save % so far. His backups have gone .491, 2.89, .895. Granted they haven't been great backups, but that is a substantial gap. Note that Valiquette did very well last season in limited playing time (2.19/.916), which is more evidence that the Rangers were very good defensively in 2007-08.

One problem with evaluating Lundqvist is that the Rangers' official scorers are way off in their recording of shot distance, which makes Lundqvist's shot-quality neutral numbers look way better than they are. So it is hard to accurately estimate the Rangers' defensive effect, but they were definitely above average for 2007-08.

So Lundqvist has an outstanding save percentage record and way outplayed his goalie teammates. That does not suggest a team created goalie. I think the evidence suggests that Lundqvist is a quality NHL goalie, and while his defence made him look better than he was in 2007-08, his 2005-06 and 2006-07 seasons were both very good.

Anonymous said...

considering that roy played half his career during the same "defensive" era brodeur did, that league average save % of .895 seems a little low. i also picked up on the fact that once a again you cherry picked both hasek and brodeur's individual "league averages" along with roy's to make your case. i'll leave it to you though to decide whether or not you want to tell your readers how your manipulating the numbers.

Anonymous said...

Anon - No, Roy began his career in 1986... Brodeur almost 10 yrs later.

Anonymous said...

no anon 2, roy 85-03, brodeur 93-now. roy played 8 years before brodeur, and 10 seasons in which their time overlapped.

The Contrarian Goaltender said...

Anonymous: Every goalie in the above table had a league average calculated in exactly the same way. The league average save percentage from 1985-86 to 1992-93 was .882, which is why Roy's league average number was substantially lower than Brodeur's.

I appreciate when people point out errors, but saying something seems a little low and using that as a basis to accuse me of falsifying data for my own ends when the information is readily available to check on the Internet seems to me to be a touch unreasonable.

Anonymous said...

Anon - that's what I said; Brodeur began his career almost 10 yrs later.

Bruce said...

Tier 4 (Overrated):

Chris Osgood .907 .905 33.1 .914 .915 -2.5 30.6
Grant Fuhr .887 .887 0.0 .900 .895 20.1 20.1
Mike Vernon .890 .892 -39.8 .896 .900 -14.1 -53.9


Fuhr's reputation was largely gathered by his performance in the playoffs. Note that his Goals Saved rating in the post-season (+20.1) not only leaves his fellow Tier 4 goalies in the dust, it's higher than all ten goalies you list in Tier 3. (In fact, it's more than the top two in this category combined.) Also, note the postseason gulf between Fuhr (+20.1) and Moog (-18.9), and it seems strange to see the latter listed in a higher tier than the former. There's a reason Sather went to Fuhr in the clutch.

My own observations of Fuhr as an Oiler season ticket holder throughout the 80s (and admitted fan), is that he faced a very high quality of shot due to the Oilers tendency to trade odd-man rushes, and that his attention tended to wander late in games. Don't know how many times I saw him give up a couple softies in the last five minutes to turn a 7-2 blowout into a 7-4 blowout. But with a one-goal lead the man was as tough to beat as any goalie I've ever seen.

Bruce said...

Interesting study, btw. The weakness of course is that it relies solely on Sv% as an indicator of a goalie's effectiveness. As we have discussed at length elsewhere, the goalie can have an impact on the flow of play other than stopping the puck. A goalie who excels at getting the puck moving in the right direction (Brodeur, Belfour) is apt to be underrated if measured only in the Sv% category. It's arguable whether they are quite as good as Hasek and Roy, but they are a lot closer than your results indicate in my view. Meanwhile, Marty Turco, an excellent starting goalie, apparently doesn't rate at all.

While comparing goalies to league averages over their careers is certainly legtimate, another method is to compare goalies to their contemporaries over the same groups of seasons. I have recently done such a study of the three post-lockout seasons here.

My study included a basket of stats beyond just Sv%, although I did attempt to give that particular category a greater weight as it certainly is an important measuring stick.

The Contrarian Goaltender said...

Yes, I noticed Fuhr's excellent playoff results as well. I would really like to see some game sheets from the 1980s to see if Fuhr did actually lock down the one-goal leads that everyone says he did. I also do wonder whether the Oilers' shot-quality allowed was quite as bad as it is often made out to be, especially from 1987 onwards when their shots against and goals against totals dropped substantially.

By the way, there are a couple of things to take into account when you look at the Moog/Fuhr split. I don't have playoff stats from before 1984. Grant Fuhr got shellacked in the 1982 postseason, whereas Moog helped the Oilers to the second round in 1981 and the Finals in 1983. As a whole I don't think Moog did any worse than Fuhr in the playoffs in Edmonton, the difference comes from Moog's poor play in Boston and Dallas. Moog also played more playoff games later in his career than Fuhr did.

Still, I think it is pretty clear that over their playoff careers Fuhr had better results than Moog. How much that had to do with Gretzky and Messier and how much that had to do with Fuhr remains an open question.

The Contrarian Goaltender said...

Concerning your study, Bruce, I think your rankings are reasonable, but I don't like using all the different categories to rank goalies because it results in double or triple-counting the same two basic things: games played (which affects all the totals numbers) and save percentage (which is at the core of all the efficiency statistics).

For example, ranking goalies based on wins or saves is more or less just ranking them based on games played, since a goalie who played twice as many games as another goalie will almost certainly finish ahead in both categories even if he is a worse goalie playing on a worse team. Similarly, GAA is just (1 - Save Percentage) x Shots per game, and so the only extra info you get from GAA is how many shots the goalie faced. I think we both agree that the majority of that is team-determined, so by equating GAA and save percentage you are more or less double-counting save percentage and then adding extra bonus points or penalties based on the strength of a goalie's teammates. There might be a small shot prevention effect encapsulated in the stat, but it is overwhelmed by the noise that at this point I'd rather ignore GAA than use it as flawed evidence of a potential effect.

I do agree with how you increased the save percentage weighting, as it should be considered the most important goalie stat. But I still think shutouts are arbitrary, as you know, and I don't know why you would use saves as a measurement. If you want to measure durability, I think minutes are more appropriate, otherwise you are penalizing goalies who play on strong defensive teams and therefore face fewer shots.

I would rather look at statistics that attempt to adjust for team context, but if I was going to do a similar ranking based on unadjusted numbers I would try to select categories that do not substantially overlap, (e.g. minutes played, winning percentage, save percentage), and then weight each category appropriately. You will probably get similar results, my quick test at one possible weighting went Brodeur-Lundqvist-Luongo, but I think the method is superior.

Bruce said...

CG: Thanks for your comments. When the spirit so moves you, feel free to leave such feedback on my blog in future, I could use the traffic. :)

Concerning your study, Bruce, I think your rankings are reasonable, but I don't like using all the different categories to rank goalies because it results in double or triple-counting the same two basic things: games played (which affects all the totals numbers) and save percentage (which is at the core of all the efficiency statistics).

Right-o. I did try to make clear that my crude little study was A method, not The method, simply using the info available to me at that moment. Your point is well-taken: in my lingo the two main categories are quality (percentages) and quantity (counting stats). Thus the various attempts to weight the results.

I do agree with how you increased the save percentage weighting, as it should be considered the most important goalie stat.

Method 1 = 60% quantity, 40% quality, (20% Sv%)
Method 2 = 40% quantity, 60% quality, (33% Sv%)
Method 3 = 30% quantity, 70% quality, (50% Sv%)

But I still think shutouts are arbitrary, as you know, and I don't know why you would use saves as a measurement.

Because they were there, listed in the little THN spreadsheet. As for shutouts, recall how the little shutout leaders study I did a while back did a pretty darn decent job of identifying who the top (reputation) keepers were over the years. In general I give the category more value than you, but note I ranked it 4th out of 5 when weighting, and in method 3 gave it just 10% value.

If you want to measure durability, I think minutes are more appropriate, otherwise you are penalizing goalies who play on strong defensive teams and therefore face fewer shots.

Agreed, although I thought you of all people would measure workload on actual shots faced, not minutes spent standing around. :) Again, I stress it was a crude study using data that was available, which happened to be Sv rather than MP. I could do a bunch o' work determining the latter, and I'll bet it wouldn't affect the derived rankings at all, especially at the lowest weight that I gave that particular stat.

I would rather look at statistics that attempt to adjust for team context, but if I was going to do a similar ranking based on unadjusted numbers I would try to select categories that do not substantially overlap, (e.g. minutes played, winning percentage, save percentage), and then weight each category appropriately. You will probably get similar results, my quick test at one possible weighting went Brodeur-Lundqvist-Luongo, but I think the method is superior.

Probably. My study was hardly intended to be definitive. Fact remains it generated a pretty darn good looking list of best goalies since the lockout, give or take a Backstrom. The point being that if you do a number of different crude studies, either they will mostly agree with each other -- as yours and mine seem to have in the current instance -- or you should be able to identify a reason why they don't (i.e. a weakness in at least one of the methods). But to my mind, multiple methods that reach the same conclusions are much stronger than any one method.

seventieslord said...

[quote]I don't have playoff stats from before 1984.[/quote]

Are you aware of "The Hockey Compendium"? Playoff stats from 1952 and onwards exist for all goaltenders. Actually, starting in 1984 they stopped including every single goalie who played at all, and just included the ones who met a certain minutes threshold for the season, and advised to check the Official Guide and Record Books of those years for more info. I did that, and made a spreadsheet of them, combined with 1952-1983, and 2002-2008 since the book was published in 2001. Would this be of any use to you?

The Contrarian Goaltender said...

Yes I am aware of The Hockey Compendium, I read it several years ago but I don't currently have a copy. I definitely wouldn't mind looking at your spreadsheet to fill in the gaps in some of my studies, if you could email it to me that would be appreciated.

Anonymous said...

Compelling stats, but I think J-S Giguere should be in tier 2 or even tier 1, as the finest goalie after the Dominator.

Steven said...

I don't agree with this analysis at all. Roy started in an era where almost nobody else played the butterfly for a while until they copied him.

So naturally, their SV% would be lower. That doesn't put Roy in Tier 1. If anything, it would make Roy just as much a fraud as you claim Brodeur to be. In fact, look at Roy's career SV%, it is lower than Brodeurs.

The whole elite defensemen in front of Brodeur argument has not proven to be true either. As after all the elite d-men left NJ, Brodeur's Career SV% is now slowly rising.

I'll give you that Hasek was probably the goalie with the best reflexes in the league, but theres more to goalies than that.

Host PPH said...

I think that it is quite impressive that he can take from there and go forward. I don't see so much difference.