As I have written about many times on this blog, NHL save percentages are quite dependent on the strength of the team in front of the goaltender. If this is the case in a league with a high degree of parity, we should logically expect the team effect to be even more significant when teams are not on a level playing field.
Looking for a more unbalanced competitive scenario than the NHL provides, I chose to investigate international hockey results. There are goaltending statistics available for most of the major international tournaments since 2000 on the IIHF website. I looked at senior men's competitions (world championships, Olympics and World Cups), the U20 junior men's world championship, and women's competitions (world championships and Olympics).
After calculating the total shots against per game and save percentages for each country in each category, I ran the correlation coefficients for shots against vs. save percentage:
Men: -0.817
Women: -0.803
Juniors: -0.802
This shows a very high correlation between shots against and save percentage, a typical result for an unbalanced competitive environment. All three tournaments suffer from a lack of competitive balance. There are a few teams that are contenders to win, a few teams that will make a game of it against the best teams, and then there is the rest of the world which gets completely dominated by the best teams. The best teams give up few chances to score against, while the worst teams give up lots of high quality scoring chances.
The drop-off in talent is not equal, however. There are a lot more teams that have the capability of winning a men's world championship than a women's world championship. If we look at the top 8 teams only, and again compare the correlation coefficients between save percentage and shots against, we see a distinctly different result on the men's side:
Men: -0.014
Women: -0.765
Junior: -0.682
For the best men's teams, there is no relationship between save percentage and shots against. This is similar to the NHL, where there are a number of different teams on a fairly level playing field. Among the top womens and junior teams, the teams that do the best job of preventing shots are also much more likely to allow easier shots against. The better teams also probably have better goaltending, but the effect is much too strong for that to account for the entire difference. It is much more likely that a country like Switzerland would be able to be competitive with Canada in goaltending than that they would have competitive in overall team strength, since they would only need to develop one elite athlete as a goaltender compared to developing an entire team of top players.
Now let's move on to look at what kind of expected save percentage results we can expect for various countries in international play. On the men's side, the top teams are very close to each other in save percentage. There isn't a single team that has a huge advantage in terms of goaltending, and somewhat surprisingly Canada did not lead in any of save percentage, GAA, or fewest shots against per game:
1. Finland, .923, 1.85, 24.0
2. Canada, .921, 2.09, 26.3
3. USA, .919, 2.23, 27.4
4. Czech Rep., .914, 2.10, 24.5
4. Russia, .914, 2.20, 25.7
6. Switzerland, .910, 2.69, 29.9
7. Sweden, .909, 2.19, 23.9
8. Slovakia, 906, 2.28, 24.3
This supports Finland's emergence as a goalie hotbed. On the other hand, Sweden is a country that has sometimes struggled with goaltending despite a strong defence (although they are in better shape now with Lundqvist). Slovakia's strong team defence results suggest that they employed a shot prevention strategy, which was a smart move given the unremarkable goaltenders they have had.
About 40% of the Canadian sample is made up of Brodeur and Luongo, the two most-used Canadian goalies in recent international tournaments. The two of them also had very similar stats in the sample (Luongo: 2.03, .926, Brodeur: 2.01, .925). Brodeur faced slightly fewer shots per game than Luongo (26.8 to 27.5), but Brodeur's shots per game were still slightly above average for Canadian goalies, which doesn't support the view that he has a huge impact on shots against (although the international ice could possibly have something to do with that as well).
In the junior tournaments, we start to see more of a separation between the best teams and the rest of the pack:
1. Canada, .936, 1.63, 25.5
2. Russia, .920, 2.08, 26.2
3. Czech Rep., .910, 2.35, 26.1
3. Finland, .910, 2.51, 27.8
5. Sweden, .903, 2.43, 25.1
5. USA, .903, 2.59, 26.7
7. Slovakia, .895, 3.06, 29.3
8. Switzerland, .886, 3.30, 29.0
Canada almost always has the best defensive team in the world juniors, so playing goalie for Canada is a big advantage. Dustin Tokarski was pretty mediocre this year in net for Canada, but still ended up with a .906 save percentage.
In the women's tournament, there is a huge gap between the top teams and the rest:
1. Canada, .947, 0.87, 16.3
2. USA, .920, 1.26, 15.7
3. Switzerland, .904, 3.50, 36.2
4. Finland, .894, 2.73, 25.8
5. Sweden, .882, 3.27, 27.7
5. Russia, .882, 3.45, 29.6
7. Germany, .881, 4.14, 34.9
7. China, .881, 4.54, 38.2
I don't know much about women's hockey, but there are a few European goalies who had some pretty impressive results considering the strength of their teams: Sweden's Kim Martin (.897 on 562 SA), Finland's Noora Raty (.911 on 293 SA), and especially Switzerland's Florence Schelling (.920 on 488 SA). On the other hand if you play for Canada or the United States, you pretty much just have to stand there and watch your team score.
It is probably fair to say that Canadian junior and women goalies are better than average, but most of the gap between Canada and everyone else in those tournaments is likely a result of shot quality against. If Canada had received merely average goaltending, then to achieve the observed save percentages the shot quality against would have needed to be 40% easier than average for the Canadian junior teams and 52% easier than average for the Canadian women's teams. I doubt it was that high, but the true figure still certainly falls well outside of the typical 10% boundaries seen in NHL competition.
Overall, the data support the theory that shots against and save percentage are positively correlated in hockey when the competition is unbalanced. In that type of environment it is much easier to play goalie for the best teams, and that should be taken into account when evaluating, for example, Canadian goalies at the world junior hockey championships. When the competition is tight, then the shots/save percentage relationship largely disappears.
It is likely that this relationship holds throughout the history of the NHL as well. In the few seasons we have of save percentage data from post-expansion era in the 1970s, for example, shots against and save percentage were positively correlated. I would expect similar results for the early years of the NHL, and during much of the Original Six era. That makes it even more important to take team strength into consideration when evaluating goalies from these periods.
Subscribe to:
Post Comments (Atom)
17 comments:
So there is a team effect on save percentage? Wow.
Some teams allow better quality shots than others - save percentage does not account for shot quality in any way. That is team dependent
That said, many of your weak international teams (Slovaks, Swiss) have low saves percentages because they had bad goalies (especially in the junior tournament where one David Aebischer cant man the forts for a decade).
I think the data shows that in general, countries that aregood enough to produce good teams that don't allow many shots are also good enough to produce good goalies.
Some teams allow better quality shots than others - save percentage does not account for shot quality in any way. That is team dependent
That is not correct, save percentage is strongly affected by shot quality. Easier shot quality against means that goalies are more likely to have a higher save percentage. In situations of equal goaltending, save percentage can be used as a proxy for shot quality.
I think the data shows that in general, countries that aregood enough to produce good teams that don't allow many shots are also good enough to produce good goalies.
I disagree. I think the gap in goaltending talent across the elite teams is relatively small, and that the gap in overall team talent is where the primary differential lies. Look at the closeness of results at the senior men's level. If the difference in stats was entirely due to goaltending talent, then how are the rest of the countries making up the gap from the junior level to the senior level? The only way that could be the case is if Canadian goalies were regressing in their early 20s while European goalies were substantially improving.
It is simply a lot more likely that the depth of elite talent in Canada means that their junior teams are much stronger, and the other countries can only compete when they have the ability to draw on multiple birth years of talent.
Let's look at the world juniors, and only look at goalies who are either NHL regulars or are top-level prospects with some NHL experience:
Canada .941
(Mason, Bernier, Price, Fleury, Leclaire, Auld)
USA .903
(Schneider, Howard, DiPietro)
Finland .907
(Niittymaki, Lehtonen, Rask)
Sweden .917
(Lundqvist)
Slovakia .904
(Budaj, Halak)
Russia .950
(Bryzgalov, Varlamov)
Russia is an outlier here, mainly because of Bryzgalov's ridiculous 2000 tournament (102 saves on 105 shots for a .971 save %). However, even when looking only at the best goalies there is still a substantial spread among the various teams, and the numbers are pretty similar to the overall numbers for each country. Canada's goalie stats are still far superior to the other countries, yet their goalies are not that much better than the rest. The most logical explanation for this is substantially different shot quality against.
PSH:Some teams allow better quality shots than others - save percentage does not account for shot quality in any way. That is team dependent
CG:That is not correct, save percentage is strongly affected by shot quality. Easier shot quality against means that goalies are more likely to have a higher save percentage. In situations of equal goaltending, save percentage can be used as a proxy for shot quality.
PSH again: We are agreeing for the most part. We are both saying shot quality is not taken into account by saves percentages.
Where we are disagreeing is in the claim that goaltending quality is roughly equal. It isn't.
One proof of that is that you can offer 6! Canadian goalies on your list and 1-3 on the other teams. That is because the other goalies they used were often not so good.
To even begin to compare things is Junior is entirely faulty. There are a number of players that have either excelled in Junior or been nobodies in Junior and then played the exact opposite upon reaching the next level.
Players are still developing in Junior, for god sakes they are 16 17 and 18 for the most part. Even at the "elite" level such as international tournaments these kids are still far from the top of their development.
This is especially the case with goaltenders, to even begin to look at goaltending stats in Juniors is faulty.
To even begin to compare things is Junior is entirely faulty. There are a number of players that have either excelled in Junior or been nobodies in Junior and then played the exact opposite upon reaching the next level.
Players are still developing in Junior, for god sakes they are 16 17 and 18 for the most part. Even at the "elite" level such as international tournaments these kids are still far from the top of their development.
This is especially the case with goaltenders, to even begin to look at goaltending stats in Juniors is faulty.
Run along. You clearly haven't the foggiest idea what is being discussed here.
CG - it's amazing to me how many raving, anonymous trolls you get in this space. Unless, of course, it's the same guy over and over again.
I haven't stopped by in a while, but, once again, this is very good work CG. I think the trickiest thing here of course is sample size. We're talking about 30-50 games per sample I'm guessing. When you move it to the individual level it would seem even more difficult to quantify. That said, I think your overall point is sound.
CG,
Have you done any comprehensive research on correlation between save percentage, and shot amounts (both SF and SA) at the NHL level?
What about research on the correlation between team, and persistence of save percentage?
I thought I remember JLikens saying somewhere that he found a slight negative correlation between SF and S%. Intuitively that correlation makes sense to me. This is pure speculation here, but I would think a team that shoots less is, on average, taking higher quality shots. And on the flip side, how often do you see a team lead the league in both SA and save percentage?
If save percentage is highly correlated with team factors (e.g, isn't it curious that both Tim Thomas and Manny Fernandez have insanely high save percentages playing for Claude Julien?), doesn't it seem a little unfair to make vehement statements about individual goaltender talent and use save percentage as almost the sole piece of evidence? (Not only do you do that quite a bit on this blog, but I think a lot of us hockey stat guys are sometimes guilty of it.)
For example, what if Brodeur's mediocre save percentage during the heyday of the Devils' neutral-zone trapping, ridiculously low shot-allowing prime had a lot to do with the team system? It's a little odd that amidst the post-lockout rule change NHL, the Devils lost a few very good defensemen, started allowing more shots, got coached by Claude Julien, and suddenly Brodeur had a much higher save percentage.
Isn't it possible that there's an explanation other than "Brodeur got better"?
Hey Sunny M., I think I'll give a bit of an answer for CG here:
Have you done any comprehensive research on correlation between save percentage, and shot amounts (both SF and SA) at the NHL level?
There are a few links that CG did:
http://brodeurisafraud.blogspot.com/2008/11/es-save-while-tied.html
http://brodeurisafraud.blogspot.com/2008/11/shots-attempts-against.html
The links on the ES Save % while tied article link to mc79hockeys discussion on this matter, which looks like the genesis of CG's look into this matter.
What about research on the correlation between team, and persistence of save percentage?
I'm not sure what you mean. Is it something like how Anderson/Vokoun and Thomas/Fernandez have similar stats behind the same team, which makes you think that the save %'s are inflated? If so, I give a response later in this post.
I thought I remember JLikens saying somewhere that he found a slight negative correlation between SF and S%. Intuitively that correlation makes sense to me. This is pure speculation here, but I would think a team that shoots less is, on average, taking higher quality shots. And on the flip side, how often do you see a team lead the league in both SA and save percentage?
From what I remember seeing, that negative correlation was weak at best, and has a lot to do with situational play. I think this i the link you were thinking of, so I won't put words in his mouth:
http://objectivenhl.blogspot.com/2008/10/goaltending-save-percentage-and-shot.html
If save percentage is highly correlated with team factors (e.g, isn't it curious that both Tim Thomas and Manny Fernandez have insanely high save percentages playing for Claude Julien?), doesn't it seem a little unfair to make vehement statements about individual goaltender talent and use save percentage as almost the sole piece of evidence? (Not only do you do that quite a bit on this blog, but I think a lot of us hockey stat guys are sometimes guilty of it.)
That's why save percentage isn't that great of a stat, and most often a shot quality-adjusted save percentage is preferred. In the case of Boston, it does have a lot to do with the system, but as someone who's seen a good amount of Bruins games, it's not only the system; those guys are flat out doing their jobs.
That all being said, I do agree that CG does rely too much on save %, though I can't fault him for it, since it's the easiest "individual" goalie stat to find, and good for back-of-the-envelope calculations. It's just that it's not perfect as a measure of goalie ability until all the other factors are taken in, such as the links I provided above go in to.
For example, what if Brodeur's mediocre save percentage during the heyday of the Devils' neutral-zone trapping, ridiculously low shot-allowing prime had a lot to do with the team system? It's a little odd that amidst the post-lockout rule change NHL, the Devils lost a few very good defensemen, started allowing more shots, got coached by Claude Julien, and suddenly Brodeur had a much higher save percentage.
Isn't it possible that there's an explanation other than "Brodeur got better"?
May be, but again here are some of CG's looks into this topic:
http://brodeurisafraud.blogspot.com/2007/02/brodeur-in-2006-07.html
http://brodeurisafraud.blogspot.com/2007/04/system-part-2.html
http://brodeurisafraud.blogspot.com/2007/05/disciplined-devils.html
Also, paraphrasing something that CG has said before, logically it wouldn't make sense for a team to stop the easier shots, and allow tough shots to go through. In fact, it is significantly more logical that a team that stops shots from going through to also not give up too many prime scoring opportunity shots, because of the way in which shots weren't being allowed. In NJ's case, the low shots against came because they didnt allow anyone to enter their zone at all, which suggests to me that when teams entered it wasn't going to be a breakaway or an odd man rush; most likely, teams would try to get a point shot, or some other fairly low-percentage shot.
Not sure if those are exactly how CG would answer it, but seems sensible from what I've read. Also hopefully the HTML tagging I did works, and apologies if you have to deal with strange letters everywhere ;).
2 shutouts in his 3 games back, Brodeur is a god among men... You seem to me.. well.. jealous.
You need an example of how rebound control and puck handling can have a significant effect on a team, check out the third goal the Devils scored today against Philly.
You need an example of how rebound control and puck handling can have a significant effect on a team, check out the third goal the Devils scored today against Philly.
Anon, it's not that rebound control is unimportant. The big issue here is how much of a quantitative effect rebound control has on goal prevention, and NOT whether it matters at all. I think CG's general point is the fact that things like rebound control and stickhandling are given somewhat "mystical" properties and values, where it may seem like every time a goalie is handling the puck that a goal is being prevented, or every time a goalie allows a rebound is a dangerous shot; however, the reality is that even a goalie who can barely play the puck is probably only going to see a couple of more shots against a game, and rebounds typically end up in harmless areas of the ice, and the defense is typically able to clear the puck without much concern. Also, it's not like having great rebound control means that every rebound is safe; it just means you give up less rebounds, and those that you give up will typically be less juicy. Taking these into account, the important thing is to see how valuable skills are to a goalies overall repertoire: in other words, seeing the relative value of every skill a goalie may have, and fitting the "soft" skills into the evaluation correctly, as opposed to saying using stick-handling and rebound control as a trump card in goaltender evaluation.
FatMan: Great summary. I don't dispute that I rely a lot on save percentage, and no doubt have taken it too far on a few occasions, but I wouldn't say it is correct that I prioritize save percentage for reasons of availability or convenience. It is because there isn't much evidence that goalies have a major impact on either shot prevention or team offensive play.
Having said that, the evidence is nevertheless clear that there is some effect, and that is what I am researching right now (Brodeur's prolonged absence gave us a good test case). I think it is just as much an error to overvalue the "soft" goalie skills as it is to undervalue them, so I am being very cautious about adding them in to my evaluative systems before more work is done.
Sunny Mehta:
e.g, isn't it curious that both Tim Thomas and Manny Fernandez have insanely high save percentages playing for Claude Julien?
I wouldn't describe that as curious at all. Or at least no more curious than the insanely high save percentages put up by Backstrom and Harding in Minnesota, or Clemmensen and Weekes in New Jersey. Save percentage depends on the team, most definitely.
There are goalies who have put up great save percentages on great defensive teams. All my work on save percentage and shots against has shown that you can't generalize - team situations differ. It is possible that a team that allows few shots against gives up higher quality shots than normal, but I highly doubt that was the case for the New Jersey Devils, given my subjective observations and the available shot quality numbers.
My position on Brodeur is not entirely dependent on save percentage, although I have built most of my case using save percentage because I view that as the best goalie statistic for evaluation (and even better if adjusted for shot quality or team). My efforts to adjust a variety of other stats for team context have repeatedly slotted Brodeur in with the other good goalies of his generation, and certainly do not show him to be an outlier like somebody like Hasek.
Again, if I come across as overly harsh towards Brodeur it is because I am usually discussing him in an all-time context. He is clearly one of the best goalies of his generation, but I see him as still lacking a bit when compared with the best to ever play.
Isn't it possible that there's an explanation other than "Brodeur got better"?
It's possible, but if so it has to be something that is not quantified by his performance statistics or shot quality metrics. Those numbers all pretty much agree that Brodeur got a lot better sometime around January 1, 2006.
Is it just coincidence Martin Brodeur has 2 SO in 3 games, and 4 in 13 overall, while Clemmenson and Weekes combined for only 2 in 50+ games? Judging by the way Marty has looked both now, and prior to the injury, plus the play/numbers of Clemmenson and Weekes, plus his performance the past couple years, could it not at least be inferred that had he not been injured he could have put up the greatest single season in NHL history? His projected totals would put him at around 55 wins, 15+ shutouts, a low 2 GAA and .920-.930 save percentage.
I don't dispute that I rely a lot on save percentage, and no doubt have taken it too far on a few occasions, but I wouldn't say it is correct that I prioritize save percentage for reasons of availability or convenience. It is because there isn't much evidence that goalies have a major impact on either shot prevention or team offensive play.
Just to be clear, by "prioritize save % because of availibity or convenience," I meant that you use it in lieu of ES save %, or SQNS%, not GAA or Wins or some other junk like that.
Having said that, the evidence is nevertheless clear that there is some effect, and that is what I am researching right now (Brodeur's prolonged absence gave us a good test case). I think it is just as much an error to overvalue the "soft" goalie skills as it is to undervalue them, so I am being very cautious about adding them in to my evaluative systems before more work is done.
My second post in the comments was same thought ;)
"My efforts to adjust a variety of other stats for team context have repeatedly slotted Brodeur in with the other good goalies of his generation, and certainly do not show him to be an outlier like somebody like Hasek."
Yeah I totally agree with that assessment.
"Again, if I come across as overly harsh towards Brodeur it is because I am usually discussing him in an all-time context. He is clearly one of the best goalies of his generation, but I see him as still lacking a bit when compared with the best to ever play."
Again I agree with your position on his status in terms of best of all time. And I think your analysis pertaining to that is almost always objective and fair. On the other hand, lol, titling the blog "Brodeur is a Fraud" is probably what comes across as overly harsh and unfair. FWIW, while you'd probably lose some sex appeal, I think a new title would do better justice to not only Marty, but your work. (Might cut down on the raving, anonymous trolls too.) :)
Post a Comment