Wednesday, February 25, 2009

Save Percentages in Unbalanced Environments

As I have written about many times on this blog, NHL save percentages are quite dependent on the strength of the team in front of the goaltender. If this is the case in a league with a high degree of parity, we should logically expect the team effect to be even more significant when teams are not on a level playing field.

Looking for a more unbalanced competitive scenario than the NHL provides, I chose to investigate international hockey results. There are goaltending statistics available for most of the major international tournaments since 2000 on the IIHF website. I looked at senior men's competitions (world championships, Olympics and World Cups), the U20 junior men's world championship, and women's competitions (world championships and Olympics).

After calculating the total shots against per game and save percentages for each country in each category, I ran the correlation coefficients for shots against vs. save percentage:

Men: -0.817
Women: -0.803
Juniors: -0.802

This shows a very high correlation between shots against and save percentage, a typical result for an unbalanced competitive environment. All three tournaments suffer from a lack of competitive balance. There are a few teams that are contenders to win, a few teams that will make a game of it against the best teams, and then there is the rest of the world which gets completely dominated by the best teams. The best teams give up few chances to score against, while the worst teams give up lots of high quality scoring chances.

The drop-off in talent is not equal, however. There are a lot more teams that have the capability of winning a men's world championship than a women's world championship. If we look at the top 8 teams only, and again compare the correlation coefficients between save percentage and shots against, we see a distinctly different result on the men's side:

Men: -0.014
Women: -0.765
Junior: -0.682

For the best men's teams, there is no relationship between save percentage and shots against. This is similar to the NHL, where there are a number of different teams on a fairly level playing field. Among the top womens and junior teams, the teams that do the best job of preventing shots are also much more likely to allow easier shots against. The better teams also probably have better goaltending, but the effect is much too strong for that to account for the entire difference. It is much more likely that a country like Switzerland would be able to be competitive with Canada in goaltending than that they would have competitive in overall team strength, since they would only need to develop one elite athlete as a goaltender compared to developing an entire team of top players.

Now let's move on to look at what kind of expected save percentage results we can expect for various countries in international play. On the men's side, the top teams are very close to each other in save percentage. There isn't a single team that has a huge advantage in terms of goaltending, and somewhat surprisingly Canada did not lead in any of save percentage, GAA, or fewest shots against per game:

1. Finland, .923, 1.85, 24.0
2. Canada, .921, 2.09, 26.3
3. USA, .919, 2.23, 27.4
4. Czech Rep., .914, 2.10, 24.5
4. Russia, .914, 2.20, 25.7
6. Switzerland, .910, 2.69, 29.9
7. Sweden, .909, 2.19, 23.9
8. Slovakia, 906, 2.28, 24.3

This supports Finland's emergence as a goalie hotbed. On the other hand, Sweden is a country that has sometimes struggled with goaltending despite a strong defence (although they are in better shape now with Lundqvist). Slovakia's strong team defence results suggest that they employed a shot prevention strategy, which was a smart move given the unremarkable goaltenders they have had.

About 40% of the Canadian sample is made up of Brodeur and Luongo, the two most-used Canadian goalies in recent international tournaments. The two of them also had very similar stats in the sample (Luongo: 2.03, .926, Brodeur: 2.01, .925). Brodeur faced slightly fewer shots per game than Luongo (26.8 to 27.5), but Brodeur's shots per game were still slightly above average for Canadian goalies, which doesn't support the view that he has a huge impact on shots against (although the international ice could possibly have something to do with that as well).

In the junior tournaments, we start to see more of a separation between the best teams and the rest of the pack:

1. Canada, .936, 1.63, 25.5
2. Russia, .920, 2.08, 26.2
3. Czech Rep., .910, 2.35, 26.1
3. Finland, .910, 2.51, 27.8
5. Sweden, .903, 2.43, 25.1
5. USA, .903, 2.59, 26.7
7. Slovakia, .895, 3.06, 29.3
8. Switzerland, .886, 3.30, 29.0

Canada almost always has the best defensive team in the world juniors, so playing goalie for Canada is a big advantage. Dustin Tokarski was pretty mediocre this year in net for Canada, but still ended up with a .906 save percentage.

In the women's tournament, there is a huge gap between the top teams and the rest:

1. Canada, .947, 0.87, 16.3
2. USA, .920, 1.26, 15.7
3. Switzerland, .904, 3.50, 36.2
4. Finland, .894, 2.73, 25.8
5. Sweden, .882, 3.27, 27.7
5. Russia, .882, 3.45, 29.6
7. Germany, .881, 4.14, 34.9
7. China, .881, 4.54, 38.2

I don't know much about women's hockey, but there are a few European goalies who had some pretty impressive results considering the strength of their teams: Sweden's Kim Martin (.897 on 562 SA), Finland's Noora Raty (.911 on 293 SA), and especially Switzerland's Florence Schelling (.920 on 488 SA). On the other hand if you play for Canada or the United States, you pretty much just have to stand there and watch your team score.

It is probably fair to say that Canadian junior and women goalies are better than average, but most of the gap between Canada and everyone else in those tournaments is likely a result of shot quality against. If Canada had received merely average goaltending, then to achieve the observed save percentages the shot quality against would have needed to be 40% easier than average for the Canadian junior teams and 52% easier than average for the Canadian women's teams. I doubt it was that high, but the true figure still certainly falls well outside of the typical 10% boundaries seen in NHL competition.

Overall, the data support the theory that shots against and save percentage are positively correlated in hockey when the competition is unbalanced. In that type of environment it is much easier to play goalie for the best teams, and that should be taken into account when evaluating, for example, Canadian goalies at the world junior hockey championships. When the competition is tight, then the shots/save percentage relationship largely disappears.

It is likely that this relationship holds throughout the history of the NHL as well. In the few seasons we have of save percentage data from post-expansion era in the 1970s, for example, shots against and save percentage were positively correlated. I would expect similar results for the early years of the NHL, and during much of the Original Six era. That makes it even more important to take team strength into consideration when evaluating goalies from these periods.

Monday, February 23, 2009

Washington's Goalie Mistake

In the past offseason, the Washington Capitals had the choice of resigning deadline acquisition Cristobal Huet, or bringing in another goalie as a replacement. The Caps attempted to sign Huet, but were not able to meet his asking price of $5.6 million per season, and had to settle on paying Jose Theodore $9 million over 2 years as a backup plan. So far this has not worked out particularly well for the Capitals as Huet, despite getting off to a slow start in Chicago, has significantly outplayed Theodore.

This has been unsurprising to anyone familiar with recent save percentage history for the two goalies. Theodore has played 2,321 minutes this season and has faced 1,086 shots against. He has stopped 90.1% of shots against, and Hockey Numbers estimates his shot quality neutral save percentage to be .899. Huet is at .918 in raw save percentage and .923 in adjusted. Both these marks are about in line with recent performance - in a post from before this season I used results from the past three seasons to predict both adusted and raw save percentage numbers. The adjusted save percentage predictions were .917 for Huet and .898 for Theodore, both of which are pretty similar to actual results so far.

If Huet faced Theodore's shots this season and put up the same raw save percentage in Washington as he did in Chicago, he would have allowed 89 goals, or 19 fewer than Theodore. If we evaluate the hypothetical using adjusted save percentages, we could expect Huet to allow about 82 goals, or 26 fewer than Theodore has. The effect on Washington would be to improve their goal differential to +55, which would rank them right behind Boston (+63) and San Jose (+59) as the third-best team in the league.

According to some reports, Washington tried hard to sign Huet but he wanted to test the waters of free agency. Still, it is hard to imagine that the Capitals front office doesn't have at least some regret about their decision to go with Theodore - I assume that Ted Leonsis would be quite willing to pay an extra $1.125 million this season (the difference in cap hits between Huet and Theodore) for a 20+ reduction in goals allowed for and as a result a team that, with the division they play in, could possibly have been leading the NHL in points right now.

Saturday, February 14, 2009

Shots Against by Team

At the request of one of my commenters, here are the shots against faced per game by team since 1993-94. To find these numbers, I used Hockey Reference. Using the power play feature, you can sort the data according to specific criteria, such as all goalies who have played for a particular franchise since 1993-94. That gives us a page such as this one for the Los Angeles Kings. The next step is to copy those numbers into a spreadsheet, total them up, and repeat the process for every team in the league.

Here are the results, sorted in order of fewest to most shots per 60 minutes of play:

1.New Jersey71,16730,44525.7
4.St. Louis70,96131,23226.4
7.San Jose70,72932,54827.6
16.Tampa Bay70,88133,82628.6
19.N.Y. Rangers71,00634,31329.0
23.Los Angeles70,71835,02829.7
24.N.Y. Islanders70,83235,16929.8

Thursday, February 12, 2009

Atlantic vs. Northeast Continued

Here is some more data on the Atlantic Division vs. the Northeast Division, the average goals and average shooting percentage per team by division (not including New Jersey and Buffalo):

Atlantic: 268 goals, 10.6%
Northeast: 263 goals, 10.4%

Atlantic: 131 goals, 9.8%
Northeast: 148 goals, 10.4%

Atlantic: 252 goals, 10.1%
Northeast: 267 goals, 10.7%

Atlantic: 237 goals, 9.7%
Northeast: 244 goals, 10.0%

Atlantic: 204 goals, 9.3%
Northeast: 215 goals, 9.6%

Atlantic: 221 goals, 10.1%
Northeast: 226 goals, 9.7%

Atlantic: 223 goals, 9.7%
Northeast: 224 goals, 9.5%

Atlantic: 239 goals, 10.1%
Northeast: 235 goals, 10.1%

Atlantic: 222 goals, 9.9%
Northeast: 228 goals, 10.0%

It is possible that if the Atlantic Division was stronger, its teams would have weaker numbers since they had to play more games inside the division. If this was true, the effect would have to take place in the extra games played against the Atlantic compared to the Northeast. In this period, the New Jersey Devils played 211 games against their own division and 185 games against the Northeast. Those extra divisional games represent just 4% of the total games played, so that suggests that the overall effect on goalie stats would be very slight even if there was a significant disparity between divisions.

There were a few seasons (1994-95, 1998-99, 1999-00, 2000-01) with a more balanced schedule where the Devils played an equal number of games against Atlantic teams and Northeast teams. In those 4 seasons, the average shooting percentage for each division was the same, and Buffalo's divisional opponents actually averaged slightly more goals per game than New Jersey's.

This evidence supports the view that divisional effects in the Eastern Conference in the mid- to late-1990s were minimal.

Wednesday, February 11, 2009

Divisional Effects on the Dominator

I received the suggestion lately that one of the reasons for the large save percentage discrepancy between Brodeur and Hasek was that there was stronger opposition in the Atlantic Division than in the Northeast. I decided to run the numbers for both goalies to check this out. I decided to look at games from 2000-01 and earlier for both goalies, as that was Hasek's last season in Buffalo.

Brodeur: .911 vs. division, .911 vs. rest of East, .916 vs. West
Hasek: .934 vs. division, .925 vs. rest of East, .925 vs. West

There wasn't much effect for Brodeur, but Hasek's result was surprising to me. The schedule was more balanced in the 1990s, so the divisional effects we see with the current NHL schedule shouldn't have been as much of a factor.

Yahoo Sports gives a breakdown of every goalies' record against all other teams (here's the link to Hasek's page). If we isolate the rest of the teams in Hasek's division, they don't look too out of the ordinary for him, with one clear exception:

Boston: .927
Montreal: .935
Ottawa: .950
Toronto: .928

In his career, Dominik Hasek has gone 24-9-5 with a 1.49 GAA and a .950 save percentage against the Ottawa Senators.

The type of Ottawa teams Hasek would have faced are very different, from the lousy expansion team of 1992-93 to the President's Trophy winners of 2002-03. Thanks to, shot and save results are available for every game against Ottawa. The Senators were a low-scoring team until 1998-99, when they won their division with 103 points. It makes the most sense to divide Ottawa games into two periods: the expansion team period from 1992-93 to 1997-98, and the dominant team period from 1998-99 to 2000-01.

From 1992-93 to 1997-98, Dominik Hasek was 17-4-1 against Ottawa, with a miniscule 1.33 GAA and a .955 save percentage. That is about what one would expect for an expansion team up against the greatest goalie of all time, and it supports the theory that Hasek fattened up on some weak teams.

Except that Hasek's success against Ottawa continued even when the Senators got good. Here are his results from 1998-99 to 2000-01: 4-3-4, 1.41, .957. His save percentage actually improved as the Senators did, although his team obviously provided almost no goal support. Hasek also dominated Ottawa in the playoffs, going 5-1, 1.55, .952 in the one-and-a-half series he played against the Sens.

Playing against the Senators was not much of an advantage for Hasek, because of the balanced schedule. Martin Brodeur played against Ottawa almost as many times in this period (28 games) as Hasek did (32). From 1993-94 to 1997-98, Brodeur was 12-3-2 against Ottawa with a 1.46 GAA and .937 save percentage. The difference is that Brodeur wasn't able to handle an improved Ottawa team as well as Hasek was. The Devils lost to the Senators in the 1998 playoffs, although Brodeur played pretty well (.927, 1.95), and Brodeur then went 4-5-2, 2.96, .893 against the Senators over the next three seasons.

I don't think playing in different divisions had much of an effect on the statistics of Brodeur and Hasek. The schedule back then was balanced so their strength of opposition was relatively similar. Both racked up wins and shutouts against the expansion Senators, the difference was that Hasek continued his domination even as Ottawa rose to be one of the best teams in the league.

Sometimes players have unusual success against a particular franchise, and that seems to be true in the case of Hasek vs. Ottawa. Hasek's repeated success against Ottawa is the main reason why he had such strong intra-division numbers in the 1990s. Little wonder then that the Senators went out and acquired Hasek in 2004. They had definitely "seen him good".

Monday, February 9, 2009

New Jersey Cowbell

A clever recurring reference among some Oiler and Flames blogs is to the "cowbell" of a particular fanbase or coach. You can read the history of this expression at the blog that originated it, Five Hole Fanatics, but in essence it refers to a specific skill or attribute that gets a disproportionate weighting in player evaluation (the name came from the famous SNL skit where Christopher Walken was obsessively focused on the sound of the cowbell).

The absence of Martin Brodeur is making it pretty obvious that New Jersey fans have a couple of cowbells in terms of evaluating goalie play. They are rebound control and puckhandling. Take a look at this HFBoards thread where Devils fan after Devils fan piles on Scott Clemmensen for his poor rebound control and his lack of Brodeur-esque puckhandling skills.

Apparently whether or not Clemmensen actually stops the puck is pretty unimportant, it is the style with which he does it and the way he handles it that is critical. I don't mean to downplay rebound control too much, since obviously poor rebounds create goals against, but the majority of rebounds are cleared away by the defence, and the majority of rebounds the other team does actually get to are stopped again by the goalie.

If Devils fans are correct in their assessment, then that strongly suggests Clemmensen has been better at making the first save than Brodeur. I don't have the exact comparison, but let's say for example that Brodeur gives up one rebound shot per game while Clemmensen gives up three, and that goals are scored on 25% of rebounds. That allows us to estimate the first shot save percentage for each of them:

Clemmensen: 25.6 1st SA/60, 3 reb/60, 2.36 GAA, .937 est. 1st shot sv%
Brodeur: 24.7 1st SA/60, 1 reb/60, 2.16 GAA, .923 est. 1st shot sv%

I think the other teams this season have been throwing a lot more perimeter shots on net against Clemmensen than they have against Brodeur, which might explain both the shot differential and why we could expect Clemmensen to have a strong first-shot save percentage.

This leads us to a hypothesis of goaltending play: Given how few rebound shots are actually taken in a typical hockey game (about 3 per game for both teams combined, according to one study), the effect of a goalie who is great at controlling rebounds might not just be allowing fewer second chance shots, but also facing fewer total shots by deterring the opposition from shooting in the first place. One of the main reasons to shoot from a bad angle or a long distance is to try to create a rebound, and if the shooters don't think they will get one then they are more likely to pass instead. The game score evidence that I have been working with lately shows that shots are to some degree discretionary (teams shoot substantially more often when trailing, for example), so I am starting to think that perhaps the main cause of the observed shot differentials between goalies may not be the direct effect of them giving up a lot of extra rebound shots or not being able to clear the zone, but because for whatever reason the other team thinks it is good strategy to put more or fewer pucks on net. Even if this is true is still could be argued that excellence in those types of goalie skills do effectively "prevent" or "create" shots against, but because the effect is indirect it would not be as easy to isolate it as it would be to, for example, count rebound shots.

Speaking of counting rebound shots, I am interested in seeing The Forechecker's rebound numbers for this season to see if perception matches reality in New Jersey. If we could isolate Brodeur's rebounds per game vs. Clemmensen's, that should be able to give us a sense of the direct impact of a goalie in terms of rebound control given the discrepancy between the two of them in that particular skill. Whether great rebound control can impact goal prevention indirectly through shot prevention is a topic that requires more study.

Wednesday, February 4, 2009

Playoff Performance

Hockey analyst James Benesh was kind enough to give me a complete copy of all playoff save percentages since 1952 recently, which allows me to update some of the playoff numbers that I began here as part of a larger look at save percentages.

One thing I didn't like about my previous method was comparing against the average playoff save percentage each season. This because the average will be influenced by the teams that advance deep into the playoffs. For example, the 1985 playoffs had an average save percentage of .882, in large part because the Oilers destroyed everyone. In 1986 the Oilers were upset, less high-scoring teams went to the Finals, and the average playoff save percentage rose to .895. For teams that went out in the first round and never had to play the Oilers or Habs, the context likely wasn't all that different, so it doesn't make tons of sense to me to adjust for it. Another factor is that a goalie who goes to the Stanley Cup Finals makes up quite a large part of the average. Goalies can face one-sixth or more of the total shots in an individual playoff season, so this means that an outstanding season is probably understated when compared to league average, since the goalie is at least in part being compared to himself.

My post about the advantage of goalies never having to face their own All-Star shooters suggested adjusting for each goalie's competition as well. I ended up making both an era adjustment and an opponent adjustment. The era adjustment was based on the average shooting percentage of all teams that qualified for the playoffs that season. The opponent adjustment was based on the regular season shooting percentage of all opponents faced in each playoff season for any given team.

A couple of other points: Since I need shot data for the adjustments, I am limited to the period from 1968-2008. Also, the adjustments are made from total number of games played by the team in the playoffs. For teams that platooned their goalies, the adjustment will not be exactly correct. It is not possible to be more accurate for many of these seasons as I have no way of finding out which goalies played in which game. I don't think it would have much of an impact over the course of a career, but if there was for example a case where one goalie played the first round against Winnipeg and another one played the second round against Edmonton, the adjustment would be a bit unfair.

Here are the results for goalies with at least 60 games played in the playoffs. ("LgAvg Sh%" is the shooting percentage of the average playoff team throughout the goalie's career to show the era they played in, while "Adj Opp Sh%" is the era-adjusted opponent shooting percentage for the opponents faced by the goalie's team in each of their playoff seasons).

RankGoalieLgAvg Sh%Adj Opp Sh%Career Sv%Adj Sv%
1.John Vanbiesbrouck11.5%12.3%.915.922
2.Dominik Hasek10.2%11.9%.925.919
3.Ken Dryden11.3%12.0%.915.918
4.Bernie Parent11.2%11.7%.917.917
5.Billy Smith12.7%11.8%.905.917
6.Patrick Roy11.1%11.5%.918.916
7.Ed Belfour10.5%11.5%.920.914
8.Kirk McLean11.4%12.0%.907.911
8.Martin Brodeur10.2%11.6%.919.911
8.Grant Fuhr12.0%11.8%.903.911
11.Mike Richter11.0%11.9%.909.909
11.Curtis Joseph10.4%11.4%.917.909
13.Chris Osgood10.1%12.1%.914.908
14.Tony Esposito11.3%12.0%.903.907
15.Felix Potvin11.1%11.3%.910.906
16.Tom Barrasso11.4%11.5%.902.903
17.Kelly Hrudey12.2%11.9%.891.902
18.Ron Hextall11.6%11.6%.897.900
18.Mike Vernon11.5%11.8%.896.900
18.Jon Casey11.5%11.9%.895.900
18.Gerry Cheevers10.8%11.8%.902.900
22.Andy Moog12.0%11.8%.890.899
23.Mike Liut12.8%11.2%.888.898
23.Don Beaupre12.3%11.8%.886.898
25.Pete Peeters12.7%11.6%.881.895
26.Ed Giacomin10.4%12.4%.895.894

This does not take into account quality of team defence. Some of these goalies no doubt had a much easier job than others. In fact, most of the top 10 probably had the advantage of playing on strong defensive teams. I would expect that Dryden, Parent, Smith, Roy, Belfour, and Brodeur faced easier than average shot quality against. Having said that they did finish in the top 10, ahead of some other guys who also played for strong teams, which implies their performances were likely above average.

Take Billy Smith, for instance. There is lots of evidence that the Islanders allowed easier than average shot quality against (the defensive talent on the team, the observation that Smith's playing partner usually posted better regular season stats, and the fact that Smith's main playing partner, Chico Resch, had a .919 adjusted playoff save percentage). However, even if Smith faced shots that were 10% easier than average, which is the typical high range of what we see in modern hockey during the regular season, he would still show up at .908 which is just outside of the top 10. The caveat is that if there was one goalie who had a shot quality adjustment far outside the norm I would expect it to be Smith, given that he played the large majority of his playoff games in a 5 year stretch when his excellent two-way team dominated the rest of the league in the playoffs.

Two of the biggest surprises are John Vanbiesbrouck and Kirk McLean. Each of them had one long playoff run that contributes to their strong numbers. It looks like Vanbiesbrouck also played well outside of Florida, he just kept running into better teams. Vanbiesbrouck's 12.3% adjusted opponent shooting percentage was the second highest on the list.

Near the bottom of the list are two Hall of Famers, Gerry Cheevers and Ed Giacomin. Since Cheevers was inducted on the grounds of his playoff performance in Boston, these results suggest that he was mostly carried by his teammates and was not a great goaltender. If you do a similar adjustment to shots against per game (i.e. adjust for era and opposition), Cheevers ranks #2 of all 26 goalies in fewest shots against. If you want to know who finished #1, the title of this blog will give you a clue.

Giacomin is another guy that I don't think is deserving of the Hall of Fame. He had a few good years in New York on a strong Rangers team, but after that his career fizzled and he was outplayed by his teammates more often than not. In the playoffs, Giacomin's record is very poor. The numbers show that he faced the strongest opposition of any of the goalies, but even taking that into account he finished dead last in save percentage.

I had a debate about Tony Esposito a while back where we ended up concluding that Esposito's playoff performance was probably subpar. These results suggest Esposito was actually about average, with a .907 adjusted save percentage. If Esposito was a choker, what does that make Ed Giacomin with his .894?

Looking at the goalies who played in less than 60 games, there were a couple of interesting results. Mike Palmateer did very well, with a .919 adjusted save percentage. Gilles Meloche, a goalie who was constantly limited by the abilities of his teammates, came in tied with Esposito at .907, which is a pretty decent result.

For current goalies, Kolzig (.922), Giguere (.916), Lalime (.913), Kiprusoff (.912), and Khabibulin (.911) have good records, Nabokov's is fairly ordinary (.905), and Theodore (.897) and Turco (.897) fared worse. As always, be sure to keep team factors in mind when interpreting those results.

Tuesday, February 3, 2009

Alone in the Universe, Apparently

From Yahoo Sports:

"5. New Jersey Devils (32-15-3, Previous: 7) – The Devils were hoping to play well enough to maintain a playoff spot when top goalie Martin Brodeur went down Nov. 1. But no one in the universe imagined they’d go 26-13-1 without the future Hall of Famer."

I didn't go on record with a prediction when Brodeur went down, and I wouldn't have gone as far as to forecast a .662 winning percentage in his absence. However, you can count me as someone who certainly imagined the possibility of the New Jersey Devils getting on just fine without their franchise goaltender.

Actually, from the results of a poll I put up on this blog at the time, I know there are a number of others who had a similar opinion. I think it is abundantly clear at this point that the New Jersey Devils are one of the top contenders in the Eastern Conference, and look to be a very strong team heading into this year's playoffs whether they have a healthy Brodeur or Scott Clemmensen in net.

Yahoo Sports also has another article up on the Devils' season so far. The key quote is probably from coach Brent Sutter: “I think the misconception was always outside, that this was a one-man team. We always felt we had a good hockey team, and we just carried on. Yes, Marty is an elite level player, an elite level goaltender, but he is just one part of it.”