It has been quite noticeable in the post-lockout NHL that the teams with playoff success have not generally been the teams with the best goaltending. That hasn't made much of a difference for many hockey fans, who continue to consider goalies with Cup rings to be the most clutch in the league. I'm not saying that goalies like Marc-Andre Fleury, Cam Ward, or Chris Osgood have played poorly in the playoffs. Not at all. However, the difference between them and a bunch of other guys is nothing more than the quality of their teammates.
There have been some goalies who have been very good in the regular season, but have not had the same level of recent playoff team success. Three of the best examples would be Roberto Luongo, Martin Brodeur, and Henrik Lundqvist. I decided to compare the numbers since the lockout for each of these three goalies with the supposedly clutch group of Fleury, Ward and Osgood, to see whether that might explain the discrepancy in their win/loss records. The numbers given are goal support per 60 minutes, shots against per 60 minutes, and save percentage:
Osgood: 3.12, 24.9, .928
Fleury: 2.95, 29.1, .916
Ward: 2.61, 28.7, .917
Luongo: 2.06, 29.9, .930
Lundqvist: 2.43, 28.8, .907
Brodeur: 2.56, 30.1, .917
We can use these numbers to calculate the playoff win threshold for each goalie, that is the save percentage they would need to record for their team to score as many goals as they allowed. Not surprisingly, the top three on the list are the guys who haven't been winning, and the bottom three are the guys who have.
1. Luongo .931
2. Lundqvist .916
3. Brodeur .915
4. Ward .909
5. Fleury .899
6. Osgood .875
Chris Osgood's advantage in Detroit is downright unfair. Marc-Andre Fleury is a fine young goalie, but there are not many other teams that would have won the Cup with a .908 save percentage from their starter. On the other hand, Martin Brodeur is not getting much help from his teammates, and somewhat amazingly Roberto Luongo's team has been outscored in the playoffs despite his .930 save percentage.
If I was about to play game 7 of the Stanley Cup Finals, I'd take any of Luongo, Lundqvist or Brodeur ahead of Ward, Fleury or Osgood. Team records don't matter, only the quality of the individual goaltender.
Thursday, October 29, 2009
Tuesday, October 27, 2009
The Lockout
It seems intuitive that both an excessive workload and an excessive amount of time off would be negative factors for a goaltender. The 2004-05 NHL lockout is an interesting case of the latter. Many of the goalies ended up playing somewhere that season, of course, but few of them played the same number of games that they would have if they had remained with their NHL clubs. There were also some significant changes to the game when they returned, including smaller goalie equipment, a stricter penalty standard, and rules that created a more open game.
Generally goalies have fairly consistent save percentage results, especially when taking team and situational factors into account. There were a number of goalies that had substantially different results after the lockout compared to how they did before, more of them than we likely would expect just to occur at random. Here are some of the most extreme results, with their even-strength save percentages from 1999-2004 compared to 2006-2009:
David Aebischer: .932 before, .908 after
Andrew Raycroft: .932 before, .905 after
Patrick Lalime: .918 before, .900 after
Marty Turco: .931 before, .916 after
Jose Theodore: .922 before, .909 after
Martin Brodeur: .918 before, .926 after
Tomas Vokoun: .919 before, .933 after
Cristobal Huet: .909 before, .926 after
Some goalies seemed to fall off a cliff after the lockout, while others got significantly better. Is there anything that might explain this result?
If an extended layoff has a negative impact, then the number of games played during the lockout is likely to be the significant variable. I decided to compare goalies who played during the lockout with goalies who decided to sit it out. I decided to look only at goalies who had a significant amount of playing time both before and after the lockout, and who were between 26 and 32 years old during the lockout season. This avoids picking out goalies like Sean Burke and Curtis Joseph, who did much worse after the lockout, or goalies like Rick DiPietro or Marc-Andre Fleury, who did much better after the lockout, since all of them obviously had age as a big factor.
I came up with a sample of 26 goalies that met the criteria. I then broke them down by guys who played during the lockout vs. guys who did not:
Played: .917 before, .909 after, -.008
Did not: .922 before, .916 after, -.006
The two groups had almost the exact same average age, so that wasn't a factor. This suggests that sitting out did not have much of an effect.
Some of the goalies played only a handful of games before getting injured or returning home. There was likely little difference between doing that and not playing at all, so I divided them up by goalies who played 15 games or more vs. goalies who played fewer than 15 games:
15 games or more: .919 before, .912 after, -.007
Fewer than 15 GP: .921 before, .916 after, -.005
Again, only a slight difference.
I also took a look at whether the younger guys were affected more than the older guys.
Age 26-28 during the lockout:
Played: .921 before, .918 after, -.003
Did not: .918 before, .910 after, -.008
Age 29-32 during the lockout:
Played: .916 before, .909 after, -.007
Did not: .922 before, .915 after, -.007
We're getting into some smaller sample sizes in the last one, but the results suggest the interesting idea that sitting out a season impacts a veteran less than it does a guy still in his prime. Every single goalie 30 years old or younger during the lockout who did not play that season did worse after the lockout than they did before. The entire list goes as follows: Biron (-.005), Denis (-.010), Esche (-.017), Grahame (-.009), Johnson (-.001), Lalime (-.018), Thibault (-.016), Weekes (-.014). Having said that, some slight decline would be expected for this group because of age factors.
When these results are combined with one of my earlier posts that showed that October is usually the worst month for goalies, I think there is some evidence to suggest that extended layoffs have a slight negative effect on goalie performance.
I'm still not sure why some of goalies were much improved after the lockout while others fell off a cliff. Cristobal Huet looks like the classic late-bloomer who finally got his shot, but both Brodeur and Vokoun were veteran NHL goalies in 2005 who went from good in the early '00s to top 5 guys post-lockout. It seems unlikely that we can point to their teams as a major factor, as both were playing on the same team as before. Nashville might have improved a bit defensively, but New Jersey got worse.
On the other side, Andrew Raycroft had some strong junior numbers, some nice age 21 and 22 AHL results (.916 and .917), and then a terrific Calder year with a .940 EV SV%. He looked set for a promising career, but then he went and played 11 games in Finland during the lockout and was never the same goalie again.
I wonder whether playing for a different team in a different country had an effect on the guys like Raycroft who played overseas during the lockout and seemed to have lost something from their games when the NHL resumed. One theory is that whoever was the goalie coach for the Swedish team Djurgardens in 2004-05 was not very good at his job, considering he had Jose Theodore and Marty Turco pass through that season, both of whom went from pre-lockout stars to post-lockout mediocrity. I doubt that really had much to do with anything, but that's at least an interesting coincidence.
Another theory is that the equipment reduction had something to do with it, but that doesn't really seem to fit the results. There were some athletic, reaction-type goalies who nosedived, like Turco, and some butterfly blockers who got better, like Giguere and Huet.
I'm not really sure what was going on, if anything. I would guess that some of the goalies who struggled post-lockout were guys who didn't stay in shape, but that doesn't apply to all of them (Martin Gerber and Jussi Markkanen were two guys with pretty decent pre-lockout numbers who played 50+ games in the lockout season, and saw their numbers drop substantially in the new NHL). It looks like there are too many variables in play that we can't conclude much at the macro level, other than to say that if there is another labour stoppage in 2011-12 I would likely advise goalies that they are probably a bit better off playing somewhere rather than sitting at home, and that they might want to be a bit wary of what the Swedish coaches tell them.
Generally goalies have fairly consistent save percentage results, especially when taking team and situational factors into account. There were a number of goalies that had substantially different results after the lockout compared to how they did before, more of them than we likely would expect just to occur at random. Here are some of the most extreme results, with their even-strength save percentages from 1999-2004 compared to 2006-2009:
David Aebischer: .932 before, .908 after
Andrew Raycroft: .932 before, .905 after
Patrick Lalime: .918 before, .900 after
Marty Turco: .931 before, .916 after
Jose Theodore: .922 before, .909 after
Martin Brodeur: .918 before, .926 after
Tomas Vokoun: .919 before, .933 after
Cristobal Huet: .909 before, .926 after
Some goalies seemed to fall off a cliff after the lockout, while others got significantly better. Is there anything that might explain this result?
If an extended layoff has a negative impact, then the number of games played during the lockout is likely to be the significant variable. I decided to compare goalies who played during the lockout with goalies who decided to sit it out. I decided to look only at goalies who had a significant amount of playing time both before and after the lockout, and who were between 26 and 32 years old during the lockout season. This avoids picking out goalies like Sean Burke and Curtis Joseph, who did much worse after the lockout, or goalies like Rick DiPietro or Marc-Andre Fleury, who did much better after the lockout, since all of them obviously had age as a big factor.
I came up with a sample of 26 goalies that met the criteria. I then broke them down by guys who played during the lockout vs. guys who did not:
Played: .917 before, .909 after, -.008
Did not: .922 before, .916 after, -.006
The two groups had almost the exact same average age, so that wasn't a factor. This suggests that sitting out did not have much of an effect.
Some of the goalies played only a handful of games before getting injured or returning home. There was likely little difference between doing that and not playing at all, so I divided them up by goalies who played 15 games or more vs. goalies who played fewer than 15 games:
15 games or more: .919 before, .912 after, -.007
Fewer than 15 GP: .921 before, .916 after, -.005
Again, only a slight difference.
I also took a look at whether the younger guys were affected more than the older guys.
Age 26-28 during the lockout:
Played: .921 before, .918 after, -.003
Did not: .918 before, .910 after, -.008
Age 29-32 during the lockout:
Played: .916 before, .909 after, -.007
Did not: .922 before, .915 after, -.007
We're getting into some smaller sample sizes in the last one, but the results suggest the interesting idea that sitting out a season impacts a veteran less than it does a guy still in his prime. Every single goalie 30 years old or younger during the lockout who did not play that season did worse after the lockout than they did before. The entire list goes as follows: Biron (-.005), Denis (-.010), Esche (-.017), Grahame (-.009), Johnson (-.001), Lalime (-.018), Thibault (-.016), Weekes (-.014). Having said that, some slight decline would be expected for this group because of age factors.
When these results are combined with one of my earlier posts that showed that October is usually the worst month for goalies, I think there is some evidence to suggest that extended layoffs have a slight negative effect on goalie performance.
I'm still not sure why some of goalies were much improved after the lockout while others fell off a cliff. Cristobal Huet looks like the classic late-bloomer who finally got his shot, but both Brodeur and Vokoun were veteran NHL goalies in 2005 who went from good in the early '00s to top 5 guys post-lockout. It seems unlikely that we can point to their teams as a major factor, as both were playing on the same team as before. Nashville might have improved a bit defensively, but New Jersey got worse.
On the other side, Andrew Raycroft had some strong junior numbers, some nice age 21 and 22 AHL results (.916 and .917), and then a terrific Calder year with a .940 EV SV%. He looked set for a promising career, but then he went and played 11 games in Finland during the lockout and was never the same goalie again.
I wonder whether playing for a different team in a different country had an effect on the guys like Raycroft who played overseas during the lockout and seemed to have lost something from their games when the NHL resumed. One theory is that whoever was the goalie coach for the Swedish team Djurgardens in 2004-05 was not very good at his job, considering he had Jose Theodore and Marty Turco pass through that season, both of whom went from pre-lockout stars to post-lockout mediocrity. I doubt that really had much to do with anything, but that's at least an interesting coincidence.
Another theory is that the equipment reduction had something to do with it, but that doesn't really seem to fit the results. There were some athletic, reaction-type goalies who nosedived, like Turco, and some butterfly blockers who got better, like Giguere and Huet.
I'm not really sure what was going on, if anything. I would guess that some of the goalies who struggled post-lockout were guys who didn't stay in shape, but that doesn't apply to all of them (Martin Gerber and Jussi Markkanen were two guys with pretty decent pre-lockout numbers who played 50+ games in the lockout season, and saw their numbers drop substantially in the new NHL). It looks like there are too many variables in play that we can't conclude much at the macro level, other than to say that if there is another labour stoppage in 2011-12 I would likely advise goalies that they are probably a bit better off playing somewhere rather than sitting at home, and that they might want to be a bit wary of what the Swedish coaches tell them.
Monday, October 26, 2009
Cause or Effect?
Kurt Overhardt is Ryan Kesler's agent. Kesler will be a restricted free agent at the end of this season, so Overhardt is working on getting him signed to a fat new extension. And either Overhardt is trying to twist the numbers to make his client look good, or he is simply mixing up cause and effect.
As the Vancouver Province reports based on Overhardt's research, the Vancouver Canucks had a better record last year in games where Ryan Kesler had more ice time than either Daniel or Henrik Sedin. That seems surprising on first glance, considering that the Sedins scored a lot more points than Kesler did and also had much better plus/minus numbers (Daniel +24, Henrik +22, Kesler +8). But when you think about it a bit deeper it seems obvious to me that this has more to do with the relative roles of Kesler and the Sedins than their effectiveness as hockey players.
If Vancouver is winning, who is more likely to be on the ice, their best offensive players or a Selke-nominee who is one of the league's best defensive forwards? Similarly, if the Canucks are behind, is the coach going to turn to Kesler, with his career high of 59 points in a season, or to the Sedins, who have each averaged nearly a point per game since the lockout?
In short, I doubt the Canucks win because Kesler plays a lot, it's more likely that Kesler plays a lot when the Canucks are winning. Or perhaps more precisely, since Kesler usually played more than the Sedins did by virtue of pulling more special teams duty, the Sedins weren't as likely to play big minutes except when the team was losing. Either way, it means that attributing the team's record to Kesler is a big stretch.
Good try by the agent, though. However, Mike Gillis seems like a sharp GM to me so I'm not sure he'll buy that line of argument, even though he no doubt sees the value of a great young two-way player like Kesler and will probably end up finding the necessary resources to eventually get the deal done.
As the Vancouver Province reports based on Overhardt's research, the Vancouver Canucks had a better record last year in games where Ryan Kesler had more ice time than either Daniel or Henrik Sedin. That seems surprising on first glance, considering that the Sedins scored a lot more points than Kesler did and also had much better plus/minus numbers (Daniel +24, Henrik +22, Kesler +8). But when you think about it a bit deeper it seems obvious to me that this has more to do with the relative roles of Kesler and the Sedins than their effectiveness as hockey players.
If Vancouver is winning, who is more likely to be on the ice, their best offensive players or a Selke-nominee who is one of the league's best defensive forwards? Similarly, if the Canucks are behind, is the coach going to turn to Kesler, with his career high of 59 points in a season, or to the Sedins, who have each averaged nearly a point per game since the lockout?
In short, I doubt the Canucks win because Kesler plays a lot, it's more likely that Kesler plays a lot when the Canucks are winning. Or perhaps more precisely, since Kesler usually played more than the Sedins did by virtue of pulling more special teams duty, the Sedins weren't as likely to play big minutes except when the team was losing. Either way, it means that attributing the team's record to Kesler is a big stretch.
Good try by the agent, though. However, Mike Gillis seems like a sharp GM to me so I'm not sure he'll buy that line of argument, even though he no doubt sees the value of a great young two-way player like Kesler and will probably end up finding the necessary resources to eventually get the deal done.
Friday, October 23, 2009
Are Shootouts An Indicator of Skill?
With four full seasons of shootout results in the books, we can start to make sense of the results. I wanted to check if better goalies tended to have better shootout results. I calculated the correlation between shootout save percentage and overall save percentage for every goalie who faced at least 50 shootouts against.
Correlation: 0.004
That's about as close to zero as you are going to find in a real life sample. What that suggests is that shootout performance tells you nothing at all about how good a goalie is.
This is reinforced by looking at the leaderboard. Among the goalies who have faced 50+ shootouts in their careers heading into this season, the top 3 were Johan Hedberg (.820), Mathieu Garon (.812), and Jose Theodore (.790). Henrik Lundqvist, Rick DiPietro, Kari Lehtonen and Tim Thomas were all good goalies who had good shootout results as well (all .740 or better). Roberto Luongo (.716) and Martin Brodeur (.715) were both above average, but not by a lot, while Tomas Vokoun was right about at average (.670). Goalies who were below average at stopping shootouts included J.S. Giguere (.652), Cristobal Huet (.604), Miikka Kiprusoff (.600), Ilya Bryzgalov (.597), Niklas Backstrom (.568), and Evgeni Nabokov (.568). The worst goalie was Vesa Toskala (.512).
Conclusion: Shootout skill is distinct from overall goalie skill. Shootout results don't provide much evidence of a goalie's overall abilities. All they do is measure how good a goalie is at stopping breakaways.
For this reason, I don't consider shootout performance at all when evaluating goalies. Other analysts do, and I understand the reasons for it. All things being equal it is better to have somebody who is good at stopping shootouts in the current NHL where shootout success directly leads to standings points. If I was a GM then I'd probably take it into account. For predicting future team success it also makes sense to include shootout skill in your prediction. I just see shootouts as a sideshow that is separate from the actual game, a rare game situation that has little meaning in an overall sense and that is based on a distinct skill set. My objective is to identify the best and the worst goalies at playing hockey, i.e. the game in its most common state with 4 or 5 skaters on each side. Shootouts don't give us much useful information to that end, so to me they don't matter.
Correlation: 0.004
That's about as close to zero as you are going to find in a real life sample. What that suggests is that shootout performance tells you nothing at all about how good a goalie is.
This is reinforced by looking at the leaderboard. Among the goalies who have faced 50+ shootouts in their careers heading into this season, the top 3 were Johan Hedberg (.820), Mathieu Garon (.812), and Jose Theodore (.790). Henrik Lundqvist, Rick DiPietro, Kari Lehtonen and Tim Thomas were all good goalies who had good shootout results as well (all .740 or better). Roberto Luongo (.716) and Martin Brodeur (.715) were both above average, but not by a lot, while Tomas Vokoun was right about at average (.670). Goalies who were below average at stopping shootouts included J.S. Giguere (.652), Cristobal Huet (.604), Miikka Kiprusoff (.600), Ilya Bryzgalov (.597), Niklas Backstrom (.568), and Evgeni Nabokov (.568). The worst goalie was Vesa Toskala (.512).
Conclusion: Shootout skill is distinct from overall goalie skill. Shootout results don't provide much evidence of a goalie's overall abilities. All they do is measure how good a goalie is at stopping breakaways.
For this reason, I don't consider shootout performance at all when evaluating goalies. Other analysts do, and I understand the reasons for it. All things being equal it is better to have somebody who is good at stopping shootouts in the current NHL where shootout success directly leads to standings points. If I was a GM then I'd probably take it into account. For predicting future team success it also makes sense to include shootout skill in your prediction. I just see shootouts as a sideshow that is separate from the actual game, a rare game situation that has little meaning in an overall sense and that is based on a distinct skill set. My objective is to identify the best and the worst goalies at playing hockey, i.e. the game in its most common state with 4 or 5 skaters on each side. Shootouts don't give us much useful information to that end, so to me they don't matter.
Wednesday, October 21, 2009
Win Thresholds for Cup Finalists
I tested my win threshold stat out on the recent Stanley Cup Finalists, to see which teams seemed to rely the most and the least on goaltending (numbers are adjusted for scoring level):
1998 Detroit .883, Washington .906
1999 Dallas .879, Buffalo .915
2000 New Jersey .884, Dallas .906
2001 Colorado .874, New Jersey .860
2002 Detroit .883, Carolina .901
2003 New Jersey .886, Anaheim .913
2004 Tampa Bay .877, Calgary .900
2006 Carolina .892, Edmonton .887
2007 Anaheim .889, Ottawa .887
2008 Detroit .867, Pittsburgh .903
2009 Pittsburgh .895, Detroit .871
Not surprisingly, all of the recent Stanley Cup Champions have low win thresholds. They would all have been good teams even with subpar goaltending. That doesn't mean they would have won the Cup anyway with a mediocre goalie. In that case most of them likely would not have won, although one could probably make an argument for the two teams that employed Chris Osgood. The average team from 1997-98 to 2008-09 had a win threshold of .904. The average Cup champs had a win threshold of .883. Clearly, winning the Cup is a team effort.
Most of the Cup finalists also have good numbers, including some of the surprise Finalists. Only two of the teams had a number that was well above average. Those two were the two teams that relied the most on goaltending to get where they ended up, the 1999 Buffalo Sabres and the 2003 Anaheim Mighty Ducks.
This suggests that teams that strongly outplay the opposition are generally better Cup candidates than teams that have top goalies but do not excel at scoring or shot prevention. We can look at teams like Chicago and Washington and wonder about their goaltending, but that is the type of team that has won the Cup recently while teams like Vancouver and New Jersey have not. Percentages can play a big role in a short playoff series, but on the other hand a goalie can only do so much. At some point his teammates will have to pick up some of the slack if they want to end up winning a championship.
1998 Detroit .883, Washington .906
1999 Dallas .879, Buffalo .915
2000 New Jersey .884, Dallas .906
2001 Colorado .874, New Jersey .860
2002 Detroit .883, Carolina .901
2003 New Jersey .886, Anaheim .913
2004 Tampa Bay .877, Calgary .900
2006 Carolina .892, Edmonton .887
2007 Anaheim .889, Ottawa .887
2008 Detroit .867, Pittsburgh .903
2009 Pittsburgh .895, Detroit .871
Not surprisingly, all of the recent Stanley Cup Champions have low win thresholds. They would all have been good teams even with subpar goaltending. That doesn't mean they would have won the Cup anyway with a mediocre goalie. In that case most of them likely would not have won, although one could probably make an argument for the two teams that employed Chris Osgood. The average team from 1997-98 to 2008-09 had a win threshold of .904. The average Cup champs had a win threshold of .883. Clearly, winning the Cup is a team effort.
Most of the Cup finalists also have good numbers, including some of the surprise Finalists. Only two of the teams had a number that was well above average. Those two were the two teams that relied the most on goaltending to get where they ended up, the 1999 Buffalo Sabres and the 2003 Anaheim Mighty Ducks.
This suggests that teams that strongly outplay the opposition are generally better Cup candidates than teams that have top goalies but do not excel at scoring or shot prevention. We can look at teams like Chicago and Washington and wonder about their goaltending, but that is the type of team that has won the Cup recently while teams like Vancouver and New Jersey have not. Percentages can play a big role in a short playoff series, but on the other hand a goalie can only do so much. At some point his teammates will have to pick up some of the slack if they want to end up winning a championship.
Saturday, October 17, 2009
Close Games, Part 2
After getting some feedback on my post looking into New Jersey's record in close games, I decided to do a bit more research. It turns out that the reason for their success was mainly because they did well in overtime and in the shootout. Here are the top 5 teams in regulation one-goal victories and regulation one-goal winning percentages since the lockout:
Regulation one-goal wins:
1. Calgary, 61
2. San Jose, 54
3. New Jersey, 53
4. Detroit, 52
5. Carolina, 51
Regulation one-goal winning percentage:
1. Carolina, .585
2. Detroit, .578
3. Calgary, .573
4. New Jersey, .572
5. San Jose, .567
By these numbers New Jersey is still good at winning close games, but they are not head and shoulders above the rest of the league. Here are the numbers for overtime and shootout wins:
Wins in overtime and shootout combined:
1. New Jersey, 56
2. Atlanta, 50
3. Dallas, 49
4. N.Y. Rangers, 48
5. Buffalo, 45
Winning percentage in games that go into OT:
1. New Jersey, .659
2. Dallas, .613
3. Atlanta, .602
4. Buffalo, .570
5. Colorado, .569
New Jersey was by far the best team in the league in games tied after 60 minutes. In this light, the team's recent playoff performances perhaps don't seem as disappointing. Their regular season records were largely influenced by their ability to perform well in 4 on 4 overtime and in shootouts. Unfortunately for them, the Devils weren't able to take advantage of those situations in the playoffs.
Another variable brought up by someone in the comments was empty net goals. This was indeed a factor that helped boost the Devils' number of one goal wins, since New Jersey has been one of the worst teams in the league at scoring empty net goals since the lockout. New Jersey scored 19 times with the other goalie pulled, which was tied for the second-lowest total in the league behind only the weak Toronto Maple Leafs, a team that faced many fewer empty net chances than the Devils. Assuming they never scored two empty netters in any one game, New Jersey scored an empty net goal in 14% of their regulation wins, the second worst percentage in the league behind only San Jose's 13%.
The Devils allowed 25 empty netters against, or an ENG against in 23% of their regulation losses, which ranked them slightly worse than the league average of 22%.
I am not sure how many of the empty netters came when leading/trailing by one goal and how many came when there was a two goal margin on the scoreboard. I decided to assume that empty net goals scored came with a one and two goal lead came in the same proportion as the team's number of one and two goal wins, e.g. a team with the same number of one goal and two goal wins would score half of their empty netters in each situation. It is likely a few teams would by chance have a very different ratio, but that probably puts most teams in the ballpark. Combined with the OT/shootout numbers, that allows us to estimate a team's regulation-only one-goal game record with empty-netters removed.
I'll refer to any game that goes to overtime or is decided in regulation by a one goal margin (empty netters excluded) as a close game. Here are the close game records for all teams since the lockout (not including 2009-10), along with their close game points percentage (games tied after regulation count as 1 point), the team's winning percentage in games decided by 2 goals or more, the total points earned in overtime and shootouts, and the percentage of close games that went to OT.
A few teams have interesting profiles here. Carolina and Calgary are teams that do much better in close games, and have tended to win the close ones in regulation. Over the last two seasons, both teams have seen both their shot ratio and percentages improve in the third period, so perhaps it could be argued that these teams have shown some clutch ability. In contrast, Phoenix and Ottawa also don't make it overtime that often, but they tend to lose the close ones and would be better off in the standings if they could hold on a bit longer to earn a few more loser points. The Coyotes and Senators both saw their third period percentages tumble over the last two years. I'm not sure whether that is a sign of poor performance late in games or simply bad luck that led to losses.
In this table New Jersey doesn't look much different from other teams, other than their league-leading total of 141 overtime and shootout points. Their rivals the New York Rangers were only one point behind. The Rangers were not as good at picking up the extra point, but they took a lot of loser points since they played more overtime games than any other team. That suggests the Rangers have made aiming for shootouts part of their team strategy. However, it looks to me like that strategy might have been suboptimal for them. Either that or the Rangers did a poor job of carrying it out, because a team that went to OT less often but won more games in regulation would have ended up with more points at the end of the day.
The Rangers took 74 points from one goal wins, 92 points from making it to overtime, and 48 points for winning by OT or shootout for a total of 214 points from close games. Given the same number of close games a typical team would have gone to overtime only 76 times, but would have won half the remaining games for a total of 52 regulation wins. That means they would only need to win 35 out of their 76 overtime games (46%) to earn more points than the Rangers did.
The Rangers had a pretty good record in games decided by 2 goals or more. They also had the third best winning percentage in the league when trailing after 2 periods, yet had a slightly below average winning percentage when leading after 2 periods. All that tends to reinforce the theory that the team would have been better off going for more wins in regulation rather than sitting back on a lead or trying to take a tie game into overtime, because it looks like too often they saw that strategy backfire.
Regulation one-goal wins:
1. Calgary, 61
2. San Jose, 54
3. New Jersey, 53
4. Detroit, 52
5. Carolina, 51
Regulation one-goal winning percentage:
1. Carolina, .585
2. Detroit, .578
3. Calgary, .573
4. New Jersey, .572
5. San Jose, .567
By these numbers New Jersey is still good at winning close games, but they are not head and shoulders above the rest of the league. Here are the numbers for overtime and shootout wins:
Wins in overtime and shootout combined:
1. New Jersey, 56
2. Atlanta, 50
3. Dallas, 49
4. N.Y. Rangers, 48
5. Buffalo, 45
Winning percentage in games that go into OT:
1. New Jersey, .659
2. Dallas, .613
3. Atlanta, .602
4. Buffalo, .570
5. Colorado, .569
New Jersey was by far the best team in the league in games tied after 60 minutes. In this light, the team's recent playoff performances perhaps don't seem as disappointing. Their regular season records were largely influenced by their ability to perform well in 4 on 4 overtime and in shootouts. Unfortunately for them, the Devils weren't able to take advantage of those situations in the playoffs.
Another variable brought up by someone in the comments was empty net goals. This was indeed a factor that helped boost the Devils' number of one goal wins, since New Jersey has been one of the worst teams in the league at scoring empty net goals since the lockout. New Jersey scored 19 times with the other goalie pulled, which was tied for the second-lowest total in the league behind only the weak Toronto Maple Leafs, a team that faced many fewer empty net chances than the Devils. Assuming they never scored two empty netters in any one game, New Jersey scored an empty net goal in 14% of their regulation wins, the second worst percentage in the league behind only San Jose's 13%.
The Devils allowed 25 empty netters against, or an ENG against in 23% of their regulation losses, which ranked them slightly worse than the league average of 22%.
I am not sure how many of the empty netters came when leading/trailing by one goal and how many came when there was a two goal margin on the scoreboard. I decided to assume that empty net goals scored came with a one and two goal lead came in the same proportion as the team's number of one and two goal wins, e.g. a team with the same number of one goal and two goal wins would score half of their empty netters in each situation. It is likely a few teams would by chance have a very different ratio, but that probably puts most teams in the ballpark. Combined with the OT/shootout numbers, that allows us to estimate a team's regulation-only one-goal game record with empty-netters removed.
I'll refer to any game that goes to overtime or is decided in regulation by a one goal margin (empty netters excluded) as a close game. Here are the close game records for all teams since the lockout (not including 2009-10), along with their close game points percentage (games tied after regulation count as 1 point), the team's winning percentage in games decided by 2 goals or more, the total points earned in overtime and shootouts, and the percentage of close games that went to OT.
| Rank | Team | Close W-L-OT | Close W% | 2G+ W% | OT/SO | % OT |
| 1. | Detroit | 71-37-66 | .598 | .740 | 94 | 38% |
| 2. | Calgary | 82-50-58 | .584 | .536 | 79 | 31% |
| 3. | Carolina | 70-43-64 | .576 | .497 | 99 | 36% |
| 4. | San Jose | 65-42-69 | .565 | .658 | 101 | 39% |
| 5. | New Jersey | 65-42-85 | .560 | .522 | 141 | 44% |
| 6. | Nashville | 64-46-71 | .550 | .537 | 109 | 39% |
| 7. | Philadelphia | 59-50-76 | .524 | .441 | 107 | 41% |
| 8. | Montreal | 51-43-74 | .524 | .519 | 112 | 44% |
| 9. | Anaheim | 61-53-80 | .521 | .597 | 119 | 41% |
| 10. | Vancouver | 58-54-78 | .511 | .540 | 121 | 41% |
| 11. | Buffalo | 46-44-79 | .506 | .591 | 124 | 47% |
| 12. | Florida | 53-54-80 | .497 | .461 | 113 | 43% |
| 13. | Toronto | 46-47-79 | .497 | .442 | 115 | 46% |
| 14. | Minnesota | 57-59-75 | .495 | .533 | 115 | 39% |
| 15. | Dallas | 50-52-80 | .495 | .582 | 129 | 44% |
| 16. | Phoenix | 56-58-54 | .494 | .358 | 84 | 32% |
| 17. | Boston | 49-54-79 | .486 | .507 | 114 | 43% |
| 18. | Columbus | 49-54-72 | .486 | .359 | 111 | 41% |
| 19. | Ottawa | 44-49-58 | .483 | .644 | 79 | 38% |
| 20. | Edmonton | 50-57-78 | .481 | .413 | 121 | 42% |
| 21. | Colorado | 56-66-65 | .473 | .496 | 102 | 35% |
| 22. | Pittsburgh | 40-50-83 | .471 | .516 | 124 | 48% |
| 23. | Washington | 44-56-77 | .466 | .470 | 112 | 44% |
| 24. | Rangers | 37-51-92 | .461 | .581 | 140 | 51% |
| 25. | St. Louis | 44-59-82 | .459 | .378 | 113 | 44% |
| 26. | Tampa Bay | 46-62-74 | .456 | .414 | 110 | 41% |
| 27. | Atlanta | 35-52-83 | .450 | .430 | 133 | 49% |
| 28. | Islanders | 42-60-74 | .449 | .375 | 112 | 42% |
| 29. | Los Angeles | 46-65-68 | .447 | .389 | 99 | 38% |
| 30. | Chicago | 39-60-79 | .441 | .447 | 116 | 44% |
A few teams have interesting profiles here. Carolina and Calgary are teams that do much better in close games, and have tended to win the close ones in regulation. Over the last two seasons, both teams have seen both their shot ratio and percentages improve in the third period, so perhaps it could be argued that these teams have shown some clutch ability. In contrast, Phoenix and Ottawa also don't make it overtime that often, but they tend to lose the close ones and would be better off in the standings if they could hold on a bit longer to earn a few more loser points. The Coyotes and Senators both saw their third period percentages tumble over the last two years. I'm not sure whether that is a sign of poor performance late in games or simply bad luck that led to losses.
In this table New Jersey doesn't look much different from other teams, other than their league-leading total of 141 overtime and shootout points. Their rivals the New York Rangers were only one point behind. The Rangers were not as good at picking up the extra point, but they took a lot of loser points since they played more overtime games than any other team. That suggests the Rangers have made aiming for shootouts part of their team strategy. However, it looks to me like that strategy might have been suboptimal for them. Either that or the Rangers did a poor job of carrying it out, because a team that went to OT less often but won more games in regulation would have ended up with more points at the end of the day.
The Rangers took 74 points from one goal wins, 92 points from making it to overtime, and 48 points for winning by OT or shootout for a total of 214 points from close games. Given the same number of close games a typical team would have gone to overtime only 76 times, but would have won half the remaining games for a total of 52 regulation wins. That means they would only need to win 35 out of their 76 overtime games (46%) to earn more points than the Rangers did.
The Rangers had a pretty good record in games decided by 2 goals or more. They also had the third best winning percentage in the league when trailing after 2 periods, yet had a slightly below average winning percentage when leading after 2 periods. All that tends to reinforce the theory that the team would have been better off going for more wins in regulation rather than sitting back on a lead or trying to take a tie game into overtime, because it looks like too often they saw that strategy backfire.
Thursday, October 15, 2009
Close Games
It is rare that professional teams can outperform their peers in a particular area for a long time. If a team has a strategic advantage, the rest of the league will study them and adjust their coaching strategy to compensate. If a specific talent or skillset is undervalued, a clever GM might be able to gain a short-term advantage, but again if the other teams are following along the market should correct the valuation.
As a result, when I run across a team that is a substantial outlier in any area then I take notice, and when that outlying result comes in an area that I mostly attribute to luck then it is especially interesting.
Here are the top 5 teams in games won by a one-goal margin since the lockout (not including this season so far):
1. New Jersey, 109
2. Dallas, 89
3. Vancouver, 87
3. Nashville, 87
5. San Jose, 86
5. Carolina, 86
5. Anaheim, 86
And here are the top 5 teams in winning percentage in games decided by one goal:
1. New Jersey, .740
2. Carolina, .709
3. Nashville, .690
4. Detroit, .673
5. San Jose. 670
The Devils are at the top of both of those lists, and it's not even close. That begs the question, just what are they doing differently than everyone else in the league?
The simple answer, and the one that 90% of journalists would probably respond with, is that the Devils have Martin Brodeur in net. There is, however, one fairly significant piece of evidence that suggests there is more to the story than that, namely New Jersey's record in one-goal games in 2008-09. Despite losing Brodeur to injury for almost 4 months, the Devils posted the best close game record they have ever had, going a remarkable 25-5-4 in one-goal games. Compared to an average team, New Jersey picked up an extra 12 points by winning the nailbiters, which made the difference between them winning their division and finishing as the #7 seed.
I looked at New Jersey's record since the lockout when leading, trailing, and tied after 2 periods. If they were a particularly clutch team, we would expect them to have a lot of wins in games that were tied heading into the third. The Devils did do well in that situation with a .644 winning percentage, good for 6th best in the league since the lockout. However, the Devils actually played a relatively low number of games that were tied after two, which was somewhat surprising for a low-scoring team. The team won a total of 36 games where they were tied after 2 periods, which was right about the average number (the Devils ranked 16th in the league).
New Jersey was also pretty good at coming from behind. Their winning percentage of .250 ranked 6th in the league, well above the average of .200. The Devils won 26 games that they trailed after 2 periods, which was tied for 4th in the league. It is likely that many of those wins would have been one-goal wins, although the average team won 20 so this would only account for part of their close game success.
By far most of New Jersey's wins came in games they were already leading after two periods. The Devils went 130-6-7 when ahead after two, for a .934 winning percentage that was the league's best. Only Detroit, San Jose and Ottawa converted a higher number of second intermission advantages into victories. However, none of those teams had anywhere close to as many one goal wins as New Jersey.
This indicates that New Jersey's terrific one-goal game record is mostly from their ability to get ahead and hold onto the lead. The Devils outplay the other team early, get a lead, and then try to close out the game by protecting that margin rather than trying to extend it, a strategy that if successful leads to a lot of one-goal victories.
I have scoring data broken down by period for the last two seasons from the Hockey Summary Project to support this thesis. New Jersey's offence dropped in the third, with the Devils ranking 25th in the NHL in third period goals (empty-netters removed). Their shots taken also dropped, from an average of 10.3 shots per period in the first two (6th in the league) to an average of 9.4 shots per period in the third (15th). This was from a greater focus on defensive play, as the team also was able to cut down on the number of shots against (from 9.5 per period to 8.8 per period in the third).
Despite fewer shots against, the team's GAA actually went up in the third period, from 2.27 in the first two to 2.34 in the third. This was because the team save percentage dropped from .920 to .911. This suggests that shot prevention, rather than clutch goaltending, was the main reason the team was so effective at preserving leads. Keep in mind that New Jersey had a relatively high success rate in mounting comebacks, which shows that they had the ability to score if they wanted to. Instead, the Devils traded offence for defence, and the reason they won so many games by a single goal was because they scored fewer late insurance goals than other strong teams.
New Jersey was actually outscored 128-124 in the third period over the last two seasons. The team's positive goal differential came entirely from its success in the first two periods. I looked at a few of the other top teams in winning close games as well as some of the worst teams, and their records usually could not be explained by their third period save percentages or shot ratios. In fact, few teams were particularly clutch in terms of their percentages. In general the teams that outshot and/or out-"percentaged" their opponents in the first two periods had good records in close games, while teams that got outshot usually did not.
Announcers and writers often focus on the late "big save" that supposedly "won the game". However, most of the time that is giving too much credit to the goalie. The reason that save looks important is that the team had already built a lead in the game. The goalie does have to make the saves to keep the team ahead, of course, and if they are facing sustained pressure sometimes they need to be excellent to keep the other team off the scoreboard, but since the average shot has a 91% chance of being stopped the odds are very much in favour of the leading team. Because of this goalies have a very high success rate in holding off the other team late in the game, in the same way that baseball closers usually manage to "save" the game when they enter in the ninth inning with their team already in front.
New Jersey has won a lot of close games by outshooting and outscoring the opposition early, and then locking down the game to reduce scoring chances in the third period. Their goaltending should get credit for its strong overall performance, but it does not appear to warrant any additional recognition for "making the big saves".
It looks like the Devils have kept their "competitive advantage" going this year, with all three wins this season coming by a shootout or by a one goal margin. They will likely need to continue to excel at winning the close ones to stay competitive in a tough Atlantic Division.
As a result, when I run across a team that is a substantial outlier in any area then I take notice, and when that outlying result comes in an area that I mostly attribute to luck then it is especially interesting.
Here are the top 5 teams in games won by a one-goal margin since the lockout (not including this season so far):
1. New Jersey, 109
2. Dallas, 89
3. Vancouver, 87
3. Nashville, 87
5. San Jose, 86
5. Carolina, 86
5. Anaheim, 86
And here are the top 5 teams in winning percentage in games decided by one goal:
1. New Jersey, .740
2. Carolina, .709
3. Nashville, .690
4. Detroit, .673
5. San Jose. 670
The Devils are at the top of both of those lists, and it's not even close. That begs the question, just what are they doing differently than everyone else in the league?
The simple answer, and the one that 90% of journalists would probably respond with, is that the Devils have Martin Brodeur in net. There is, however, one fairly significant piece of evidence that suggests there is more to the story than that, namely New Jersey's record in one-goal games in 2008-09. Despite losing Brodeur to injury for almost 4 months, the Devils posted the best close game record they have ever had, going a remarkable 25-5-4 in one-goal games. Compared to an average team, New Jersey picked up an extra 12 points by winning the nailbiters, which made the difference between them winning their division and finishing as the #7 seed.
I looked at New Jersey's record since the lockout when leading, trailing, and tied after 2 periods. If they were a particularly clutch team, we would expect them to have a lot of wins in games that were tied heading into the third. The Devils did do well in that situation with a .644 winning percentage, good for 6th best in the league since the lockout. However, the Devils actually played a relatively low number of games that were tied after two, which was somewhat surprising for a low-scoring team. The team won a total of 36 games where they were tied after 2 periods, which was right about the average number (the Devils ranked 16th in the league).
New Jersey was also pretty good at coming from behind. Their winning percentage of .250 ranked 6th in the league, well above the average of .200. The Devils won 26 games that they trailed after 2 periods, which was tied for 4th in the league. It is likely that many of those wins would have been one-goal wins, although the average team won 20 so this would only account for part of their close game success.
By far most of New Jersey's wins came in games they were already leading after two periods. The Devils went 130-6-7 when ahead after two, for a .934 winning percentage that was the league's best. Only Detroit, San Jose and Ottawa converted a higher number of second intermission advantages into victories. However, none of those teams had anywhere close to as many one goal wins as New Jersey.
This indicates that New Jersey's terrific one-goal game record is mostly from their ability to get ahead and hold onto the lead. The Devils outplay the other team early, get a lead, and then try to close out the game by protecting that margin rather than trying to extend it, a strategy that if successful leads to a lot of one-goal victories.
I have scoring data broken down by period for the last two seasons from the Hockey Summary Project to support this thesis. New Jersey's offence dropped in the third, with the Devils ranking 25th in the NHL in third period goals (empty-netters removed). Their shots taken also dropped, from an average of 10.3 shots per period in the first two (6th in the league) to an average of 9.4 shots per period in the third (15th). This was from a greater focus on defensive play, as the team also was able to cut down on the number of shots against (from 9.5 per period to 8.8 per period in the third).
Despite fewer shots against, the team's GAA actually went up in the third period, from 2.27 in the first two to 2.34 in the third. This was because the team save percentage dropped from .920 to .911. This suggests that shot prevention, rather than clutch goaltending, was the main reason the team was so effective at preserving leads. Keep in mind that New Jersey had a relatively high success rate in mounting comebacks, which shows that they had the ability to score if they wanted to. Instead, the Devils traded offence for defence, and the reason they won so many games by a single goal was because they scored fewer late insurance goals than other strong teams.
New Jersey was actually outscored 128-124 in the third period over the last two seasons. The team's positive goal differential came entirely from its success in the first two periods. I looked at a few of the other top teams in winning close games as well as some of the worst teams, and their records usually could not be explained by their third period save percentages or shot ratios. In fact, few teams were particularly clutch in terms of their percentages. In general the teams that outshot and/or out-"percentaged" their opponents in the first two periods had good records in close games, while teams that got outshot usually did not.
Announcers and writers often focus on the late "big save" that supposedly "won the game". However, most of the time that is giving too much credit to the goalie. The reason that save looks important is that the team had already built a lead in the game. The goalie does have to make the saves to keep the team ahead, of course, and if they are facing sustained pressure sometimes they need to be excellent to keep the other team off the scoreboard, but since the average shot has a 91% chance of being stopped the odds are very much in favour of the leading team. Because of this goalies have a very high success rate in holding off the other team late in the game, in the same way that baseball closers usually manage to "save" the game when they enter in the ninth inning with their team already in front.
New Jersey has won a lot of close games by outshooting and outscoring the opposition early, and then locking down the game to reduce scoring chances in the third period. Their goaltending should get credit for its strong overall performance, but it does not appear to warrant any additional recognition for "making the big saves".
It looks like the Devils have kept their "competitive advantage" going this year, with all three wins this season coming by a shootout or by a one goal margin. They will likely need to continue to excel at winning the close ones to stay competitive in a tough Atlantic Division.
Monday, October 12, 2009
Situation-Adjusted Save Percentage
My conclusion from a recent look at special teams performance was that a goalie's play while his team is killing a penalty is important for evaluation, but that his performance when his team was on the power play was not. The latter conclusion was based on the lack of correlation between EV SV% and PP SV%, which suggests that the results are either highly influenced by luck or the rest of the team.
Gabe Desjardins over at Puck Prospectus agrees with me, and has a nice article up with a more advanced look at evaluating goalies based on only their EV and PK play. His method looks like a solid one for comprehensive goalie evaluation, but I thought of developing a much simpler formula that can be used as a quick-and-dirty way to take special teams into account.
Over the last decade, 76.3% of the shots have come at EV, 3.7% on the PP, and 20.0% on the PK. That number is skewed up slightly by the 2005-06 season, as in both of the last two seasons 19.7% of the shots taken were by a team on the power play. If we ignore the PP shots, a good approximation of the average EV/PK split is 80/20. By assigning an 80% weighting to the goalie's EV SV% and a 20% weighting to his PK SV% we can quickly adjust for special teams factors.
This adjustment doesn't actually make much of a difference for most goalies, but it does impact goalies who either faced a disproportionate number of shots on the penalty kill or for whatever reason allowed a unusual number of shorthanded goals. Here are the top 20 in situation-adjusted save percentage since the lockout (min. 100 GP):
1. Niklas Backstrom, .923
2. Tomas Vokoun, .922
3. Roberto Luongo, .920
4. Tim Thomas, .919
5. Cristobal Huet, .918
5. Henrik Lundqvist, .918
5. Dominik Hasek, .918
8. Martin Brodeur, .917
8. J.S. Giguere, .917
10. Miikka Kiprusoff, .914
10. Chris Mason, .914
12. Manny Fernandez, .913
13. Martin Biron, .912
13. Ryan Miller, .912
13. Kari Lehtonen, .912
16. Ilja Bryzgalov, .910
16. Marc-Andre Fleury, .910
16. Ray Emery, .910
19. Rick Dipietro, 909
19. Dwayne Roloson, .909
19. Manny Legace, .909
I think the best use of this adjustment would be as a quick check when comparing goalies. Let's say you were voting on the 2009 Calder Trophy and you wanted to compare the performances of Steve Mason and Pekka Rinne. If you look at the raw save percentages it was pretty close, with Mason at .916 and Rinne at .917. What that doesn't show, however, is that Rinne faced an unusually low number of shots against on the penalty kill. If we multiply their EV SV% by 80% and their PK SV% by 20%, Mason edges ahead .917 to .914.
The formula also adjusts for goalies who were lucky or unlucky with shorthanded scoring chances against. Henrik Lundqvist and Cam Ward both had .916 save percentages last season. However, Lundqvist allowed 11 shorthanded goals against compared to Ward's 5, as the Rangers allowed a lot of shots and presumably a lot of scoring chances against on the power play. If we look at EV and PK play only, Lundqvist jumps to .919 while Ward falls to .915.
To simply adjust for special teams factors remember the "80/20 rule", and you'll be able to pick out the goalies who have the burden or good fortune of facing heavy or light work on the penalty kill.
Gabe Desjardins over at Puck Prospectus agrees with me, and has a nice article up with a more advanced look at evaluating goalies based on only their EV and PK play. His method looks like a solid one for comprehensive goalie evaluation, but I thought of developing a much simpler formula that can be used as a quick-and-dirty way to take special teams into account.
Over the last decade, 76.3% of the shots have come at EV, 3.7% on the PP, and 20.0% on the PK. That number is skewed up slightly by the 2005-06 season, as in both of the last two seasons 19.7% of the shots taken were by a team on the power play. If we ignore the PP shots, a good approximation of the average EV/PK split is 80/20. By assigning an 80% weighting to the goalie's EV SV% and a 20% weighting to his PK SV% we can quickly adjust for special teams factors.
This adjustment doesn't actually make much of a difference for most goalies, but it does impact goalies who either faced a disproportionate number of shots on the penalty kill or for whatever reason allowed a unusual number of shorthanded goals. Here are the top 20 in situation-adjusted save percentage since the lockout (min. 100 GP):
1. Niklas Backstrom, .923
2. Tomas Vokoun, .922
3. Roberto Luongo, .920
4. Tim Thomas, .919
5. Cristobal Huet, .918
5. Henrik Lundqvist, .918
5. Dominik Hasek, .918
8. Martin Brodeur, .917
8. J.S. Giguere, .917
10. Miikka Kiprusoff, .914
10. Chris Mason, .914
12. Manny Fernandez, .913
13. Martin Biron, .912
13. Ryan Miller, .912
13. Kari Lehtonen, .912
16. Ilja Bryzgalov, .910
16. Marc-Andre Fleury, .910
16. Ray Emery, .910
19. Rick Dipietro, 909
19. Dwayne Roloson, .909
19. Manny Legace, .909
I think the best use of this adjustment would be as a quick check when comparing goalies. Let's say you were voting on the 2009 Calder Trophy and you wanted to compare the performances of Steve Mason and Pekka Rinne. If you look at the raw save percentages it was pretty close, with Mason at .916 and Rinne at .917. What that doesn't show, however, is that Rinne faced an unusually low number of shots against on the penalty kill. If we multiply their EV SV% by 80% and their PK SV% by 20%, Mason edges ahead .917 to .914.
The formula also adjusts for goalies who were lucky or unlucky with shorthanded scoring chances against. Henrik Lundqvist and Cam Ward both had .916 save percentages last season. However, Lundqvist allowed 11 shorthanded goals against compared to Ward's 5, as the Rangers allowed a lot of shots and presumably a lot of scoring chances against on the power play. If we look at EV and PK play only, Lundqvist jumps to .919 while Ward falls to .915.
To simply adjust for special teams factors remember the "80/20 rule", and you'll be able to pick out the goalies who have the burden or good fortune of facing heavy or light work on the penalty kill.
Saturday, October 10, 2009
The Win Threshold
I am firmly against comparing goalies based on wins. This is not because wins aren't important or desirable. Wins are what every player and team wants more than anything else. What makes wins a poor stat is that every team situation is different. If every goalie played the same schedule with identical teammates in front of them, then we could just give the Vezina to the guy with the most wins at the end of the season. In real life, goalies do not compete on a level playing field.
The two most important team factors that affect a goalie's ability to win are his goal support and the number of shots he has to face. More goal support means that more of his mistakes are covered up, and facing fewer shots against means fewer opportunities to allow goals.
Last year the Detroit Red Wings had the league's best offence, scoring 3.52 goals per game. They also allowed the second fewest shots against per game with 27.7. On the other end of the scale, the New York Islanders finished second last in both goals for (2.42) and shots against (33.5) per game. Quite obviously an Islander goalie would need to be much better than a Red Wing goalie for their teams to have the same chance at winning, because he would have to make up for his team scoring one less goal per game and he would have to do it while facing an extra half-dozen shots against.
We can calculate what I'll call the "win threshold" for the goalies on each team by taking (shots against - goals for) / shots against. This gives us the save percentage that would result in the team ending up with an equal number of goals for and goals against over the course of the season. If the goalie's save percentage is above that number, the team is likely to win more than the lose, while anything below the threshold means that the team should end up sub-.500 (or sub-.550 in the shootout era).
In 2008-09, Detroit's win threshold was .873, which was the lowest in the league. The Islanders' win threshold was .928, which was not only the highest mark in the league but also the highest of any team since the lockout.
Expressed a different way, Detroit is likely to win if their opponents have a shooting percentage of 12.6% or worse. The New York Islanders are likely to win only if their opponents have a shooting percentage of 7.1% or worse. The shooting percentage against Detroit needs to be almost 80% higher than the percentage against the Islanders for the teams to have the same likelihood of winning the game.
Naturally, comparing win totals on goalies playing on the Islanders to goalies playing on the Red Wings is completely senseless. It would be like comparing two students in terms of how many course credits they attained, where the first student passes their courses if they achieve a mark of 50% or better while the second student only passes if they score 90% or higher. With that advantage, the first student is much more likely to pass his courses and achieve a higher overall number of passes. Even if the second student is exceptional and the first student is mediocre, it is likely that the first student will have a similar or better score because of their inherent advantage.
I ran the formula for every team since the 1997-98 season, including an adjustment for average league goals and shots per game.
Top 10 since 1997-98:
1. 2001 Devils, .860
2. 2004 Senators, .866
3. 2000 Blues, .867
4. 2003 Senators, .868
5. 2008 Red Wings, .869
6. 1998 Blues, .872
6. 1999 Blues, .872
6. 1998 Stars, .872
6. 2006 Red Wings, .872
10. 2001 Avalanche, .874
10. 2003 Blues, .874
10. 2009 Red Wings, .874
Bottom 10 since 1997-98:
1. 1998 Lightning, .937
2. 2002 Thrashers, .935
2. 2000 Thrashers, .935
4. 2003 Panthers, .934
5. 2002 Blue Jackets, .933
6. 2002 Panthers, .932
7. 1999 Lightning, .931
7. 2000 Islanders,. 931
7. 2004 Panthers, .931
10. 2001 Wild, .930
10. 2004 Blue Jackets, .930
If you ever wondered how Roman Turek managed to get 42 wins in a season, or how Patrick Lalime won 39, here's your answer. On the other hand, note that four of Roberto Luongo's teams show up in the bottom 10. Why didn't the Florida Panthers make the playoffs? Because the team was terrible. It had nothing to do with the goaltending.
Note that these are team totals that need to be achieved, which make it even more difficult for goalies on the worst teams than it appears at first glance. If they have a backup who plays around 20-25 games at .900, then the starting goalie would need to be at .940 or better for the team to have a goal differential of zero. Even then, the team is unlikely to make the playoffs without scoring more goals than they allow.
The average win/loss record of the teams in the top 10 list was 48-22-12. The average win/loss record of the teams in the bottom 10 was 22-46-14. What is interesting, however, is that the goaltending performance was quite similar:
Save % of top 10 teams: .905
Save % of bottom 10 teams: .904
The teams on the top list didn't win because of great goaltending or because their goalie gave them clutch saves. They won because their teams scored a lot of goals and didn't allow many shots against. Similarly, the teams on the bottom list lost because they struggled to score and allowed too many shots against, not because their goaltenders were poor. This is further proof that win totals are a team stat, and should not be used to evaluate individual goalies.
The two most important team factors that affect a goalie's ability to win are his goal support and the number of shots he has to face. More goal support means that more of his mistakes are covered up, and facing fewer shots against means fewer opportunities to allow goals.
Last year the Detroit Red Wings had the league's best offence, scoring 3.52 goals per game. They also allowed the second fewest shots against per game with 27.7. On the other end of the scale, the New York Islanders finished second last in both goals for (2.42) and shots against (33.5) per game. Quite obviously an Islander goalie would need to be much better than a Red Wing goalie for their teams to have the same chance at winning, because he would have to make up for his team scoring one less goal per game and he would have to do it while facing an extra half-dozen shots against.
We can calculate what I'll call the "win threshold" for the goalies on each team by taking (shots against - goals for) / shots against. This gives us the save percentage that would result in the team ending up with an equal number of goals for and goals against over the course of the season. If the goalie's save percentage is above that number, the team is likely to win more than the lose, while anything below the threshold means that the team should end up sub-.500 (or sub-.550 in the shootout era).
In 2008-09, Detroit's win threshold was .873, which was the lowest in the league. The Islanders' win threshold was .928, which was not only the highest mark in the league but also the highest of any team since the lockout.
Expressed a different way, Detroit is likely to win if their opponents have a shooting percentage of 12.6% or worse. The New York Islanders are likely to win only if their opponents have a shooting percentage of 7.1% or worse. The shooting percentage against Detroit needs to be almost 80% higher than the percentage against the Islanders for the teams to have the same likelihood of winning the game.
Naturally, comparing win totals on goalies playing on the Islanders to goalies playing on the Red Wings is completely senseless. It would be like comparing two students in terms of how many course credits they attained, where the first student passes their courses if they achieve a mark of 50% or better while the second student only passes if they score 90% or higher. With that advantage, the first student is much more likely to pass his courses and achieve a higher overall number of passes. Even if the second student is exceptional and the first student is mediocre, it is likely that the first student will have a similar or better score because of their inherent advantage.
I ran the formula for every team since the 1997-98 season, including an adjustment for average league goals and shots per game.
Top 10 since 1997-98:
1. 2001 Devils, .860
2. 2004 Senators, .866
3. 2000 Blues, .867
4. 2003 Senators, .868
5. 2008 Red Wings, .869
6. 1998 Blues, .872
6. 1999 Blues, .872
6. 1998 Stars, .872
6. 2006 Red Wings, .872
10. 2001 Avalanche, .874
10. 2003 Blues, .874
10. 2009 Red Wings, .874
Bottom 10 since 1997-98:
1. 1998 Lightning, .937
2. 2002 Thrashers, .935
2. 2000 Thrashers, .935
4. 2003 Panthers, .934
5. 2002 Blue Jackets, .933
6. 2002 Panthers, .932
7. 1999 Lightning, .931
7. 2000 Islanders,. 931
7. 2004 Panthers, .931
10. 2001 Wild, .930
10. 2004 Blue Jackets, .930
If you ever wondered how Roman Turek managed to get 42 wins in a season, or how Patrick Lalime won 39, here's your answer. On the other hand, note that four of Roberto Luongo's teams show up in the bottom 10. Why didn't the Florida Panthers make the playoffs? Because the team was terrible. It had nothing to do with the goaltending.
Note that these are team totals that need to be achieved, which make it even more difficult for goalies on the worst teams than it appears at first glance. If they have a backup who plays around 20-25 games at .900, then the starting goalie would need to be at .940 or better for the team to have a goal differential of zero. Even then, the team is unlikely to make the playoffs without scoring more goals than they allow.
The average win/loss record of the teams in the top 10 list was 48-22-12. The average win/loss record of the teams in the bottom 10 was 22-46-14. What is interesting, however, is that the goaltending performance was quite similar:
Save % of top 10 teams: .905
Save % of bottom 10 teams: .904
The teams on the top list didn't win because of great goaltending or because their goalie gave them clutch saves. They won because their teams scored a lot of goals and didn't allow many shots against. Similarly, the teams on the bottom list lost because they struggled to score and allowed too many shots against, not because their goaltenders were poor. This is further proof that win totals are a team stat, and should not be used to evaluate individual goalies.
Thursday, October 8, 2009
The Underrated Allan Bester
I discussed recently how goalies are often evaluated based on a few memories. For some goalies it means they are forever remembered as winners. For others, it means that they are defined by one bad goal against. The latter group is often very underrated. For example, all our memories of ten years of league average save percentages and a previously stellar international career vanished in a flash as soon as that puck bounced in off of Tommy Salo's head.
Another example of this phenomenon is Allan Bester. Bester is remembered as a joke among many Toronto Maple Leafs fans. Take this quote off a Leaf blog, for example:
"Raycroft is the worst starting goalie I’ve seen play for the Leafs since Allan Bester. Remember him? He attempted suicide one night. No, really. He jumped in front of a bus! But it went between his legs." (Leaf Club)
This is of course a reference to the the overtime goal scored by Sergio Momesso in the 1990 playoffs at Maple Leaf Gardens. Momesso's shot beat Bester five-hole, a memory that Leaf fans will forever associate with the tiny (5'7, 155) netminder.
However, the numbers paint a very different picture of Allan Bester. According to the statistics, Bester outperformed his teammates in every league that he ever played in, and was easily the best goalie the Leafs had in the 1980s. Despite this, the Momesso goal was essentially the end of Bester's NHL career.
Bester broke into the league early as a 19 year old right out of junior, and had all of his NHL success in his early twenties. With fellow youngster Ken Wregget, Bester made up a promising goalie tandem for the Leafs in the mid-1980s. The 1988-89 season was probably the high point of Bester's career, as he finished in the top 10 in save percentage on a team that missed the playoffs. Bester was even named to Team Canada's world championship squad, although he was unable to participate because of injury.
In 1989-90 Bester had an off-year, and then he struggled in the playoffs. Everything started to unravel after that. He was displaced in the Leaf net in 1990-91 by rookie Peter Ing and then got traded to the Detroit Red Wings for a draft pick.
The Wings buried Bester in the AHL, but he played well in the minors. In 1992, the Adirondack Red Wings won the Calder Cup, and Bester was named playoff MVP. The next year he outplayed Chris Osgood, who was seen as Detroit's goalie of the future. Without an opportunity to move up in the Red Wings organization, Bester looked to catch on with one of the expansion teams. He was passed over in the expansion draft, but managed to catch on with the Anaheim Mighty Ducks as a free agent.
Unfortunately for Allan Bester, Anaheim had the best goaltending of any of the expansion teams, with the solid tandem of Guy Hebert and Ron Tugnutt. Bester was competing with Mikhail Shtalenkov for the #3 role in the organization, and despite similar IHL numbers the team decided to give the NHL opportunities to Shtalenkov. Bester continued to play in the IHL until he got one last cup of coffee at the NHL level as an injury replacement for the Dallas Stars in 1996. He acquitted himself well in a 10 game stint, but returned to the IHL where he remained until he retired in 1997-98 at the age of 33.
Having established the biography, let's look at the numbers. I compared Bester's numbers to his teammates for every season of his professional career, except when he only played a few games which would not be a representative sample.
Allan Bester solidly outplayed his teammates at the NHL level:
Bester: 4.00 GAA, .883 save %, .432 win %, 2.5 SO/70 GP
Others: 4.30 GAA, .869 save %, .370 win %, 0.5 SO/70 GP
Bester has a clear edge in every stat, including a large edge in shutouts. For goalies playing on bad teams, shutouts are often a fairly good indicator of dominance, since they aren't able to post the easy shutouts that goalies on winning teams often get. On the mid-'80s Leafs, the only way anybody was going to get a shutout was through an outstanding performance, and Bester had a very respectable 2.5 shutouts per 70 games played (I prefer to express shutout rates per 70 games because the per-game rates get pretty small). This was five times the rate of his teammates. The raw total was 7 shutouts in the equivalent of 178 full games, compared to his teammates' total of just 2 in 309.
Question to Maple Leaf fans of that era: Does one bad playoff goal make up for a winning percentage difference of .062 compared to Toronto's other goalies, the equivalent of a 5 extra wins over a full season?
In the AHL, Bester again outplayed his teammates:
Bester: .487 win %, 3.74 GAA, 1.4 SO/70 GP
Others: .456 win %, 4.05 GAA, 1.5 SO/70 GP
Bester's playing partners included Wregget, Tim Bernhardt, Rick St. Croix, Damian Rhodes, Scott King and Chris Osgood. Bester spent his prime age 27 and 28 years in the minors as a Red Wing, outplaying his teammates and being named the MVP of a Calder Cup winning team, yet only getting to play 31 minutes in the NHL.
One contributing factor to this was likely the overall improvement in league goaltending during the early- to mid-1990s. A new wave of goalies was entering the league, bringing with them the modern butterfly style and displacing many veterans who weren't able to keep up. Bester certainly wasn't the only goalie of his age that got caught up in a numbers game, but having said that he was still putting up numbers that were as good as or better than several young goalies who were headed for the NHL. It is possible that his declining performance in 1989-90 and 1990-91 showed that he no longer had the ability to compete at the NHL level, but his minor league performance suggests that it is also possible that he had an off-year and didn't get a chance to play his way back into form.
In the IHL, Bester was in his thirties but was still usually the best goalie on the team. Some of his partners were scrubs, but others were legit goalies (e.g. Shtalenkov, Essensa).
Bester: .630 win %, 3.22 GAA, 1.9 SO/70 GP
Others: .563 win %, 3.43 GAA, 1.6 SO/70 GP
Just as winning goalies shouldn't be judged based on one shining playoff run, losing goalies should not be judged based on one bad goal. Allan Bester was likely at least a league average goalie, and for a short time probably even better than that.
I'll leave you with one final stat: From 1983-84 to 1989-90, Allan Bester finished 13th in the NHL in save percentage among goalies with at least 150 GP, playing on one of the worst teams in the league. I'll post a section of the standings so we can see the goalies just above and below him:
11. Tom Barrasso, .883
12. Don Beaupre, .883
13. Allan Bester, .883
14. Mike Liut, .883
15. Billy Smith, .882
16. Grant Fuhr, .881
17. Kirk McLean, .881
18. Mike Vernon, .881
19. Pete Peeters, .881
In the 1980s those 9 goalies combined for 5 First Team All-Stars, 4 Second Team All-Stars, and 9 Stanley Cups. Every single one of them had long NHL careers. And yet nobody gave Bester another NHL shot.
I think the scouts got it wrong on this one. Bester may have been small and he may have let a few through the wickets, but the evidence suggests that he was a legitimate pro goalie and he deserved another shot at an NHL job after the age of 25. He should be remembered for what he was, the best Toronto Maple Leafs goalie of the 1980s.
Another example of this phenomenon is Allan Bester. Bester is remembered as a joke among many Toronto Maple Leafs fans. Take this quote off a Leaf blog, for example:
"Raycroft is the worst starting goalie I’ve seen play for the Leafs since Allan Bester. Remember him? He attempted suicide one night. No, really. He jumped in front of a bus! But it went between his legs." (Leaf Club)
This is of course a reference to the the overtime goal scored by Sergio Momesso in the 1990 playoffs at Maple Leaf Gardens. Momesso's shot beat Bester five-hole, a memory that Leaf fans will forever associate with the tiny (5'7, 155) netminder.
However, the numbers paint a very different picture of Allan Bester. According to the statistics, Bester outperformed his teammates in every league that he ever played in, and was easily the best goalie the Leafs had in the 1980s. Despite this, the Momesso goal was essentially the end of Bester's NHL career.
Bester broke into the league early as a 19 year old right out of junior, and had all of his NHL success in his early twenties. With fellow youngster Ken Wregget, Bester made up a promising goalie tandem for the Leafs in the mid-1980s. The 1988-89 season was probably the high point of Bester's career, as he finished in the top 10 in save percentage on a team that missed the playoffs. Bester was even named to Team Canada's world championship squad, although he was unable to participate because of injury.
In 1989-90 Bester had an off-year, and then he struggled in the playoffs. Everything started to unravel after that. He was displaced in the Leaf net in 1990-91 by rookie Peter Ing and then got traded to the Detroit Red Wings for a draft pick.
The Wings buried Bester in the AHL, but he played well in the minors. In 1992, the Adirondack Red Wings won the Calder Cup, and Bester was named playoff MVP. The next year he outplayed Chris Osgood, who was seen as Detroit's goalie of the future. Without an opportunity to move up in the Red Wings organization, Bester looked to catch on with one of the expansion teams. He was passed over in the expansion draft, but managed to catch on with the Anaheim Mighty Ducks as a free agent.
Unfortunately for Allan Bester, Anaheim had the best goaltending of any of the expansion teams, with the solid tandem of Guy Hebert and Ron Tugnutt. Bester was competing with Mikhail Shtalenkov for the #3 role in the organization, and despite similar IHL numbers the team decided to give the NHL opportunities to Shtalenkov. Bester continued to play in the IHL until he got one last cup of coffee at the NHL level as an injury replacement for the Dallas Stars in 1996. He acquitted himself well in a 10 game stint, but returned to the IHL where he remained until he retired in 1997-98 at the age of 33.
Having established the biography, let's look at the numbers. I compared Bester's numbers to his teammates for every season of his professional career, except when he only played a few games which would not be a representative sample.
Allan Bester solidly outplayed his teammates at the NHL level:
Bester: 4.00 GAA, .883 save %, .432 win %, 2.5 SO/70 GP
Others: 4.30 GAA, .869 save %, .370 win %, 0.5 SO/70 GP
Bester has a clear edge in every stat, including a large edge in shutouts. For goalies playing on bad teams, shutouts are often a fairly good indicator of dominance, since they aren't able to post the easy shutouts that goalies on winning teams often get. On the mid-'80s Leafs, the only way anybody was going to get a shutout was through an outstanding performance, and Bester had a very respectable 2.5 shutouts per 70 games played (I prefer to express shutout rates per 70 games because the per-game rates get pretty small). This was five times the rate of his teammates. The raw total was 7 shutouts in the equivalent of 178 full games, compared to his teammates' total of just 2 in 309.
Question to Maple Leaf fans of that era: Does one bad playoff goal make up for a winning percentage difference of .062 compared to Toronto's other goalies, the equivalent of a 5 extra wins over a full season?
In the AHL, Bester again outplayed his teammates:
Bester: .487 win %, 3.74 GAA, 1.4 SO/70 GP
Others: .456 win %, 4.05 GAA, 1.5 SO/70 GP
Bester's playing partners included Wregget, Tim Bernhardt, Rick St. Croix, Damian Rhodes, Scott King and Chris Osgood. Bester spent his prime age 27 and 28 years in the minors as a Red Wing, outplaying his teammates and being named the MVP of a Calder Cup winning team, yet only getting to play 31 minutes in the NHL.
One contributing factor to this was likely the overall improvement in league goaltending during the early- to mid-1990s. A new wave of goalies was entering the league, bringing with them the modern butterfly style and displacing many veterans who weren't able to keep up. Bester certainly wasn't the only goalie of his age that got caught up in a numbers game, but having said that he was still putting up numbers that were as good as or better than several young goalies who were headed for the NHL. It is possible that his declining performance in 1989-90 and 1990-91 showed that he no longer had the ability to compete at the NHL level, but his minor league performance suggests that it is also possible that he had an off-year and didn't get a chance to play his way back into form.
In the IHL, Bester was in his thirties but was still usually the best goalie on the team. Some of his partners were scrubs, but others were legit goalies (e.g. Shtalenkov, Essensa).
Bester: .630 win %, 3.22 GAA, 1.9 SO/70 GP
Others: .563 win %, 3.43 GAA, 1.6 SO/70 GP
Just as winning goalies shouldn't be judged based on one shining playoff run, losing goalies should not be judged based on one bad goal. Allan Bester was likely at least a league average goalie, and for a short time probably even better than that.
I'll leave you with one final stat: From 1983-84 to 1989-90, Allan Bester finished 13th in the NHL in save percentage among goalies with at least 150 GP, playing on one of the worst teams in the league. I'll post a section of the standings so we can see the goalies just above and below him:
11. Tom Barrasso, .883
12. Don Beaupre, .883
13. Allan Bester, .883
14. Mike Liut, .883
15. Billy Smith, .882
16. Grant Fuhr, .881
17. Kirk McLean, .881
18. Mike Vernon, .881
19. Pete Peeters, .881
In the 1980s those 9 goalies combined for 5 First Team All-Stars, 4 Second Team All-Stars, and 9 Stanley Cups. Every single one of them had long NHL careers. And yet nobody gave Bester another NHL shot.
I think the scouts got it wrong on this one. Bester may have been small and he may have let a few through the wickets, but the evidence suggests that he was a legitimate pro goalie and he deserved another shot at an NHL job after the age of 25. He should be remembered for what he was, the best Toronto Maple Leafs goalie of the 1980s.
Tuesday, October 6, 2009
Shot Quality Refined
I have to throw up a link to the latest work by Ken Krzywicki on shot quality. He tackles the issue of arena reporting bias, and has updated his model to take that into account. I think there are some very good reasons to question the importance and/or the accuracy of shot quality measures, but I still think it is possible that some of the problems are a result of the data or the models rather than the theory that different teams allow different types of scoring chances. That shot quality differences are observed in the game of hockey is beyond doubt, the debate is about whether those differences are significant at the NHL level.
To me, the revised model makes more sense, because we see a smaller spread of values at the team level. The best team has an expected save percentage of .915, while the worst is at .904. Compare that to the .900 - .918 which would be predicted based on the raw numbers. Half of the teams are within .002 of league average, which matches our expectation that most teams do not have significant shot quality effects.
I would like to see the shots separated out by situation into EV and PK. I think that would allow us to better be able to test the results. However, the model must be projecting some even strength variations, since the correlation between expected save percentage and power plays against in 2008-09 was -0.03.
Some of the team results are a bit surprising. In particular, Boston is rated as having allowing the 4th most difficult shots against, which seems to strongly contradict the numbers of Thomas and Fernandez, the Bruins' low number of penalties taken, and the coach and team style of play.
Alan Ryder also has a method for evaluating shot quality in his annual NHL review. He says he has tried to make some adjustments for rink bias as well, yet some his numbers differ from Krzywicki's. One of them is in his measurement for Boston whom Ryder grades as about average, which I suspect might be a more accurate assessment.
By comparing Krzywicki's numbers with Ryder's, and also keeping in mind each team's actual save percentage as an additional check, I think there are about ten teams that we can target as possible shot quality outliers in 2008-09, with New Jersey, Phoenix, Buffalo, Minnesota and Columbus the candidates for easier than average shot quality and Toronto, Dallas, Carolina, Montreal and Calgary possibly allowing harder than average shots against.
Of those, a few could be the rest of measurement error, score effects, randomness, etc., but I think is likely that there are a few teams that must be ahead or behind the rest of the league. To me, Minnesota, New Jersey, Toronto and Dallas are the most likely candidates for 2008-09. For those four teams, the shot quality results match both the subjective perception and the actual results in the crease.
If this is correct, that would have implications for our evaluation of some goalies. That doesn't mean that goalies like Turco or Toskala can be let off the hook, however. The expected save percentage of an outlier team is still likely somewhere in the range of .003-.005 above or below average, which means that outstanding or terrible save percentages cannot be explained by the rest of the team and must fall mostly on the shoulders of the goalie that posted them.
To me, the revised model makes more sense, because we see a smaller spread of values at the team level. The best team has an expected save percentage of .915, while the worst is at .904. Compare that to the .900 - .918 which would be predicted based on the raw numbers. Half of the teams are within .002 of league average, which matches our expectation that most teams do not have significant shot quality effects.
I would like to see the shots separated out by situation into EV and PK. I think that would allow us to better be able to test the results. However, the model must be projecting some even strength variations, since the correlation between expected save percentage and power plays against in 2008-09 was -0.03.
Some of the team results are a bit surprising. In particular, Boston is rated as having allowing the 4th most difficult shots against, which seems to strongly contradict the numbers of Thomas and Fernandez, the Bruins' low number of penalties taken, and the coach and team style of play.
Alan Ryder also has a method for evaluating shot quality in his annual NHL review. He says he has tried to make some adjustments for rink bias as well, yet some his numbers differ from Krzywicki's. One of them is in his measurement for Boston whom Ryder grades as about average, which I suspect might be a more accurate assessment.
By comparing Krzywicki's numbers with Ryder's, and also keeping in mind each team's actual save percentage as an additional check, I think there are about ten teams that we can target as possible shot quality outliers in 2008-09, with New Jersey, Phoenix, Buffalo, Minnesota and Columbus the candidates for easier than average shot quality and Toronto, Dallas, Carolina, Montreal and Calgary possibly allowing harder than average shots against.
Of those, a few could be the rest of measurement error, score effects, randomness, etc., but I think is likely that there are a few teams that must be ahead or behind the rest of the league. To me, Minnesota, New Jersey, Toronto and Dallas are the most likely candidates for 2008-09. For those four teams, the shot quality results match both the subjective perception and the actual results in the crease.
If this is correct, that would have implications for our evaluation of some goalies. That doesn't mean that goalies like Turco or Toskala can be let off the hook, however. The expected save percentage of an outlier team is still likely somewhere in the range of .003-.005 above or below average, which means that outstanding or terrible save percentages cannot be explained by the rest of the team and must fall mostly on the shoulders of the goalie that posted them.
Monday, October 5, 2009
Winning Means the Rest Doesn't Count
Goalies are rated far too much based on their team success. One reason for this is that we all have limited memories, which causes certain moments to be more easily recalled while the rest is forgotten. For some goalies that's a good thing, because we can clearly recall them doing things like stopping Pavel Bure on a penalty shot in the Finals or making a save with 2 seconds left in game 7 to help their team win the Cup. For others we remember only the weak moments that they would rather forget. As a result, goalies who have not experienced team success often see a few of their worst performances get scrutinized to the near-exclusion of all else, while similar performances by other goalies get ignored because they have a ring.
Patrick Roy was a terrific playoff goalie, but he had his fair share of weak performances. If you play 17 playoff seasons you are naturally going to have a few that are forgettable, of course, but some of Roy's flops were actually quite spectacular (losing his job in 1987 and 1988, his problems against the Bourque/Neely Bruins, a few first-round upsets against much weaker opponents, and of course the famous game 7 of the 2002 Western Conference Finals). Yet his fans will often talk about 4 Cups and 3 Conn Smythes, as if that's all that happened. Again, Roy's playoff record is strong by any measure, but it obviously looks even stronger when you only focus on the best of the best.
For other goalies more of a focus is put on their negatives than their positives. Glenn Hall is generally considered a playoff underachiever and is blamed by many for the Hawks' relative lack of playoff success in the 1960s, despite a strong effort in helping his team win the 1961 Cup and even though Chicago was outshot in nearly every playoff series that decade. Another one would be Curtis Joseph, who often led his team past better teams with excellent performances in the early rounds, but usually gets blamed because of his teams' failure to perform well deeper in the playoffs.
If we adjust for era and opponents and compare the playoff numbers for Roy and Joseph, Roy is something like a .916 playoff goalie while Joseph is at .909. Adjust for power plays against and the gap might be even smaller. That's a significant difference over a decent sample size, certainly enough to conclude that Roy was better, but the actual gap is just one goal every 143 shots. You have to be pretty good at subjective evaluation to notice a difference of one extra goal saved every five games.
If we rely on our memories, however, then the differences are usually exaggerated. If I think of Roy and Joseph I might recall Roy's performances in the Stanley Cup Finals in 1993 and 2001, while what comes to mind for Joseph is how he played in the Conference Finals in 1999 and 2002. Based on that my memory is going to tell me that Roy was a lot better, because I'm effectively comparing a .934 goalie against an .890 goalie. That's the equivalent of Tim Thomas vs. Vesa Toskala in 2008-09. That is a gap that you can easily see with your eyes, and the result is substantially overrating the difference between playoff Roy and playoff Joseph.
There are a few goalies in the league today who benefit from the "only memorable playoff successes count" method of goalie evaluation. Marc-Andre Fleury is an obvious one, with his big save on Lidstrom still fresh in everyone's memories. Yet perhaps the goalie who benefits the most is Cam Ward. Many fans overlook his mostly mediocre regular season play (essentially everything up until December of 2008) in favour of the 6 playoff series of above average play that have built Ward something of a reputation as a money goalie. Some are claiming that Ward is one of the top 5 goalies in the NHL and deserves a chance to become Canada's starter in 2010, neither of which is supported by his history. Almost nobody took Ward to task for his performance in the Pittsburgh series despite getting shelled in four straight games. That's the power of reputation: If you're considered "clutch", then you can get blown out and the fans blame the rest of the team. If you're considered unclutch, then the fans blame you.
Ward just signed a contract extension for $37.8 million over 6 years. That's more than he deserves based on his past record, but Ward has shown some real improvement over the last few seasons (his EV SV% numbers have gone .899-.917-.926). Ward probably got rushed to the NHL before he was ready, and when he caught lightning in a bottle in the 2006 playoffs his team unwisely decided to throw him even further over his head by handing him the starting job a couple years before he deserved it. His performance over the last two seasons is likely more indicative of what he can be expected to do until the end of his this contract. Still, we don't have a lot to go on to be able to assess Ward's true skill level. Is he still improving? Was last season a good year or a typical year? All that remains to be seen.
In the new NHL, teams are paying players for their projected performance instead of their past performance, which is what I'm sure Carolina was thinking when they inked Ward to his big deal. If his 2008-09 numbers represent his true skill, then Ward is a top-10 goalie in the league. I'm not a big fan of paying out $5-6M+ on goalies, unless it is to an elite goalie with an established track record, because there always seems to be Craig Anderson-types available who have the potential to give you 90% of the performance for 20% of the cost and you don't want to get stuck paying big money to a goalie giving you average performance. However, if Ward is legitimately a .925 puckstopper at EV than it shouldn't be too bad of a contract for Carolina. If it turns out he's more of a .917 guy then that's a big overpayment.
I think it is pretty likely that Ward has become and will continue to be at the very least a solid NHL goalie, but I'm not at all convinced that he is some incredible pressure performer. If you look at his playoff performances throughout his career (junior, AHL, NHL), the only time that his playoff results look way out of line compared to his regular season numbers is his surprise Cup run in 2006. He did do pretty well in the playoffs in junior, but in the 2005 AHL playoffs he had one good series and one not-so-good one (.895 save percentage in a second-round loss to a lower-seeded team). Ward's international record is also fairly spotty, albeit in a very small sample, with a .901 combined save percentage in world championship games against the other top hockey nations (USA, Russia, Sweden, Finland).
To me Ward's performance in 2006, given what he did directly before and after it, is less a sign of his clutch ability than evidence of what kind of a charmed run the Hurricanes were fortunate enough to go on for the entire 2005-06 season. Referring back to the recent topic of percentages, I'm not sure a team has ever seen their shooting percentage jump more from one season to the next then those Hurricanes did when they improved from 7.2% in 2003-04 to 11.2% in 2005-06. No wonder the team went from dead last in goals and out of the playoffs to the 3rd best offence and the 4th best record in the league.
Winning once or twice does not mean that the rest shouldn't count. If you want to evaluate goalies on their playoff or international play, then first of all be aware of the sample size issues. Secondly, evaluate them based on their individual success, not their team success. And finally, be sure to take their entire performance into account, not just the very best or the very worst.
Patrick Roy was a terrific playoff goalie, but he had his fair share of weak performances. If you play 17 playoff seasons you are naturally going to have a few that are forgettable, of course, but some of Roy's flops were actually quite spectacular (losing his job in 1987 and 1988, his problems against the Bourque/Neely Bruins, a few first-round upsets against much weaker opponents, and of course the famous game 7 of the 2002 Western Conference Finals). Yet his fans will often talk about 4 Cups and 3 Conn Smythes, as if that's all that happened. Again, Roy's playoff record is strong by any measure, but it obviously looks even stronger when you only focus on the best of the best.
For other goalies more of a focus is put on their negatives than their positives. Glenn Hall is generally considered a playoff underachiever and is blamed by many for the Hawks' relative lack of playoff success in the 1960s, despite a strong effort in helping his team win the 1961 Cup and even though Chicago was outshot in nearly every playoff series that decade. Another one would be Curtis Joseph, who often led his team past better teams with excellent performances in the early rounds, but usually gets blamed because of his teams' failure to perform well deeper in the playoffs.
If we adjust for era and opponents and compare the playoff numbers for Roy and Joseph, Roy is something like a .916 playoff goalie while Joseph is at .909. Adjust for power plays against and the gap might be even smaller. That's a significant difference over a decent sample size, certainly enough to conclude that Roy was better, but the actual gap is just one goal every 143 shots. You have to be pretty good at subjective evaluation to notice a difference of one extra goal saved every five games.
If we rely on our memories, however, then the differences are usually exaggerated. If I think of Roy and Joseph I might recall Roy's performances in the Stanley Cup Finals in 1993 and 2001, while what comes to mind for Joseph is how he played in the Conference Finals in 1999 and 2002. Based on that my memory is going to tell me that Roy was a lot better, because I'm effectively comparing a .934 goalie against an .890 goalie. That's the equivalent of Tim Thomas vs. Vesa Toskala in 2008-09. That is a gap that you can easily see with your eyes, and the result is substantially overrating the difference between playoff Roy and playoff Joseph.
There are a few goalies in the league today who benefit from the "only memorable playoff successes count" method of goalie evaluation. Marc-Andre Fleury is an obvious one, with his big save on Lidstrom still fresh in everyone's memories. Yet perhaps the goalie who benefits the most is Cam Ward. Many fans overlook his mostly mediocre regular season play (essentially everything up until December of 2008) in favour of the 6 playoff series of above average play that have built Ward something of a reputation as a money goalie. Some are claiming that Ward is one of the top 5 goalies in the NHL and deserves a chance to become Canada's starter in 2010, neither of which is supported by his history. Almost nobody took Ward to task for his performance in the Pittsburgh series despite getting shelled in four straight games. That's the power of reputation: If you're considered "clutch", then you can get blown out and the fans blame the rest of the team. If you're considered unclutch, then the fans blame you.
Ward just signed a contract extension for $37.8 million over 6 years. That's more than he deserves based on his past record, but Ward has shown some real improvement over the last few seasons (his EV SV% numbers have gone .899-.917-.926). Ward probably got rushed to the NHL before he was ready, and when he caught lightning in a bottle in the 2006 playoffs his team unwisely decided to throw him even further over his head by handing him the starting job a couple years before he deserved it. His performance over the last two seasons is likely more indicative of what he can be expected to do until the end of his this contract. Still, we don't have a lot to go on to be able to assess Ward's true skill level. Is he still improving? Was last season a good year or a typical year? All that remains to be seen.
In the new NHL, teams are paying players for their projected performance instead of their past performance, which is what I'm sure Carolina was thinking when they inked Ward to his big deal. If his 2008-09 numbers represent his true skill, then Ward is a top-10 goalie in the league. I'm not a big fan of paying out $5-6M+ on goalies, unless it is to an elite goalie with an established track record, because there always seems to be Craig Anderson-types available who have the potential to give you 90% of the performance for 20% of the cost and you don't want to get stuck paying big money to a goalie giving you average performance. However, if Ward is legitimately a .925 puckstopper at EV than it shouldn't be too bad of a contract for Carolina. If it turns out he's more of a .917 guy then that's a big overpayment.
I think it is pretty likely that Ward has become and will continue to be at the very least a solid NHL goalie, but I'm not at all convinced that he is some incredible pressure performer. If you look at his playoff performances throughout his career (junior, AHL, NHL), the only time that his playoff results look way out of line compared to his regular season numbers is his surprise Cup run in 2006. He did do pretty well in the playoffs in junior, but in the 2005 AHL playoffs he had one good series and one not-so-good one (.895 save percentage in a second-round loss to a lower-seeded team). Ward's international record is also fairly spotty, albeit in a very small sample, with a .901 combined save percentage in world championship games against the other top hockey nations (USA, Russia, Sweden, Finland).
To me Ward's performance in 2006, given what he did directly before and after it, is less a sign of his clutch ability than evidence of what kind of a charmed run the Hurricanes were fortunate enough to go on for the entire 2005-06 season. Referring back to the recent topic of percentages, I'm not sure a team has ever seen their shooting percentage jump more from one season to the next then those Hurricanes did when they improved from 7.2% in 2003-04 to 11.2% in 2005-06. No wonder the team went from dead last in goals and out of the playoffs to the 3rd best offence and the 4th best record in the league.
Winning once or twice does not mean that the rest shouldn't count. If you want to evaluate goalies on their playoff or international play, then first of all be aware of the sample size issues. Secondly, evaluate them based on their individual success, not their team success. And finally, be sure to take their entire performance into account, not just the very best or the very worst.
Subscribe to:
Posts (Atom)
