Monday, November 25, 2013
Thursday, October 17, 2013
Back in the Blogging Business (But With a Change of Address)
It's been quite a while since I posted in this space, and the main reason for that is that I was one of the fans who saw their interest in professional hockey really fade in response to the lockout last fall. That disinterest persisted for some time, I didn't end up watching a lot of games for much of the season and the chaotic, shortened 48-game schedule made it pretty tough to figure out whether we were seeing meaningful results or variance, especially with a lot of the top-tier goalies having down years (the list of guys who came in with save percentages below league average in 2012-13 includes Quick, Rinne, Luongo, Price, Ward, and Backstrom).
Despite all that, I happen to be a Montreal Canadiens fan, and the chance to watch a surprisingly good team on the upswing was mainly what pulled me back in. Unfortunately, it was more or less just in time to suffer through one of the most comprehensively frustrating playoff series I've ever witnessed, but c'est la vie, and at least my work here has made me as aware as anyone of just how fickle goaltending results can be over small sample sizes.
Anyway, in the interim I have been working on a few other things, one of which was joining in on the HFBoards' History of Hockey Top 40 Goaltenders project, a fun exercise that helped me fill in some of the historical knowledge I wasn't quite up to speed on and dig into a bunch of new numbers. Here's the link to my final voting record. You might notice that #7 happens to be the namesake of this blog, which brings us to the next order of business: Moving over to contrariangoaltender.blogspot.com.
There are several reasons for this:
2. It was far more interesting to put a microscope on the abilities of a 34-year old Vezina winner than a 41-year old whose best days are clearly behind him.
3. I suspect the trade-off in terms of clickbait vs. perceived loss of credibility has changed to the point where I'd rather not keep it. Plus I killed my audience by taking a year off anyway so what's the downside?
In short, if you haven't read the entire record of my mini-obsession with Martin Brodeur to trace the evolving perspective of his career in this space, there is good reason to believe that a basic save percentage analysis underrates him to the point that he does deserve to considered one of the top 6-8 goalies ever. This is because Brodeur adds value in terms of non-save skills (I believe primarily through puckhandling and keeping the play going to reduce faceoffs in his own zone), and because his home town scorekeeper cost him several points on his save percentage through undercounting shots relative to other rinks around the league. Goaltending is about finding small edges that add up over time, and once those two things are factored in, Brodeur's initially good-but-not-necessarily-elite save percentage record looks a lot more impressive.
There's also the fact that starting with the month I started blogging here in January 2007, Brodeur managed to record up a .917 save percentage on his next ~7000 shots against, beating league average by .008 while playing for a team that was still well-disciplined and effective defensively but no longer the completely airtight lockdown unit it had been during the Cup years. And that's not even taking into account the probability that number was actually understated because of shot prevention and the stingy scorer in New Jersey. As far as counterarguments go, that was a pretty strong one. And to round it off, all this came in Brodeur's age 34-37 seasons, at a time when his career should have been expected to be on a downswing, yet he still managed to actually beat league average by a higher margin than he did from ages 26-33 (.911, +.005 vs. league average).
I think there is still some uncertainty as to the exact nature of EV shot quality effects on the Devils throughout Brodeur's career, particularly given his statistical drop-off in the early '00s during a more open team style of play and the numbers that some of the team's mediocre backups were able to record (albeit usually in small sample sizes against weak opposition). It's also very difficult to tease out exactly how much value the best puckhandling goalies contribute relative to their peers. Despite the extensive focus on his statistical record here and in other places, I think Brodeur remains an interesting and unusual analytics puzzle. For now, though, as he enters what could be his last NHL season at the age of 41 with his heir apparent already on the Devils roster, I simply have to give the guy his due.
My archive will still be up here, and I'm hoping to revisit some of my favourite old posts with updated numbers to reflect more recent seasons. Thanks to everybody who has read my writing here or at Hockey Prospectus, and if there's anyone still hanging around I hope you will consider checking me out at the new address.
Anyway, in the interim I have been working on a few other things, one of which was joining in on the HFBoards' History of Hockey Top 40 Goaltenders project, a fun exercise that helped me fill in some of the historical knowledge I wasn't quite up to speed on and dig into a bunch of new numbers. Here's the link to my final voting record. You might notice that #7 happens to be the namesake of this blog, which brings us to the next order of business: Moving over to contrariangoaltender.blogspot.com.
There are several reasons for this:
1. Martin Brodeur is one of the greatest goaltenders of all-time. He may be a overrated in some circles, particularly among members of the mainstream media, but he is certainly not a fraud.
3. I suspect the trade-off in terms of clickbait vs. perceived loss of credibility has changed to the point where I'd rather not keep it. Plus I killed my audience by taking a year off anyway so what's the downside?
In short, if you haven't read the entire record of my mini-obsession with Martin Brodeur to trace the evolving perspective of his career in this space, there is good reason to believe that a basic save percentage analysis underrates him to the point that he does deserve to considered one of the top 6-8 goalies ever. This is because Brodeur adds value in terms of non-save skills (I believe primarily through puckhandling and keeping the play going to reduce faceoffs in his own zone), and because his home town scorekeeper cost him several points on his save percentage through undercounting shots relative to other rinks around the league. Goaltending is about finding small edges that add up over time, and once those two things are factored in, Brodeur's initially good-but-not-necessarily-elite save percentage record looks a lot more impressive.
There's also the fact that starting with the month I started blogging here in January 2007, Brodeur managed to record up a .917 save percentage on his next ~7000 shots against, beating league average by .008 while playing for a team that was still well-disciplined and effective defensively but no longer the completely airtight lockdown unit it had been during the Cup years. And that's not even taking into account the probability that number was actually understated because of shot prevention and the stingy scorer in New Jersey. As far as counterarguments go, that was a pretty strong one. And to round it off, all this came in Brodeur's age 34-37 seasons, at a time when his career should have been expected to be on a downswing, yet he still managed to actually beat league average by a higher margin than he did from ages 26-33 (.911, +.005 vs. league average).
I think there is still some uncertainty as to the exact nature of EV shot quality effects on the Devils throughout Brodeur's career, particularly given his statistical drop-off in the early '00s during a more open team style of play and the numbers that some of the team's mediocre backups were able to record (albeit usually in small sample sizes against weak opposition). It's also very difficult to tease out exactly how much value the best puckhandling goalies contribute relative to their peers. Despite the extensive focus on his statistical record here and in other places, I think Brodeur remains an interesting and unusual analytics puzzle. For now, though, as he enters what could be his last NHL season at the age of 41 with his heir apparent already on the Devils roster, I simply have to give the guy his due.
My archive will still be up here, and I'm hoping to revisit some of my favourite old posts with updated numbers to reflect more recent seasons. Thanks to everybody who has read my writing here or at Hockey Prospectus, and if there's anyone still hanging around I hope you will consider checking me out at the new address.
Friday, July 6, 2012
The Value of Subjective Evaluation
I've written many times about how I do not put a high degree of emphasis on subjective evaluations of goaltenders because there are extraneous and often subconscious factors that can impact how a performance is judged. Even on the individual game level, where the task should presumably be at its very easiest, luck and the performance of the goalie's teammates still have a big impact on how they are rated.
TSN analyst and former NHL goaltender Jamie McLennan scored every goaltending performance in the 2012 Stanley Cup playoffs based on a subjective scale from 1-5, which gives an interesting point of comparison between what the numbers show in terms of saves and goals against and what an informed expert concludes from their individual judgment. I do not question McLennan's scouting ability or his knowledge of goaltending, but the data set he has provided has some interesting properties that force me to question how much value his analysis actually adds.
Beyond those two, comparing the save numbers to the game scores indicates that McLennan thought that Marc-Andre Fleury, Cory Schneider, Jimmy Howard and Martin Brodeur were all a bit better than their numbers suggest, while Tim Thomas, Jose Theodore and Corey Crawford were all a bit worse. That makes sense for Fleury, awful performances do tend to disproportionately drag down a goalie's overall averages, as well as Brodeur who is noted for his non-save skills (although McLennan also openly admitted to giving the veteran a few sympathy marks for his play in the Stanley Cup Finals).
I would subjectively agree that Thomas and Crawford may both have been a bit worse than their numbers suggest, particularly if you factor in situational leverage and their impact on win probability. A previous post of mine looked at save percentage by game score, and Thomas and Crawford were certainly outplayed by their opposite numbers when the game was close. How much of that is randomness and how much of that was each goalie's fault is an open question, but I'm not surprised that someone subjectively rating goalies would take their situational performance into account to some extent. Still, if I turn the analysis around to predict save percentage based on game score, McLennan was subjectively downgrading both goalies by only about .006-.008, not a very significant change at all for a 150-200 shot sample.
Imagine two individuals, one who didn't watch a second of playoff hockey but was armed with detailed stat sheets showing the save efficiency of every goalie, and one who also didn't watch a second of playoff hockey but still somehow faithfully tuned in every night to hear Jamie McLennan's Post2Post segment on TSN. Which one would be in better position to rate the goaltenders in the 2012 playoffs? I think it probably would be the McLennan fan, but their advantage would be very slight as the two would still agree on almost everything.
Over and over, analysis tends to show that the subjective factors often hyped by broadcasters and hockey insiders (shot quality, clutch saves, etc.) really do not have that much of an impact at the end of the day. That's not always the most intuitive finding for us narrative-obsessed sports fans, but it's hard to argue with the evidence. I think we should be careful to completely dismiss potential factors that over time could have a significant impact, particularly at the career level, but given that nearly every observer seems to vastly overrate the value of goaltending contributions that are not encapsulated in save percentage I think it reinforces the idea that a goalie's save rate should remain the primary and most trusted method of evaluation. Subjective opinions from people who know what they are talking about should not be disregarded, but they should at least be treated with some level of skepticism and compared with the statistical record to test their validity before given much weighting in the final evaluation.
TSN analyst and former NHL goaltender Jamie McLennan scored every goaltending performance in the 2012 Stanley Cup playoffs based on a subjective scale from 1-5, which gives an interesting point of comparison between what the numbers show in terms of saves and goals against and what an informed expert concludes from their individual judgment. I do not question McLennan's scouting ability or his knowledge of goaltending, but the data set he has provided has some interesting properties that force me to question how much value his analysis actually adds.
- In 86 playoff games, McLennan never gave the losing goalie a better score than the winning goalie. He gave the two the same score only 6 times, meaning that 93% of the time the goalie on the winning team was judged to have played a better game. Eleven times the two goalies had a save percentage within .005, suggesting there was likely very little difference in their play, yet the winning goalie routinely received a higher degree of recognition. When you factor in score effects that typically end up inflating the winning goalie's numbers slightly, it is likely that the losing goalie often managed to match or outperform the winner, yet they never got a better score and only rarely were even graded on the same level as their counterpart
- The correlation between a goalie's average game score and his overall save percentage was 0.924, which is an extremely high degree of correlation
- Goalies received a 5 every single time they had a save percentage of .970 or better, regardless of how many shots they faced
- If you compare the save percentage rankings with the game score rankings, only 4 out of the 17 goalies with at least 3 games played in the 2012 playoffs have a ranking differential of more than two (Niemi, Howard, Brodeur and Holtby)
- Using regression, each goalie's average game score can be predicted from their overall save percentage. Every goalie's predicted average score was within 0.25 of their actual average game score, with the exception of Jon Quick, who was 0.37 higher, and Antti Niemi, who came in a whopping 0.67 lower
Beyond those two, comparing the save numbers to the game scores indicates that McLennan thought that Marc-Andre Fleury, Cory Schneider, Jimmy Howard and Martin Brodeur were all a bit better than their numbers suggest, while Tim Thomas, Jose Theodore and Corey Crawford were all a bit worse. That makes sense for Fleury, awful performances do tend to disproportionately drag down a goalie's overall averages, as well as Brodeur who is noted for his non-save skills (although McLennan also openly admitted to giving the veteran a few sympathy marks for his play in the Stanley Cup Finals).
I would subjectively agree that Thomas and Crawford may both have been a bit worse than their numbers suggest, particularly if you factor in situational leverage and their impact on win probability. A previous post of mine looked at save percentage by game score, and Thomas and Crawford were certainly outplayed by their opposite numbers when the game was close. How much of that is randomness and how much of that was each goalie's fault is an open question, but I'm not surprised that someone subjectively rating goalies would take their situational performance into account to some extent. Still, if I turn the analysis around to predict save percentage based on game score, McLennan was subjectively downgrading both goalies by only about .006-.008, not a very significant change at all for a 150-200 shot sample.
Imagine two individuals, one who didn't watch a second of playoff hockey but was armed with detailed stat sheets showing the save efficiency of every goalie, and one who also didn't watch a second of playoff hockey but still somehow faithfully tuned in every night to hear Jamie McLennan's Post2Post segment on TSN. Which one would be in better position to rate the goaltenders in the 2012 playoffs? I think it probably would be the McLennan fan, but their advantage would be very slight as the two would still agree on almost everything.
Over and over, analysis tends to show that the subjective factors often hyped by broadcasters and hockey insiders (shot quality, clutch saves, etc.) really do not have that much of an impact at the end of the day. That's not always the most intuitive finding for us narrative-obsessed sports fans, but it's hard to argue with the evidence. I think we should be careful to completely dismiss potential factors that over time could have a significant impact, particularly at the career level, but given that nearly every observer seems to vastly overrate the value of goaltending contributions that are not encapsulated in save percentage I think it reinforces the idea that a goalie's save rate should remain the primary and most trusted method of evaluation. Subjective opinions from people who know what they are talking about should not be disregarded, but they should at least be treated with some level of skepticism and compared with the statistical record to test their validity before given much weighting in the final evaluation.
Thursday, June 28, 2012
Why Couldn't Mats Sundin Score on the Power Play?
Mats Sundin was named to the Hall of Fame in his first year of eligibility, creating some controversy as many thought he was not fully deserving of that honour. Some observers seemed especially miffed that Sundin was inducted ahead of Brendan Shanahan.
I don't really see the injustice there to be honest as I think Sundin is fully deserving of the Hall. If you gave me the choice of Sundin or Shanahan I would have taken Sundin at virtually any point during their careers. Sundin's record of consistent production is pretty strong, and I don't particularly care about his lack of team success or failure to win any major trophies. His international scoring record, in particular, is fantastic (18 goals, 21 assists for 39 points in 30 games played in best-on-best tournaments, plus 18-26-44 in 35 games in his world championships career).
Many have wondered why Sundin was unable to duplicate the same excellent results in the NHL, particularly in the playoffs. North American observers are sometimes quick to resort to the traditional explanation that European players are more motivated to perform well in international tournaments than in Stanley Cup postseason games, but I'm pretty skeptical of that being a major factor. I think a better explanation is that Mats Sundin was one of the best 5 on 5 forwards in the world and was playing with better linemates for Tre Kronor than he was in Toronto.
Sundin has a very impressive record of even strength scoring. From 1996-97 to 2001-02, he finished in the top 20 in the league in even strength scoring in six consecutive seasons while amassing 338 points at even strength over that span, the second-best total in the league behind only Jaromir Jagr. When you take into account Sundin's usual lack of top linemates and coach Pat Quinn's favoured strategy of rolling four lines that had the byproduct of reducing the available ice time for his #1 centre relative to other stars around the league, Sundin's scoring rates are even more impressive.
Here are the even strength and power play scoring rates during the regular season and playoffs for 8 of the top centers in the league from 1997-98 to 2003-04 (plus Brendan Shanahan, given all the recent discussion of whether he was more deserving than Sundin):
Sundin ranks second to only Forsberg in both the regular season and the playoffs in his rate of even strength point production. Sundin also had the best even strength goalscoring rate in the regular season as well as the fourth-best pace in the playoffs. However, the former Leaf captain ranks dead last in power play scoring in the regular season, and only Lindros (who had all of 7 playoff games played during this period) ranks below him in playoff power play scoring.
I checked the participation rates for the centers (percentage of team goals while a player was on the ice on which they recorded either a goal or an assist) to see if there were any major discrepancies:
Forsberg: 86.4% EV, 69.8% PP
Sundin: 83.9% EV, 64.4% PP
Sakic: 87.1% EV, 66.1% PP
Lindros: 81.5% EV, 64.4% PP
Turgeon: 85.7% EV, 70.0% PP
Modano: 83.8% EV, 64.7% PP
Yashin: 83.3% EV, 74.7% PP
Yzerman: 78.5% EV, 59.3% PP
I don't really see the injustice there to be honest as I think Sundin is fully deserving of the Hall. If you gave me the choice of Sundin or Shanahan I would have taken Sundin at virtually any point during their careers. Sundin's record of consistent production is pretty strong, and I don't particularly care about his lack of team success or failure to win any major trophies. His international scoring record, in particular, is fantastic (18 goals, 21 assists for 39 points in 30 games played in best-on-best tournaments, plus 18-26-44 in 35 games in his world championships career).
Many have wondered why Sundin was unable to duplicate the same excellent results in the NHL, particularly in the playoffs. North American observers are sometimes quick to resort to the traditional explanation that European players are more motivated to perform well in international tournaments than in Stanley Cup postseason games, but I'm pretty skeptical of that being a major factor. I think a better explanation is that Mats Sundin was one of the best 5 on 5 forwards in the world and was playing with better linemates for Tre Kronor than he was in Toronto.
Sundin has a very impressive record of even strength scoring. From 1996-97 to 2001-02, he finished in the top 20 in the league in even strength scoring in six consecutive seasons while amassing 338 points at even strength over that span, the second-best total in the league behind only Jaromir Jagr. When you take into account Sundin's usual lack of top linemates and coach Pat Quinn's favoured strategy of rolling four lines that had the byproduct of reducing the available ice time for his #1 centre relative to other stars around the league, Sundin's scoring rates are even more impressive.
Here are the even strength and power play scoring rates during the regular season and playoffs for 8 of the top centers in the league from 1997-98 to 2003-04 (plus Brendan Shanahan, given all the recent discussion of whether he was more deserving than Sundin):
| Player | RS EV | RS PP | PO EV | PO PP |
| Forsberg | 2.94 | 6.36 | 3.11 | 5.43 |
| Sundin | 2.66 | 4.04 | 2.35 | 3.23 |
| Sakic | 2.64 | 5.46 | 1.94 | 5.20 |
| Lindros | 2.59 | 4.27 | 1.72 | 1.72 |
| Turgeon | 2.53 | 4.96 | 2.24 | 4.94 |
| Modano | 2.36 | 4.98 | 2.05 | 4.95 |
| Yashin | 2.28 | 4.75 | 0.99 | 3.59 |
| Yzerman | 2.26 | 4.98 | 1.99 | 5.08 |
| Shanahan | 2.23 | 5.24 | 1.95 | 3.88 |
Sundin ranks second to only Forsberg in both the regular season and the playoffs in his rate of even strength point production. Sundin also had the best even strength goalscoring rate in the regular season as well as the fourth-best pace in the playoffs. However, the former Leaf captain ranks dead last in power play scoring in the regular season, and only Lindros (who had all of 7 playoff games played during this period) ranks below him in playoff power play scoring.
I checked the participation rates for the centers (percentage of team goals while a player was on the ice on which they recorded either a goal or an assist) to see if there were any major discrepancies:
Forsberg: 86.4% EV, 69.8% PP
Sundin: 83.9% EV, 64.4% PP
Sakic: 87.1% EV, 66.1% PP
Lindros: 81.5% EV, 64.4% PP
Turgeon: 85.7% EV, 70.0% PP
Modano: 83.8% EV, 64.7% PP
Yashin: 83.3% EV, 74.7% PP
Yzerman: 78.5% EV, 59.3% PP
Sundin's rates are pretty typical in both game situations. His power play rate is slightly below the group average, but is identical to that of Lindros and very close to Modano's. Yzerman's PP number is interesting, given that it is much lower than the others. To add to that, Brendan Shanahan's participation rate in the same unit was 61.4%. These numbers suggest that the Red Wings' dominant power play unit was more of a team effort than, say, the Colorado Avalanche power play which was very dependent on Sakic and Forsberg.
Was Mats Sundin a poor performer on the power play, or was he merely a victim of a poor special teams unit in Toronto? It's probably at least a bit of both. Sundin apparently wasn't good enough to singlehandedly lift his team's unit above average, but he did score 47 points with the man advantage with Quebec in 1992-93 as a member of a standout PP lineup also consisting of Sakic and Steve Duchesne. Sundin also saw his rate jump in 2002-03 to 5.1 PPP/60 after several seasons in a row in the 3s or low 4s. Over the remainder of his career Sundin never again dropped below 5.2. Maybe he finally figured out how to score as a 31-year old, or maybe adding teammates like Nolan, Mogilny and Nieuwendyk had an impact and helped boost Sundin's scoring statistics.
It is interesting to compare the power play numbers for Sundin's Maple Leafs with Brendan Shanahan's Detroit Red Wings during the same period (1998 to 2004). Putting the top 10 in power play goals for each team side by side really illustrates the difference in quality, and does seem to suggest that Sundin would have most likely been able to rack up a lot more points if he had better teammates to share the load with the man advantage.
Detroit Red Wings, Power Play Goals ('98-04):
1. Brendan Shanahan, 81
2. Steve Yzerman, 52
3. Nicklas Lidstrom, 46
4. Sergei Fedorov, 46
5. Tomas Holmstrom, 43
6. Brett Hull, 29
7. Martin Lapointe, 25
8. Igor Larionov, 21
9. Vyacheslav Kozlov, 20
10. Luc Robitaille, 16
Toronto Maple Leafs, Power Play Goals ('98-04):
1. Mats Sundin, 69
2. Gary Roberts, 28
3. Sergei Berezin, 27
4. Bryan McCabe, 22
5. Steve Thomas, 21
6. Darcy Tucker, 21
7. Jonas Hoglund, 17
8. Igor Korolev, 14
8. Alexander Mogilny, 14
10. Mikael Renberg, 13
I think Sundin was disadvantaged by team factors, particularly from 1997 to 2002, which also happens to be his peak period of even strength scoring. As the team's best player he should shoulder some of the blame for Toronto being so mediocre with the man advantage, but results from earlier and later in his career show that when Sundin did have the good fortune to play together with star linemates then he was able to post better power play scoring numbers. Pumping up his PP scoring stats could have moved Sundin from the 75-85 point range to a consistent 90+, which would have made him a more significant factor in the overall scoring race and in turn would have seen him viewed in a much more positive light today.
One final stat to compare Sundin and Shanahan: In the 18 seasons where both of them played in the NHL, Shanahan scored more even strength points than Sundin only twice. I'd take Sundin over Shanny every single time, and I think he's a deserving first ballot Hall of Famer.
Was Mats Sundin a poor performer on the power play, or was he merely a victim of a poor special teams unit in Toronto? It's probably at least a bit of both. Sundin apparently wasn't good enough to singlehandedly lift his team's unit above average, but he did score 47 points with the man advantage with Quebec in 1992-93 as a member of a standout PP lineup also consisting of Sakic and Steve Duchesne. Sundin also saw his rate jump in 2002-03 to 5.1 PPP/60 after several seasons in a row in the 3s or low 4s. Over the remainder of his career Sundin never again dropped below 5.2. Maybe he finally figured out how to score as a 31-year old, or maybe adding teammates like Nolan, Mogilny and Nieuwendyk had an impact and helped boost Sundin's scoring statistics.
It is interesting to compare the power play numbers for Sundin's Maple Leafs with Brendan Shanahan's Detroit Red Wings during the same period (1998 to 2004). Putting the top 10 in power play goals for each team side by side really illustrates the difference in quality, and does seem to suggest that Sundin would have most likely been able to rack up a lot more points if he had better teammates to share the load with the man advantage.
Detroit Red Wings, Power Play Goals ('98-04):
1. Brendan Shanahan, 81
2. Steve Yzerman, 52
3. Nicklas Lidstrom, 46
4. Sergei Fedorov, 46
5. Tomas Holmstrom, 43
6. Brett Hull, 29
7. Martin Lapointe, 25
8. Igor Larionov, 21
9. Vyacheslav Kozlov, 20
10. Luc Robitaille, 16
Toronto Maple Leafs, Power Play Goals ('98-04):
1. Mats Sundin, 69
2. Gary Roberts, 28
3. Sergei Berezin, 27
4. Bryan McCabe, 22
5. Steve Thomas, 21
6. Darcy Tucker, 21
7. Jonas Hoglund, 17
8. Igor Korolev, 14
8. Alexander Mogilny, 14
10. Mikael Renberg, 13
I think Sundin was disadvantaged by team factors, particularly from 1997 to 2002, which also happens to be his peak period of even strength scoring. As the team's best player he should shoulder some of the blame for Toronto being so mediocre with the man advantage, but results from earlier and later in his career show that when Sundin did have the good fortune to play together with star linemates then he was able to post better power play scoring numbers. Pumping up his PP scoring stats could have moved Sundin from the 75-85 point range to a consistent 90+, which would have made him a more significant factor in the overall scoring race and in turn would have seen him viewed in a much more positive light today.
One final stat to compare Sundin and Shanahan: In the 18 seasons where both of them played in the NHL, Shanahan scored more even strength points than Sundin only twice. I'd take Sundin over Shanny every single time, and I think he's a deserving first ballot Hall of Famer.
Wednesday, June 6, 2012
Darryl Sutter and Goaltending
Darryl Sutter is getting a lot of credit for his impact on the Los Angeles Kings this season, which is perhaps not too surprising given that the team was 15-14-4 prior to his hire and 40-15-11 since, playoffs included. Jon Quick is all over the headlines for his ridiculous postseason numbers, and certainly much of that is deserved, but there is also some historical evidence that suggests Sutter may be in the class of coaches that have a positive impact on the statistics of their goaltenders.
Throughout his career, Sutter's goaltenders have routinely been above average. Sutter-led teams have only posted a below-average save percentage in two out of his dozen seasons as an NHL coach, and in every one of the remaining ten his team was at least .006 above the league benchmark in save rate:
Throughout his career, Sutter's goaltenders have routinely been above average. Sutter-led teams have only posted a below-average save percentage in two out of his dozen seasons as an NHL coach, and in every one of the remaining ten his team was at least .006 above the league benchmark in save rate:
1993-94: .902 in Chicago (.895 avg)
1994-95: .907 in Chicago (.901 avg)
1997-98: .896 in San Jose (.906 avg)
1998-99: .915 in San Jose (.908 avg)
1999-00: .911 in San Jose (.904 avg)
2000-01: .914 in San Jose (.903 avg)
2001-02: .918 in San Jose (.908 avg)
2002-03: .897 in San Jose/Calgary (.909 avg)
2003-04: .919 in Calgary (.911 avg)
2005-06: .917 in Calgary (.901 avg)
2011-12: .930 in Los Angeles (.914 avg)
Overall: .910 under Sutter, .903 league average
Sutter had Belfour for three seasons, Kiprusoff for two and Quick this past year, so he was partially lucky to benefit from some good goaltending. However, the numbers before and after he arrived in the different towns seem to suggest that there was a consistent save percentage effect as a result of Sutter's hiring. I looked at the full season prior to Sutter being hired and the full season after he was fired, with partial in-season results before he was hired/after he was fired also included in the before and after sample:
Chicago (before & after): 4352 SA, .898 save %, .893 average, +.005
Chicago under Sutter: 5940 SA, .903 save %, .892 average, +.011
San Jose (before & after): 6434 SA, .906 save %, .908 average, -.002
San Jose under Sutter: 11781 SA, .910 save %, .906 average, +.004
Calgary (before & after): 5738 SA, .909 save %, .907 average, +.002
Calgary under Sutter: 5545 SA, .914 save %, .906 average, +.008
Los Angeles (before): 3258 SA, .917 save %, .913 average, +.004
Los Angeles under Sutter: 1271 SA, .930 save %, .914 average, +.016
That's a consistent bump of .006 at each of Sutter's first three stops, with his L.A. numbers looking even better so far.
I should point out however that selection bias probably has an impact here, given that teams with low save percentages would be more likely to fire coaches and hire replacements.
Combined before sample: .901 save percentage, .905 league average, -.004
Combined sample with Sutter: .910 save percentage, .903 league average, +.007
Combined after sample: .914 save percentage, .906 league average, +.008
Looking at it this way gives a potentially much less charitable interpretation of Sutter's true impact: Maybe he simply coached some good goalies and was the beneficiary of teams bouncing back from poor seasons. On the other hand, it does seem reasonable that the impact of a coach would continue in at least some fashion even after they leave town. If Sutter played a key role in developing young players or established a style of play that the team continued to use under his successor then he could be partially credited for some of those continuing effects.
It is always important to take regression to the mean into account when evaluating coaches, or else guys hired by underachieving teams will look like heroes nearly every time as the team's results gradually change to more closely match their overall level of talent. Sutter is not a miracle worker, and I think that Jack Johnson for Jeff Carter trade had a larger impact on the Kings' amazing transformation into a playoff juggernaut than their mid-season coaching change. On the other hand, Los Angeles is certainly doing a great job of protecting their goaltender through great defensive play at the moment, and that reflects well on their head coach.
In the competitive world of the NHL, slight edges can sometimes end up being important, and Sutter's coaching may just be providing Jonathan Quick the defensive advantage needed to help the Kings' goaltender complete his memorable season with a great postseason run that currently has him holding the highest official playoff save percentage of all-time among goalies that participated in more than one playoff round.
I should point out however that selection bias probably has an impact here, given that teams with low save percentages would be more likely to fire coaches and hire replacements.
Combined before sample: .901 save percentage, .905 league average, -.004
Combined sample with Sutter: .910 save percentage, .903 league average, +.007
Combined after sample: .914 save percentage, .906 league average, +.008
Looking at it this way gives a potentially much less charitable interpretation of Sutter's true impact: Maybe he simply coached some good goalies and was the beneficiary of teams bouncing back from poor seasons. On the other hand, it does seem reasonable that the impact of a coach would continue in at least some fashion even after they leave town. If Sutter played a key role in developing young players or established a style of play that the team continued to use under his successor then he could be partially credited for some of those continuing effects.
It is always important to take regression to the mean into account when evaluating coaches, or else guys hired by underachieving teams will look like heroes nearly every time as the team's results gradually change to more closely match their overall level of talent. Sutter is not a miracle worker, and I think that Jack Johnson for Jeff Carter trade had a larger impact on the Kings' amazing transformation into a playoff juggernaut than their mid-season coaching change. On the other hand, Los Angeles is certainly doing a great job of protecting their goaltender through great defensive play at the moment, and that reflects well on their head coach.
In the competitive world of the NHL, slight edges can sometimes end up being important, and Sutter's coaching may just be providing Jonathan Quick the defensive advantage needed to help the Kings' goaltender complete his memorable season with a great postseason run that currently has him holding the highest official playoff save percentage of all-time among goalies that participated in more than one playoff round.
Monday, May 28, 2012
Shootout Records Don't Predict Playoff Success
There was an interesting statistical pattern this year in the first round of the playoffs: Only one of the goalies on a winning team had a better shootout win/loss record this year than his counterpart on the losing team.
Boston vs. Washington: Thomas 7-1, Holtby 0-1, Caps win
Florida vs. New Jersey: Theodore 4-7, Brodeur 7-2, Devils win
Pittsburgh vs. Philadelphia: Fleury 9-2, Bryzgalov 3-5, Flyers win
Vancouver vs. Los Angeles: Luongo 6-6 (Schneider 2-1), Quick 6-8, Kings win
St. Louis vs. San Jose: Halak 3-7 (Eliott 1-3), Niemi 8-4, Blues win
Phoenix vs. Chicago: Smith 6-8, Crawford 6-4, Coyotes win
Nashville vs. Detroit: Rinne 4-5, Howard 7-2, Predators win
Combined totals:
Winning goalies: 34-42 in shootouts this year
Losing goalies: 55-28 in shootouts this year
This result was surely largely influenced by randomness, especially given that the trend somewhat reversed itself in subsequent rounds with Brodeur and the Devils continuing to win. However, some matchups probably looked closer than they were in the standings because the weaker team had the benefit of a regular season shootout edge. The Flyers, for example, would have had a better record than the Penguins if all shootouts were counted as ties (although the Penguins' goal differential advantage indicates that Pittsburgh probably still should have been a slight favourite in that series).
The shootout looks like it is here to stay in the NHL, but it does still seem unfair that it has an impact on the regular season standings that are used to determine playoff seedings. When comparing two teams to make predictions for a playoff series, shootout results should obviously be disregarded. That said, regular season records aren't even the best way to predict which playoff teams to bet on anyway (as the Kings and Devils are currently demonstrating), with metrics such as score-tied Fenwick and goal differential showing more predictive power anyway.
This result was surely largely influenced by randomness, especially given that the trend somewhat reversed itself in subsequent rounds with Brodeur and the Devils continuing to win. However, some matchups probably looked closer than they were in the standings because the weaker team had the benefit of a regular season shootout edge. The Flyers, for example, would have had a better record than the Penguins if all shootouts were counted as ties (although the Penguins' goal differential advantage indicates that Pittsburgh probably still should have been a slight favourite in that series).
The shootout looks like it is here to stay in the NHL, but it does still seem unfair that it has an impact on the regular season standings that are used to determine playoff seedings. When comparing two teams to make predictions for a playoff series, shootout results should obviously be disregarded. That said, regular season records aren't even the best way to predict which playoff teams to bet on anyway (as the Kings and Devils are currently demonstrating), with metrics such as score-tied Fenwick and goal differential showing more predictive power anyway.
Tuesday, May 22, 2012
Mike Smith: Good Season or Good Goalie?
Case A: 2011-12 Was A Fluke Season
The fact that goalie results are heavily influenced by randomness and have a lot of variance from year-to-year has been clearly established by statistical analysis. It would be very unlikely for an ordinary starting goalie to hit .930 just by luck, but it is not beyond the realm of possibility. If the goalie is perhaps slightly above-average and aided by some relevant team factors then that explanation becomes a lot more compelling. The sudden massive improvement in numbers for Mike Smith makes it reasonable to question where his true talent lies. After all, it is not very often that a 30-year old goalie is able to bump up his career save percentage by .008 in the course of a single season.
The most commonly advanced argument by those seeking to downgrade Mike Smith is the team argument that claims that Dave Tippett is a defensive genius that helps all goaltenders succeed in Phoenix. It's generally a good idea to be skeptical of shot quality arguments, but on the other hand there are three goalies now that have superior numbers under Tippett then they managed everywhere else since the lockout (regular season and playoffs combined):
Ilya Bryzgalov:
Without Tippett: 7517 SA, .911
With Tippett: 4481 SA, .919
Jason LaBarbera:
Without Tippett: 2703 SA, .905
With Tippett: 1474 SA, .916
Mike Smith:
Without Tippett: 4430 SA, .907
With Tippett: 2617 SA, .934
It is probably also worth noting that Smith played in the Pacific Division, a division that had a higher average save percentage than any other division in the league. Part of that is surely because the Pacific has some great, top-flight goalies, but the league-leading number was also likely influenced at least to some degree by the tight style of play that is more typical for teams on the West Coast.
In the playoffs and regular season combined this year, Smith has faced just 15% of his shots against on the penalty kill. Throughout the rest of his career his average was 19% (although part of the decrease was because penalties were down in 2011-12). Smith also posted a .909 save percentage on the PK, which vastly surpasses his previous career PK rate of .874. This year was only the second time in his career that Smith's PK rate even went above .880.
If Smith had faced 19% of his shots against on the PK with a PK save percentage of .880, his seasonal save percentage would have dropped .006 to .924. Assuming that Phoenix has some additional shot quality effect at even strength relative to the rest of the league, he may only have played at a level of around .920 in a neutral team situation. Obviously Smith had a terrific campaign, but perhaps those potential team factors caused some skeptical GMs around the league to drop him out of the top three in Vezina voting, even though his numbers suggest he probably deserved that ranking (Smith led the league in GVT, a measure that historically tracks very well with Vezina winners).
One large negative indicator for Mike Smith heading into this season is that he had an extreme home/road split. Prior to this year he had a career save percentage of .918 at home compared to just .896 on the road. That's usually not a good sign, given that road save percentages are subject to less potential scorer bias since they are spread across a number of different arenas. This year, Smith's degree of improvement on the road (+.030 to .926) was almost twice as much as the increase in his home save percentage (+.016 to .934). It is perhaps interesting to note that both LaBarbera (.922 on the road) and Bryzgalov (.921) have excellent road numbers under Tippett in Phoenix (both actually did better away from home than they did in Glendale). Phoenix has also finished in the top eight in road record in each of the past three seasons, despite finishing 16th in points at home this season for the second year in a row.
To summarize, Smith's track record doesn't support anything close to a season of 67 GP at .930, and there is evidence that suggests Phoenix may be a place that helps its goalies out a bit. As a result, Smith is probably headed for a major regression in his numbers for 2012-13.
Case B: Mike Smith Is A Good Goalie
Smith has been very impressive in the playoffs, causing some to argue that his game has developed and he is in the process of joining the game's goaltending elite. The caveat is that subjective arguments are affected by all kinds of observer biases, and sometimes it is hard to differentiate between a good goalie and a lucky goalie or a hot goalie, but Smith's numbers have been so outstanding that it is likely they do represent a significant improvement, even taking into account some of the other factors that may be in play.
One of the biggest question marks in evaluating Mike Smith is determining how much of an impact the concussion he suffered late in 2008 had on his career development. It's certainly possible to make the numbers fit a narrative that portrays Smith as a promising goalie early in his career before he was brought low by a concussion, and that it shouldn't be that surprising that Smith has blossomed into a quality NHL starter now that he has fully recovered.
Smith was concussed in December 2008. His career save percentage as of December 31, 2008 was .913, which is very strong taking into account the league average of .907 from 2007 to 2009. Smith's 2008-09 season on a really bad Tampa team looks quite impressive in retrospect. The Lightning fired Barry Melrose in mid-season, ended up second last overall, and went 10-22-9 with a 3.62 GAA with their backup goalies in the net (Smith himself was 14-18-9 and 2.62). Unfortunately Smith's campaign was cut short by his concussion symptoms.
How much did the effects of his head injury affect Smith in 2009-10 and 2010-11? It's difficult to tell, particularly because he faced a relatively small sample size of 1600 shots. In 2010-11 he would likely have been much closer to full recovery, but rarely played in a backup role behind Dwayne Roloson. Smith's combined numbers for those two seasons, together with January 2009 when he was playing with concussion symptoms before getting shut down for the year, come to .899 on 2096 SA, which is well below where he was prior to the injury.
Smith have simply have been somewhat lucky early in his career and then unlucky later on, we've seen that happen many times over a couple of 2000 shot samples. However, it is also entirely possible that his injury had a major impact on his results and that he has had above-average talent since the start of his career.
It has been widely reported that Phoenix goalie coach Sean Burke wanted to bring in Mike Smith because Burke saw potential there. Maybe Burke was correct that Smith was a star in waiting, although a cautionary note is that those are the types of things that are easy to say in hindsight if you want to make a specific talent evaluator or coach look great, mainly because they don't account for any of the misses (if Smith didn't work out then nobody would be writing stories about how Sean Burke got it wrong).
Smith's two-year, $4 million deal doesn't exactly suggest that Phoenix thought they were going to be getting Vezina-calibre goaltending, but at the same time with LaBarbera in the backup role and no real other options on the farm the Coyotes were clearly betting their season on Smith. If they didn't think he was going to be at least average it is questionable whether they would have made that move. Goalies don't always follow linear career paths, and maybe the change of scenery combined with coaching and his personal development helped Smith make the jump and join the games' elite.
The fact that goalie results are heavily influenced by randomness and have a lot of variance from year-to-year has been clearly established by statistical analysis. It would be very unlikely for an ordinary starting goalie to hit .930 just by luck, but it is not beyond the realm of possibility. If the goalie is perhaps slightly above-average and aided by some relevant team factors then that explanation becomes a lot more compelling. The sudden massive improvement in numbers for Mike Smith makes it reasonable to question where his true talent lies. After all, it is not very often that a 30-year old goalie is able to bump up his career save percentage by .008 in the course of a single season.
The most commonly advanced argument by those seeking to downgrade Mike Smith is the team argument that claims that Dave Tippett is a defensive genius that helps all goaltenders succeed in Phoenix. It's generally a good idea to be skeptical of shot quality arguments, but on the other hand there are three goalies now that have superior numbers under Tippett then they managed everywhere else since the lockout (regular season and playoffs combined):
Ilya Bryzgalov:
Without Tippett: 7517 SA, .911
With Tippett: 4481 SA, .919
Jason LaBarbera:
Without Tippett: 2703 SA, .905
With Tippett: 1474 SA, .916
Mike Smith:
Without Tippett: 4430 SA, .907
With Tippett: 2617 SA, .934
It is probably also worth noting that Smith played in the Pacific Division, a division that had a higher average save percentage than any other division in the league. Part of that is surely because the Pacific has some great, top-flight goalies, but the league-leading number was also likely influenced at least to some degree by the tight style of play that is more typical for teams on the West Coast.
In the playoffs and regular season combined this year, Smith has faced just 15% of his shots against on the penalty kill. Throughout the rest of his career his average was 19% (although part of the decrease was because penalties were down in 2011-12). Smith also posted a .909 save percentage on the PK, which vastly surpasses his previous career PK rate of .874. This year was only the second time in his career that Smith's PK rate even went above .880.
If Smith had faced 19% of his shots against on the PK with a PK save percentage of .880, his seasonal save percentage would have dropped .006 to .924. Assuming that Phoenix has some additional shot quality effect at even strength relative to the rest of the league, he may only have played at a level of around .920 in a neutral team situation. Obviously Smith had a terrific campaign, but perhaps those potential team factors caused some skeptical GMs around the league to drop him out of the top three in Vezina voting, even though his numbers suggest he probably deserved that ranking (Smith led the league in GVT, a measure that historically tracks very well with Vezina winners).
One large negative indicator for Mike Smith heading into this season is that he had an extreme home/road split. Prior to this year he had a career save percentage of .918 at home compared to just .896 on the road. That's usually not a good sign, given that road save percentages are subject to less potential scorer bias since they are spread across a number of different arenas. This year, Smith's degree of improvement on the road (+.030 to .926) was almost twice as much as the increase in his home save percentage (+.016 to .934). It is perhaps interesting to note that both LaBarbera (.922 on the road) and Bryzgalov (.921) have excellent road numbers under Tippett in Phoenix (both actually did better away from home than they did in Glendale). Phoenix has also finished in the top eight in road record in each of the past three seasons, despite finishing 16th in points at home this season for the second year in a row.
To summarize, Smith's track record doesn't support anything close to a season of 67 GP at .930, and there is evidence that suggests Phoenix may be a place that helps its goalies out a bit. As a result, Smith is probably headed for a major regression in his numbers for 2012-13.
Case B: Mike Smith Is A Good Goalie
Smith has been very impressive in the playoffs, causing some to argue that his game has developed and he is in the process of joining the game's goaltending elite. The caveat is that subjective arguments are affected by all kinds of observer biases, and sometimes it is hard to differentiate between a good goalie and a lucky goalie or a hot goalie, but Smith's numbers have been so outstanding that it is likely they do represent a significant improvement, even taking into account some of the other factors that may be in play.
One of the biggest question marks in evaluating Mike Smith is determining how much of an impact the concussion he suffered late in 2008 had on his career development. It's certainly possible to make the numbers fit a narrative that portrays Smith as a promising goalie early in his career before he was brought low by a concussion, and that it shouldn't be that surprising that Smith has blossomed into a quality NHL starter now that he has fully recovered.
Smith was concussed in December 2008. His career save percentage as of December 31, 2008 was .913, which is very strong taking into account the league average of .907 from 2007 to 2009. Smith's 2008-09 season on a really bad Tampa team looks quite impressive in retrospect. The Lightning fired Barry Melrose in mid-season, ended up second last overall, and went 10-22-9 with a 3.62 GAA with their backup goalies in the net (Smith himself was 14-18-9 and 2.62). Unfortunately Smith's campaign was cut short by his concussion symptoms.
How much did the effects of his head injury affect Smith in 2009-10 and 2010-11? It's difficult to tell, particularly because he faced a relatively small sample size of 1600 shots. In 2010-11 he would likely have been much closer to full recovery, but rarely played in a backup role behind Dwayne Roloson. Smith's combined numbers for those two seasons, together with January 2009 when he was playing with concussion symptoms before getting shut down for the year, come to .899 on 2096 SA, which is well below where he was prior to the injury.
Smith have simply have been somewhat lucky early in his career and then unlucky later on, we've seen that happen many times over a couple of 2000 shot samples. However, it is also entirely possible that his injury had a major impact on his results and that he has had above-average talent since the start of his career.
It has been widely reported that Phoenix goalie coach Sean Burke wanted to bring in Mike Smith because Burke saw potential there. Maybe Burke was correct that Smith was a star in waiting, although a cautionary note is that those are the types of things that are easy to say in hindsight if you want to make a specific talent evaluator or coach look great, mainly because they don't account for any of the misses (if Smith didn't work out then nobody would be writing stories about how Sean Burke got it wrong).
Smith's two-year, $4 million deal doesn't exactly suggest that Phoenix thought they were going to be getting Vezina-calibre goaltending, but at the same time with LaBarbera in the backup role and no real other options on the farm the Coyotes were clearly betting their season on Smith. If they didn't think he was going to be at least average it is questionable whether they would have made that move. Goalies don't always follow linear career paths, and maybe the change of scenery combined with coaching and his personal development helped Smith make the jump and join the games' elite.
Conclusion
It seems to be an absolute rock-solid bet that Mike Smith's numbers are going to regress next season, and that his true talent level doesn't quite measure up to the level of performance he has displayed this year. On the other hand, it seems clear that Smith's concussion had a negative impact on his career, and as a result his career numbers are at least somewhat understated.
There is a chance that Smith is now one of the game's best goalies, and there is a chance that this was his career season. The most likely case is that Smith is not an elite goalie who can be counted on to remain near the top of the league year after year, but he may be an above-average starter who still has a number of useful years ahead of him. Smith is probably still a good candidate to put up another season of around .920 or maybe even a little better next year, given Phoenix's team discipline and the fact that he outperformed league average by .006 over a three season stretch earlier in his career when he was likely not as good as he is right now.
There is a chance that Smith is now one of the game's best goalies, and there is a chance that this was his career season. The most likely case is that Smith is not an elite goalie who can be counted on to remain near the top of the league year after year, but he may be an above-average starter who still has a number of useful years ahead of him. Smith is probably still a good candidate to put up another season of around .920 or maybe even a little better next year, given Phoenix's team discipline and the fact that he outperformed league average by .006 over a three season stretch earlier in his career when he was likely not as good as he is right now.
Subscribe to:
Posts (Atom)
