I discussed recently how goalies are often evaluated based on a few memories. For some goalies it means they are forever remembered as winners. For others, it means that they are defined by one bad goal against. The latter group is often very underrated. For example, all our memories of ten years of league average save percentages and a previously stellar international career vanished in a flash as soon as that puck bounced in off of Tommy Salo's head.
Another example of this phenomenon is Allan Bester. Bester is remembered as a joke among many Toronto Maple Leafs fans. Take this quote off a Leaf blog, for example:
"Raycroft is the worst starting goalie I’ve seen play for the Leafs since Allan Bester. Remember him? He attempted suicide one night. No, really. He jumped in front of a bus! But it went between his legs." (Leaf Club)
This is of course a reference to the the overtime goal scored by Sergio Momesso in the 1990 playoffs at Maple Leaf Gardens. Momesso's shot beat Bester five-hole, a memory that Leaf fans will forever associate with the tiny (5'7, 155) netminder.
However, the numbers paint a very different picture of Allan Bester. According to the statistics, Bester outperformed his teammates in every league that he ever played in, and was easily the best goalie the Leafs had in the 1980s. Despite this, the Momesso goal was essentially the end of Bester's NHL career.
Bester broke into the league early as a 19 year old right out of junior, and had all of his NHL success in his early twenties. With fellow youngster Ken Wregget, Bester made up a promising goalie tandem for the Leafs in the mid-1980s. The 1988-89 season was probably the high point of Bester's career, as he finished in the top 10 in save percentage on a team that missed the playoffs. Bester was even named to Team Canada's world championship squad, although he was unable to participate because of injury.
In 1989-90 Bester had an off-year, and then he struggled in the playoffs. Everything started to unravel after that. He was displaced in the Leaf net in 1990-91 by rookie Peter Ing and then got traded to the Detroit Red Wings for a draft pick.
The Wings buried Bester in the AHL, but he played well in the minors. In 1992, the Adirondack Red Wings won the Calder Cup, and Bester was named playoff MVP. The next year he outplayed Chris Osgood, who was seen as Detroit's goalie of the future. Without an opportunity to move up in the Red Wings organization, Bester looked to catch on with one of the expansion teams. He was passed over in the expansion draft, but managed to catch on with the Anaheim Mighty Ducks as a free agent.
Unfortunately for Allan Bester, Anaheim had the best goaltending of any of the expansion teams, with the solid tandem of Guy Hebert and Ron Tugnutt. Bester was competing with Mikhail Shtalenkov for the #3 role in the organization, and despite similar IHL numbers the team decided to give the NHL opportunities to Shtalenkov. Bester continued to play in the IHL until he got one last cup of coffee at the NHL level as an injury replacement for the Dallas Stars in 1996. He acquitted himself well in a 10 game stint, but returned to the IHL where he remained until he retired in 1997-98 at the age of 33.
Having established the biography, let's look at the numbers. I compared Bester's numbers to his teammates for every season of his professional career, except when he only played a few games which would not be a representative sample.
Allan Bester solidly outplayed his teammates at the NHL level:
Bester: 4.00 GAA, .883 save %, .432 win %, 2.5 SO/70 GP
Others: 4.30 GAA, .869 save %, .370 win %, 0.5 SO/70 GP
Bester has a clear edge in every stat, including a large edge in shutouts. For goalies playing on bad teams, shutouts are often a fairly good indicator of dominance, since they aren't able to post the easy shutouts that goalies on winning teams often get. On the mid-'80s Leafs, the only way anybody was going to get a shutout was through an outstanding performance, and Bester had a very respectable 2.5 shutouts per 70 games played (I prefer to express shutout rates per 70 games because the per-game rates get pretty small). This was five times the rate of his teammates. The raw total was 7 shutouts in the equivalent of 178 full games, compared to his teammates' total of just 2 in 309.
Question to Maple Leaf fans of that era: Does one bad playoff goal make up for a winning percentage difference of .062 compared to Toronto's other goalies, the equivalent of a 5 extra wins over a full season?
In the AHL, Bester again outplayed his teammates:
Bester: .487 win %, 3.74 GAA, 1.4 SO/70 GP
Others: .456 win %, 4.05 GAA, 1.5 SO/70 GP
Bester's playing partners included Wregget, Tim Bernhardt, Rick St. Croix, Damian Rhodes, Scott King and Chris Osgood. Bester spent his prime age 27 and 28 years in the minors as a Red Wing, outplaying his teammates and being named the MVP of a Calder Cup winning team, yet only getting to play 31 minutes in the NHL.
One contributing factor to this was likely the overall improvement in league goaltending during the early- to mid-1990s. A new wave of goalies was entering the league, bringing with them the modern butterfly style and displacing many veterans who weren't able to keep up. Bester certainly wasn't the only goalie of his age that got caught up in a numbers game, but having said that he was still putting up numbers that were as good as or better than several young goalies who were headed for the NHL. It is possible that his declining performance in 1989-90 and 1990-91 showed that he no longer had the ability to compete at the NHL level, but his minor league performance suggests that it is also possible that he had an off-year and didn't get a chance to play his way back into form.
In the IHL, Bester was in his thirties but was still usually the best goalie on the team. Some of his partners were scrubs, but others were legit goalies (e.g. Shtalenkov, Essensa).
Bester: .630 win %, 3.22 GAA, 1.9 SO/70 GP
Others: .563 win %, 3.43 GAA, 1.6 SO/70 GP
Just as winning goalies shouldn't be judged based on one shining playoff run, losing goalies should not be judged based on one bad goal. Allan Bester was likely at least a league average goalie, and for a short time probably even better than that.
I'll leave you with one final stat: From 1983-84 to 1989-90, Allan Bester finished 13th in the NHL in save percentage among goalies with at least 150 GP, playing on one of the worst teams in the league. I'll post a section of the standings so we can see the goalies just above and below him:
11. Tom Barrasso, .883
12. Don Beaupre, .883
13. Allan Bester, .883
14. Mike Liut, .883
15. Billy Smith, .882
16. Grant Fuhr, .881
17. Kirk McLean, .881
18. Mike Vernon, .881
19. Pete Peeters, .881
In the 1980s those 9 goalies combined for 5 First Team All-Stars, 4 Second Team All-Stars, and 9 Stanley Cups. Every single one of them had long NHL careers. And yet nobody gave Bester another NHL shot.
I think the scouts got it wrong on this one. Bester may have been small and he may have let a few through the wickets, but the evidence suggests that he was a legitimate pro goalie and he deserved another shot at an NHL job after the age of 25. He should be remembered for what he was, the best Toronto Maple Leafs goalie of the 1980s.
Thursday, October 8, 2009
Tuesday, October 6, 2009
Shot Quality Refined
I have to throw up a link to the latest work by Ken Krzywicki on shot quality. He tackles the issue of arena reporting bias, and has updated his model to take that into account. I think there are some very good reasons to question the importance and/or the accuracy of shot quality measures, but I still think it is possible that some of the problems are a result of the data or the models rather than the theory that different teams allow different types of scoring chances. That shot quality differences are observed in the game of hockey is beyond doubt, the debate is about whether those differences are significant at the NHL level.
To me, the revised model makes more sense, because we see a smaller spread of values at the team level. The best team has an expected save percentage of .915, while the worst is at .904. Compare that to the .900 - .918 which would be predicted based on the raw numbers. Half of the teams are within .002 of league average, which matches our expectation that most teams do not have significant shot quality effects.
I would like to see the shots separated out by situation into EV and PK. I think that would allow us to better be able to test the results. However, the model must be projecting some even strength variations, since the correlation between expected save percentage and power plays against in 2008-09 was -0.03.
Some of the team results are a bit surprising. In particular, Boston is rated as having allowing the 4th most difficult shots against, which seems to strongly contradict the numbers of Thomas and Fernandez, the Bruins' low number of penalties taken, and the coach and team style of play.
Alan Ryder also has a method for evaluating shot quality in his annual NHL review. He says he has tried to make some adjustments for rink bias as well, yet some his numbers differ from Krzywicki's. One of them is in his measurement for Boston whom Ryder grades as about average, which I suspect might be a more accurate assessment.
By comparing Krzywicki's numbers with Ryder's, and also keeping in mind each team's actual save percentage as an additional check, I think there are about ten teams that we can target as possible shot quality outliers in 2008-09, with New Jersey, Phoenix, Buffalo, Minnesota and Columbus the candidates for easier than average shot quality and Toronto, Dallas, Carolina, Montreal and Calgary possibly allowing harder than average shots against.
Of those, a few could be the rest of measurement error, score effects, randomness, etc., but I think is likely that there are a few teams that must be ahead or behind the rest of the league. To me, Minnesota, New Jersey, Toronto and Dallas are the most likely candidates for 2008-09. For those four teams, the shot quality results match both the subjective perception and the actual results in the crease.
If this is correct, that would have implications for our evaluation of some goalies. That doesn't mean that goalies like Turco or Toskala can be let off the hook, however. The expected save percentage of an outlier team is still likely somewhere in the range of .003-.005 above or below average, which means that outstanding or terrible save percentages cannot be explained by the rest of the team and must fall mostly on the shoulders of the goalie that posted them.
To me, the revised model makes more sense, because we see a smaller spread of values at the team level. The best team has an expected save percentage of .915, while the worst is at .904. Compare that to the .900 - .918 which would be predicted based on the raw numbers. Half of the teams are within .002 of league average, which matches our expectation that most teams do not have significant shot quality effects.
I would like to see the shots separated out by situation into EV and PK. I think that would allow us to better be able to test the results. However, the model must be projecting some even strength variations, since the correlation between expected save percentage and power plays against in 2008-09 was -0.03.
Some of the team results are a bit surprising. In particular, Boston is rated as having allowing the 4th most difficult shots against, which seems to strongly contradict the numbers of Thomas and Fernandez, the Bruins' low number of penalties taken, and the coach and team style of play.
Alan Ryder also has a method for evaluating shot quality in his annual NHL review. He says he has tried to make some adjustments for rink bias as well, yet some his numbers differ from Krzywicki's. One of them is in his measurement for Boston whom Ryder grades as about average, which I suspect might be a more accurate assessment.
By comparing Krzywicki's numbers with Ryder's, and also keeping in mind each team's actual save percentage as an additional check, I think there are about ten teams that we can target as possible shot quality outliers in 2008-09, with New Jersey, Phoenix, Buffalo, Minnesota and Columbus the candidates for easier than average shot quality and Toronto, Dallas, Carolina, Montreal and Calgary possibly allowing harder than average shots against.
Of those, a few could be the rest of measurement error, score effects, randomness, etc., but I think is likely that there are a few teams that must be ahead or behind the rest of the league. To me, Minnesota, New Jersey, Toronto and Dallas are the most likely candidates for 2008-09. For those four teams, the shot quality results match both the subjective perception and the actual results in the crease.
If this is correct, that would have implications for our evaluation of some goalies. That doesn't mean that goalies like Turco or Toskala can be let off the hook, however. The expected save percentage of an outlier team is still likely somewhere in the range of .003-.005 above or below average, which means that outstanding or terrible save percentages cannot be explained by the rest of the team and must fall mostly on the shoulders of the goalie that posted them.
Monday, October 5, 2009
Winning Means the Rest Doesn't Count
Goalies are rated far too much based on their team success. One reason for this is that we all have limited memories, which causes certain moments to be more easily recalled while the rest is forgotten. For some goalies that's a good thing, because we can clearly recall them doing things like stopping Pavel Bure on a penalty shot in the Finals or making a save with 2 seconds left in game 7 to help their team win the Cup. For others we remember only the weak moments that they would rather forget. As a result, goalies who have not experienced team success often see a few of their worst performances get scrutinized to the near-exclusion of all else, while similar performances by other goalies get ignored because they have a ring.
Patrick Roy was a terrific playoff goalie, but he had his fair share of weak performances. If you play 17 playoff seasons you are naturally going to have a few that are forgettable, of course, but some of Roy's flops were actually quite spectacular (losing his job in 1987 and 1988, his problems against the Bourque/Neely Bruins, a few first-round upsets against much weaker opponents, and of course the famous game 7 of the 2002 Western Conference Finals). Yet his fans will often talk about 4 Cups and 3 Conn Smythes, as if that's all that happened. Again, Roy's playoff record is strong by any measure, but it obviously looks even stronger when you only focus on the best of the best.
For other goalies more of a focus is put on their negatives than their positives. Glenn Hall is generally considered a playoff underachiever and is blamed by many for the Hawks' relative lack of playoff success in the 1960s, despite a strong effort in helping his team win the 1961 Cup and even though Chicago was outshot in nearly every playoff series that decade. Another one would be Curtis Joseph, who often led his team past better teams with excellent performances in the early rounds, but usually gets blamed because of his teams' failure to perform well deeper in the playoffs.
If we adjust for era and opponents and compare the playoff numbers for Roy and Joseph, Roy is something like a .916 playoff goalie while Joseph is at .909. Adjust for power plays against and the gap might be even smaller. That's a significant difference over a decent sample size, certainly enough to conclude that Roy was better, but the actual gap is just one goal every 143 shots. You have to be pretty good at subjective evaluation to notice a difference of one extra goal saved every five games.
If we rely on our memories, however, then the differences are usually exaggerated. If I think of Roy and Joseph I might recall Roy's performances in the Stanley Cup Finals in 1993 and 2001, while what comes to mind for Joseph is how he played in the Conference Finals in 1999 and 2002. Based on that my memory is going to tell me that Roy was a lot better, because I'm effectively comparing a .934 goalie against an .890 goalie. That's the equivalent of Tim Thomas vs. Vesa Toskala in 2008-09. That is a gap that you can easily see with your eyes, and the result is substantially overrating the difference between playoff Roy and playoff Joseph.
There are a few goalies in the league today who benefit from the "only memorable playoff successes count" method of goalie evaluation. Marc-Andre Fleury is an obvious one, with his big save on Lidstrom still fresh in everyone's memories. Yet perhaps the goalie who benefits the most is Cam Ward. Many fans overlook his mostly mediocre regular season play (essentially everything up until December of 2008) in favour of the 6 playoff series of above average play that have built Ward something of a reputation as a money goalie. Some are claiming that Ward is one of the top 5 goalies in the NHL and deserves a chance to become Canada's starter in 2010, neither of which is supported by his history. Almost nobody took Ward to task for his performance in the Pittsburgh series despite getting shelled in four straight games. That's the power of reputation: If you're considered "clutch", then you can get blown out and the fans blame the rest of the team. If you're considered unclutch, then the fans blame you.
Ward just signed a contract extension for $37.8 million over 6 years. That's more than he deserves based on his past record, but Ward has shown some real improvement over the last few seasons (his EV SV% numbers have gone .899-.917-.926). Ward probably got rushed to the NHL before he was ready, and when he caught lightning in a bottle in the 2006 playoffs his team unwisely decided to throw him even further over his head by handing him the starting job a couple years before he deserved it. His performance over the last two seasons is likely more indicative of what he can be expected to do until the end of his this contract. Still, we don't have a lot to go on to be able to assess Ward's true skill level. Is he still improving? Was last season a good year or a typical year? All that remains to be seen.
In the new NHL, teams are paying players for their projected performance instead of their past performance, which is what I'm sure Carolina was thinking when they inked Ward to his big deal. If his 2008-09 numbers represent his true skill, then Ward is a top-10 goalie in the league. I'm not a big fan of paying out $5-6M+ on goalies, unless it is to an elite goalie with an established track record, because there always seems to be Craig Anderson-types available who have the potential to give you 90% of the performance for 20% of the cost and you don't want to get stuck paying big money to a goalie giving you average performance. However, if Ward is legitimately a .925 puckstopper at EV than it shouldn't be too bad of a contract for Carolina. If it turns out he's more of a .917 guy then that's a big overpayment.
I think it is pretty likely that Ward has become and will continue to be at the very least a solid NHL goalie, but I'm not at all convinced that he is some incredible pressure performer. If you look at his playoff performances throughout his career (junior, AHL, NHL), the only time that his playoff results look way out of line compared to his regular season numbers is his surprise Cup run in 2006. He did do pretty well in the playoffs in junior, but in the 2005 AHL playoffs he had one good series and one not-so-good one (.895 save percentage in a second-round loss to a lower-seeded team). Ward's international record is also fairly spotty, albeit in a very small sample, with a .901 combined save percentage in world championship games against the other top hockey nations (USA, Russia, Sweden, Finland).
To me Ward's performance in 2006, given what he did directly before and after it, is less a sign of his clutch ability than evidence of what kind of a charmed run the Hurricanes were fortunate enough to go on for the entire 2005-06 season. Referring back to the recent topic of percentages, I'm not sure a team has ever seen their shooting percentage jump more from one season to the next then those Hurricanes did when they improved from 7.2% in 2003-04 to 11.2% in 2005-06. No wonder the team went from dead last in goals and out of the playoffs to the 3rd best offence and the 4th best record in the league.
Winning once or twice does not mean that the rest shouldn't count. If you want to evaluate goalies on their playoff or international play, then first of all be aware of the sample size issues. Secondly, evaluate them based on their individual success, not their team success. And finally, be sure to take their entire performance into account, not just the very best or the very worst.
Patrick Roy was a terrific playoff goalie, but he had his fair share of weak performances. If you play 17 playoff seasons you are naturally going to have a few that are forgettable, of course, but some of Roy's flops were actually quite spectacular (losing his job in 1987 and 1988, his problems against the Bourque/Neely Bruins, a few first-round upsets against much weaker opponents, and of course the famous game 7 of the 2002 Western Conference Finals). Yet his fans will often talk about 4 Cups and 3 Conn Smythes, as if that's all that happened. Again, Roy's playoff record is strong by any measure, but it obviously looks even stronger when you only focus on the best of the best.
For other goalies more of a focus is put on their negatives than their positives. Glenn Hall is generally considered a playoff underachiever and is blamed by many for the Hawks' relative lack of playoff success in the 1960s, despite a strong effort in helping his team win the 1961 Cup and even though Chicago was outshot in nearly every playoff series that decade. Another one would be Curtis Joseph, who often led his team past better teams with excellent performances in the early rounds, but usually gets blamed because of his teams' failure to perform well deeper in the playoffs.
If we adjust for era and opponents and compare the playoff numbers for Roy and Joseph, Roy is something like a .916 playoff goalie while Joseph is at .909. Adjust for power plays against and the gap might be even smaller. That's a significant difference over a decent sample size, certainly enough to conclude that Roy was better, but the actual gap is just one goal every 143 shots. You have to be pretty good at subjective evaluation to notice a difference of one extra goal saved every five games.
If we rely on our memories, however, then the differences are usually exaggerated. If I think of Roy and Joseph I might recall Roy's performances in the Stanley Cup Finals in 1993 and 2001, while what comes to mind for Joseph is how he played in the Conference Finals in 1999 and 2002. Based on that my memory is going to tell me that Roy was a lot better, because I'm effectively comparing a .934 goalie against an .890 goalie. That's the equivalent of Tim Thomas vs. Vesa Toskala in 2008-09. That is a gap that you can easily see with your eyes, and the result is substantially overrating the difference between playoff Roy and playoff Joseph.
There are a few goalies in the league today who benefit from the "only memorable playoff successes count" method of goalie evaluation. Marc-Andre Fleury is an obvious one, with his big save on Lidstrom still fresh in everyone's memories. Yet perhaps the goalie who benefits the most is Cam Ward. Many fans overlook his mostly mediocre regular season play (essentially everything up until December of 2008) in favour of the 6 playoff series of above average play that have built Ward something of a reputation as a money goalie. Some are claiming that Ward is one of the top 5 goalies in the NHL and deserves a chance to become Canada's starter in 2010, neither of which is supported by his history. Almost nobody took Ward to task for his performance in the Pittsburgh series despite getting shelled in four straight games. That's the power of reputation: If you're considered "clutch", then you can get blown out and the fans blame the rest of the team. If you're considered unclutch, then the fans blame you.
Ward just signed a contract extension for $37.8 million over 6 years. That's more than he deserves based on his past record, but Ward has shown some real improvement over the last few seasons (his EV SV% numbers have gone .899-.917-.926). Ward probably got rushed to the NHL before he was ready, and when he caught lightning in a bottle in the 2006 playoffs his team unwisely decided to throw him even further over his head by handing him the starting job a couple years before he deserved it. His performance over the last two seasons is likely more indicative of what he can be expected to do until the end of his this contract. Still, we don't have a lot to go on to be able to assess Ward's true skill level. Is he still improving? Was last season a good year or a typical year? All that remains to be seen.
In the new NHL, teams are paying players for their projected performance instead of their past performance, which is what I'm sure Carolina was thinking when they inked Ward to his big deal. If his 2008-09 numbers represent his true skill, then Ward is a top-10 goalie in the league. I'm not a big fan of paying out $5-6M+ on goalies, unless it is to an elite goalie with an established track record, because there always seems to be Craig Anderson-types available who have the potential to give you 90% of the performance for 20% of the cost and you don't want to get stuck paying big money to a goalie giving you average performance. However, if Ward is legitimately a .925 puckstopper at EV than it shouldn't be too bad of a contract for Carolina. If it turns out he's more of a .917 guy then that's a big overpayment.
I think it is pretty likely that Ward has become and will continue to be at the very least a solid NHL goalie, but I'm not at all convinced that he is some incredible pressure performer. If you look at his playoff performances throughout his career (junior, AHL, NHL), the only time that his playoff results look way out of line compared to his regular season numbers is his surprise Cup run in 2006. He did do pretty well in the playoffs in junior, but in the 2005 AHL playoffs he had one good series and one not-so-good one (.895 save percentage in a second-round loss to a lower-seeded team). Ward's international record is also fairly spotty, albeit in a very small sample, with a .901 combined save percentage in world championship games against the other top hockey nations (USA, Russia, Sweden, Finland).
To me Ward's performance in 2006, given what he did directly before and after it, is less a sign of his clutch ability than evidence of what kind of a charmed run the Hurricanes were fortunate enough to go on for the entire 2005-06 season. Referring back to the recent topic of percentages, I'm not sure a team has ever seen their shooting percentage jump more from one season to the next then those Hurricanes did when they improved from 7.2% in 2003-04 to 11.2% in 2005-06. No wonder the team went from dead last in goals and out of the playoffs to the 3rd best offence and the 4th best record in the league.
Winning once or twice does not mean that the rest shouldn't count. If you want to evaluate goalies on their playoff or international play, then first of all be aware of the sample size issues. Secondly, evaluate them based on their individual success, not their team success. And finally, be sure to take their entire performance into account, not just the very best or the very worst.
Wednesday, September 30, 2009
Percentages
There are two ways to win hockey games. Option A is outshooting the other team and winning through a greater volume of scoring chances. Option B is playing even or getting outshot and relying on superior percentages to end up ahead on the scoreboard.
In the long run, teams that outshoot generally have a better chance to achieve success. Over the course of a single season, however, it can be the team that rides a hot streak in terms of percentages that ends up on top of the standings.
The problem with outshooting numbers are that they are often skewed by the score (see JLikens' preview of the Colorado Avalanche for a more detailed explanation of how the score affects shot results). This effect is especially large in the third period, where teams like Colorado and Toronto had strong outshooting results because most of the time they were losing the hockey game. On the flip side, top teams like Detroit and San Jose noticeably shut it down in the third period, trading off some of their outshooting advantage to lower the overall number of scoring chances and increase the likelihood of preserving their lead.
To avoid being misled by the score effects, it is likely better for predictive purposes to look at the data set from the first and second periods only. Using the data from the Hockey Summary Project, I separated out each team's performance in the first and second periods only over the last two seasons. The teams are ranked in order of the projected number of points they could be expected to put up if they record the same shot for/against ratio, shooting percentage and save percentage in the first two periods in 2009-10.
These numbers are not meant to be exact predictions, just historical data. Many teams have turned over much of their rosters and/or coaching staffs since 2007-08, which may mean that some of the numbers aren't very useful for assessing the 2009-10 squads. However, the SF/SA ratio should still have some value in predicting results. The correlation between SF/SA over the first two periods in 2007-08 compared to 2008-09 was 0.79. If a team is returning most of its key players and has the same coach or a similar coaching philosophy there is good reason to expect a similar SF/SA in 2009-10.
The numbers for Detroit and San Jose show the advantage of majorly outshooting the opposition. Detroit demonstrated last year during Chris Osgood's early-season struggles that they could win despite weak goaltending. As long as the Red Wings and Sharks continue to pepper opposing goalies with shots they will win games even with mediocre percentages. In contrast, Edmonton and Atlanta will have almost no chance of making the playoffs unless they drastically improve their territorial play, because it is very hard to compensate for the handicap of facing 20% more shots against.
This chart allows us to see the teams that are relying on percentages rather than outshooting for their success. These teams are more likely to be susceptible to large swings in the standings depending on the luck of their shooters/goaltenders. For example, Montreal's fall from Conference Champs in 2008 to 8th seed in the 2009 playoffs was almost entirely driven by percentages (while Boston's ascent was also pretty much the same story, in reverse). Along with the Habs, Pittsburgh, Minnesota, Florida and Philadelphia are the teams that have been most dependent on having the percentages in their favour.
I'd expect Pittsburgh to be a better outshooting team this year, based on the way they performed late in the season and in the playoffs under Dan Bylsma. If they are they should finish at or near the top of the Eastern Conference.
Philadelphia could also see their outshooting numbers rise in 2009-10 because of additions and the continued maturation of some of their young stars, although I expect a drop in team save percentage might very well wipe out some of those gains. The Flyers are a popular pick among many pundits heading into this season, but I'm not sure they are an elite team yet. However, these numbers might also be selling Philadelphia a bit short, given that they are for the first two periods only and the Flyers were a strong third period team over the last two seasons.
On the other hand, there are several teams that have been outshooting the opposition but have not had as much luck with the percentages. These teams include the New York Rangers, Carolina, Columbus, and Toronto. We saw with both Carolina and Columbus in the second half of 2008-09 what happens when an outshooting team goes from poor percentages to good percentages. The improved goaltending of Cam Ward and better Blue Jacket shooting took both teams into the playoffs. If Columbus continues to improve their scoring ability they could move into the second tier of good teams in the Western Conference, behind Detroit and San Jose.
The Toronto Maple Leafs are an interesting team, since goaltending was the glaring problem that played a big role in sinking their last two seasons. The Leafs were .010 behind every other team in save percentage, which naturally meant they usually were playing from behind. League average goaltending this season probably makes Toronto a playoff contender. It appears the hopes of Leaf fans are riding on the shoulders of Vesa Toskala, or, perhaps more likely, Swedish rookie Jonas Gustavsson.
The other team that was held back by goaltending in 2008-09 was the Dallas Stars. Marty Turco is past his prime but he is still likely to be better this season, and the Stars should be back in the playoff picture.
The Buffalo Sabres have looked like a playoff team in terms of their underlying numbers for the past two seasons. They are slightly above average in all three of shot ratio, shooting percentage and save percentage. Unless somebody has a better explanation I'm inclined to attribute their lack of playoff qualification to bad luck, and the Sabres should have a good shot at returning to the postseason.
I'd say that Edmonton, Phoenix and Colorado are unlikely to make the playoffs in the West. In the East it should be safe to write off the Islanders, and Atlanta is facing a steep uphill climb unless they get some major improvement from their young players. Other than that the picture looks pretty wide open. The elite teams should end up at the top of their respective conferences again (Detroit, San Jose, Chicago, Pittsburgh, Washington), but percentages will play a big role in determining which of the rest of the teams get a chance to play for the Cup.
In the long run, teams that outshoot generally have a better chance to achieve success. Over the course of a single season, however, it can be the team that rides a hot streak in terms of percentages that ends up on top of the standings.
The problem with outshooting numbers are that they are often skewed by the score (see JLikens' preview of the Colorado Avalanche for a more detailed explanation of how the score affects shot results). This effect is especially large in the third period, where teams like Colorado and Toronto had strong outshooting results because most of the time they were losing the hockey game. On the flip side, top teams like Detroit and San Jose noticeably shut it down in the third period, trading off some of their outshooting advantage to lower the overall number of scoring chances and increase the likelihood of preserving their lead.
To avoid being misled by the score effects, it is likely better for predictive purposes to look at the data set from the first and second periods only. Using the data from the Hockey Summary Project, I separated out each team's performance in the first and second periods only over the last two seasons. The teams are ranked in order of the projected number of points they could be expected to put up if they record the same shot for/against ratio, shooting percentage and save percentage in the first two periods in 2009-10.
| Rank | Team | Proj Pts | SF/SA | Sh% | Sv% |
| 1. | Detroit | 113 | 1.42 | 8.8% | .902 |
| 2. | San Jose | 109 | 1.26 | 8.8% | .910 |
| 3. | Boston | 104 | 0.97 | 9.6% | .920 |
| 4. | Montreal | 103 | 0.91 | 10.6% | .916 |
| 5. | New Jersey | 102 | 1.09 | 8.4% | .920 |
| 6. | Chicago | 100 | 1.05 | 9.7% | .909 |
| 7. | Pittsburgh | 99 | 0.93 | 10.3% | .914 |
| 7. | Dallas | 99 | 1.04 | 10.1% | .905 |
| 9. | Washington | 97 | 1.14 | 9.0% | .904 |
| 10. | Minnesota | 96 | 0.88 | 9.7% | .920 |
| 10. | Florida | 96 | 0.91 | 8.5% | .928 |
| 10. | Buffalo | 96 | 1.04 | 9.3% | .909 |
| 13. | N.Y. Rangers | 94 | 1.14 | 7.6% | .916 |
| 14. | Vancouver | 92 | 0.96 | 9.4% | .912 |
| 15. | Ottawa | 91 | 0.98 | 9.5% | .908 |
| 16. | Philadelphia | 90 | 0.89 | 9.9% | .912 |
| 16. | Carolina | 90 | 1.10 | 8.6% | .905 |
| 18. | Anaheim | 89 | 0.98 | 9.0% | .911 |
| 19. | St. Louis | 88 | 0.97 | 9.2% | .908 |
| 20. | Calgary | 87 | 1.01 | 9.3% | .903 |
| 21. | Edmonton | 83 | 0.82 | 10.3% | .909 |
| 22. | Phoenix | 82 | 0.93 | 8.5% | .914 |
| 22. | Colorado | 82 | 1.03 | 8.7% | .901 |
| 24. | Columbus | 81 | 1.10 | 7.7% | .906 |
| 24. | Tampa Bay | 81 | 0.96 | 9.1% | .904 |
| 26. | Nashville | 77 | 0.97 | 8.3% | .907 |
| 27. | Los Angeles | 75 | 0.97 | 8.6% | .901 |
| 28. | Toronto | 73 | 1.05 | 8.5% | .891 |
| 29. | N.Y. Islanders | 72 | 0.94 | 7.7% | .911 |
| 30. | Atlanta | 67 | 0.79 | 9.5% | .903 |
These numbers are not meant to be exact predictions, just historical data. Many teams have turned over much of their rosters and/or coaching staffs since 2007-08, which may mean that some of the numbers aren't very useful for assessing the 2009-10 squads. However, the SF/SA ratio should still have some value in predicting results. The correlation between SF/SA over the first two periods in 2007-08 compared to 2008-09 was 0.79. If a team is returning most of its key players and has the same coach or a similar coaching philosophy there is good reason to expect a similar SF/SA in 2009-10.
The numbers for Detroit and San Jose show the advantage of majorly outshooting the opposition. Detroit demonstrated last year during Chris Osgood's early-season struggles that they could win despite weak goaltending. As long as the Red Wings and Sharks continue to pepper opposing goalies with shots they will win games even with mediocre percentages. In contrast, Edmonton and Atlanta will have almost no chance of making the playoffs unless they drastically improve their territorial play, because it is very hard to compensate for the handicap of facing 20% more shots against.
This chart allows us to see the teams that are relying on percentages rather than outshooting for their success. These teams are more likely to be susceptible to large swings in the standings depending on the luck of their shooters/goaltenders. For example, Montreal's fall from Conference Champs in 2008 to 8th seed in the 2009 playoffs was almost entirely driven by percentages (while Boston's ascent was also pretty much the same story, in reverse). Along with the Habs, Pittsburgh, Minnesota, Florida and Philadelphia are the teams that have been most dependent on having the percentages in their favour.
I'd expect Pittsburgh to be a better outshooting team this year, based on the way they performed late in the season and in the playoffs under Dan Bylsma. If they are they should finish at or near the top of the Eastern Conference.
Philadelphia could also see their outshooting numbers rise in 2009-10 because of additions and the continued maturation of some of their young stars, although I expect a drop in team save percentage might very well wipe out some of those gains. The Flyers are a popular pick among many pundits heading into this season, but I'm not sure they are an elite team yet. However, these numbers might also be selling Philadelphia a bit short, given that they are for the first two periods only and the Flyers were a strong third period team over the last two seasons.
On the other hand, there are several teams that have been outshooting the opposition but have not had as much luck with the percentages. These teams include the New York Rangers, Carolina, Columbus, and Toronto. We saw with both Carolina and Columbus in the second half of 2008-09 what happens when an outshooting team goes from poor percentages to good percentages. The improved goaltending of Cam Ward and better Blue Jacket shooting took both teams into the playoffs. If Columbus continues to improve their scoring ability they could move into the second tier of good teams in the Western Conference, behind Detroit and San Jose.
The Toronto Maple Leafs are an interesting team, since goaltending was the glaring problem that played a big role in sinking their last two seasons. The Leafs were .010 behind every other team in save percentage, which naturally meant they usually were playing from behind. League average goaltending this season probably makes Toronto a playoff contender. It appears the hopes of Leaf fans are riding on the shoulders of Vesa Toskala, or, perhaps more likely, Swedish rookie Jonas Gustavsson.
The other team that was held back by goaltending in 2008-09 was the Dallas Stars. Marty Turco is past his prime but he is still likely to be better this season, and the Stars should be back in the playoff picture.
The Buffalo Sabres have looked like a playoff team in terms of their underlying numbers for the past two seasons. They are slightly above average in all three of shot ratio, shooting percentage and save percentage. Unless somebody has a better explanation I'm inclined to attribute their lack of playoff qualification to bad luck, and the Sabres should have a good shot at returning to the postseason.
I'd say that Edmonton, Phoenix and Colorado are unlikely to make the playoffs in the West. In the East it should be safe to write off the Islanders, and Atlanta is facing a steep uphill climb unless they get some major improvement from their young players. Other than that the picture looks pretty wide open. The elite teams should end up at the top of their respective conferences again (Detroit, San Jose, Chicago, Pittsburgh, Washington), but percentages will play a big role in determining which of the rest of the teams get a chance to play for the Cup.
Friday, September 25, 2009
Conn Smythe Trophies
I think the worst commonly used argument for ranking Patrick Roy over Dominik Hasek is the oft-invoked "4 Cups, 3 Conn Smythes vs. 1 Cup, 0 Conn Smythes". I've dealt with the issue of team success pretty much all along in this space so you've heard those arguments before, but I also don't like it when players are evaluated based on the number of Conn Smythes they win.
There isn't a more situational award in all of hockey than the Conn Smythe Trophy. I'm not trying to diminish the importance of playing great in the playoffs, I just don't believe that one's Conn Smythe haul is a good way to evaluate playoff success or to compare the relative performance of different players. The binary nature of the award (winner gets 1, all others get 0, only 1 award handed out every year) makes it more arbitrary. If you ranked all the playoff performances of the 1990s for every single player, for example, you might have three players from 1993 end up in the top 10 (Patrick Roy, Wayne Gretzky, Doug Gilmour). Obviously only one of them can win that year's MVP award, meaning that the others end up, by "Smythe only accounting", ranked below players from weaker postseasons that weren't at their level of play. Was Cam Ward in 2006 better than Dominik Hasek in 1999? John Vanbiesbrouck in 1996? Martin Brodeur in 2003? Olaf Kolzig in 1998? Ward's the only one of those guys with a Smythe, even though I wouldn't rate his playoff as even close to any of the others.
The rest of the team is obviously a big factor, since in almost all cases the Conn Smythe Trophy winner is from the Cup champion. To win the Stanley Cup, a player needs to have great teammates around them. There have been a number of absolutely terrific playoff performances by players whose teams ended up losing in the second round or the Conference Finals. Gilmour in 1993 and Peter Forsberg in 2002 are two of the most commonly cited examples, and we can also point to the more recent efforts of Alex Ovechkin and Jonas Hiller in the 2009 playoffs. As such, by simply counting Smythes you are at least partially giving double credit for team success. Being the best player on a Stanley Cup winner is a notable achievement, but that by no means signifies that player was the best player on any team.
A player's Conn Smythe chances depend a lot on who else happens to be playing great in that particular playoff season. In the 2009 playoffs, Sidney Crosby scored 15 goals and 31 points, which is more than anyone has recorded in either category since Joe Sakic in 1996. That sounds pretty deserving of some hardware, but unfortunately for Sid the Kid his teammate Evgeni Malkin scored 36 points in the same postseason, which means Malkin gets a Smythe while Crosby gets nothing. In other years there have been no singularly outstanding candidates and the writers more or less picked somebody out of a hat (2007 is a good example, the Ducks likely split the votes among a number of good candidates). If Crosby had the same playoffs as he did this year in 2006, 2007, or 2008 he would have easily won the Conn Smythe, probably even if his team lost the Final. Therefore, giving no credit to the second place guy makes little sense.
I also dislike the excessive focus on the Stanley Cup Finals in Smythe voting. Over a short series, luck has a big impact. A player can be driving puck possession and creating scoring chances and yet not have his shots or his linemates' shots hit the net for a few games in the Finals, throwing him out of Smythe contention. Again Crosby is a good example, as in the 2009 Finals he scored 1 goal on 16 shots while linemates Bill Guerin and Chris Kunitz combined for 0 goals on 32 shots. Sometimes the puck luck is against you, and that's pretty much the only conclusion you can make when a top line shoots 2% and hits more posts than they score goals over an entire series. Crosby's 3 points in 7 games cost him the Conn Smythe, but it looked to me like he was doing the same thing he had been doing all playoffs. The difference was that he was playing against tougher competition and the shots weren't going in.
The last thing is that, like any awards voting, sometimes the voters make debatable choices. I don't agree with the voters' choice of Cam Ward over Rod Brind'Amour or Chris Pronger in 2006, or Joe Nieuwendyk over Hasek in 1999, to give just a couple of examples.
If I was running the league, I would make the Conn Smythe voting results public so we could see who was finishing in 2nd and 3rd place. I would also adopt all-star voting for the Stanley Cup playoffs, like they do in international tournaments and at the end of the regular season. This would allow credit to be given to the players who were the best at their position but who may not have ended up with a ring because their teammates weren't as good.
Here is my attempt at putting together postseason All-Star teams for each season since the lockout:
G: Jonas Hiller
D: Nicklas Lidstrom, Brent Seabrook
F: Evgeni Malkin, Sidney Crosby, Alex Ovechkin
2008:
G: Marc-Andre Fleury
D: Nicklas Lidstrom, Sergei Gonchar
F: Henrik Zetterberg, Sidney Crosby, Johan Franzen
2007:
G: Roberto Luongo
D: Nicklas Lidstrom, Chris Pronger
F: Daniel Alfredsson, Sami Pahlsson, Dany Heatley
2006:
G: Dwayne Roloson
D: Chris Pronger, Scott Niedermayer
F: Eric Staal, Rod Brind'Amour, Patrik Elias
I doubt the Conn Smythe voters would have agreed with me on those teams (I left 2 of the 4 trophy winners off of my teams entirely). If you disagree with my selections, by all means take a shot at it in the comments. However, regardless of disagreements over a few individual cases, awarding All-Stars would give players like Crosby and Lidstrom the credit they deserve for their outstanding playoff performances.
Monday, September 21, 2009
Don't Always Believe Your Lying Eyes
How do sportswriters vote for the year's best goalie? I bet they'll tell you they evaluate lots of different things, like how good he is at stopping the puck, how good he is at controlling rebounds, his mental toughness, how much he helps his team win and a bunch of other criteria that sound pretty good. They'll watch lots of hockey and check out a lot of different goalies, enough that they should be in a pretty good position to be able to evaluate them. And then who do they often end up picking? The goalie on the best team, or at least the best defensive team. That's what they've been doing for years.
If you asked sportswriters the same question back in the 1960s they'd probably give you a similar rundown, describing in detail some technical aspects of Jacques Plante that made him great, or how competitive Terry Sawchuk was in the clutch, or how much Glenn Hall helped his teams win. Some of them no doubt simply copied down the wins leaders, but I bet that a majority of the journalists sat down every year with their detailed opinions and jot notes and went through their memories of the games they had watched and tried to evaluate a dozen different subjective things. They then mailed off their ballots to league headquarters, certain that they had taken everything into account and identified the league's best goaltender.
Were they sincere? I think they were. Yet give me a random stat sheet from one of the seasons back then, with the names of the goalies blacked out, and from that alone I can predict with over 80% accuracy who the voters thought was the best goalie that season. If it wasn't for one particular outlying goalie it would be almost a sure bet.
From 1935 to 1970, there is a very simple algorithm to determine the First Team All-Star goalie. Look at all the goalies who played in at least 75% of their team's games, rank them by goals against average, and take the guy at the top of the list. That's the First Team All-Star. That solution worked in 30 out of 36 seasons, all of them except for 1957, 1958, 1960, 1964, 1968 and 1969. Here are the goalies who were named First Team All-Star ahead of the high-minutes GAA leader in those seasons:
1957: Glenn Hall
1958: Glenn Hall
1960: Glenn Hall
1964: Glenn Hall
1968: Gump Worsley**
1969: Glenn Hall
**-Led the league in GAA by a wide margin but did not play in 75% of his team's games during a season where all of the top 4 guys in GAA were in platoon situations.
Glenn Hall was quite obviously the exception to the rule. That he was the only guy to get named the league's best goalie without playing on one of the league's top defensive teams reflects well on him, and it's even more impressive that he did it 5 times. That shows that he was considered a difference-maker and is evidence of why he is considered one of the best goalies to ever play the game.
Other than Hall, though, it was pretty much a succession of GAA leaders that were named the season's best. Whether or not the voters made a conscious, stat-based decision to rely on GAA in their rankings or whether their subjective views were impacted by the fact that the goalies they watched were less likely to allow goals than everyone else is unclear, I'd guess it was a combination of the two but in the end the result is the same.
That certainly does not mean that the voters never got it right, of course. There were some seasons, perhaps even quite a few seasons, when the best goalie played on the best team or where strong goaltending made the difference between a good and a great defensive team. It remains quite obvious to anyone with a reasonable view of a goalie's role within a hockey team that the best goalie does not always allow the fewest goals against.
The voters were no less predictable in the 1970s, but what had changed was that it became common to platoon goalies. As a result, very few goalies played in 75% or more of the team's games. The voters still, whether consciously or subconscously, voted in the GAA leaders. In every season from 1971 to 1979 the same goalie won the Vezina (which under the old definition went to the starting goalie on the team that allowed the fewest goals) and was named the First Team All-Star.
I'd say the voting has improved at least somewhat in recent years, perhaps partly due to the introduction of the modern Vezina Trophy that allowed GMs to vote on the year's best goalie as well as the sportswriters. Yet even the game's insiders are not immune, as the starting goalie on the team with the fewest goals against is still overrepresented in both recent All-Star and Vezina voting. If you are the starting goalie on the team with the fewest goals against and you play in at least 50 games, you're still almost guaranteed to be named First or Second All-Star at the end of the season. The only post-expansion exceptions to that rule both came in the last decade (Ed Belfour in '99 and Roman Cechmanek in '03, although Cechmanek shared the Jennings with First Team All-Star Martin Brodeur so it's debatable whether that one should even count as an oversight). These results suggest that the bias towards goalies on strong defensive teams is still alive and well.
Some people will tell you that all goalie statistics are flawed because the rest of the team affects them, which means that the best way to evaluate goalies is to watch them play. It is true that a goalie's statistics are affected by his teammates, but what they are failing to take into account is that the rest of the team also affects how a goalie is viewed. This is especially true in the NHL where the margins between goaltenders are so narrow. I'd bet for many sportswriters and most hockey fans this effect is considerably more significant than any team-to-team differences in shot quality. A goalie on a good team that doesn't let in many goals and makes a lot of saves with his team in the lead usually looks better than a goalie on a bad team who faces more shots, allows more goals, and spends more time with his team trailing.
I think that at the NHL level there is some value to goalie scouting, but that there is likely more value in numerical analysis. That's especially true if fans or teams are not able to watch and grade the goalie in every game, which means they will end up seeing certain guys good and certain guys bad. There are other issues with evaluating goalies by watching them, such as the impact of a goalie's style and whether a viewer can accurately assess the importance of various goalie skills (I'd suggest things like rebound control and puckhandling tend to get overrated since they are more obvious and easier to grade than skills like positioning, post play, a goalie's ability to block screened or close-in shots, etc.).
If I was forced to pick only one of the two methods of evaluation (stats vs. watching), I'd prefer to take the numbers. Of course I'd rather watch a guy as well to see if there are any nuances that can be added to what the statistical record is telling me, but the moral of the story is this: Always be aware that your eyes can lie.
If you asked sportswriters the same question back in the 1960s they'd probably give you a similar rundown, describing in detail some technical aspects of Jacques Plante that made him great, or how competitive Terry Sawchuk was in the clutch, or how much Glenn Hall helped his teams win. Some of them no doubt simply copied down the wins leaders, but I bet that a majority of the journalists sat down every year with their detailed opinions and jot notes and went through their memories of the games they had watched and tried to evaluate a dozen different subjective things. They then mailed off their ballots to league headquarters, certain that they had taken everything into account and identified the league's best goaltender.
Were they sincere? I think they were. Yet give me a random stat sheet from one of the seasons back then, with the names of the goalies blacked out, and from that alone I can predict with over 80% accuracy who the voters thought was the best goalie that season. If it wasn't for one particular outlying goalie it would be almost a sure bet.
From 1935 to 1970, there is a very simple algorithm to determine the First Team All-Star goalie. Look at all the goalies who played in at least 75% of their team's games, rank them by goals against average, and take the guy at the top of the list. That's the First Team All-Star. That solution worked in 30 out of 36 seasons, all of them except for 1957, 1958, 1960, 1964, 1968 and 1969. Here are the goalies who were named First Team All-Star ahead of the high-minutes GAA leader in those seasons:
1957: Glenn Hall
1958: Glenn Hall
1960: Glenn Hall
1964: Glenn Hall
1968: Gump Worsley**
1969: Glenn Hall
**-Led the league in GAA by a wide margin but did not play in 75% of his team's games during a season where all of the top 4 guys in GAA were in platoon situations.
Glenn Hall was quite obviously the exception to the rule. That he was the only guy to get named the league's best goalie without playing on one of the league's top defensive teams reflects well on him, and it's even more impressive that he did it 5 times. That shows that he was considered a difference-maker and is evidence of why he is considered one of the best goalies to ever play the game.
Other than Hall, though, it was pretty much a succession of GAA leaders that were named the season's best. Whether or not the voters made a conscious, stat-based decision to rely on GAA in their rankings or whether their subjective views were impacted by the fact that the goalies they watched were less likely to allow goals than everyone else is unclear, I'd guess it was a combination of the two but in the end the result is the same.
That certainly does not mean that the voters never got it right, of course. There were some seasons, perhaps even quite a few seasons, when the best goalie played on the best team or where strong goaltending made the difference between a good and a great defensive team. It remains quite obvious to anyone with a reasonable view of a goalie's role within a hockey team that the best goalie does not always allow the fewest goals against.
The voters were no less predictable in the 1970s, but what had changed was that it became common to platoon goalies. As a result, very few goalies played in 75% or more of the team's games. The voters still, whether consciously or subconscously, voted in the GAA leaders. In every season from 1971 to 1979 the same goalie won the Vezina (which under the old definition went to the starting goalie on the team that allowed the fewest goals) and was named the First Team All-Star.
I'd say the voting has improved at least somewhat in recent years, perhaps partly due to the introduction of the modern Vezina Trophy that allowed GMs to vote on the year's best goalie as well as the sportswriters. Yet even the game's insiders are not immune, as the starting goalie on the team with the fewest goals against is still overrepresented in both recent All-Star and Vezina voting. If you are the starting goalie on the team with the fewest goals against and you play in at least 50 games, you're still almost guaranteed to be named First or Second All-Star at the end of the season. The only post-expansion exceptions to that rule both came in the last decade (Ed Belfour in '99 and Roman Cechmanek in '03, although Cechmanek shared the Jennings with First Team All-Star Martin Brodeur so it's debatable whether that one should even count as an oversight). These results suggest that the bias towards goalies on strong defensive teams is still alive and well.
Some people will tell you that all goalie statistics are flawed because the rest of the team affects them, which means that the best way to evaluate goalies is to watch them play. It is true that a goalie's statistics are affected by his teammates, but what they are failing to take into account is that the rest of the team also affects how a goalie is viewed. This is especially true in the NHL where the margins between goaltenders are so narrow. I'd bet for many sportswriters and most hockey fans this effect is considerably more significant than any team-to-team differences in shot quality. A goalie on a good team that doesn't let in many goals and makes a lot of saves with his team in the lead usually looks better than a goalie on a bad team who faces more shots, allows more goals, and spends more time with his team trailing.
I think that at the NHL level there is some value to goalie scouting, but that there is likely more value in numerical analysis. That's especially true if fans or teams are not able to watch and grade the goalie in every game, which means they will end up seeing certain guys good and certain guys bad. There are other issues with evaluating goalies by watching them, such as the impact of a goalie's style and whether a viewer can accurately assess the importance of various goalie skills (I'd suggest things like rebound control and puckhandling tend to get overrated since they are more obvious and easier to grade than skills like positioning, post play, a goalie's ability to block screened or close-in shots, etc.).
If I was forced to pick only one of the two methods of evaluation (stats vs. watching), I'd prefer to take the numbers. Of course I'd rather watch a guy as well to see if there are any nuances that can be added to what the statistical record is telling me, but the moral of the story is this: Always be aware that your eyes can lie.
Sunday, September 13, 2009
Investigating Special Teams Skill
Is penalty kill goaltending a distinct skill from even strength goaltending? We know that skaters show differences in ability on special teams compared to at even strength, which suggests that 4 on 5 hockey is a different game than 5 on 5. If so, maybe there are differences in goaltending results as well. Just anecdotally, there are a few goalies that seem to do better at even strength than on the penalty kill. J.S. Giguere comes to mind as someone I've run across recently who fits that profile. However, we would expect a few goalies in the entire population to have some weird results just by chance or because of possible team factors, and even if a few guys show a difference in ability it does not mean that is typical around the league. We need to look at some overall numbers over a larger sample size to get some better evidence for or against this potential effect.
I checked the correlation between even strength save percentage and penalty kill save percentage at the team level, using the 10 seasons' worth of special teams numbers from NHL.com. The correlation coefficient was 0.71, which is a high result that suggests a fairly close relationship.
With a correlation that high, we should be able to use EV SV% to predict PK SV% with some reasonable accuracy. I used Excel to generate a linear equation to predict PK SV% from EV SV%, and compared each team's results with the expected. The predictions were pretty good. Only 7 out of 30 teams had a difference between actual and expected PK SV% greater than .006. They were the following:
Minnesota: .886 PK, .878 EXP (+.008), .925 EV
Detroit: .875 PK, .866 EXP (+.009), .917 EV
Washington: .875 PK, .864 EXP (+.011), .915 EV
Columbus: .872 PK, .862 EXP (+.010), .914 EV
Boston: .861 PK, .869 EXP (-.008), .919 EV
Phoenix: .858 PK, .869 EXP (-.011), .919 EV
Atlanta: .849 PK, .859 EXP (-.010), .911 EV
There's Minnesota again, with yet another result that suggests they are doing something abnormal shot quality-wise. Another familiar team with some shot quality outlier evidence behind them are the Atlanta Thrashers, who also make their usual appearance at the bottom of the table.
It could be that some teams were excellent or weak for a few seasons, but over a longer time frame that tends to get mostly washed out in the average. A few teams appear to have persistently higher or lower results than expected, which suggests that the penalty kill situation is very similar to the one at even strength: For most of the league there is little shot quality effect, although there are apparently a few outliers at each extreme. It seems that for the most part goalie performance is similar at even strength and on the penalty kill. Either that or teams tend to have similar shot quality against at EV and on the PK. One other factor that would impact PK shot quality is the number of 5 on 3 chances against. I suspect this is one of the reasons Minnesota ranks so high. With their excellent team discipline they were less likely to have to face 5 on 3 situations.
I also checked the correlation between EV SV% and PP SV%, and it was essentially zero (0.01). The sample size for power play shots is a lot smaller, as the most shots any team faced over the last 10 seasons while on the power play was 1,096, which is about half a season's worth of shots for a 70+ game starting goalie. I'm not sure a goalie's save results while his team is on the power play are particularly meaningful.
What these results suggest to me is that we should rate goalies based on their performance both at even strength and on the penalty kill. Focusing on EV only would be underrating the Lundqvist/Luongo types that are likely adding a lot of value through their play on the penalty kill. It does not appear that there is much of a skill difference by game situation for goalies. It is more difficult for all goalies to make a save when their team is down a man, but a better goalie is still generally more likely to make that save than a weaker goalie. We certainly need to take into account the number of power play shots against for each goalie, to level the playing field for goalies on teams with higher or lower penalty totals. Just as at EV, it is likely that in today's NHL most teams around the league are pretty similar in terms of shot quality against on special teams, although there are probably a few outliers that we need to keep in mind. Looking at situational save percentages for even strength and 4 on 5 play only would probably be the best way to go.
I checked the correlation between even strength save percentage and penalty kill save percentage at the team level, using the 10 seasons' worth of special teams numbers from NHL.com. The correlation coefficient was 0.71, which is a high result that suggests a fairly close relationship.
With a correlation that high, we should be able to use EV SV% to predict PK SV% with some reasonable accuracy. I used Excel to generate a linear equation to predict PK SV% from EV SV%, and compared each team's results with the expected. The predictions were pretty good. Only 7 out of 30 teams had a difference between actual and expected PK SV% greater than .006. They were the following:
Minnesota: .886 PK, .878 EXP (+.008), .925 EV
Detroit: .875 PK, .866 EXP (+.009), .917 EV
Washington: .875 PK, .864 EXP (+.011), .915 EV
Columbus: .872 PK, .862 EXP (+.010), .914 EV
Boston: .861 PK, .869 EXP (-.008), .919 EV
Phoenix: .858 PK, .869 EXP (-.011), .919 EV
Atlanta: .849 PK, .859 EXP (-.010), .911 EV
There's Minnesota again, with yet another result that suggests they are doing something abnormal shot quality-wise. Another familiar team with some shot quality outlier evidence behind them are the Atlanta Thrashers, who also make their usual appearance at the bottom of the table.
It could be that some teams were excellent or weak for a few seasons, but over a longer time frame that tends to get mostly washed out in the average. A few teams appear to have persistently higher or lower results than expected, which suggests that the penalty kill situation is very similar to the one at even strength: For most of the league there is little shot quality effect, although there are apparently a few outliers at each extreme. It seems that for the most part goalie performance is similar at even strength and on the penalty kill. Either that or teams tend to have similar shot quality against at EV and on the PK. One other factor that would impact PK shot quality is the number of 5 on 3 chances against. I suspect this is one of the reasons Minnesota ranks so high. With their excellent team discipline they were less likely to have to face 5 on 3 situations.
I also checked the correlation between EV SV% and PP SV%, and it was essentially zero (0.01). The sample size for power play shots is a lot smaller, as the most shots any team faced over the last 10 seasons while on the power play was 1,096, which is about half a season's worth of shots for a 70+ game starting goalie. I'm not sure a goalie's save results while his team is on the power play are particularly meaningful.
What these results suggest to me is that we should rate goalies based on their performance both at even strength and on the penalty kill. Focusing on EV only would be underrating the Lundqvist/Luongo types that are likely adding a lot of value through their play on the penalty kill. It does not appear that there is much of a skill difference by game situation for goalies. It is more difficult for all goalies to make a save when their team is down a man, but a better goalie is still generally more likely to make that save than a weaker goalie. We certainly need to take into account the number of power play shots against for each goalie, to level the playing field for goalies on teams with higher or lower penalty totals. Just as at EV, it is likely that in today's NHL most teams around the league are pretty similar in terms of shot quality against on special teams, although there are probably a few outliers that we need to keep in mind. Looking at situational save percentages for even strength and 4 on 5 play only would probably be the best way to go.
Subscribe to:
Posts (Atom)
