Wednesday, December 30, 2009

Book Review: HockeyNomics

Stats analysis has slowly but steadily crept its way into the world of professional sports over the last two decades, and hockey has been no exception. I'm not someone who needs to be convinced of the value of statistical analysis (obviously), but the numbers work done in the hockey mainstream continues to be pretty simplistic.

HockeyNomics is Darcy Norman's attempt to popularize and spread the idea of "study[ing] NHL stats based on science, not just opinion". I found the book to be a useful introduction to the world of statistical analysis in hockey. When I say "statistical analysis", I really mean "proper statistical analysis", because there are lots of fantasy hockey GMs out there who can quote lots of numbers that really say very little at all.

Norman quotes hockey analyst Alan Ryder in the book as saying, "Hockey is awash with meaningless and, even worse, misleading statistics" (p. 34). As a goalie analyst, I've run into lots of sportswriters who write screeds against judging goalies based on stats, and then fill out their Vezina ballot based on the league leaders in wins and shutouts. Similarly, points make up such a huge part of a skater's rating, even though the player's usage (situational ice time, defensive vs. offensive zone faceoffs) and situation (strength of linemates and opposition) can have a big impact on those numbers. And that's not even starting to discuss the player's defensive play. Many people seem to evaluate a player's penalty killing skill, for example, by the number of shorthanded goals they score, which is at best an awfully incomplete analysis.

Nobody watches every game, so even the most hard-core anti-numbers crowd is probably going to be influenced by at least some numerical evidence. The trick is convincing fans to focus on the good numbers (e.g. possession metrics, save %, rate stats) instead of the bad (scoring numbers, wins, career stats).

The book does not get into any really heavy mathematical work (Norman says a few times that doing so would be outside of his scope). The limited scope means that some parts had to be simplified. For example, the Ovechkin-Crosby debate was limited to a discussion of which player was likely to be more high-scoring over the course of their careers. A full analysis could have tried to factor in additional variables like linemates, ice time, or defensive play. Those who follow the likes of Vic Ferrari, Tyler Dellow, Gabe Desjardins or Puck Prospectus on a regular basis might want to see some deeper analysis or think that the book didn't go far enough in a few spots.

Norman discusses a number of interesting topics. I thought the Poisson modeling to identify the best goalscoring season was an interesting method. The typical way to adjust is to use the average goals scored per game to create an adjusted goals figure, but that doesn't seem to work in all cases since the scoring totals of the top forwards and the league average goals scored have not always followed each other in perfect lockstep. In some seasons in the 1980s league scoring was very high, yet other than Gretzky the top forwards in the league were only scoring around 110 points. In the early 1990s the league scoring level had dropped, but for a few seasons in 1992-93 and 1993-94 many of the top forwards put up terrific numbers.

The book also gets into various measures of drafting success. I personally feel that player development is more important than drafting, and that how teams develop their 18, 19 and 20 year old players should be considered as well. There is a high degree of consensus on high picks, so it seems very unlikely that we would see the observed divergence in terms of player performances based simply on what names get called out on draft day. Part of it may just be a matter of overall team-building strategy: Some teams rely heavily on drafted talent and are content to develop players at the NHL level, while others either have much less turnover or fill many of their roster spots with players that were developed elsewhere.

The last chapter is of particular interest, since it deals with the question of whether Martin Brodeur is overrated. I don't want to give too much away, but Norman quotes me a few times and that should tell you which side he ends up on. To be honest, a regular reader of this blog would be familiar with all the arguments presented in the chapter. It is important once again to point out that "overrated" simply means that a player is rated more highly than he deserves to be. Martin Brodeur is a unique athlete in terms of his durability, his playing style and his ability to contribute to his team in ways other than simply making saves. Despite this many fans consider him to be the best goalie ever, even though Brodeur is most likely the third best goalie of the last 20 years.

HockeyNomics gives a quick summary of the most relevant arguments, but I think there are a couple of issues that still need to be resolved to properly evaluate Brodeur. They include a more nuanced analysis of his overall non-save impact, properly assessing any scorer bias effects from playing in New Jersey, and improving shot quality metrics. My confidence in shot quality measures has been shaken somewhat over the last few months, but I'm still pretty sure that there is some signal in the noise. It may be only useful for analyzing outlier teams, but if anybody was an outlier in terms of defensive skill it surely was the New Jersey Devils.

HockeyNomics is a decent read for anyone interested in hockey, but if you want deep analysis or heavy number-crunching you might find it a bit light in spots. It might be a good gift idea for that friend or relative who either buys into just a bit too much of the conventional wisdom in hockey, or perhaps knows the numbers but tends to focus on the wrong ones.

Saturday, December 26, 2009

Why Do So Many Dumb Things Get Written About Chris Osgood?

The official website of the National Hockey League named Chris Osgood as the Second Team All-Star goalie on the NHL's All-Decade Team (hat tip to an anonymous poster on this blog for passing along the link).

That is so mind-boggling that it really shouldn't require much further comment for any rational individual. But if you want to see some numbers just to drill the point home, here are the GAAs of Chris Osgood and Roberto Luongo in both the regular season and the playoffs from 1999-00 to 2009-10:

Regular Season: Osgood 2.56, Luongo 2.56
Playoffs: Osgood 2.05, Luongo 2.09

Even if one was so crazy as to use goal prevention as the sole criterion, it's still barely possible to make the argument for Osgood over Luongo, when you consider that Luongo played in a lot more games. Think for more than a nanosecond about their respective teammates and team situations and it's completely obvious who is far better.

If you want to argue about the value of save percentages for someone like Martin Brodeur, who brings more to the table than just stopping pucks that's fine, but this is Chris Osgood. Over the last decade Osgood's numbers read .912 at EV and .871 on the PK. Luongo's .929 at EV and .887 on the PK. And we're talking about a sample size of over 13,000 shots for Osgood and nearly 18,000 shots for Luongo, i.e. no doubt at all that there is a massive gulf between the two goalies.

Multiply the save percentage differential over Luongo's workload, and you get a difference of 275 goals, or nearly 30 goals per season. That's the equivalent of something like 50-55 wins. A team would be 10 points better in the standings every year with Luongo in net than with Osgood. On a per-game basis, that's a difference of about half a goal per game. Yet somehow a difference in playoff winning percentage of .073 between the two goalies is enough to make up for all of that.

Basically, this guy is saying that the scoring proficiency of a goalie's team means an awful lot more than saving an extra half a goal per game, based on the time-worn journalistic principle of assigning all team results to the account of their goaltender. Yet I'm sure nobody would dare rank one skater ahead of another if the first guy had averaged 30 fewer points per season than the other guy for an entire decade, regardless of any difference between them in terms of Cups or team success.

For illustrative purposes, here are the players who averaged 0.5 PPG less over the past 10 years than All-Decade First Team All-Star Joe Sakic:

Pierre-Marc Bouchard
Jonathan Cheechoo
Ulf Dahlen
Joe Pavelski
Gary Roberts
Todd White

And the same thing for fellow First Team All-Star Jarome Iginla:

Chris Higgins
Jan Hlavac
Bobby Holik
Sami Kapanen
Steve Konowalchuk
Sergei Zholtok

Now obviously scoring totals are not complete assessments of player values, but I think the point is pretty obvious. I'm not sure Osgood would make my Fifteenth All-Decade Team, as I'd take any of Luongo, Brodeur, Giguere, Kiprusoff, Hasek, Roy, Vokoun, Kolzig, Belfour, Nabokov, Lundqvist, Turco, Roloson, Khabibulin, Thomas or Theodore ahead of him. Maybe a couple of others as well.

Monday, December 14, 2009

Goalie and Faceoffs

Some of the most interesting new statistics introduced by stats guys in the blogosphere have been numbers that track how many defensive and offensive zone faceoffs players are put on the ice for. For skaters, this helps adjust for the way they are being used by their coach, since for example it is an advantage for an offensive player to start their shift in the other team's end rather than having to move the puck 180 feet down the ice to get it into a scoring position. Faceoff numbers can also be used to measure which players are driving possession, by seeing whether a player ends their shift more often in the offensive or defensive zone.

I haven't seen anyone apply these numbers yet to goalies. Since goalies don't change lines, the starting and ending shift numbers are of course meaningless. However, on many shots goalies have the opportunity to either freeze or play the puck, and that choice affects the number of defensive zone draws their teams face. Also, if certain goalies are able to contribute to their teams in other ways, such as for example through their puckhandling skills, then it might show up by their team taking more draws at the other end of the rink.

Vic Ferrari at timeonice has the faceoff zone start numbers for every player in the league last year, including goalies. I'll stick to the convention used by Gabe Desjardins at Behind the Net where he has the offensive faceoff percentage for all players, a number calculated by ignoring the neutral zone faceoffs and dividing the number of offensive zone faceoffs by the total number of draws in offensive and defensive zone combined.

The correlation between the offensive faceoff percentage for starters and their backups was 0.51, which suggests that the rest of the team has a big impact on puck possession and faceoffs. That should be fairly uncontroversial. I'd expect with a bigger sample size for most of the backup goalies that number would be higher.

The correlation between offensive faceoff percentage and shots faced per 60 minutes was -0.57. I was expecting that relationship to be stronger, since all the Corsi evidence shows a big advantage to starting in the other team's zone, but again EV only for one season is a fairly small sample so there is likely a reasonable degree of luck in the numbers.

There were 9 teams that had a offensive faceoff percentage difference of 5% or more between their starting goalie and his backups. Here are the faceoff numbers for each of those teams, along with the offensive faceoff percentages and shots faced per 60 minutes:

Ryan Miller: 480 def, 433 off, 47%, 30.9 SA/60
Backups: 254 def, 282 off, 53%, 31.2 SA/60


Dwayne Roloson: 553 def, 513 off, 48%, 32.6 SA/60
Backups: 210 def, 240 off, 53%, 31.4 SA/60

Carey Price: 423 def, 409 off, 49%, 29.9 SA/60
Jaroslav Halak: 383 def, 269 off, 41%, 33.5 SA/60

New Jersey:
Martin Brodeur: 194 def, 268 off, 58%, 28.8 SA/60
Scott Clemmensen: 434 def, 353 off, 45%, 29.0 SA/60
Kevin Weekes: 154 def, 136 off, 47%, 30.1 SA/60

New York Rangers:
Henrik Lundqvist: 459 def, 665 off, 59%, 29.0 SA/60
Steve Valiquette: 90 def, 102 off, 53%, 30.7 SA/60

Alex Auld: 322 def, 373 off, 54%, 28.0 SA/60
Backups: 390 def, 332 off, 46%, 28.3 SA/60

Martin Biron: 584 def, 484 off, 45%, 32.4 SA/60
Antero Niittymaki: 226 def, 230 off, 50%, 31.5 SA/60

San Jose:
Evgeni Nabokov: 510 def, 521 off, 51%, 27.1 SA/60
Brian Boucher: 150 def, 202 off, 57%, 26.2 SA/60

Vesa Toskala: 342 def, 366 off, 52%, 29.8 SA/60
Backups: 278 def, 370 off, 57%, 30.0 SA/60

The Price/Halak gap is interesting and appears to account for some of the shot differential between them, but I'm not sure how much it had to do with the goalies. I'd bet the split would be in the other direction if we were looking at this season's numbers, based on how Montreal has played in front of each of them in 2009-10. Most of the others either involve backups who didn't play very much or goalies who I wouldn't expect to have much of an effect on faceoffs, although there is one notable exception.

The biggest gap between any starter and backup, by far, is in New Jersey. Knowing what we know about those goalies, I'd say that these numbers suggest a real effect. In most starter/backup scenarios, we have to at least consider the possibility of strength of schedule being a factor, but that wasn't the case here as Clemmensen was an injury replacement for Brodeur. The Devils weren't performing exactly the same all the way through the season, but all three of their goalies had similar GAAs so it is likely that they played in fairly similar environments.

The numbers indicate that Brodeur is either helping shift the play to the other end of the ice or freezing the puck less often than the other goalies. Here are the faceoff numbers for New Jersey's goalies broken down per 60 minutes of EV play:

Brodeur: 8.6 def zone, 11.9 off zone
Clemmensen: 14.2 def zone, 11.6 off zone
Weekes: 15.4 def zone, 13.6 off zone

Brodeur keeps the puck moving a lot more than the other two (and indeed, he probably keeps it moving more than any other goalie in the league). However, the team did not take substantially more faceoffs at the other end of the rink, which makes it uncertain whether Brodeur's impact translates to the offensive side of the rink (although I would certainly like to see more data on this one).

Consistently giving the puck to teammates instead of allowing the opposing team to win control of it through a faceoff should help a team, and this may account for some of the observed shot differentials between Brodeur and his backup goalies. There are other possible benefits, such as creating more changes on the fly, which could be to the benefit of a smart bench coach who wants to get his matchups. I'd expect some tradeoff in terms of increased turnovers, but Brodeur is likely pretty efficient.

I think these results also shed a bit more light on the rebound numbers discussed earlier here that showed Clemmensen allowing a lower rate of rebound shots against than Brodeur. Given that Brodeur would have been playing the puck much more often and attempting to direct his rebounds rather than simply freezing the puck, that means he would have been facing many more opportunities to turn over the puck or for the other team to steal it and get a quick shot on goal. If Brodeur was aggressive in terms of directing rebounds and playing the puck while Clemmensen was conservative (and probably helped by a defence that gave extra attention to clearing the crease), then that would explain why Brodeur's numbers don't seem as good despite his superior skill. More opportunities usually mean more errors, no matter how good you are.

This is just a cursory look from one year's worth of data, but looking at a few more seasons' worth of data could help us better identify Brodeur's effect here and see whether any other goalies seem to have a tendency towards freezing or moving the puck.

Monday, December 7, 2009

Why Offence Rules the New NHL

"Defence wins championships" is a familiar cliche that is thrown around as a truth in not only hockey circles, but by fans of virtually every team sport. At some points in hockey history it may have been true, but I believe the game has changed. In the new NHL, the evidence suggests that offence wins championships.

First of all, I'll give the small sample size warning to everything that I'm about to post. Past results do not guarantee future performance and so on. I tend to believe the numbers show a real effect since regular season results have generally fallen in line, but I'm going to be focusing on playoff results only which means that the sample is limited to 60 playoff series over the last 4 seasons.

Secondly, I'm ignoring all shootouts here. Goals for and goals against mean actual goals, not the goal awarded in the standings to the team that won the shootout. I counted all games that went into a shootout as ties for both teams, so because of that a few times I considered a team that finished lower in the standings to have had a better record than their opponent. This approach makes sense to me since shootouts don't happen in the playoffs.

It is important to realize that in the playoffs nothing happens with a high degree of certainty. Upsets regularly happen in short series. Just to establish a baseline, over the period from 1955-2009 the team with the better regular season winning percentage has won 65% of the playoff series. Since the lockout the percentage has fallen to 61%, which likely reflects the higher level of parity in today's game.

If the better overall team wins 61% of the time, how does the better offensive team do? Answer: The team with more goals scored has actually done even better, winning 37 of the last 60 series (62%). That's a much better success rate than the team with fewer goals allowed, which has won just 27 series (45%).

Often the same team will be better in both categories. For example, the 2008 Detroit Red Wings had better regular season offensive and defensive numbers than all four of their playoff opponents. Let's look only at series when a team with more goals scored plays against a team with fewer goals against, the classic offensive team vs. defensive team scenario. In those matchups, the stronger offensive team has won 24 out of 38 times (63%).

The numbers also show the value of looking at a team's win threshold. The team with the better win threshold won 38 out of 60 series (63%). Win threshold has been a better predictor of success since 2006 than a team's overall record or number of goals scored. As you can infer from that result, a team with a higher win threshold was slightly more likely to win than an opponent with a lower win threshold even when the latter had a better win/loss record, although this advantage was slight (11/21, 52%).

When we focus on goaltending, we also see that the post-lockout playoffs have been determined primarily by the play of skaters, rather than by the play of the masked men. The team with the better regular season save percentage has won just 25 out of 60 series (42%). When a team with a better save percentage has played against a team with a better record, the team with the better goaltending won just 7 out of 25 series.

If you look at each position individually then goalie is the most important position in hockey, but the 18 skaters as a group are collectively much more important than the goalie. A good measure of the effectiveness of the skaters is a team's win threshold. Since the lockout, when a team with a better win threshold went up against a team with a better save percentage, the team with the better skaters won 26 out of 39 times (67%). Given the uncertainty of playoff results in hockey, that is a very high probability.

There is another cliche that goes something like, "In the playoffs you don't need a great goalie, you just need a hot goalie". If you have a dominant team then you might not even need that, but for most teams that is probably not far from the truth. The abundance of good goalies in today's NHL means that a lot of teams have a goalie that is capable of excellent play for a month or two. The best goalies are not able to be the difference makers that they perhaps once were, and that means that goaltending ranks well behind a team's offensive ability in terms of predicting their success.

Wednesday, December 2, 2009

Clutch Play

"TCG's other big problem is that he completely ignores the human element of sport, and approaches all statistical problems from the assumption that players perform the same regardless of game situation, again without ever really grappling with the problem. It is essentially a hockey version of Moneyball, Billy Beane's now comical baseball analysis which included the theory that there is no such thing as clutch hitting...I find attempts to erase the human element of sport under a pile of statistics not only patently false, but also vaguely disgusting." (HFBoards)

I just wanted to post a few words about my outlook on clutch play. The poster above is essentially correct in my base assumption regarding clutch performance in hockey players. However, I don't think it's fair to say I've never grappled with the problem. I simply think that the high degree of uncertainty means that it does not make sense to use clutch performance as an evaluative tool for goalies. Here are the six primary reasons:

1. Most of hockey is played with the score close. Allowing a goal against in a close game results in a significant downgrade in a team's win probability, and therefore most of a goalie's workload should be considered to be a clutch situation. From that, it follows that a goalie's overall performance should closely approximate his clutch performance, since most of the sample comes from situations with a high penalty for allowing a goal against.

2. If we define clutch situations more narrowly, we run into small sample size issues. For example, if we were to evaluate goalies based entirely on their performance in third periods in the playoffs, then we only have several hundred shots to work with even for experienced netminders. A more common split is to simply discuss regular season performance and playoff performance separately, but for the most part over a larger sample size the two results converge, or are generally within a typical margin of error for the size of the sample. Playoff results are also fraught with additional perils, including extreme opposition effects, a different style of play, and greater playing-to-the-score effects.

3. The best way to evaluate a method or statistic is to see how well it predicts the future. If we want to include clutch play in our predictions for active netminders, then sample size is even more of a concern. It is a simple fact of random chance and the variability of athletic endeavours that some goalies are going to start their playoff careers hot while others start their playoff careers cold, regardless of their level of talent, preparation or mental fortitude. Without a large sample size to work with, we're essentially guessing at this point whether someone like Cam Ward has a true ability to raise his game in important situations or whether he had a couple of well-timed hot streaks. If he continues to play on a marginally talented team that frequently misses the playoffs, we may never know with a high degree of confidence which viewpoint is correct.

4. Players and teams have the option of changing their style of play, their matchups, their shoot/pass tendencies, and their offensive/defensive bias to match the game situation. Goalies do not have those same strategic options. This suggests that changes in results or percentages in response to the game situation are primarily driven by players, not goalies.

5. Nearly every goalie who has been identified as clutch by subjective evaluators played on a dominant team. That correlation certainly suggests that many observers are conflating team effects with goaltender performance. It is possible that they are correct, but it does not seem very probable, given that on the whole team strength is a much better predictor of playoff success than goalie strength. If there were goalies who played on weak teams who did not have significant team success yet were universally praised as clutch, then I would have more confidence in the ability of observers to rate "clutchness".

6. Subjective evaluations of goalies often strongly emphasize a goalie's performance in important situations, his team success, and whether or not someone believes he is a "winner". It is conventional hockey wisdom that you need a clutch goalie to win and that the best goalies win the most games. Therefore, it seems extremely likely that there should be a selection bias against goalies who can't perform under pressure, that scouts will pick out the most clutch goalies to advance to higher levels of play. If a goalie who is in truth a "choker" or a "loser" does make it all the way to the best league in the world, then that would reflect somewhat poorly on the ability of observers to subjectively evaluate clutch play.

In summary, I'm not a clutch play disbeliever, merely a clutch play skeptic. I have done studies of clutch performance because I think it is a worthwhile topic to investigate, but I don't think the evidence suggests it is particularly significant or that it is accurately estimated by observers. I'm certainly not saying that sports psychologists are quacks, or that all athletes perform at exactly the same level in every situation. Players are indeed human, and there are too many top-level athletes who failed under pressure to discount the human element entirely. However, I think we need to focus primarily on the most significant data, and I see NHL goaltending as an area where major clutch differences are structurally unlikely (see points 1, 4 and 6 above). We also have to be very careful about poor logic when switching back and forth from the general case to the specific case, e.g. some athletes choke, Goalie A choked in a big game, therefore Goalie A is a choker who will always choke.

Let's assume there is some small variance in clutch skill among NHL goalies. Precisely measuring that skill will be very difficult, whether you are evaluating players subjectively, objectively, or using both methods. Either way you are going to make mistakes, because chance happens and the future is unknown. If you want to be like a television broadcaster and subjectively praise players for their mental toughness and because "all they do is win", then you are going to hype some guys who simply went on a hot streak and have nothing but regression to the mean in their future. You're also going to dump on some players for their lack of fortitude who, unbeknownst to you, are going to tear up the league in future playoff seasons.

On the other hand, if you view the world the way I do, you run the risk of failing to correctly praise a player as clutch, or at least you won't do so until their careers are mostly over and they have proven that they have that ability. You will also continue to predict great things for players who have good overall records but have poor clutch performances in their early careers. Some of these players might continue to perform poorly under pressure, but many of them will see their future pressure performances improve to match their overall ability.

If someone thinks that this perspective is promoting an agenda or in some way ignoring evidence, I'd like to point out that I took the same approach on shot prevention effects. I always thought there was a small effect, but I thought it was likely fairly insignificant and I wasn't going to commit to anything at all until I had evidence of what it was.

Was I wrong to state that goalies have no effect on shots against? Absolutely, as I think there is some very good evidence that goalies can affect the number of shots against, and the observed variance among NHL goalies seems to be about 1 shot above or below average. However, there are tons of people (including Martin Brodeur himself) who are demonstrably wrong on the other side of the equation because they overestimate the effect. Guessing too high is just as wrong as guessing too low. And by doing some in-depth research on the issue from my devil's advocate position and debating others who disagreed with me, I think we've come to greater learning than we otherwise would have if I'd merely accepted the consensus opinion or came to some quick subjective estimate and left it there.

Similarly, it's highly probable that I would be wrong to state that there is absolutely no difference in clutch skill among NHL goalies. However, in the presence of uncertainty that's still the position I am generally going to take, because I don't know exactly where to draw the line and I think that most people are drawing theirs too far on the opposite side of the true marker. That makes us both wrong, but my bet is that I'm closer to being right.

It's doubtful we'll ever develop perfect metrics or track them perfectly, and even if we do there are still going to be some limitations like being constrained by sample size. In the great clutch debate, that means it is likely always going to come down to picking which error you want to make. I'd rather assume a player is not clutch and wait for proof that they are, then assume that they are and wait for proof that they aren't. I think the general correlation between regular season and playoff performance, and the observed regression to the mean of many players who at one point or another were considered playoff over- or underachievers makes my position the one that is less likely to make mistakes. I am quite aware that there will probably still be mistakes, but I'm willing to accept the trade-off. And if we can ever prove the magnitude of clutch skill for NHL goalies, then I will update my position accordingly.

Tuesday, November 24, 2009

Let Down by Goaltending?

I often point out how many overrate the importance of goaltending, and that plenty of good teams have had success with average goalies. However, that certainly does not mean bad goaltending cannot be very costly in some situations. I think there have been quite a few examples of very good teams who were dominant in the regular season but found themselves at a disadvantage when they went up against strong playoff opponents getting much better play between the pipes.

To try to identify some teams that were let down by goaltending, I looked at which teams had the best adjusted win threshold over five consecutive years without winning a Stanley Cup in that time span:

1. Boston Bruins (1974-1978) .849
2. Detroit Red Wings (1992-1996), .853
3. Boston Bruins (1980-1984), .863
4. New York Rangers (1971-1975), .864
5. St. Louis Blues (1998-2002), .865
6. Philadelphia Flyers (1976-1980), .865
7. Buffalo Sabres (1974-1978), .866
8. Ottawa Senators (2001-2006), .868
9. Calgary Flames (1990-1994), .871
10. Philadelphia Flyers (1996-2000), .871

Four of these teams won a Cup either shortly before or shortly after this particular streak, but six of them never won a single Championship despite putting together an excellent group of skaters.

Most of these teams may very well have won with a better netminder, but there are two legitimate reasons why that might not have been the case: Strength of opposition and the playoff performance of the rest of the team.

Sometimes even great teams are left playing second fiddle to an even greater team. The 1970s Bruins and 1970s Sabres teams had the misfortune of running up against the great Flyers and Habs teams of that decade. The 1980s Calgary Flames are another classic example. Despite averaging an .870 adjusted win threshold for an entire decade (1984-85 to 1993-94), the Flames won just a single Stanley Cup. I can sympathize with having to compete against a prime Wayne Gretzky, but the early '90s Flames don't have the same ready-made excuse of simply having a more powerful cross-province rival. That suggests more scrutiny needs to fall on the performance of Mike Vernon.

In other cases, it might have been the skaters that underperformed in the playoffs. For the most part, good regular season teams end up being good playoff teams, but there are some exceptions. One pretty clear example of that is the case of the Ottawa Senators. Ottawa's team save percentage actually improved from .910 in the regular season to .921 in the playoffs between 2001 and 2006. The team lost because they stopped scoring: Ottawa scored nearly a full goal per game higher in the regular season (3.27) than in the playoffs (2.29). That means Ottawa's adjusted win threshold went from an excellent .871 to a very average .905. Lalime and Emery may have had a few bad moments, but the goalies should not be the scapegoats for the Senators' spring collapses.

What do you think? Were these 10 teams let down by goaltending? Are there any other examples that did not make the list?

Sunday, November 15, 2009

Why Goalie Equipment Was Not Responsible for the Scoring Drop

I'm interrupting a series of posts on win thresholds to continue a discussion we had in one of the comment threads about goalie equipment. A lot of people make a big deal about the size of goalie equipment in the NHL, and the NHL is currently working on yet another attempt to "fix" this problem. Many see large goalie pads as the main reason for the low scoring rate, and feel that if more restrictions were put on equipment then the league would become significantly more high-scoring. I've seen some fans go so far as to argue that the main difference between the 1980s scoring environment and the scoring environment today is that goalies wear much larger equipment.

I think those claims are completely unfounded. The reasons for the rapid improvement in goalie numbers over the last two decades have been improved goalie technique and better defensive play. Goalie equipment size has been a minor factor.

The main proof for this is the way that the goalie crop turned over in the late '80s and early '90s. The old guys were phased out of the game, a new wave took over, and there was a rapid increase in the league-average save percentage. All of this happened before goalies started increasing the size of their equipment.

Patrick Roy is generally credited with popularizing the butterfly style. Roy played his first NHL game on February 23, 1985. At that time goalies spent most of their time on their feet, they relied on their limbs rather than their body to make saves, and they made skate saves and stacked the pads and hugged the post with their arms and did all the other traditional things goalies had been doing for years. Let's look at what happened to that group of netminders.

There were 66 goalies who played a game in that same 1984-85 season, including Roy. Their average age was 25.7 years old. Just 25 of them (38%, a little over one-third) were still in the NHL in the 1989-90 season, a mere five years later. Only 8 of those 25 were still in the NHL a decade later in the 1994-95 season. Only 2 of of those 8 performed at a high level compared to league average in the 1990s, Roy and John Vanbiesbrouck. Roy was the only one of them that won a Vezina Trophy in the 1990s, and he won his last award in 1992.

Let's fast forward a decade to that shortened 1994-95 season, and do the same analysis for that group of goalies. Remember, this is still before the equipment got huge. Here's a picture of Patrick Roy in action against the Quebec Nordiques during the 1993-94 season, for example. He has hardly any thigh-rise on his pads and his pants, chest protector and jersey all fit fairly snugly.

In 1994-95, 68 goalies played in the NHL, with an average age of 26.3. Jump ahead 5 years to 1999-00, as we did for the prior group, and we find that 44 of them are still in the league (65%, or about two-thirds). Not only that, but a dozen of them were still playing 11 years later when the league came back from the lockout for the 2005-06 season. Six out of the 9 Vezinas awarded from 2000-2009 went to goalies who were active in 1994-95, and that group included many of the top goalies of the 2000s.

Having established that the goalies from 1994-95 had much more longevity than the goalies from 1984-85, let's look at the average save percentages for those two seasons. I'll also throw in the average for 2003-04, the season with both the lowest average goals per game and the highest average save percentage of the so-called Dead Puck Era.

1984-85: .874
1994-95: .901
2003-04: .911

Power plays per game were at about the same level for all three seasons, 7.9 per game in 1984-85, 8.6 in 1994-95, and 8.3 in 2003-04. Some of the increase in save percentage would have been from defensive play, but most of it was because the goalies were better. League-wide defensive play also improved from 1994-95 to 2003-04, yet the league average save percentage only went up by .010, despite goalie equipment getting much larger. It is obvious that the more significant change happened between 1985 and 1995, not between 1995 and 2004.

We can also get a sense of the changing dynamic from 1985 to 1995 by comparing the save percentages of the goalies who were active in both periods. I chose to look at three year averages to avoid small sample sizes, including the year before and the year after for each goalie (i.e. 1984-86 and 1994-1996). There were 8 goalies in the group, but I dropped Roy since he only played 1 game in 1984-85 and therefore would really only have one season count. Roy played the modern technique anyway so his progress isn't really meaningful to what we want to track, which is how the older goalies adapted their games to a changing league.

1984-86: .886
1994-96: .900

There is a substantial improvement in the numbers. It is important to note, though, that this improvement was mostly being driven by the younger goalies in the group. The goalies who saw their numbers jump the most were Ken Wregget (.871 to .900), Tom Barrasso (.886 to .898), and especially John Vanbiesbrouck (.882 to .915), all of whom were 21 years old or younger in 1984-85. Not only were these goalies young and still not at their prime in the mid-'80s, but they already used some modern techniques or were able to adapt to the changing game.

Let's look at the older group of goalies, which includes Andy Moog, Grant Fuhr, Don Beaupre, and Kelly Hrudey. These goalies did not change their styles as much over the same period:

1984-86: .889
1994-96: .896

Despite seeing their save percentage numbers rise as a result of better defensive play in front of them, this group of four lost a ton of ground to the rest of the league during this period. They went from +.015 compared to league average in the mid-'80s to .002 below league average in the mid-'90s.

Let's compare that to the 1995 group. We want to look at the goalies who were still playing a decade later, so I picked out the 12 that played in the post-lockout NHL. I decided to compare their 1994-96 results to their numbers from 2002-04, since we want to look at seasons with no equipment restrictions.

1994-96: .907
2002-04: .911

That's just a slight increase. To be fair, we should remove Dominik Hasek, who is older than the rest of the group and is skewing the numbers with a mostly age-related decline from .926 to .914. Without the Dominator the group goes from .904 to .911, an increase of +.007. In a more defensive league with larger equipment, the increase in numbers is exactly the same as the increase we saw from the standup goalies from 1985-1995. In both cases, the league improved defensively over the period. Larger equipment would have had some small effect, but certainly not the game-changing impact that some would have you believe.

Finally, recent years have also showed us that there is not much of a relationship between equipment size and goalie play. After the lockout in 2005 there were new restrictions placed on goalie leg pads (reduced from 12" to 11") as well as glove and blocker sizes. Yet today the average save percentage in all game situations is back up to the same levels as it was in the early 2000s, despite the removal of most of the clutching and grabbing.

Just to recap, the league average save percentage went up by .037 from 1985 to 2004. Most of that improvement (.027) was already made by 1995, which was before the league-wide increase in goalie equipment size. The rest of the improvement coincided with an increasingly defensive league. When we look at the standup goalies who played in both 1985 and 1995, and compare their performance to goalies who played in both 1995 and 2006, we find that both groups increased their numbers by a similar amount even though only the latter group benefitted from huge equipment.

If I had to estimate and rank the factors that led to the change in league save percentage between 1985 and today, I would rank them in this order:

1. Improved goaltending technique (~.015-.020)
2. Improved defensive play (~.010-.015)
3. Goalie equipment size (~.005-.010)

The post-lockout crackdown on goalie equipment was still probably a good idea, but the continued focus on equipment size is in my view excessive and not something that will yield significant rewards. If the league wants to increase scoring, there are better alternatives to pursue.

Thursday, November 12, 2009

The Goalies With the Hardest Jobs

Having seen which goalies had to do the least for their teams to win, let's look at the reverse side of the coin. Which goalies needed to be at their absolute best almost every game to keep their teams ahead on the scoreboard?

1. Ron Low, .920
2. Denis Herron, .915
3. Cesare Maniago, .914
4. Roberto Luongo, .913
5. Marc Denis, .912
6. Gary Smith, .911
7. Doug Favell, .911
8. Greg Millen, .911
9. Kevin Weekes, .910
10. Guy Hebert, .910
11. Bruce Gamble, .909
12. Jeff Hackett, .909
13. Jim Rutherford, .908
14. Gilles Meloche, .908
15. Mike Dunham, .908
16. Tomas Vokoun, .908
17. Mike Palmateer, .907
18. Ron Tugnutt, .907
19. Dwayne Roloson, .906
20. Manny Fernandez, .906

Just a reminder that these numbers are purely based on save percentage, and don't take into account the number of penalties taken or any shot quality effects. Some of these goalies would have therefore in reality had an easier or tougher time of it. Luongo, for example, has faced a high rate of power plays against throughout his career. On the other hand, Roloson and Fernandez wouldn't make the list if special teams were accounted for.

In contrast to the low win threshold group there are no Hall of Famers on this list, although Luongo should be considered to be on a Hall of Fame track by anyone but the most extreme Cup counter. If we extend the list, the first Hall of Famer that we would hit would be Gump Worsley at .904. The only other enshrined goalie with a number above .900 (the league average baseline that all numbers were adjusted to) is Tony Esposito's .901.

One good reason that we don't see many of the top goalies on this list is that good goalies usually end up on the better teams. Worsley would have made the top 10 based on his years in New York and Minnesota only, but he also played a few seasons on some great Montreal teams. As a Sabre Dominik Hasek would have narrowly missed making this list, but playing for the Senators and Red Wings pulled down his overall number.

I was expecting to see Ron Low first on this list since he is famous for having the worst career winning percentage in history. We also see all 3 of the "bad team goalies" named by Ken Dryden in "The Game" show up in the top 7 (Herron, Smith, Favell). If Dryden needed a fourth example he'd probably have picked Meloche, who also makes an appearance.

All of the goalies on this list were good enough to stick around in the NHL long enough to play at least 300 games. Some weren't particularly good, but most were solid goalies in poor situations. I think several of them are probably quite underrated based on their numbers, in particular Maniago, Hebert, Palmateer, and Vokoun. Just a caveat, though, that the observed relationship between team results and goalie results means that more work needs to be done trying to disentangle team effects from goalie performance for the 1960s and 1970s goalies.

Wednesday, November 11, 2009

The Goalies With the Easiest Jobs

Which goalies had the lowest career win thresholds? Win thresholds measure how effective a team is at scoring goals and preventing shots, so a low win threshold means that a team provides a high margin of error for a goalie. Even if they are playing poorly, the other team has a limited number of chances to score, and even if a few pucks end up in the back of the net the rest of the team might just outscore the opposition anyway. It's a nice safety margin to have for any goalie. The problem comes when people try to compare goalies based on win/loss records without taking those situations into account.

I ran the numbers for every goalie with 300+ games played since 1955 (for goalies who played part of their careers before 1955 I was only able to include the portion that came after because of the lack of data). It doesn't take a vast knowledge of hockey history to know that Ken Dryden is going to come out ranked #1, but the identities of the other goalies at the top of the list is of some interest. The numbers are adjusted to a baseline of .900. Here are the 20 goalies with the lowest career win threshold:

1. Ken Dryden, .850
2. Gerry Cheevers, .861
3. Michel Larocque, .868
4. Gilles Gilbert, .873
5. Chris Osgood, .873
6. Mike Vernon, .875
7. Andy Moog, .879
8. Patrick Lalime, .880
9. Jacques Plante, .881
10. Rick Wamsley, .881
11. Tim Cheveldae, .882
12. Pete Peeters, .882
13. Grant Fuhr, .882
14. Roman Turek, .883
15. Ed Giacomin, .884
16. Marty Turco, .885
17. Martin Brodeur, .886
18. Manny Legace, .886
19. Wayne Stephenson, .886
20. Ron Hextall, .887

There are 3 teams that are mainly responsible for the top 6 goalies on the list: The 1970s Canadiens, the 1970s Bruins, and the 1990s Red Wings. All three teams were dominant teams that scored a lot of goals and did not allow many shots at their own end of the ice.

Six of the 20 goalies are in the Hall of Fame or will be when they retire (Dryden, Cheevers, Plante, Fuhr, Giacomin, Brodeur). If we increased the minimum games played to qualify we would catch a couple more (Ed Belfour .887, Patrick Roy .888). A few others, like Vernon, Moog, and Tom Barrasso (who just missed making the list) have been in Hall of Fame discussions or at least have sparked Hall of Fame debate. There are even some crazy Detroit fans out there who will do their best to try to convince you of the merits of Chris Osgood.

Playing on a strong team helps pad a goalie's win totals, and it brings them extra attention. It also sets them up for the postseason success that is an excessively important criteria in the way most people rank goalies (and regrettably, this group apparently includes the Hall of Fame voters). Win thresholds illustrate the unfairness of comparing win totals across goalies, and when combined with other performance statistics they can help us get a better sense of who was driving results on their team and who was just along for the ride.

The goalies with the hardest jobs are coming tomorrow. Stay tuned to see how many Hall of Famers are on that list...

Sunday, November 8, 2009

Team Effects on Goalies

I've been playing around with my win threshold stat lately. Through the Hockey Summary Project and as a result of a new book by Sebastien Tremblay, we now have shots against data for every NHL season since 1954-55. That opens the door for a complete statistical analysis, both at the team level and at the individual goalie level.

Since the primary focus of this blog is team effects on goalies, that was the first area I looked at. Here are the correlation coefficients between each team's win threshold and save percentage (both figures adjusted for league scoring levels):

The Original Six Era (1955-1967): -0.545
The Expansion Era (1968-1979): -0.394
The Open Eighties (1980-1990): -0.306
The Talent Influx* (1991-1997): -0.133
The Dead Puck Era (1998-2004): -0.081
Post-Lockout NHL (2006-2009): 0.014

Over most of the NHL's history, the teams that scored the most goals and prevented the most shots have also tended to have the goalies with the best numbers. It is only over the last two decades that we have seen increasing independence between team results and goalie results. That is not to say that there is none of that in recent years, simply that when looking at the overall picture we would expect more of a team impact on the numbers of a goalie playing in the 1960s or 1970s than on a goalie playing today.

Part of this could have been that the better teams had more of a tendency to develop or acquire the top goaltenders in the past. However, the evidence to me suggests that shot quality effects are mostly determined by differences in skill rather than differences in style of play, and therefore shot quality effects are going to be largest in an unbalanced league with large differences in skill between the top and bottom teams. That is supported by the data above, since in today's salary-capped league we don't see the same degree of goalie/team stat dependence, whereas in leagues that were more unbalanced because of factors like expansion, territorial rights or management competence, the goalie and team numbers are far more intertwined.

I have a few posts coming up on the topic of win thresholds, including which goalies had the lowest and highest career numbers, the importance of goaltending through different periods of league history, and how well win threshold numbers predict playoff results.

(*-I'm planning to look at a few different metrics broken down by era, and I wasn't really sure what to call the transitional period between the high-scoring 1980s and the low-scoring late '90s/early '00s. I'm not aware of any term that is in common use to describe the period, but if there is hopefully somebody can let me know. In my opinion the most defining part of the early 1990s was the migration of new talent into the NHL that really broke the Canadian dominance of the league. More Americans started playing the game at the highest level, while the fall of communism and escalating player salaries also attracted more of the top European players. At the same time, there was an overall improvement in the level of goaltending around the league, with a new generation of goalies breaking in and mostly displacing the generation before them.)

Tuesday, November 3, 2009

Dominik Hasek's Decline

It is very difficult to be a great player for a long time at the highest level of hockey. Aging, the force of competition, and overall evolution of the game usually end up gradually pulling them back to the pack, a process that can be accelerated by injuries, coaching strategies or team environments. We have seen some of the league's best players go from special to ordinary because of injuries, Eric Lindros being one of the most extreme examples. When that happens it is obvious to everyone that the player is a shadow of their former selves.

Sometimes the descent is a little more difficult to pick out. There have been players who went from special to merely very good because of injury. Wayne Gretzky's injury in the 1991 Canada Cup has often been used to explain his late career scoring drop, for example. For Gretzky, a "drop" still meant being among the league leaders in assists every year, but in his later career the Great One was not nearly the same goalscorer or even-strength offensive player that he was in his prime.

I think one overlooked example of this latter type of injury was Dominik Hasek's groin injury in 2000. The numbers suggest the injury was what ended the Dominator's peak. Many people consider the 2000, 2001, and 2002 seasons to be part of Hasek's prime, but I don't believe that is correct.

To demonstrate this point, we need to look at the situational breakdowns. There were a couple of things going on in the early '00s that were inflating Hasek's numbers: A league that was gradually becoming more low-scoring, and a Buffalo Sabres team with improved team discipline that took fewer penalties. Let's look at Hasek's even-strength save percentages:

1998-99: .946
1999-00: .923
2000-01: .924
2001-02: .925

We don't have official even-strength save percentages from before 1998-99, but based on my estimates from Hasek's overall save percentage and the number of power plays the Sabres faced I'm quite sure they were much closer to .946 than .925. In 1997-98, for example, has the goals against by situation, although the shots faced totals aren't correct. However, if we assume that Hasek faced the same number of even strength shots against in 1997-98 as he would in 1998-99, his even strength save percentage would have been .938. That assumption is unrealistic, given that Hasek played more minutes and faced more shots per game in 1997-98 than he did in 1998-99. Hasek was probably performing at .940+ in terms of EV SV% in '97-98, and he was likely in or near that range for the entire period from 1993-94 to 1998-99.

If old age was the reason for the drop from '90s Hasek to '00s Hasek, then we would have expected a more gradual decline. There was essentially a sudden drop from a high peak to a lower plateau, which fits the pattern of a player reduced by an injury.

Hasek's even-strength numbers recovered post-lockout to the .930 range. Perhaps that year off allowed him to heal some of the old battle scars and revert to the Dominator of old, at least for half a season. After that, another injury at the Olympics and the effects of old age caught up with him, even though Hasek was still an effective goalie pretty much right up until the end.

I have heard it argued that comparing Hasek's Buffalo numbers with his numbers in Detroit show that it is more difficult to dominate as a goaltender on a strong team, but I think that is wrong. Hasek's numbers dropped in Detroit partly because the Red Wings took a lot more penalties than the Sabres, and partly because he was not the same goalie that he was at his peak.

It is not surprising that Hasek ran into injury trouble, given that his style had to have been one of the most physically demanding styles of any goalie to ever play. He usually played a straight butterfly on the first shot, which applies the usual stress on the groin and hips, and then he also added his special brand of post-save moves and close-save tactics that required extreme agility and flexibility. Unfortunately, injuries often follow athletes who play a style nobody else is able to play (think of guys like Orr, Lindros, Forsberg, etc.), and the same thing was true for the Dominator.

Thursday, October 29, 2009

Playoff Wins Are a Bad Stat

It has been quite noticeable in the post-lockout NHL that the teams with playoff success have not generally been the teams with the best goaltending. That hasn't made much of a difference for many hockey fans, who continue to consider goalies with Cup rings to be the most clutch in the league. I'm not saying that goalies like Marc-Andre Fleury, Cam Ward, or Chris Osgood have played poorly in the playoffs. Not at all. However, the difference between them and a bunch of other guys is nothing more than the quality of their teammates.

There have been some goalies who have been very good in the regular season, but have not had the same level of recent playoff team success. Three of the best examples would be Roberto Luongo, Martin Brodeur, and Henrik Lundqvist. I decided to compare the numbers since the lockout for each of these three goalies with the supposedly clutch group of Fleury, Ward and Osgood, to see whether that might explain the discrepancy in their win/loss records. The numbers given are goal support per 60 minutes, shots against per 60 minutes, and save percentage:

Osgood: 3.12, 24.9, .928
Fleury: 2.95, 29.1, .916
Ward: 2.61, 28.7, .917
Luongo: 2.06, 29.9, .930
Lundqvist: 2.43, 28.8, .907
Brodeur: 2.56, 30.1, .917

We can use these numbers to calculate the playoff win threshold for each goalie, that is the save percentage they would need to record for their team to score as many goals as they allowed. Not surprisingly, the top three on the list are the guys who haven't been winning, and the bottom three are the guys who have.

1. Luongo .931
2. Lundqvist .916
3. Brodeur .915
4. Ward .909
5. Fleury .899
6. Osgood .875

Chris Osgood's advantage in Detroit is downright unfair. Marc-Andre Fleury is a fine young goalie, but there are not many other teams that would have won the Cup with a .908 save percentage from their starter. On the other hand, Martin Brodeur is not getting much help from his teammates, and somewhat amazingly Roberto Luongo's team has been outscored in the playoffs despite his .930 save percentage.

If I was about to play game 7 of the Stanley Cup Finals, I'd take any of Luongo, Lundqvist or Brodeur ahead of Ward, Fleury or Osgood. Team records don't matter, only the quality of the individual goaltender.

Tuesday, October 27, 2009

The Lockout

It seems intuitive that both an excessive workload and an excessive amount of time off would be negative factors for a goaltender. The 2004-05 NHL lockout is an interesting case of the latter. Many of the goalies ended up playing somewhere that season, of course, but few of them played the same number of games that they would have if they had remained with their NHL clubs. There were also some significant changes to the game when they returned, including smaller goalie equipment, a stricter penalty standard, and rules that created a more open game.

Generally goalies have fairly consistent save percentage results, especially when taking team and situational factors into account. There were a number of goalies that had substantially different results after the lockout compared to how they did before, more of them than we likely would expect just to occur at random. Here are some of the most extreme results, with their even-strength save percentages from 1999-2004 compared to 2006-2009:

David Aebischer: .932 before, .908 after
Andrew Raycroft: .932 before, .905 after
Patrick Lalime: .918 before, .900 after
Marty Turco: .931 before, .916 after
Jose Theodore: .922 before, .909 after

Martin Brodeur: .918 before, .926 after
Tomas Vokoun: .919 before, .933 after
Cristobal Huet: .909 before, .926 after

Some goalies seemed to fall off a cliff after the lockout, while others got significantly better. Is there anything that might explain this result?

If an extended layoff has a negative impact, then the number of games played during the lockout is likely to be the significant variable. I decided to compare goalies who played during the lockout with goalies who decided to sit it out. I decided to look only at goalies who had a significant amount of playing time both before and after the lockout, and who were between 26 and 32 years old during the lockout season. This avoids picking out goalies like Sean Burke and Curtis Joseph, who did much worse after the lockout, or goalies like Rick DiPietro or Marc-Andre Fleury, who did much better after the lockout, since all of them obviously had age as a big factor.

I came up with a sample of 26 goalies that met the criteria. I then broke them down by guys who played during the lockout vs. guys who did not:

Played: .917 before, .909 after, -.008
Did not: .922 before, .916 after, -.006

The two groups had almost the exact same average age, so that wasn't a factor. This suggests that sitting out did not have much of an effect.

Some of the goalies played only a handful of games before getting injured or returning home. There was likely little difference between doing that and not playing at all, so I divided them up by goalies who played 15 games or more vs. goalies who played fewer than 15 games:

15 games or more: .919 before, .912 after, -.007
Fewer than 15 GP: .921 before, .916 after, -.005

Again, only a slight difference.

I also took a look at whether the younger guys were affected more than the older guys.

Age 26-28 during the lockout:

Played: .921 before, .918 after, -.003
Did not: .918 before, .910 after, -.008

Age 29-32 during the lockout:

Played: .916 before, .909 after, -.007
Did not: .922 before, .915 after, -.007

We're getting into some smaller sample sizes in the last one, but the results suggest the interesting idea that sitting out a season impacts a veteran less than it does a guy still in his prime. Every single goalie 30 years old or younger during the lockout who did not play that season did worse after the lockout than they did before. The entire list goes as follows: Biron (-.005), Denis (-.010), Esche (-.017), Grahame (-.009), Johnson (-.001), Lalime (-.018), Thibault (-.016), Weekes (-.014). Having said that, some slight decline would be expected for this group because of age factors.

When these results are combined with one of my earlier posts that showed that October is usually the worst month for goalies, I think there is some evidence to suggest that extended layoffs have a slight negative effect on goalie performance.

I'm still not sure why some of goalies were much improved after the lockout while others fell off a cliff. Cristobal Huet looks like the classic late-bloomer who finally got his shot, but both Brodeur and Vokoun were veteran NHL goalies in 2005 who went from good in the early '00s to top 5 guys post-lockout. It seems unlikely that we can point to their teams as a major factor, as both were playing on the same team as before. Nashville might have improved a bit defensively, but New Jersey got worse.

On the other side, Andrew Raycroft had some strong junior numbers, some nice age 21 and 22 AHL results (.916 and .917), and then a terrific Calder year with a .940 EV SV%. He looked set for a promising career, but then he went and played 11 games in Finland during the lockout and was never the same goalie again.

I wonder whether playing for a different team in a different country had an effect on the guys like Raycroft who played overseas during the lockout and seemed to have lost something from their games when the NHL resumed. One theory is that whoever was the goalie coach for the Swedish team Djurgardens in 2004-05 was not very good at his job, considering he had Jose Theodore and Marty Turco pass through that season, both of whom went from pre-lockout stars to post-lockout mediocrity. I doubt that really had much to do with anything, but that's at least an interesting coincidence.

Another theory is that the equipment reduction had something to do with it, but that doesn't really seem to fit the results. There were some athletic, reaction-type goalies who nosedived, like Turco, and some butterfly blockers who got better, like Giguere and Huet.

I'm not really sure what was going on, if anything. I would guess that some of the goalies who struggled post-lockout were guys who didn't stay in shape, but that doesn't apply to all of them (Martin Gerber and Jussi Markkanen were two guys with pretty decent pre-lockout numbers who played 50+ games in the lockout season, and saw their numbers drop substantially in the new NHL). It looks like there are too many variables in play that we can't conclude much at the macro level, other than to say that if there is another labour stoppage in 2011-12 I would likely advise goalies that they are probably a bit better off playing somewhere rather than sitting at home, and that they might want to be a bit wary of what the Swedish coaches tell them.

Monday, October 26, 2009

Cause or Effect?

Kurt Overhardt is Ryan Kesler's agent. Kesler will be a restricted free agent at the end of this season, so Overhardt is working on getting him signed to a fat new extension. And either Overhardt is trying to twist the numbers to make his client look good, or he is simply mixing up cause and effect.

As the Vancouver Province reports based on Overhardt's research, the Vancouver Canucks had a better record last year in games where Ryan Kesler had more ice time than either Daniel or Henrik Sedin. That seems surprising on first glance, considering that the Sedins scored a lot more points than Kesler did and also had much better plus/minus numbers (Daniel +24, Henrik +22, Kesler +8). But when you think about it a bit deeper it seems obvious to me that this has more to do with the relative roles of Kesler and the Sedins than their effectiveness as hockey players.

If Vancouver is winning, who is more likely to be on the ice, their best offensive players or a Selke-nominee who is one of the league's best defensive forwards? Similarly, if the Canucks are behind, is the coach going to turn to Kesler, with his career high of 59 points in a season, or to the Sedins, who have each averaged nearly a point per game since the lockout?

In short, I doubt the Canucks win because Kesler plays a lot, it's more likely that Kesler plays a lot when the Canucks are winning. Or perhaps more precisely, since Kesler usually played more than the Sedins did by virtue of pulling more special teams duty, the Sedins weren't as likely to play big minutes except when the team was losing. Either way, it means that attributing the team's record to Kesler is a big stretch.

Good try by the agent, though. However, Mike Gillis seems like a sharp GM to me so I'm not sure he'll buy that line of argument, even though he no doubt sees the value of a great young two-way player like Kesler and will probably end up finding the necessary resources to eventually get the deal done.

Friday, October 23, 2009

Are Shootouts An Indicator of Skill?

With four full seasons of shootout results in the books, we can start to make sense of the results. I wanted to check if better goalies tended to have better shootout results. I calculated the correlation between shootout save percentage and overall save percentage for every goalie who faced at least 50 shootouts against.

Correlation: 0.004

That's about as close to zero as you are going to find in a real life sample. What that suggests is that shootout performance tells you nothing at all about how good a goalie is.

This is reinforced by looking at the leaderboard. Among the goalies who have faced 50+ shootouts in their careers heading into this season, the top 3 were Johan Hedberg (.820), Mathieu Garon (.812), and Jose Theodore (.790). Henrik Lundqvist, Rick DiPietro, Kari Lehtonen and Tim Thomas were all good goalies who had good shootout results as well (all .740 or better). Roberto Luongo (.716) and Martin Brodeur (.715) were both above average, but not by a lot, while Tomas Vokoun was right about at average (.670). Goalies who were below average at stopping shootouts included J.S. Giguere (.652), Cristobal Huet (.604), Miikka Kiprusoff (.600), Ilya Bryzgalov (.597), Niklas Backstrom (.568), and Evgeni Nabokov (.568). The worst goalie was Vesa Toskala (.512).

Conclusion: Shootout skill is distinct from overall goalie skill. Shootout results don't provide much evidence of a goalie's overall abilities. All they do is measure how good a goalie is at stopping breakaways.

For this reason, I don't consider shootout performance at all when evaluating goalies. Other analysts do, and I understand the reasons for it. All things being equal it is better to have somebody who is good at stopping shootouts in the current NHL where shootout success directly leads to standings points. If I was a GM then I'd probably take it into account. For predicting future team success it also makes sense to include shootout skill in your prediction. I just see shootouts as a sideshow that is separate from the actual game, a rare game situation that has little meaning in an overall sense and that is based on a distinct skill set. My objective is to identify the best and the worst goalies at playing hockey, i.e. the game in its most common state with 4 or 5 skaters on each side. Shootouts don't give us much useful information to that end, so to me they don't matter.

Wednesday, October 21, 2009

Win Thresholds for Cup Finalists

I tested my win threshold stat out on the recent Stanley Cup Finalists, to see which teams seemed to rely the most and the least on goaltending (numbers are adjusted for scoring level):

1998 Detroit .883, Washington .906
1999 Dallas .879, Buffalo .915
2000 New Jersey .884, Dallas .906
2001 Colorado .874, New Jersey .860
2002 Detroit .883, Carolina .901
2003 New Jersey .886, Anaheim .913
2004 Tampa Bay .877, Calgary .900
2006 Carolina .892, Edmonton .887
2007 Anaheim .889, Ottawa .887
2008 Detroit .867, Pittsburgh .903
2009 Pittsburgh .895, Detroit .871

Not surprisingly, all of the recent Stanley Cup Champions have low win thresholds. They would all have been good teams even with subpar goaltending. That doesn't mean they would have won the Cup anyway with a mediocre goalie. In that case most of them likely would not have won, although one could probably make an argument for the two teams that employed Chris Osgood. The average team from 1997-98 to 2008-09 had a win threshold of .904. The average Cup champs had a win threshold of .883. Clearly, winning the Cup is a team effort.

Most of the Cup finalists also have good numbers, including some of the surprise Finalists. Only two of the teams had a number that was well above average. Those two were the two teams that relied the most on goaltending to get where they ended up, the 1999 Buffalo Sabres and the 2003 Anaheim Mighty Ducks.

This suggests that teams that strongly outplay the opposition are generally better Cup candidates than teams that have top goalies but do not excel at scoring or shot prevention. We can look at teams like Chicago and Washington and wonder about their goaltending, but that is the type of team that has won the Cup recently while teams like Vancouver and New Jersey have not. Percentages can play a big role in a short playoff series, but on the other hand a goalie can only do so much. At some point his teammates will have to pick up some of the slack if they want to end up winning a championship.

Saturday, October 17, 2009

Close Games, Part 2

After getting some feedback on my post looking into New Jersey's record in close games, I decided to do a bit more research. It turns out that the reason for their success was mainly because they did well in overtime and in the shootout. Here are the top 5 teams in regulation one-goal victories and regulation one-goal winning percentages since the lockout:

Regulation one-goal wins:
1. Calgary, 61
2. San Jose, 54
3. New Jersey, 53
4. Detroit, 52
5. Carolina, 51

Regulation one-goal winning percentage:
1. Carolina, .585
2. Detroit, .578
3. Calgary, .573
4. New Jersey, .572
5. San Jose, .567

By these numbers New Jersey is still good at winning close games, but they are not head and shoulders above the rest of the league. Here are the numbers for overtime and shootout wins:

Wins in overtime and shootout combined:
1. New Jersey, 56
2. Atlanta, 50
3. Dallas, 49
4. N.Y. Rangers, 48
5. Buffalo, 45

Winning percentage in games that go into OT:
1. New Jersey, .659
2. Dallas, .613
3. Atlanta, .602
4. Buffalo, .570
5. Colorado, .569

New Jersey was by far the best team in the league in games tied after 60 minutes. In this light, the team's recent playoff performances perhaps don't seem as disappointing. Their regular season records were largely influenced by their ability to perform well in 4 on 4 overtime and in shootouts. Unfortunately for them, the Devils weren't able to take advantage of those situations in the playoffs.

Another variable brought up by someone in the comments was empty net goals. This was indeed a factor that helped boost the Devils' number of one goal wins, since New Jersey has been one of the worst teams in the league at scoring empty net goals since the lockout. New Jersey scored 19 times with the other goalie pulled, which was tied for the second-lowest total in the league behind only the weak Toronto Maple Leafs, a team that faced many fewer empty net chances than the Devils. Assuming they never scored two empty netters in any one game, New Jersey scored an empty net goal in 14% of their regulation wins, the second worst percentage in the league behind only San Jose's 13%.

The Devils allowed 25 empty netters against, or an ENG against in 23% of their regulation losses, which ranked them slightly worse than the league average of 22%.

I am not sure how many of the empty netters came when leading/trailing by one goal and how many came when there was a two goal margin on the scoreboard. I decided to assume that empty net goals scored came with a one and two goal lead came in the same proportion as the team's number of one and two goal wins, e.g. a team with the same number of one goal and two goal wins would score half of their empty netters in each situation. It is likely a few teams would by chance have a very different ratio, but that probably puts most teams in the ballpark. Combined with the OT/shootout numbers, that allows us to estimate a team's regulation-only one-goal game record with empty-netters removed.

I'll refer to any game that goes to overtime or is decided in regulation by a one goal margin (empty netters excluded) as a close game. Here are the close game records for all teams since the lockout (not including 2009-10), along with their close game points percentage (games tied after regulation count as 1 point), the team's winning percentage in games decided by 2 goals or more, the total points earned in overtime and shootouts, and the percentage of close games that went to OT.

RankTeamClose W-L-OTClose W%2G+ W%OT/SO% OT
4.San Jose65-42-69.565.65810139%
5.New Jersey65-42-85.560.52214144%
25.St. Louis44-59-82.459.37811344%
26.Tampa Bay46-62-74.456.41411041%
29.Los Angeles46-65-68.447.3899938%

A few teams have interesting profiles here. Carolina and Calgary are teams that do much better in close games, and have tended to win the close ones in regulation. Over the last two seasons, both teams have seen both their shot ratio and percentages improve in the third period, so perhaps it could be argued that these teams have shown some clutch ability. In contrast, Phoenix and Ottawa also don't make it overtime that often, but they tend to lose the close ones and would be better off in the standings if they could hold on a bit longer to earn a few more loser points. The Coyotes and Senators both saw their third period percentages tumble over the last two years. I'm not sure whether that is a sign of poor performance late in games or simply bad luck that led to losses.

In this table New Jersey doesn't look much different from other teams, other than their league-leading total of 141 overtime and shootout points. Their rivals the New York Rangers were only one point behind. The Rangers were not as good at picking up the extra point, but they took a lot of loser points since they played more overtime games than any other team. That suggests the Rangers have made aiming for shootouts part of their team strategy. However, it looks to me like that strategy might have been suboptimal for them. Either that or the Rangers did a poor job of carrying it out, because a team that went to OT less often but won more games in regulation would have ended up with more points at the end of the day.

The Rangers took 74 points from one goal wins, 92 points from making it to overtime, and 48 points for winning by OT or shootout for a total of 214 points from close games. Given the same number of close games a typical team would have gone to overtime only 76 times, but would have won half the remaining games for a total of 52 regulation wins. That means they would only need to win 35 out of their 76 overtime games (46%) to earn more points than the Rangers did.

The Rangers had a pretty good record in games decided by 2 goals or more. They also had the third best winning percentage in the league when trailing after 2 periods, yet had a slightly below average winning percentage when leading after 2 periods. All that tends to reinforce the theory that the team would have been better off going for more wins in regulation rather than sitting back on a lead or trying to take a tie game into overtime, because it looks like too often they saw that strategy backfire.

Thursday, October 15, 2009

Close Games

It is rare that professional teams can outperform their peers in a particular area for a long time. If a team has a strategic advantage, the rest of the league will study them and adjust their coaching strategy to compensate. If a specific talent or skillset is undervalued, a clever GM might be able to gain a short-term advantage, but again if the other teams are following along the market should correct the valuation.

As a result, when I run across a team that is a substantial outlier in any area then I take notice, and when that outlying result comes in an area that I mostly attribute to luck then it is especially interesting.

Here are the top 5 teams in games won by a one-goal margin since the lockout (not including this season so far):

1. New Jersey, 109
2. Dallas, 89
3. Vancouver, 87
3. Nashville, 87
5. San Jose, 86
5. Carolina, 86
5. Anaheim, 86

And here are the top 5 teams in winning percentage in games decided by one goal:

1. New Jersey, .740
2. Carolina, .709
3. Nashville, .690
4. Detroit, .673
5. San Jose. 670

The Devils are at the top of both of those lists, and it's not even close. That begs the question, just what are they doing differently than everyone else in the league?

The simple answer, and the one that 90% of journalists would probably respond with, is that the Devils have Martin Brodeur in net. There is, however, one fairly significant piece of evidence that suggests there is more to the story than that, namely New Jersey's record in one-goal games in 2008-09. Despite losing Brodeur to injury for almost 4 months, the Devils posted the best close game record they have ever had, going a remarkable 25-5-4 in one-goal games. Compared to an average team, New Jersey picked up an extra 12 points by winning the nailbiters, which made the difference between them winning their division and finishing as the #7 seed.

I looked at New Jersey's record since the lockout when leading, trailing, and tied after 2 periods. If they were a particularly clutch team, we would expect them to have a lot of wins in games that were tied heading into the third. The Devils did do well in that situation with a .644 winning percentage, good for 6th best in the league since the lockout. However, the Devils actually played a relatively low number of games that were tied after two, which was somewhat surprising for a low-scoring team. The team won a total of 36 games where they were tied after 2 periods, which was right about the average number (the Devils ranked 16th in the league).

New Jersey was also pretty good at coming from behind. Their winning percentage of .250 ranked 6th in the league, well above the average of .200. The Devils won 26 games that they trailed after 2 periods, which was tied for 4th in the league. It is likely that many of those wins would have been one-goal wins, although the average team won 20 so this would only account for part of their close game success.

By far most of New Jersey's wins came in games they were already leading after two periods. The Devils went 130-6-7 when ahead after two, for a .934 winning percentage that was the league's best. Only Detroit, San Jose and Ottawa converted a higher number of second intermission advantages into victories. However, none of those teams had anywhere close to as many one goal wins as New Jersey.

This indicates that New Jersey's terrific one-goal game record is mostly from their ability to get ahead and hold onto the lead. The Devils outplay the other team early, get a lead, and then try to close out the game by protecting that margin rather than trying to extend it, a strategy that if successful leads to a lot of one-goal victories.

I have scoring data broken down by period for the last two seasons from the Hockey Summary Project to support this thesis. New Jersey's offence dropped in the third, with the Devils ranking 25th in the NHL in third period goals (empty-netters removed). Their shots taken also dropped, from an average of 10.3 shots per period in the first two (6th in the league) to an average of 9.4 shots per period in the third (15th). This was from a greater focus on defensive play, as the team also was able to cut down on the number of shots against (from 9.5 per period to 8.8 per period in the third).

Despite fewer shots against, the team's GAA actually went up in the third period, from 2.27 in the first two to 2.34 in the third. This was because the team save percentage dropped from .920 to .911. This suggests that shot prevention, rather than clutch goaltending, was the main reason the team was so effective at preserving leads. Keep in mind that New Jersey had a relatively high success rate in mounting comebacks, which shows that they had the ability to score if they wanted to. Instead, the Devils traded offence for defence, and the reason they won so many games by a single goal was because they scored fewer late insurance goals than other strong teams.

New Jersey was actually outscored 128-124 in the third period over the last two seasons. The team's positive goal differential came entirely from its success in the first two periods. I looked at a few of the other top teams in winning close games as well as some of the worst teams, and their records usually could not be explained by their third period save percentages or shot ratios. In fact, few teams were particularly clutch in terms of their percentages. In general the teams that outshot and/or out-"percentaged" their opponents in the first two periods had good records in close games, while teams that got outshot usually did not.

Announcers and writers often focus on the late "big save" that supposedly "won the game". However, most of the time that is giving too much credit to the goalie. The reason that save looks important is that the team had already built a lead in the game. The goalie does have to make the saves to keep the team ahead, of course, and if they are facing sustained pressure sometimes they need to be excellent to keep the other team off the scoreboard, but since the average shot has a 91% chance of being stopped the odds are very much in favour of the leading team. Because of this goalies have a very high success rate in holding off the other team late in the game, in the same way that baseball closers usually manage to "save" the game when they enter in the ninth inning with their team already in front.

New Jersey has won a lot of close games by outshooting and outscoring the opposition early, and then locking down the game to reduce scoring chances in the third period. Their goaltending should get credit for its strong overall performance, but it does not appear to warrant any additional recognition for "making the big saves".

It looks like the Devils have kept their "competitive advantage" going this year, with all three wins this season coming by a shootout or by a one goal margin. They will likely need to continue to excel at winning the close ones to stay competitive in a tough Atlantic Division.

Monday, October 12, 2009

Situation-Adjusted Save Percentage

My conclusion from a recent look at special teams performance was that a goalie's play while his team is killing a penalty is important for evaluation, but that his performance when his team was on the power play was not. The latter conclusion was based on the lack of correlation between EV SV% and PP SV%, which suggests that the results are either highly influenced by luck or the rest of the team.

Gabe Desjardins over at Puck Prospectus agrees with me, and has a nice article up with a more advanced look at evaluating goalies based on only their EV and PK play. His method looks like a solid one for comprehensive goalie evaluation, but I thought of developing a much simpler formula that can be used as a quick-and-dirty way to take special teams into account.

Over the last decade, 76.3% of the shots have come at EV, 3.7% on the PP, and 20.0% on the PK. That number is skewed up slightly by the 2005-06 season, as in both of the last two seasons 19.7% of the shots taken were by a team on the power play. If we ignore the PP shots, a good approximation of the average EV/PK split is 80/20. By assigning an 80% weighting to the goalie's EV SV% and a 20% weighting to his PK SV% we can quickly adjust for special teams factors.

This adjustment doesn't actually make much of a difference for most goalies, but it does impact goalies who either faced a disproportionate number of shots on the penalty kill or for whatever reason allowed a unusual number of shorthanded goals. Here are the top 20 in situation-adjusted save percentage since the lockout (min. 100 GP):

1. Niklas Backstrom, .923
2. Tomas Vokoun, .922
3. Roberto Luongo, .920
4. Tim Thomas, .919
5. Cristobal Huet, .918
5. Henrik Lundqvist, .918
5. Dominik Hasek, .918
8. Martin Brodeur, .917
8. J.S. Giguere, .917
10. Miikka Kiprusoff, .914
10. Chris Mason, .914
12. Manny Fernandez, .913
13. Martin Biron, .912
13. Ryan Miller, .912
13. Kari Lehtonen, .912
16. Ilja Bryzgalov, .910
16. Marc-Andre Fleury, .910
16. Ray Emery, .910
19. Rick Dipietro, 909
19. Dwayne Roloson, .909
19. Manny Legace, .909

I think the best use of this adjustment would be as a quick check when comparing goalies. Let's say you were voting on the 2009 Calder Trophy and you wanted to compare the performances of Steve Mason and Pekka Rinne. If you look at the raw save percentages it was pretty close, with Mason at .916 and Rinne at .917. What that doesn't show, however, is that Rinne faced an unusually low number of shots against on the penalty kill. If we multiply their EV SV% by 80% and their PK SV% by 20%, Mason edges ahead .917 to .914.

The formula also adjusts for goalies who were lucky or unlucky with shorthanded scoring chances against. Henrik Lundqvist and Cam Ward both had .916 save percentages last season. However, Lundqvist allowed 11 shorthanded goals against compared to Ward's 5, as the Rangers allowed a lot of shots and presumably a lot of scoring chances against on the power play. If we look at EV and PK play only, Lundqvist jumps to .919 while Ward falls to .915.

To simply adjust for special teams factors remember the "80/20 rule", and you'll be able to pick out the goalies who have the burden or good fortune of facing heavy or light work on the penalty kill.

Saturday, October 10, 2009

The Win Threshold

I am firmly against comparing goalies based on wins. This is not because wins aren't important or desirable. Wins are what every player and team wants more than anything else. What makes wins a poor stat is that every team situation is different. If every goalie played the same schedule with identical teammates in front of them, then we could just give the Vezina to the guy with the most wins at the end of the season. In real life, goalies do not compete on a level playing field.

The two most important team factors that affect a goalie's ability to win are his goal support and the number of shots he has to face. More goal support means that more of his mistakes are covered up, and facing fewer shots against means fewer opportunities to allow goals.

Last year the Detroit Red Wings had the league's best offence, scoring 3.52 goals per game. They also allowed the second fewest shots against per game with 27.7. On the other end of the scale, the New York Islanders finished second last in both goals for (2.42) and shots against (33.5) per game. Quite obviously an Islander goalie would need to be much better than a Red Wing goalie for their teams to have the same chance at winning, because he would have to make up for his team scoring one less goal per game and he would have to do it while facing an extra half-dozen shots against.

We can calculate what I'll call the "win threshold" for the goalies on each team by taking (shots against - goals for) / shots against. This gives us the save percentage that would result in the team ending up with an equal number of goals for and goals against over the course of the season. If the goalie's save percentage is above that number, the team is likely to win more than the lose, while anything below the threshold means that the team should end up sub-.500 (or sub-.550 in the shootout era).

In 2008-09, Detroit's win threshold was .873, which was the lowest in the league. The Islanders' win threshold was .928, which was not only the highest mark in the league but also the highest of any team since the lockout.

Expressed a different way, Detroit is likely to win if their opponents have a shooting percentage of 12.6% or worse. The New York Islanders are likely to win only if their opponents have a shooting percentage of 7.1% or worse. The shooting percentage against Detroit needs to be almost 80% higher than the percentage against the Islanders for the teams to have the same likelihood of winning the game.

Naturally, comparing win totals on goalies playing on the Islanders to goalies playing on the Red Wings is completely senseless. It would be like comparing two students in terms of how many course credits they attained, where the first student passes their courses if they achieve a mark of 50% or better while the second student only passes if they score 90% or higher. With that advantage, the first student is much more likely to pass his courses and achieve a higher overall number of passes. Even if the second student is exceptional and the first student is mediocre, it is likely that the first student will have a similar or better score because of their inherent advantage.

I ran the formula for every team since the 1997-98 season, including an adjustment for average league goals and shots per game.

Top 10 since 1997-98:
1. 2001 Devils, .860
2. 2004 Senators, .866
3. 2000 Blues, .867
4. 2003 Senators, .868
5. 2008 Red Wings, .869
6. 1998 Blues, .872
6. 1999 Blues, .872
6. 1998 Stars, .872
6. 2006 Red Wings, .872
10. 2001 Avalanche, .874
10. 2003 Blues, .874
10. 2009 Red Wings, .874

Bottom 10 since 1997-98:
1. 1998 Lightning, .937
2. 2002 Thrashers, .935
2. 2000 Thrashers, .935
4. 2003 Panthers, .934
5. 2002 Blue Jackets, .933
6. 2002 Panthers, .932
7. 1999 Lightning, .931
7. 2000 Islanders,. 931
7. 2004 Panthers, .931
10. 2001 Wild, .930
10. 2004 Blue Jackets, .930

If you ever wondered how Roman Turek managed to get 42 wins in a season, or how Patrick Lalime won 39, here's your answer. On the other hand, note that four of Roberto Luongo's teams show up in the bottom 10. Why didn't the Florida Panthers make the playoffs? Because the team was terrible. It had nothing to do with the goaltending.

Note that these are team totals that need to be achieved, which make it even more difficult for goalies on the worst teams than it appears at first glance. If they have a backup who plays around 20-25 games at .900, then the starting goalie would need to be at .940 or better for the team to have a goal differential of zero. Even then, the team is unlikely to make the playoffs without scoring more goals than they allow.

The average win/loss record of the teams in the top 10 list was 48-22-12. The average win/loss record of the teams in the bottom 10 was 22-46-14. What is interesting, however, is that the goaltending performance was quite similar:

Save % of top 10 teams: .905
Save % of bottom 10 teams: .904

The teams on the top list didn't win because of great goaltending or because their goalie gave them clutch saves. They won because their teams scored a lot of goals and didn't allow many shots against. Similarly, the teams on the bottom list lost because they struggled to score and allowed too many shots against, not because their goaltenders were poor. This is further proof that win totals are a team stat, and should not be used to evaluate individual goalies.