Thursday, November 27, 2008

My Definition of a "Money Goalie"

Patrick Roy's number 33 was recently retired by the Montreal Canadiens, prompting journalists everywhere to find a hundred different ways to say, "Patrick Roy was a clutch goalie". One of the most common cliches in this context, of course, is "money goalie".

I think nearly all talk about goalies who are supposedly "money goalies" or are considered "clutch" is a result of the most basic goalie observation bias - the goalie who is in front looks better because every time he makes a save it looks like he is saving the game. Most games simply come down to whatever team had the better scoring chances, but in any close game you can always use hindsight and selective memory to find a save that looks like it "won" the game, and of course this is what gets focused on by the team's fans and by broadcasters and newspaper writers in the game summary. "He made the key saves at key times in the game," they say, or some other similar cliche.

However, this is not unusual. All goalies, including the bad ones, stop at least 85% of the shots against them, even in the most high-leverage situations like late in the third period. If there are 5 minutes to go in a game and your team is up by a goal, the odds are strongly in your favour even with a terrible goalie like Dan Cloutier in net. The team has to just hold the opposition to a few shots against the rest of the way and it is likely they will win based on probability alone. It is even more likely if they manage to add an insurance goal or two. That's why teams have a combined 213-19-32 record this season when leading after two periods. And then of course Cloutier gets named the game's first star for "making the big save", the save that "preserved the win"...

Of course you can also find a number of saves by the losing goalie that kept his team in it, but they aren't as likely to make the highlight reels or create a reputation for the goalie of being the guy who "makes the key saves to keep his team only a single goal down".

Playing goal is a percentages game. The difference between a Hall of Famer and an OK goalie is that the great netminder stops an extra 2-3% of shots, or about 1 out of every 50. It is easy to say that someone like Grant Fuhr stopped the vast majority of the shots against him while his team was up by a goal in the third period, but all goalies stop the vast majority of shots against them in all situations. The investigative question becomes whether Fuhr was actually playing better or whether he was just in that game situation so often that a few timely saves linger in the memory banks long after hundreds of similar scoring chances have been forgotten.

Name anyone often commonly referred to as a "money goalie", and I bet you they played on a good team. In my view, the term "money goalie" is a term that gets applied almost exclusively to goalies who play on very good teams and therefore spend a lot of time playing with the lead. Obviously goalies have some impact on the game, so a great goalie will make it more likely that his team is in the lead, but the rest of the teammates combined have a much greater impact than the goaltender alone.

This mythology usually extends further. People tell stories about Billy Smith and Gerry Cheevers and say things like, "All they ever cared about was winning. They would give up meaningless goals in regular season games, but when the chips were down they were unbeatable." Fine, I don't necessarily buy it, but that is at least plausible that somebody would increase their level of focus and effort when it mattered most. However, here is my problem with this line of reasoning: If you accept the premise that a goaltender is not completely responsible for his team's win/loss record, which I believe any reasonable person would, then it is quite likely that a goalie with such a competitive mindset would nevertheless lose a game or a playoff series, despite his best efforts. After all, goalies like Roy and Fuhr still had more early playoff exits than Stanley Cup victories, even with outstanding teammates around them. How can we be sure that goalies like Gilles Meloche, Cesare Maniago, Gary Smith, Dan Bouchard and Mike Liut didn't have the same focus on winning that Billy Smith and Gerry Cheevers allegedly did? Was it merely because they were never dealt the good fortune of playing on teams stacked with Hall of Famers?

I want to know which goalies "only cared about winning" and "came up big in the clutch" yet played on a mediocre team and had little team success. If we cannot identify anyone who meets that description, then that tells me that clutch play is difficult to identify objectively and that many of the goalies on great championship teams are almost certainly getting too much credit for their contributions to that team success.

Monday, November 24, 2008

Brian Boucher Is Exposing The NHL's Most Overrated Goalie

Q: How do you win 10 out of 12 games despite a mediocre .891 save percentage?

A: Play for the San Jose Sharks.

Evgeni Nabokov is probably the most overrated goalie in the league. He didn't deserve his Vezina runner-up finish last season, and he doesn't deserve his status as one of the league's elite goalies (for example, calls him "arguably the best goalie in the NHL"). This has become even more clear over the last couple of weeks, as Nabokov has missed several games with a knee injury and the Sharks haven't missed a beat with backup goalie Brian Boucher between the pipes.

Boucher has posted outstanding results since San Jose acquired him last season. He has gone 10-2-2 with a 1.85 GAA, .929 save percentage, and 3 shutouts. Unless Boucher has morphed into an elite goalie on the wrong side of 30, he has either been on a very lucky streak or he is playing on a strong team defence. It is probably true that Boucher has been at least somewhat lucky, and he may have been playing mostly weak opposition, but it would still be very unlikely for him to post similar results over 14 decisions on nearly every other team in the league.

Even without Boucher's success, there is still lots of evidence to argue that Nabokov is overrated. Nabokov's shot-quality adjusted save percentage numbers are very ordinary. Only one of his post-lockout seasons has been above league average (2006-07), and even that one was by a small margin.

Another problem with Nabokov is that he is erratic. His month-by-month save percentages are all over the place (e.g. last year his month-by-month line went .916, .929, .915, .890, .867, .941, .909). He almost seems to alternate between good and bad years throughout his career, and some of his seasons have been quite poor (2002-03 and 2005-06).

Nabokov makes some flashy saves and when he is on his game he can be pretty good. I can understand that there are people who caught the right sample of Sharks games could think Nabokov is a terrific goalie. However, every game counts, and over the long haul Nabokov's results are not elite.

What is interesting to me is that San Jose fans, who are not likely to be biased from a small viewing sample, seem to have a very different perspective of Nabokov than the numbers do. The opinions I have read may not be representative of all San Jose fans, but most of them that post or comment online appear to have the perception that their defence is weak and giveaway-prone, and that Nabokov has to make a lot of difficult stops. That conflicts with the shot quality numbers (San Jose has been consistently 5-7% better than average in shot quality against over the last 3 years), it conflicts with the giveaway numbers (looking at road giveaways only to remove scorer bias, San Jose ranked 3rd in the league in fewest giveaways in 2007-08), it conflicts with the results of Boucher and other Sharks goalies, and it conflicts with the shot prevention numbers (San Jose ranked 2nd in fewest shots allowed last year, and currently rank 2nd again this year). In this type of situation I think an objective analyst should at least be open to the possibility that the numbers aren't telling the whole story, but Nabokov is hardly the first high win total/low save percentage goalie to get lots of love from his local fanbase so for now I'll trust the evidence more than the hometown fans.

Nabokov is currently sidelined with an injury but is expected to return soon. If he gets nearly all of the starts the rest of the way and the Sharks continue to dominate, Nabokov could once again lead the league in wins. That may be difficult, since Boucher's success will likely result in San Jose throwing a few more starts his way, but it would be interesting to see a goalie win 45+ games with a sub-.900 save percentage, if nothing else as an interesting test case for Vezina voters.

In somewhat related news, Toronto fans seem to be souring on Vesa Toskala. This is not that surprising, since Toskala hasn't been anything special at all since the lockout, either in San Jose or Toronto. Here is a comparison of Toskala and Nabokov in San Jose:

Toskala, SJS: 65-28-10 (.680), 2.34, .914
Nabokov, SJS: 218-152-48 (.579), 2.38, .910

Now, here is Toskala so far in Toronto:

Toskala, TOR: 40-31-10 (.556), 2.86, .898

Makes one wonder how well Nabokov would do on a weaker team. I'm not sure he would do a whole let better than Toskala has. San Jose gets a lot of credit for their work in developing goalies, but I think much of it may simply be a result of the strong team and goalie-friendly climate in which they play.

Saturday, November 22, 2008

Goalies Always Get the Blame

Are NHL teams good at evaluating goalies independent of their team context? I think there is a lot of evidence that they certainly aren't perfect.

Take, for example, this information that I collected on the number of goalies each team has used in the last 10 years (since 1998-99). What stat is the best predictor of the number of goalies? Turns out it is GAA. Most observers would agree that GAA is strongly team-influenced, especially for goalies who play on very weak teams. Of the 10 best teams in GAA, only Ottawa and Philadelphia used more than 11 goalies. Of the 15 worst teams, everyone except Columbus used more than 11 goalies, and some of the teams used 20 or more.

Some of this effect is legitimately a result of goalie quality, but the fact that so many different goalies failed to make an impact on the league's weakest teams shows how tough it was to play there. It also shows how much more stable the goaltending situation tends to be on a top defensive team, and why goalies who come into the league on one of those teams have a much better chance at staying in that position for a while.

Here is the list of teams with GAA and number of goalies they have employed in the last decade:


(Post edited to correct errors)

Friday, November 21, 2008

Goalie Hyperbole, Part 1,542

Damien Cox, who has been known in the past to massively overrate the impact of goalies in hockey (which may explain his enthusiastic support of Marty Brodeur as the best ever), is at it again.

Here is the money quote:

"There have been many who have suggested the sport shouldn't be called hockey, but rather, goalie. The identity of the player who fills the crease, after all, is the major determing factor when it comes to the success or failure of a hockey club."

That must be great news, then, for the Tampa Bay Lightning, who currently sit 4th in the NHL in save percentage behind Mike Smith's goaltending. With that kind of production at such an overwhelmingly important position, they must be having a great season. Oh, wait.

Other top-10 save percentage teams languishing at the bottom of the standings include Phoenix, Ottawa and Florida. Meanwhile, the two best teams in the league are San Jose and Detroit, with their scintillating .906 and .892 save percentages respectively. I guess there must be other factors at work in NHL games other than goaltending! Who knew?

Correlation of goals for and winning percentage in 2008: .611

Correlation of save percentage and winning percentage in 2008: .374

If we are going to rename hockey, I think "scoring" should rank a lot higher on the list than "goalie".

Monday, November 17, 2008

Shots Attempts Against

With the issue of shots against featuring prominently in recent discussion here and elsewhere in the blogosphere, I thought to take a slightly different look at the issue. Generally attempts to define the shots against/save percentage relationship look at total numbers by season (like this one, for example). I was interested in how teams performed over multiple seasons, and how similar shot preventing teams did in terms of save percentage.

Based on the results from the even-strength save percentage while tied numbers, I was also interested in exploring the results of blocked shots and missed shots against. The frequency of blocked or missed shots is quite high. Over the last 8 seasons, there were 555,531 shots, 215,508 blocked shots, and 208,889 missed shots, meaning that 43% of shot attempts never make it on net. It would therefore seem that a team's commitment to shot blocking could have a big impact on the number of shots actually faced.

The "fewer shots = lower save percentage" argument is usually based on one of the following two premises: 1. Goalies who face infrequent shots lose focus and are less physically prepared for each shot that comes than those who are facing more frequent shots, or 2. Goalies who face infrequent shots are facing higher quality chances. The first one could possibly be true, but based on my personal experience I do not think it is a major factor. It is, however, a difficult one to test (it would require analysis of the play-by-play records to do it properly), so this post is directed at premise 2.

Before I continue, just a caveat: with RTSS data there is always the possibility of systematic errors. It is, in fact, quite likely that teams differ in their reporting criteria, based on what we have discovered from past shot and shot distance reporting results. I will present the data as is, but if there is reason to believe that the numbers aren't quite correct please point it out.

I took the last 8 seasons (1999-00 to 2007-08), and collected the total minutes, shots against, saves, blocked shots and missed shots for each team. I then divided the teams up into 5 groups, based on where they rank in shot attempts against (Att), defined as saves plus goals allowed plus blocked shots plus missed shots. I have also included shots against (SA) and save percentage numbers for each team.

Low Shot Group:
Anaheim: 46.4 Att, 28.2 SA, .913 Sv%
Detroit: 46.5 Att, 25.9 SA, .910 Sv%
Chicago: 46.7 Att, 28.2 SA, .899 Sv%
Dallas: 46.9 Att, 25.0 SA, .910 Sv%
San Jose: 47.1 Att, 27.0 SA, .910 Sv%
New Jersey: 47.5 Att, 25.5 SA, .912 Sv%

If a low shots against total was the only handicap preventing Martin Brodeur from posting elite save percentage numbers then we would expect the Devils' goaltending to outperform the rest of this group. The fact that Anaheim, Detroit, Dallas and San Jose have virtually identical save percentages while facing even fewer shot attempts than New Jersey seems to be strong evidence against that viewpoint.

Anaheim and Chicago do not block many shots, at least according to NHL scorers, so even with average shot totals they both move up into the top group here. Chicago is definitely the outlier, finishing far behind the other teams. Could the Chicago scorers possibly be underreporting blocks and missed shots? Or do the Hawks simply suffer from bad goaltending and/or high shot quality against?

Moderately Low Shot Group:
Boston: 48.4 Att, 28.5 SA, .903 Sv%
Nashville: 48.7 Att, 29.3 SA, .912 Sv%
Ottawa: 48.8 Att, 26.8 SA, .908 Sv%
Vancouver: 48.8 Att, 27.2 SA, .906 Sv%
St. Louis: 48.9 Att, 25.4 SA, .901 Sv%
Calgary: 48.9 Att, 27.5 SA, .907 Sv%

St. Louis ranks near the very best in shots against, but the reason seems to be that they block so many shots. Given the goaltending they have had that is maybe not too surprising, but lumping in St. Louis with the top possession teams during the study period appears to be incorrect. Ottawa is similar to the Blues. On the other hand, Nashville apparently lets a lot more shots through, and they had the best save percentage in this group.

Average Shot Group:
Colorado: 49.3 Att, 27.4 SA, .912 Sv%
Tampa Bay: 49.5 Att, 28.3 SA, .898 Sv%
Philadelphia: 49.8 Att, 27.1 SA, .908 Sv%
Toronto: 49.9 Att, 27.9 SA, .903 Sv%
Buffalo: 50.0 Att, 27.9 SA, .909 Sv%
Minnesota: 50.0 Att, 28.5 SA, .916 Sv%
Phoenix: 50.0 Att, 29.4 SA, .905 Sv%

The average group had the widest range of save percentage results, from Minnesota (best in the league) to Tampa Bay (worst in the league). Even though Minnesota blocked a normal number of shots and allowed an average number of shots on goal, their goalies had very high save percentages, which suggests that Jacques Lemaire knows how to make life easier for his goalies. Both Colorado and Buffalo had good results even after Roy and Hasek left town, which suggests that they were good defensive teams.

Moderately High Shot Group:
Columbus: 50.3 Att, 30.3 SA, .905 Sv%
Carolina: 50.4 Att, 27.5 SA, .901 Sv%
Los Angeles: 50.8 Att, 27.7 SA, .900 Sv%
Washington: 50.9 Att, 29.9 SA, .905 Sv%
Edmonton: 50.9 Att, 27.2 SA, .904 Sv%

The save percentages are very similar for all 5 teams and none of these teams had elite goalies in the period (other than maybe a couple of Kolzig seasons), so the results likely generally reflect team shot quality against. Edmonton had the second highest total of blocked and missed shots in the league while Columbus had one of the lowest, creating a difference of over 3 shots on goal per game, but the two teams had almost identical save percentages.

High Shot Group:
NY Islanders: 52.2 Att, 29.6 SA, .904 Sv%
Pittsburgh: 52.2 Att, 30.4 SA, .900 Sv%
Florida: 52.9 Att, 31.9 SA, .913 Sv%
NY Rangers: 52.9 Att, 29.1 SA, .904 Sv%
Montreal: 53.3 Att, 30.1 SA, .914 Sv%
Atlanta: 53.3 Att, 31.6 SA, .898 Sv%

For teams that give up a lot of chances, the norm appears to be mediocre save percentages rather than higher ones. Florida and Montreal are the only above-average teams and they rank far ahead of everyone else.

Here are the overall averages for each group:

Low Shots: 46.9 Att, 26.6 SA, .909 Sv%
Mod. Low: 48.8 Att, 27.5 SA, .906 Sv%
Average: 49.8 Att, 28.1 SA, .907 Sv%
Mod. High: 50.7 Att, 28.5 SA, .903 Sv%
High Shots: 52.8 Att, 30.5 SA, .906 Sv%

There is no real evidence of a pattern in terms of goaltending success. There are a few teams that have outlier results based on their groupings, either underperforming (Chicago, St. Louis, Tampa Bay) or overperforming (Nashville, Florida, Montreal, Minnesota) their expected save percentages. If we remove the best and the worst team in each group to deal with potential outliers, the save percentages by group go .911 - .906 - .907 - .903 - .905. This evidence certainly doesn't show that playing on a top defensive club hurts one's save percentage; if anything it suggests the opposite. However, save percentage is not a perfect proxy for shot quality - the goalies themselves obviously have an impact. If top teams tend to have better goaltending, for instance, then we would expect this result.

These numbers are polluted by a few variables, like suspect RTSS data and special teams play. The topic of possession, outshooting, and scoring percentages is still being investigated. However, I remain unconvinced that playing on a strong defensive team makes it tougher to put up high save percentages. I did not really evaluate possession effects here, merely shot prevention, so it could be that there is an effect for outshooting teams. In any event, even if a general relationship is established, this post is evidence that there is still substantial variability from team to team even within similar shots against groupings. This variability means that we cannot necessarily go from an established general result to the specific case for individual goaltenders or individual teams without additional supporting evidence.

These findings also show significant differences in shot-blocking tactics, which support the use of Fenwick or Corsi numbers rather than raw shot totals to evaluate team puck possession. Finally, there is some evidence of a relationship between blocked shots and goalie save efficiency (-0.30 correlation between blocked shots per game and save percentage). It remains to be determined whether this may be because shot blocking makes it harder to stop pucks, or because teams with bad goaltending simply try to make more saves themselves.

Saturday, November 15, 2008

ES Save % While Tied

There has been a very insightful investigation of the effects of possession on scoring success going on at Mc79hockey (check it out here and here). The key takeaway is that teams play to the score, and in-game tactics appear to have a large effect on both outshooting and scoring percentages.

To remove game score as a factor, Vic Ferrari wrote an app at his website to look at even-strength play with the score tied. I ran the numbers for each team in the league during the 2007-08 season. The full table is below, but here are a couple of comments:

Correlation between number of shots faced and save percentage: .097. Correlation between shot differential and save percentage: .063. Based on this sample, outshooting appears to have little effect on save percentage at even strength in a tie game.

Correlation between blocked shots and save percentage: -0.239. I'd like to see more data on this, but it seems to suggest there could be a weak negative relationship between blocked shots and save percentages (i.e. it is tougher for a goalie to play on a team that blocks more shots).

These results are more sensitive to luck/randomness since the sample size is smaller. There appears to be a clear shot quality effect, however, since most of the elite defensive teams show up near the top with weaker teams near the bottom. There is also a gap of .041 between the best and worst teams.

I compared the teams' results with Alan Ryder's shot quality numbers. Most teams had similar rankings, but there were a few teams with large discrepancies. Either those teams had very good or very bad goaltending, or they were very good or bad at protecting leads or playing from behind, or there was something else going on.

Better than expected:
Anaheim: 29th in shot quality, 1st in ES SV% while tied
N.Y. Rangers: 25th in shot quality, 6th in ES SV% while tied
Dallas: 27th in shot quality, 17th in ES SV% while tied
Montreal: 17th in shot quality, 8th in ES SV% while tied

Worse than expected:
Columbus: 1st in shot quality, 20th in ES SV% while tied
Washington: 4th in shot quality, 22nd in ES SV% while tied
Buffalo: 13th in shot quality, 30th in ES SV% while tied
Calgary: 3rd in shot quality, 18th in ES SV% while tied
Tampa Bay: 14th in shot quality, 29th in ES SV% while tied

Anaheim and Buffalo are the most interesting ones here. I don't think that either Giguere is that good or Miller is that bad. It could be that both teams have extreme results in the opposite direction when they are leading or trailing. However, I suspect that for both of those teams shot quality simply may not accurately measure the true difficulty of scoring chances against.

Assuming the RTSS shot distance figures that make up the shot quality rating are more or less correct (and these are road numbers only, to reduce rink reporting bias), the difference would therefore have to be either because of the average shot angle, the average defensive pressure on the shooter, or the quality of the average opposing shooter. Behind the Net has a shot quality measure that takes into account shot angle. According to those numbers, Giguere faced shots of about average difficulty, while Miller faced slightly easier than average shots against. It looks like the Ducks probably kept more of their shots away from the middle of the ice than normal, but that doesn't explain the entire difference.

I would suspect that as an offensive team Buffalo might be giving up more rush chances and more space to opposing forwards, which allows them more time to evaluate their options and make their shots. Anaheim's defence-first philosophy likely creates more pressured shots, which are easier to stop. Another possible explanation is that Anaheim has an elite checking line which may help their goalies by reducing shots against from the opposition's best players.

I would love to see these numbers for when teams were leading and trailing as well, to see both the overall save percentage effects and which teams were great at locking it down or opening it up.

Here are the full numbers for 2007-08 at even-strength, game tied:

3.New Jersey67541.939
4.N.Y. Islanders74546.938
6.N.Y. Rangers59940.933
16.San Jose58845.923
23.St. Louis70459.916
28.Los Angeles70865.908
29.Tampa Bay71369.903

Friday, November 14, 2008

New Jersey's Forgotten Star

Many hockey observers, especially New Jersey fans themselves, claim that the current New Jersey Devils squad is a motley crew of castoffs and journeymen, held singlehandedly together by Martin Brodeur. I must admit I don't really understand why such a large portion of a fan base seemingly comes down so hard on their own players like Devils fans do, although it probably has a lot to do with high expectations and standards resulting from New Jersey's recent history of championship teams. I especially don't understand it since those fans are wrong. New Jersey still has at least one other star player remaining on its team, a player that has been one of their most critical pieces for the last decade. That player is Patrik Elias.

Since 1999-00, Patrik Elias has scored 231 goals and 314 assists for 545 points in 593 games, with a +130 rating. The points may not seem all that impressive, but he put up those numbers playing on a team with a primary focus on defence. In the one season the reins were loosened a bit, 2000-01, Elias scored 40 goals and 96 points to finish 3rd in the league in scoring.

Elias contributes in many more ways than the scoresheet. He combines his offensive strengths with excellent defensive play, and his plus/minus rating reflects his two-way dominance. A recent development in hockey analysis has been the use of the Corsi number, which is the difference between the number of shots directed at the opposition net and the number of shots directed at the player's own net while he is on the ice. Last season, Detroit Red Wing players dominated the Corsi numbers as they were the most dominant outshooting team in the league. The best player in the league in terms of Corsi numbers who did not play for Detroit? Patrik Elias.

But didn't Elias have a bad year last year? Not really, he was just unlucky. With Elias on the ice, New Jersey took 32 shots per 60 minutes of 5-on-5 play, and allowed just 21. Elias' only problem was that when he was on the ice this season the goaltending just happened to be much better at the wrong end of the ice: the save percentage on shots by him and his teammates was .930, while it was just .904 for shots against. Elias' shooting percentage has dropped in both of the last two seasons so he may be losing his scoring touch, but this season it afflicted his linemates as well. Some of this may be attributable to a difference in shot quality, but it seems unlikely Elias and his linemates were giving up large numbers of dangerous scoring chances against. When Elias is playing, the puck is usually in the other end of the ice and that is a tremendous advantage for his team as well as for his goaltender.

Perhaps the best way to express Elias' impact on the team is to show New Jersey's record with him in the lineup compared to without. Last season, they were 3-4-1 with Elias out of the lineup, and 43-25-6 with him. In 2006-07, 4-3-0 without Elias, 45-21-9 with him. Two seasons ago, when Elias missed a substantial amount of time, the Devils were 19-18-7 without him, and 27-9-2 with him. Overall for those three seasons, the Devils were a .508 team (26-25-8) without Patrik Elias, and a .660 team (115-55-17) with him in the lineup.

New Jersey's goals for and against splits were substantially different with and without Elias:

with Elias:

2007-08: 2.47 GF, 2.28 GA
2006-07: 2.53 GF, 2.31 GA
2005-06: 2.92 GF, 2.50 GA

without Elias:

2007-08: 1.88 GF, 3.00 GA
2006-07: 2.29 GF, 2.86 GA
2005-06: 2.77 GF, 2.95 GA

How did the Devils do without Martin Brodeur over that same time period? New Jersey was 6-7-5 in games without Martin Brodeur for a .472 winning percentage, and 135-73-20 with MB30 between the pipes (.636). That's a gap of .164, compared to Elias' difference of .152 (although the "without Brodeur" sample is much smaller). New Jersey allowed a lot fewer goals with Brodeur in net (2.30 per game) than with his backups (3.20).

Interestingly, though, the defensive impact of Elias was not too far behind that of his goaltender. With Elias the team allowed 2.34 goals per game, and without him it was 2.95. Most of the games the Devils played without Elias came in 2005-06, a season where the team played poorly during the first half and got on a roll in the second half. Elias missed the entire first half and played most of the second half, which was the ideal scenario to put up positive with/without win/loss splits. It is likely that Elias' contribution explains some of the improvement, but there were probably many other factors at play. However, even if we exclude 2005-06 entirely and look at just 2006-07 and 2007-08 the results are almost exactly the same. In those two seasons, the Devils were 7-7-1 without Elias (.500) and 88-46-15 with him (.641). There was also a combined average of 2.30 goals against per game with Elias in the lineup compared to 2.93 goals per game without him. The numbers certainly suggest that Elias has a very strong impact on New Jersey team success and goal prevention. They are especially impressive when you consider that when Elias was not available, the Devils would most likely elevate an established NHLer from the second or third lines to take his spot. Because of this it would have been reasonable to expect the Elias differential to be much smaller than the gap between Brodeur and his replacements (primarily infrequently-used backup/minor-leaguer Scott Clemmensen), yet the effect on team success by Elias and Brodeur might not be that different at all.

Vic Ferrari did a post on this topic last year, investigating the difference between when a star player is playing compared to when they are out. Often teams do much worse without their best player, since it creates a ripple effect throughout the lineup - the great player benefits his teammates not only directly by playing with them, but also indirectly makes things easier for the rest of the team by taking on extra minutes and tougher opponents, as well as often drawing penalties or ending their shift in the offensive zone. Earl Sleek made a similar "trickle-down" argument for explaining the importance of Scott Niedermayer to the Ducks in the second half of '07-08.

These types of effects are a lot more difficult to see. Everybody knows when the backup goalie is in the game, but not everyone notices when a second-pairing defencemen has to suddenly start playing 25 minutes a game, or when a second-liner accustomed to playing against checking forwards finds himself moving up to play against the opposing top line, or when a rarely used forward has to be pressed into service as a replacement on the penalty kill unit. And this subtlety is another reason why the importance of goaltending tends to be overrated compared to other positions. Valuable, of course, but still generally overrated.

Is Patrik Elias more valuable to the Devils than Martin Brodeur? I don't think so, but Elias is nevertheless underrated player. Led by Elias, the Devils have a strong group of forwards that usually wins the territorial battle and outshoots the opposition. This talent is recognized by informed hockey fans - over 80% of the people who voted in a recent poll on this site predicted New Jersey would still make the playoffs despite losing Brodeur to injury. Unless Elias or several other key forwards join Marty on the injury list then I agree with that assessment.

Wednesday, November 12, 2008

More Musings on Career vs. Peak

I think one of the irreconcilable differences when it comes to doing all-time rankings is an observer's personal perspective on the weighting of peak vs. career. I dislike the expression "agree to disagree", because I think it is usually a retreat tactic employed by someone who has run out of arguments, but I think it sometimes legitimately comes into play during all-time debates when an impasse has been reached because of a fundamental difference in evaluation criteria.

Conceptually, I think there is one group of people that approaches the issue as if they were a general manager looking to build a hockey team in an expansion draft. The draft eligible group consists of every 18 year old kid that has ever played hockey, and their goal is to pick the one that provides the highest contribution to that team over their entire career. In this scenario, things like durability, consistency, and longevity are important considerations.

The other group comes at it from a totally different angle. They tend to think of themselves as the coach of a one game playoff. They can pick anybody who has ever played to be on their team, and each player will participate during the prime of their careers, at the absolute height of their skills and abilities. In this scenario, durability and longevity are very much secondary considerations.

This is an oversimplification, because I don't think you will find many people that will take either case to their extremes (e.g. take Mark Recchi over Guy Lafleur, or Jose Theodore over Ed Belfour). Both peak and longevity should have some weighting, but at the end of the day everyone is going to favour one or the other.

Anyone who has read more than a few posts on this blog will be aware that I fall into the second group. For that matter, anyone who has just read this website's title could probably guess that correctly. I've said this many times before, but I am far more interested in how good somebody was than how valuable they were. When a player has retired, and a few decades have passed, and the memories have mostly faded away, the only thing that sticks in the collective recollection of hockey fans is the talent and skills that player displayed when they were at their best. Bobby Hull unleashing a wicked slapshot, Rocket Richard cutting towards the net, Bobby Orr wheeling up ice, and so on. For anyone who is over 25, picture Wayne Gretzky as a player. Is he wearing the jersey of either St. Louis or New York? I didn't think so. When your grandchild asks you, "How good was Dominik Hasek?", he is going to be a lot more interested in how Hasek played against the Canadians and Russians in Nagano than how he did as a 40 year old platooning with Chris Osgood.

If you had a time travelling machine and you went back in time, collected every goalie who has been a top starter in the NHL at their peak, gave them all access to the same nutrition, training, equipment, etc. (to minimize era effects, as otherwise you would just take whoever is the best in the league today), and then lined them up against a wall to pick sides, I might not even glance Brodeur's way until a dozen guys are already off the board. But if I am a GM looking over the 18 year old versions of every goalie who has ever played NHL hockey and trying to figure out who I want in net for my team for the next 2 decades, I would give Brodeur a much closer look.

In summary, when I rank hockey players, I tend to divide them into tiers. Within each tier, things like longevity and durability and so on become important, because if you had to choose between two guys of similar abilities then you would rather have the guy you can count on to deliver every single night. But a flaky superstar always beats a consistent very good player because he is on a different level, and all the consistent 30 goal or 80 point seasons in the world don't bridge the gap, in my estimation. Which is why I don't consider career records to be that important, whether they are career wins, shutouts, goals, assists, passing yards, strikeouts, whatever.

Here's a discussion question, for those who like the career value approach: Let's say Chris Osgood, through some experimental genetic engineering or possibly a fortunate archaeological find, is able to stay eternally youthful and play goal in the NHL for as long as he wants. Let's also say that Osgood stays on a perennially strong team in Detroit, continues to put up his customary slightly-above-average save percentages, and averages 30 wins and 5 shutouts per season and one Stanley Cup win every 10 seasons. How many more years would Ozzy have to play before you would rate him ahead of Dominik Hasek in your all-time goalie rankings?

(In the unlikely event that there is someone reading this that doesn't like goalies, I'd suggest substituting Rod Brind'Amour, 70 points, and Mario Lemieux into the previous paragraph to get a more-or-less equivalent scenario for skaters).

Friday, November 7, 2008

Brodeur, Roy, and the Career Wins Record

The question was recently raised by someone in the Yahoo Hockey Analysis Group about whether Martin Brodeur's career win total is more impressive than Patrick Roy's, once you take into account the changes in league tiebreaking procedures (e.g. 4-on-4 OT, shootouts), the number of games played in a season, and the cyclical nature of how many games goalies play in a season. I decided to take a closer look at this question.

First of all, wins is not a very informative stat, unless you take into account the team a goalie is playing on. Goal support and the number and type of shots against all have a big impact on winning and losing. I will not be adjusting for any of those things in this post, so take the numbers with the standard critical eye reserved for goalie win totals. The objective here is to compare wins across different eras rather than to evaluate goaltenders.

First of all, for each season since 1917-18 I figured out the percentage of games that ended up in a win for one of the goalies, and used it as an adjustment factor in my analysis. In today's NHL, that figure is 100% because of the shootout. In 2003-04, before the shootout was introduced, 14% of games ended up as ties, so just 86% of the time a goalie was awarded with a win. In lower scoring eras early in the NHL's history, it was not uncommon for the percentage of games with a win to be at 80% or lower.

Another consideration was the number of games in a season. Everything was normalized to the current 82 game schedule.

The final thing I looked at was how goalies were utilized. I wanted to use a similar measurement for all the years, so I took the average of the top 6 goalies with the highest minutes played in each season. I then figured out the percentage of minutes played by the top 6, and used this as an adjustment factor. In seasons where the starters played nearly every game, such as the 1930s, the number was very close to 100%. Last season the figure was 89%. For much of the 1970s and 1980s, however, this number was much lower. For example, in 1986-87 the top 6 goalies played just 67% of the minutes.

The problem with the minutes adjustment was that it was quite variable. In the Original Six era, if one or two teams decided to platoon goaltenders for a season it would drastically reduce the overall total. I therefore decided to use a three-year average for each season, consisting of the previous season, the current season, and the following season. This smoothed the curve and allowed for, in my opinion, a better reflection of the era effect, rather than putting an excessive weighting on an individual team's roster management choices.

This allows us to calculate a season-by-season adjusted win figure, based on the win frequency, schedule length, and average level of minutes played for top goalies. Here are the all-time top 30 (up to the end of 2007-08):

RankGoalieWinsAdj Wins
1.Patrick Roy551610
2.Terry Sawchuk449558
3.Jacques Plante437555
4.Martin Brodeur538543
5.Glenn Hall407536
6.Tony Esposito423529
7.Ed Belfour484515
8.Curtis Joseph449474
9.Grant Fuhr403462
10.Rogie Vachon355447
11.Mike Vernon389443
12.Tiny Thompson284442
13.Gump Worsley335437
14.Andy Moog372434
15.Clint Benedict190430
16.Turk Broda302429
17.John Vanbiesbrouck374426
18.Harry Lumley330421
19.Tom Barrasso369415
20.Dominik Hasek389399
21.John Ross Roach219397
22.George Hainsworth246384
23.Ed Giacomin289381
24.Chris Osgood363373
25.Billy Smith305373
26.Frank Brimsek252364
27.Mike Liut294361
28.Sean Burke324352
29.Dan Bouchard286351
30.Bernie Parent271341

By this measure, Patrick Roy is still well ahead of everyone. Martin Brodeur not only has a lot more ground to make up on Roy, but he also has to first pass Plante and Sawchuk. Based on these figures and his current health situation, Brodeur probably won't pass Roy in career adjusted wins until some time in 2011.

These results show that wins were much easier to come by in the late 1990s and early 2000s. For example, coming into this season Roberto Luongo had a total of 197 wins but only 182 adjusted wins, which shows how win totals are being inflated in the current era where top goalies play nearly every game and somebody wins every single night. Of course Luongo is a textbook case of a goalie who is very underrated by wins totals because he has played most of his career on weak teams, but that is just a reminder of the limitations of the statistic.

Roy's wins record is certainly more impressive when you take into account the adjusting factors. There are many who make the mistake of comparing Brodeur's numbers directly with Roy's, but that is something that simply cannot be done without understanding the league contexts. Once you adjust for that, Patrick Roy comes out ahead in almost everything. There is no question that both Roy and Brodeur played most of their careers on outstanding teams, but at this point Roy still has to be rated the better goaltender, and that is even without taking into account their playoff records (where Roy has a large advantage over Brodeur).

This list also doesn't adjust for career length. This is a disadvantage for older goalies who tended to have shorter careers, both because they were more likely to get injured and also because there was a lot more competition for the few starting jobs that were available. That is likely the main reason that 7 of the top 10 goalies on the list played most of their careers post-1967. It is also why the longevity of Sawchuk, Plante and Hall is impressive, and why they are usually rated very highly on all-time goalie lists.

Thursday, November 6, 2008

Expected Shutouts - Original Six

There is limited shot data available prior to NHL officially began tracking shots in the 1980s, but shot data has been compiled for the period for 1955-1967. I thought to take a look at that period, which covers most of the careers of three of the general consensus best goalies ever (Hall, Plante, Sawchuk), and calculate expected shutout numbers for those three goalies, as well as contemporaries Johnny Bower and Gump Worsley.

Glenn Hall: 67 SO, 42.9 Exp SO, +24.1, +56%

Terry Sawchuk: 55 SO, 38.6 Exp SO, +16.4, +42%

Jacques Plante: 58 SO, 42.1 Exp SO, +15.9, +38%

Johnny Bower: 26 SO, 19.6 Exp SO, +6.4, +33%

Gump Worsley: 27 SO, 25.5 Exp SO, +1.5, +6%

I am a bit suprised that Sawchuk beats out Plante, especially since this sample includes nearly all of Plante's Montreal career but does not include Sawchuk's seasons from 1950-54 in Detroit where Sawchuk put up 45 shutouts. However, it is very close between the two of them and Bower for second place behind Hall, and the numbers show why all four of them are considered all-time greats.

Tuesday, November 4, 2008

How Good Is New Jersey's Defence?

How hard is it to play goal in New Jersey? We are about to get as much evidence over the next 3-4 months as we have over the last decade.

All of us, his most hardcore critics included, don't like to see players injured and wish Martin Brodeur a speedy recovery.

By the way, here are a few numbers for temporary Devils #1 goalie Kevin Weekes:

Career save percentage: .902 (league average during career: .906)

Post-lockout shot quality neutral save %: .888 in 2954 minutes played

Career SO: 19. Career Expected SO: 17.3.

vs. Backups ('99-00 through '05-06):
Weekes: .414 win %, 2.81, .905
Backups: .418 win %, 2.71, .905

Kevin Weekes is an average-to-slightly-below-average goalie, or at least he was one a couple of seasons ago. Weekes hasn't played much over the last two seasons, so the small sample size limits our ability to evaluate him, but if he can regain his pre-lockout form the Devils shouldn't be hurt too badly by goaltending. With the way the rest of the team has played in the early going, I think the Devils should still be a playoff team without Martin Brodeur. But if you disagree, feel free to vote in the poll on the right or explain why in the comments.

Sunday, November 2, 2008

Expected Shutouts, Part 2

Since we have already looked at goalies who did well by the expected shutouts measure, let's continue the discussion by looking at who didn't do so well. I have put together a list of goalies who have fewer career shutouts than expected (min. 30,000 minutes played):

GoalieSOExp SODiff%
Tommy Salo3737.1-0.10%
Chris Osgood4747.6-0.6-1%
Don Beaupre1717.6-0.6-3%
Felix Potvin3233.3-1.3-4%
Kirk McLean2223.3-1.3-6%
Ron Hextall2326.0-3.0-12%
Mike Richter2428.0-4.0-14%
Grant Fuhr2529.2-4.2-14%
Olaf Kolzig3541.2-6.2-15%
Bill Ranford1519.9-4.9-25%
Mike Vernon2738.2-11.2-29%
Ken Wregget915.4-6.4-42%

I saw Chris Osgood described recently on an Internet message board as "the definition of a league average goalie", which I think is a pretty apt description. Grant Fuhr once again puts up pretty unimpressive stats. I'm surprised Mike Richter and Olaf Kolzig rank so low, but not surprised at all by Bill Ranford or Mike Vernon.

Current goalies who have done poorly so far by this metric include: DiPietro (-5%), Roloson (-6%), Gerber (-16%), Fernandez (-18%), and especially Miller (-39%!).

Finally, let's look at which goalies have the most expected shutouts. That is, if you substituted a league average goalie in to play the same minutes, how many shutouts could we expect them to have? This table shows why the all-time shutout record is so hard to break - it requires durability, good teammates, and playing in a defensive era. Only two goalies in the post-WHA era had even an outside shot at breaking Sawchuk's record, and this list shows how the guy who eventually is going to do it had quite a bit of an advantage over the rest of his peers:

RankGoalieExp SO
1.Martin Brodeur77.6
2.Ed Belfour60.2
3.Patrick Roy47.7
4.Chris Osgood47.6
5.Curtis Joseph44.0
6.Dominik Hasek43.1
7.Olaf Kolzig41.2
8.Mike Vernon38.2
9.Tommy Salo37.1
10.Nikolai Khabibulin35.6

Saturday, November 1, 2008

Expected Shutouts

The main problem with the adjusted shutout totals I calculated in a recent post is they do not take team factors into account. A goalie playing on a strong defensive team in a higher-scoring era (like, for example, Tony Esposito) would rank a lot higher by the adjusted shutouts metric than they probably deserve.

To deal with this issue, I have developed another way of looking at shutouts. It is based on comparing a goalie's shutout performance to how many shutouts an average goalie would probably have recorded while playing the same minutes and facing the same shots. The expected number of shutouts was based on the goalie's minutes played and the predicted probability of a shutout for an average goalie, which was calculated by raising the league-average save percentage to the power of the number of shots faced per game by the subject goalie. For example, if a goalie faced 30 shots per game when the league average was .900, the expected probability of recording a shutout in any given game would be .900^30, or .042. If the goalie played 4000 minutes that season, that would project to an expected season total of 2.8.

This probability calculation is based on the assumption that the goalie would face exactly the same number of shots every game, and that each shot against is equally likely to be stopped, two assumptions which are obviously not true. However, since it is based on averages, it is likely that the method will come up with a predicted shutout total that is within a reasonable range from the actual result.

To test whether this method approximates reality, I compared the expected results against actual (not including the current season). Since 1983-84, i.e. the seasons Hockey-Reference has save percentage data, the expected shutout percentage has been 4.4% of games, and the actual has been 4.8%. This is pretty close. I expected the actual result to slightly exceed the expected result because goalies do not face the average number of shots every game and it gets exponentially easier to get a shutout whenever you face fewer shots. For example, a goalie who faces 20 shots one game and 40 the next is much more likely to get a shutout in one of those two games than a goalie who faces 30 shots per game twice in a row. This method has no corrections for shot quality, so goalies that face higher than average shot quality have a more difficult time recording shutouts regardless of their shots against totals. With these issues in mind, we need to interpret this statistic by looking at ranges and relative rankings, rather than raw totals only.

To show how this works, let's look at a couple of specific examples. Cristobal Huet has 17 career shutouts, while Dan Cloutier has 15. On the career rankings list, they look pretty similar. But are they? I calculated Cloutier's expected career shutout total to be 22.7, meaning Cloutier is 7.7 shutouts (or 34%) below average. Huet, on the other hand, has just 9.8 expected shutouts, meaning he has outperformed the metric by 7.2 shutouts, or 73%. Even though they have similar career totals, Huet is among the very best and Cloutier is among the very worst when you look at performance relative to expected.

Unfortunately I cannot apply this ranking to every goalie throughout history because we have limited shot data. I therefore had to focus on goalies who played mostly in the save percentage era, and if shot data was missing I filled in the league average shots per game for those seasons. Here are the top 20 goalies who have at least 20,000 career minutes between 1982 and 2008, ranked by ratio of shutouts to expected shutouts:

RankGoalieSOExp SODiff%
1.Dominik Hasek8143.1+37.9+88%
2.Roberto Luongo3920.8+18.2+88%
3.Mike Liut2514.4+10.6+74%
4.Pete Peeters2113.0+8.0+62%
5.Greg Hanlon138.4+4.6+55%
6.Tom Barrasso3825.1+12.9+51%
7.Greg Millen1711.6+5.4+47%
8.Guy Hebert2819.8+8.2+41%
9.Patrick Roy6647.7+18.3+38%
10.Daren Puppa1914.5+4.5+31%
11.Evgeni Nabokov4031.0+9.0+29%
12.Ed Belfour7660.2+15.8+26%
13.J.S. Giguere3024.2+5.8+24%
14.Martin Brodeur9677.6+18.4+24%
15.Patrick Lalime3529.2+5.8+20%
16.Jocelyn Thibault3932.9+6.1+19%
17.Jon Casey1613.5+2.5+19%
18.Andy Moog2823.9+4.1+17%
19.Rick Wamsley1210.3+1.7+17%
20.Curtis Joseph5144.0+7.0+16%

The two usual suspects, Hasek and Luongo, end up at the top. Martin Brodeur's record compared to average is good but not exactly dominant. Brodeur does have a high career "shutouts above average" mark with +18.4, second only to Hasek, edging out Roy's +18.3 with Luongo right behind at +18.2. Hasek crushes the field with almost 38 more shutouts than average. If Hasek played Brodeur's minutes and outperformed average by the same 88% margin, the Dominator would have had 146 shutouts. Just something to keep in mind whenever somebody tries to argue that Brodeur's eventual shutout record means that he was the best.

I'll finish for now by identifying some potentially overlooked goalies who did well but have played fewer games. I'll set 10,000 minutes as the cutoff here (note that both Kari Lehtonen (+75%) and Marc-Andre Fleury (+45%) narrowly missed the minimum but would have made this list):
GoalieSOExp SODiff%
Allan Bester72.8+4.2+150%
Bob Froese135.7+7.3+128%
Murray Bannerman84.0+4.0+100%
Clint Malarchuk126.9+5.1+74%
Cristobal Huet179.8+7.2+73%
Mathieu Garon149.1+4.9+54%
Roman Cechmanek2516.8+8.2+49%
Miikka Kiprusoff2620.0+6.0+30%
Pat Riggin84.0+4.0+29%
Henrik Lundqvist1713.2+3.8+29%
Jamie Storr1612.9+3.1+24%