Tuesday, March 27, 2007

Ten Questions

Here are 10 questions for those who claim Martin Brodeur is the best goalie ever:

1. Would the best goalie of all time never once in his career lead the league in save percentage, despite nearly always playing in front of excellent defensive teams?

2. Would the best goalie of all time play his first 9 seasons (i.e. the prime of his career) without winning any Vezinas, and only twice even being named to the NHL Second All-Star Team?

3. Would the best goalie of all time not be considered the best in the game by virtually anyone until he played over a decade in the league, and his two main rivals retired?

4. Would the best goalie of all time have a better save percentage than the other goalie only 52.6% of the time in the playoffs, that is only outplay his opponent barely more than half the time, again despite playing for an excellent team his entire career?

5. Would the best goalie of all time have an 8-19 career record in playoff overtime games?

6. Would the best goalie of all time be an unquestioned backup on his country's Olympic team in his prime in the middle of one of his best seasons, and then again start the tournament as a backup four years later, even after the leading candidate for the job decided not to play?

7. Would the best goalie of all time never finish higher than 3rd in Hart Trophy voting?

8. Would the best goalie of all time rarely steal games in either the regular season or the playoffs (just 5% of regular season wins and 3% of playoff wins came when his team was outshot by 10 shots or more)? Would he also have a below-average winning percentage in such situations, both in the regular season and in the playoffs?

9. Would the best goalie of all time never win a Conn Smythe Trophy, despite 3 Stanley Cups?

10. Would the best goalie of all time go four years in a row without making it past the second round of the playoffs, despite playing for 100-point teams in three of those seasons? Would he be solidly outplayed in playoff series by Damian Rhodes, Cam Ward, Robert Esche, and Arturs Irbe?

Thursday, March 22, 2007

East vs. West

One argument being used in favour of Eastern goaltenders this year is that it is tougher to play there. The main justification for this is that 15 of the top 20 goalscorers currently play in the Eastern Conference, led by players such as Lecavalier, Ovechkin and Heatley. If the top talent is in the East, then it follows that goalies such as Brodeur, DiPietro, et al. have a harder job than their counterparts out West.

On the surface, this seems to be correct. Looking at goals per game, Eastern teams average 3.0 goals for and 3.0 goals against per game, while Western teams average 2.8 goals for and 2.8 goals against. This means that there is 0.4 more goals per game in games involving Eastern teams.

However, we need to try to find out why this is, before rushing to attribute it to the shining talents of Crosby, Ovechkin, and company. There are six major reasons why goals would be higher in one conference than in the other: more shots, more dangerous shots, better shooters, more/less penalties, better special teams, and worse goalies. Let's look at them one by one:

Although I have heard worse goalies used as a justification, this one does not appear to hold water. Seven of the top 15 goalies in shot-quality neutral save percentage (courtesy Hockey Numbers) are from the East. With DiPietro, Huet, Lundqvist, Lehtonen, Brodeur and Kolzig, the East is not lacking in goalie talent. Looking at the top 10 in SQNSV% in each conference (West .910, East .908), it appears the West has a slight edge, but it is pretty close. Based on the average of 30 shots per game, this accounts for a difference of 0.1 goals per game.

The difficulty of shots taken is not easy to measure, and we must rely on statistical measures of shot quality to estimate it. According to one model (Hockey Numbers), even-strength shot quality in the West has been 0.98, in the East 0.99. Again, essentially the same (1% difference), and certainly not enough to justify the difference in scoring.

Better shooters is the most common explanation. This, however, is not supported by the evidence. The shooting percentage for Eastern teams is 9.8%. For Western teams it is 9.7%, which is very close. This difference of 1% can be entirely explained by the 1% difference in shot quality. Therefore, the East is not any better at scoring than the West. If anything, the opposite appears to be true, given that Western goalies are a little bit better.

There is not much difference in penalties taken either. Western teams have been shorthanded on average 361 times this season. Eastern teams have averaged 359 times shorthanded, which is a difference of .01 times per team per game.

Looking at special teams, Eastern teams have been successful on 17.8% of their power plays, Western teams only 17.5%. This small edge is counteracted by the difference in penalties taken, meaning the difference between East and West in power play goals scored is all of 4 goals.

Having dismissed five of the six factors, we turn to the last one, shots taken. Finally, here is something to take note of: Western teams average 29.1 shots for and 28.7 shots against, for a total of about 58 per game. Eastern teams average 30.2 shots for and 30.7 shots against, for a total of about 61 per game. This is a difference of 3 shots per team per game, or 5%. Given the league average scoring rate, this accounts for a difference of 0.3 goals per game, which is nearly the entire gap between East and West. The difference, it would seem, is in the style of play between the two conferences - the East tends to be more open, allowing for more shots to be taken. Western teams play more of a closed game, and are more effective at preventing shots.

Having looked at the difference for the conferences as a whole, let's turn to the top talent. Is it justifiable to claim that 75% of the league's top scorers reside in the Eastern Conference? Or are lurking variables again clouding the analysis? Evidence suggests that indeed they are, and again it is shots taken that is the prime culprit.

Of the top 20 goalscorers, 15 play in the East. These 15 players have averaged 257 shots on goal so far this season. The remaining 5 are from the West. If the difference was talent, then they would be expected to have a similar average number of shots. They do not - the 5 players have averaged only 216 shots, or 41 less than the Eastern shooters.

This is a little unfair to compare the top 5 to the top 15, so let's look at the averages for the top 10 scorers from each conference:

East: 39 goals on 265 shots, 14.7% shooting
West: 33 goals on 210 shots, 15.7% shooting

This shows clearly that the difference is not talent, but simply opportunity. The top goalscorers are found in the East because they simply take more shots. The West's best shooters have actually been more effective at scoring on the shots they have taken. There is of course skill involved in creating shots, which can be attributed to the top players in East. Eastern teams, however, also tend to rely on their best players a little more. The top 10 scorers in the East average about 1 minute more per game on the ice at even strength, and half a minute more on the power play. This helps to further explain the difference in scoring.

In conclusion, claims that the league's best shooters can be found in the Eastern Conference appear to be largely overblown. Eastern Conference players have scored more goals because they take more shots than their Western counterparts, but their scoring efficiency is not any better. They are more effective at creating shots, but this is mitigated by increased ice time as well as the overall conference statistics indicating that Eastern games are more open and have more shots taken.

In terms of ranking goalies, conference effects can be almost completely disregarded. The only significant differences for the goal differential were shots taken and goaltending play. Shots do not affect the best goaltending measures such as save percentage. For other statistics, goalies should be judged based on the teams they play behind, which has much more of an impact. For example, New Jersey and Tampa Bay allow a relatively low number of shots against, despite playing the "wide-open" East, while Phoenix and Nashville allow a high number of shots, despite playing in the "defensive" West. Therefore, there is no reason to claim a conference effect is discriminating against goalies in either the East or the West, since individual team effects are much more significant.

Tuesday, March 20, 2007

The Best Playoff Goalie

I have looked at clutch play, game-stealing ability, team support, and overall playoff performance to try to determine who is the best playoff goalie of this generation. Taking all those factors into account, the goal is to arrive at a ranking of goalies in terms of their true playoff performance.

As stated at the start of this exercise, goalies are often judged on one stat alone: Stanley Cups. Based on that statistic, the rankings go: Roy 4, Brodeur 3, Belfour 1, Hasek 1, Joseph 0. However, my analysis so far has demonstrated that those rankings are not representative of each of the goalie's true performance. Too much of it is the team effect.

One of the goalies benefitted extraordinarily from team effects. That goalie was Martin Brodeur. Advantages such as facing a mere 23.8 shots per 60 minutes, facing nearly six shots less per game than his direct opponent, and rarely being severely outshot or even outshot at all contributed to Brodeur's gaudy win total and multiple Cup rings. Brodeur's head-to-head winning percentage is significantly worse than his actual winning percentage, indicating that many times the team won the game for him. In the 2001 playoffs, for example, the Devils went all the way to game 7 of the Stanley Cup finals. Despite that, Brodeur was outplayed by his opponent in net in terms of save percentage in over half the games in that playoff season, indicating it was the team driving the success rather than the goaltender. Analysis of clutch play and individual game-winning ability were unfavourable to Brodeur relative to his peers. Brodeur's overall performance is decent, but after adjusting for his era and the team in front of him, he drops behind the other four. Therefore, Martin Brodeur ranks as the 5th best playoff goalie of his generation.

Curtis Joseph has the worst numbers of the five, but he, like Brodeur, has hidden team effects. Cujo's are the other way - he faced the most shots, almost certainly the hardest shots, and got the least goal support. He almost always had to be perfect or close to it for his team to win a game. When he was at his best, he was very tough to beat, and was second only to Hasek in terms of ability to steal a game for his team. Despite his team disadvantages, his head-to-head record was almost as good as Brodeur's. Overall, this ranks Curtis Joseph as the 4th best playoff goalie.

Ed Belfour has had a solid playoff career. He was the most clutch, didn't always have the best team in front of him, and stole a few games here and there as well. However, Hasek and Roy clearly separated themselves as the best two goalies of their generation, both in the regular season and in the playoffs. Eddie the Eagle was excellent, but never raised his game to another level the way the Dominator and St. Patrick were able to do. As a result, he takes 3rd place.

Patrick Roy has an outstanding playoff reputation, and is considered by many to be the best money goalie of all time. His performance in the 1993 playoffs is one of the top goaltending performances ever. His playoff overtime record is phenomenal. There are also of course the 4 Stanley Cups and 3 Conn Smythe Trophies. It is difficult to rank him second to anyone, given all those accomplishments, and it certainly is close. But Hasek's performance is just a little bit better overall, both in terms of save percentage and head-to-head record. Hasek was also a little more likely to steal a game on his own, and Roy had the advantage of better teams. Roy certainly has a sparkling playoff career and is one of the all-time greats, but the honour of the best playoff goalie of his generation does not go to him.

The best playoff goalie, therefore, has been Dominik Hasek. This viewpoint would be considered controversial to many because Hasek ranks behind Roy and Brodeur in terms of Stanley Cups. However, just as in the regular season, Hasek's overall performance simply overwhelms his peers. He was consistently excellent in all situations. His teams were not weaklings, but they were definitely not powerhouses either. His 1999 run to the finals was one of the all-time great playoff performances, even though Buffalo fell just short. In the only time Hasek had an outstanding team in the playoffs, he won the Stanley Cup.

Final Rankings:
1. Dominik Hasek
2. Patrick Roy
3. Ed Belfour
4. Curtis Joseph
5. Martin Brodeur

Despite his often jawdropping statistics and saves, Hasek has been largely underappreciated in his time. Often dismissed by unknowledgeable fans as "lucky" and injury-prone, he has the best regular season statistics and a slew of trophies, and this analysis indicates that he has the best playoff performance as well. Imagine what he could have accomplished if North American coaches were better at evaluating goalies, and he was able to add three or four more seasons as a starter in the early '90s. I find it difficult to argue for anyone other than Hasek as the greatest goalie in hockey history.

This analysis is further evidence of Martin Brodeur's overratedness. His mention in the discussion of the greatest goalie of all time is proof of that. His playoff stats do not merit it, nor do his regular season stats. Study after study has revealed the same basic truth: Brodeur has had a massive advantage playing for the New Jersey Devils, a team that consistently outchecked, outshot, and outscored the opposition. I have him ranked as the 5th best playoff goalie of his own generation, never mind all-time. Others may disagree with the ranking, but I think even the most charitable assessment would place him no higher than the third best playoff goalie of his time. Dominik Hasek and Patrick Roy both have him trumped in regular season and playoff play, and Hasek has a better international record as well. Longevity is the only advantage that Brodeur will have in the end, but that should by no means be the decisive factor in this debate. Brodeur has had an excellent career, and has been a key member of one of hockey's all-time best defensive teams. He is a good goalie, one of the best of his time, and merits discussion as one of the top 20 ever. If he could be viewed in that perspective, rather than being hyped up to unrealistic levels on the basis of his team's accomplishments, the entire hockey community would be better for it.

Monday, March 19, 2007

Playoff Goalies - Overall Performance

Having broken down playoff performance in some detail, we now turn to overall performance measures. Continuing with the philosophy that save percentage is the best commonly available goalie stat, we'll look at that first:

Save percentage
Hasek .926, Brodeur .921, Belfour .920, Roy .918, Cujo .908

Hasek has a decent lead, Roy would come second after factoring in era, there's not much between Brodeur and Belfour, and Cujo trails the pack. Save percentage is of course influenced by various team factors, as previously discussed.

Next we look at goalie head-to-head wins, based on which goalie had the better save percentage in each individual game. This removes the impact of shots for and against, as well as any era effects. If Roy was better than his opponent in 1986, that is the same as Brodeur outplaying his counterpart in 2003. Roy may have let in 3 goals in that game while Brodeur let in only one, but in both cases they were the better goaltender and helped give their teams a good chance to win. Team effects creep in here of course (as with all goalie stats), since goalies playing for good teams will tend to face somewhat easier shots than their opponents, thus making them more likely to win. Here, however, are the results:

Head-to-head record
Hasek 64.2%, Roy 62.2%, Brodeur 52.6%, Belfour 51.3%, Joseph 48.5%

This shows some of the era effect not visible in Roy's overall save percentage numbers. There is a huge gap between Hasek and Roy at the top and the other three. Brodeur, Belfour and Cujo all have been about even-odds to outplay the goalie at the other end. Hasek and Roy have done it nearly two-thirds of the time.

Hot game frequency:
Brodeur 52.9%, Roy 47.8%, Belfour 47.2%, Hasek 46.4%, Cujo 41.2%

Poor game frequency:
Hasek 17.5%, Roy 21.1%, Belfour 21.1%, Brodeur 23.5%, Cujo 29.8%

Roy's numbers here are impressive because they are based on modern-day save percentage cutoffs. He would rank even better if the cutoffs were season-adjusted. Belfour is very similar to Roy. Brodeur has actually been quite streaky, somewhat belying his reputation as Mr. Consistency. Hasek has been the least likely to have a poor game in the playoffs.

Head-to-head record against each other:
Hasek 14-9-1, Roy 26-18-1, Joseph 21-23, Belfour 20-26, Brodeur 14-19

This is head-to-head record, not actual wins and losses, against the other four goalies. Hasek and Roy show again how dominant they were against their peers.

Looking at overall performance, it is impossible not to conclude that Hasek and Roy were head and shoulders above their peers in terms of playoff performance. They were one-two in save percentage (after factoring in era), their head-to-head records are outstanding against their peers, and they were the least likely to let their teammates down with a poor game. Hasek finished slightly ahead of Roy in nearly all the categories, so he gets the nod in top spot. Brodeur and Belfour have very similar stats, with Brodeur having a slight edge. Cujo is a little bit beneath the others, but one of the main reasons for that is of course the team factors discussed earlier.

Overall Performance:
1. Hasek
2. Roy
3. Brodeur
4. Belfour
5. Cujo

Sunday, March 18, 2007

Playoff Goalies - Team Effects

Team effects must always be considered when evaluating goalies. Some of the key considerations are goal support, amount of shots faced, and difficulty of shots faced. Lacking detailed information about shot quality, we will have to make due with simpler measures based on simply the number of shots. Other variables are slim chance games (described in the last post), as well as cheap wins and tough losses, statistics that measure how often the win or loss was unfairly credited to the goaltender, given his actual performance.

Cujo 2.34, Belfour 2.44, Brodeur 2.46, Hasek 2.65, Roy 2.89

Cujo played for four teams, none of which provided him with a high number of goals for. Even the high-powered Red Wings let him down, scoring just 1.71 goals per game with Cujo in their net. Hasek's ranking here is surprisingly high. That is not just from his 2002 Detroit Cup run either; even on the Sabres, his career support was 2.59 goals per game. Hasek fans might argue, however, that one of the reasons for the Sabres' scoring was the ability to take additional risks because of the presence of Hasek in net. Brodeur also ranks higher than one might have thought, given the Devils reputation as a boring, low-scoring team, but some of his teams were actually excellent offensively, particularly the 2000 Cup winners. Roy's amount of time spent playing in the 1980s helped inflate his goal support, although he played most of his career on good teams.

Shots Against per 60 Minutes:
Cujo 29.1, Roy 28.1, Hasek 27.6, Belfour 26.9, Brodeur 23.8

Average Outshot Margin:
Cujo 0.9, Hasek 0.5, Roy -0.3, Belfour -0.7, Brodeur -5.7

Four of them are pretty close, and then there is Brodeur. Facing almost six shots less than the other goalie every game is certainly a nice advantage to have. In addition, Brodeur has averaged under 24 shots against per game in his playoff career, over three shots less than any of the others. Clearly, he had it a lot easier than his peers.

Slim chance games:
Belfour 17, Roy 14, Hasek 9, Cujo 9, Brodeur 1

Again, it is almost phenomenal how well the Devils have protected Brodeur throughout the last decade. A slim chance game is a game where an excellent goalie would be expected to lose to a replacement level goalie, so to win such a game requires the goalie to be outstanding AND the opposing goalie to be well below average. For Belfour and Hasek, around 10% of their games were this type of situation, a significant handicap. Brodeur has only played in one of them in his entire playoff career.

Cheap wins:
Roy 7, Belfour 6, Brodeur 5, Cujo 2, Hasek 0

Tough losses:
Belfour 18, Roy 16, Brodeur 14, Cujo 13, Hasek 6

Of the five, Curtis Joseph clearly had the biggest team handicap. He faced the most shots and had the least goals scored for him. He also had the highest rate of tough losses to games played, and very few cheap wins. Even though he faced relatively few shots per game (26.9), Ed Belfour had the second worst goal support, as well as the most slim chance games and tough losses. The stats indicate that Hasek's Sabres weren't as bad a team as many remember, but Hasek faced his share of tough situations, and never had the benefit of recording a cheap win. Roy played for some pretty good teams, and interestingly enough his statistics are quite similar behind the Canadiens and the Avalanche, indicating that those Canadiens teams were probably underrated by many. Brodeur's advantage in terms of team effects is remarkable, and a testament to the sustained excellence of the New Jersey Devils. This advantage must be taken into account when evaluating his career performance.

Overall Team Effects (ranked hardest to easiest):
1. Cujo
2. Belfour
3. Hasek
4. Roy
5. Brodeur

Saturday, March 17, 2007

Playoff Goalies - Ability to Steal a Game

Game-stealing ability is something that is very rarely quantified. It is often assumed that any goalie who is good is also proficient at stealing games, and you won't be able to watch a game involving a top level goalie without broadcasters doing their best to convince you of that point. However, the numbers indicate that goalies differ substantially in terms of how many games they actually do steal or how well they do while being outshot, even despite a similar overall performance. Just because a goalie is good, does not mean they necessarily win a lot of games singlehandedly for their team. And conversely, there are a number of mediocre goalies that seem to every now and then elevate their game enough to pull out a surprising win for their teammates. So rather than going on reputation alone, I will dig into the numbers to try to identify the goalies that really are a threat to steal victory from the jaws of certain defeat.

Stealing a game is somewhat subjectively determined. It is generally assumed to be a situation where the other team is fully deserving of the victory because of its dominant play, and it is only the goaltender's individual contribution that turns the result around. That implies a team that is outplayed, and teams that are outplayed tend to get outshot. A goalie could theoretically "steal" a game with even shot totals if he faced much harder shots (for example, more power plays, more breakaways, more odd-man rushes, etc.) than his counterpart, but I do not have shot quality data for playoff games.

To rate game-stealing ability, therefore, we will turn to stolen wins, or wins when outshot by 10 shots or more. Since some teams are outshot more often, I will also calculate the winning percentage when severely outshot to measure the goalie's effectiveness at stealing games. I also have a statistic called "slim chance games", designed to calculate the number of times when an outstanding goalie would be expected to lose to a replacement level opponent simply because his team was outshot. "Slim chance wins" are wins in such games, and they will also be considered in the analysis.

Stolen wins:
Roy 18, Cujo 14, Hasek 11, Belfour 10, Brodeur 3

Times outshot by 10+ shots:
Roy 33, Belfour 28, Cujo 24, Hasek 17, Brodeur 9

Winning % when outshot by 10+:
Hasek .647, Cujo .583, Roy .545, Belfour .357, Brodeur .333

Slim chance wins:
Roy 12, Belfour 9, Cujo 6, Hasek 5, Brodeur 1

Dominik Hasek emerges as the leader here, winning nearly two out of every three games that the Sabres were severely outshot in. Curtis Joseph also shows a clear ability to steal games. Twenty-one percent of his career playoff wins were stolen, the highest of any of the candidates. Patrick Roy has also won a large number of games for his team over the course of his illustrious playoff career.

It is very clearly revealed here how heavily Brodeur has been shielded by the Devils defence. Even taking that into account, however, he has simply not been very effective at stealing games. Only 3% of his career playoff victories were stolen wins. Even though he has mostly played well in the playoffs, he has never been leaned upon to win games on his own, and he hasn't been successful when asked to do so. It appears that his "game-stealing" reputation is hype and not reality.

Overall Game-Stealing Ability:
1. Hasek
2. Cujo
3. Roy
4. Belfour
5. Brodeur

Friday, March 16, 2007

Playoff Clutch Performance

"Clutchness" is one of the hot topics of debate in the sports world, especially between traditionalists and sabermetricians. Is there evidence of clutch performers? It is difficult to say, although studies that deny such an effect have tended to be more convincing than ones that claim clutch players exist. The viewpoint here is that there are no clutch performers, merely clutch performances, but when we are grading goalies based on their playoff performance, then those clutch performances should be taken into account. It may not be the best way to predict which goalie is going to perform the best in this year's playoffs, but in terms of their overall playoff contribution it would be unfair if it was not taken into account.

It is somewhat difficult to isolate what exactly is a "clutch" situation. Often these are defined to be situations like leading by a goal or the last five minutes of the game. These seem to be pretty arbitrary, since goals allowed in the first period count the same on the scoreboard as goals allowed in the third. Also, game dynamics often change in those situations, i.e. some teams alter their play to a more defensive style than others, which has a hidden impact on the results. In a way, in fact, a goalie's entire playoff performance could be considered to be taking place in a "clutch" situation.

One situation can pretty safely be considered clutch, however: sudden death overtime. When it comes down to next goal wins, the pressure on the goalies is at its greatest. Overtime play should therefore be a strong factor in evaluating clutch play.

To calculate overtime performance, I looked not only at record but also at estimated save percentage. This was found by finding out the rate of shots against during the entire game, and interpolating to find the estimated number of shots against during the overtime session alone.

OT Winning Percentage:
Roy .684, Belfour .525, Hasek .500, Cujo .481, Brodeur .296

OT Save %:
Belfour .944, Roy .942, Hasek .935, Cujo .926, Brodeur .902

Patrick Roy was the undisputed king of overtime hockey. Belfour edges him in save percentage, but Roy became a legend by winning in overtime, especially during the magical run of 1993. Belfour and Hasek were also very good. Martin Brodeur has been surprisingly poor in overtime. Amazingly, despite playing less than half as many OT games as Roy, Brodeur has actually lost more of them. This is a fairly sizable strike against Marty.

Goalies also face greater pressure in elimination games. The threat of the season ending should add extra meaning and pressure to the goalie's performance. It is difficult to isolate elimination games, but certainly game sevens are pressure-packed since they are elimination games for both teams. Game sevens also indicate that teams are playing at roughly the same level - a totally overmatched team is unlikely to make it to game seven. Therefore, game sevens are also considered in the analysis.

Game 7 Save %:
Hasek .946, Brodeur .928, Belfour .921, Roy .907, Cujo .900

Head-to-head record Game 7s:
Belfour .833, Hasek .667, Cujo .571, Brodeur .500, Roy .462

Hasek has only played three game sevens, and despite losing two of them has played well in all three. Brodeur and Belfour have good records in rubbermatches. Game sevens haven't been so kind to St. Patrick, who holds the record for the most career losses in game sevens. His save percentage hasn't been great either.

Combining the two, we get one concise "clutch save percentage" statistic, including performance in OT and game sevens.

Save percentage (game 7 and overtime):
Hasek .939, Belfour .936, Roy .924, Brodeur .915, Cujo .913

Hasek barely edges out Belfour for top spot. However, Belfour has played in more overtime games and game sevens. In particular, Hasek's game seven results are from a very small sample. Belfour's overtime record is a little better as well. That's enough for Eddie the Eagle to take top spot. Roy was the best in overtime, but his game seven record was weak. With era adjustments, however, his save percentage would be a little higher. That's enough to edge out the Dominator for second place. Brodeur and Cujo are pretty close overall, but Cujo gets the slight edge because of Brodeur's awful overtime record.

Overall Clutch Rankings:
1. Belfour
2. Roy
3. Hasek
4. Cujo
5. Brodeur

Thursday, March 15, 2007

Quantifying Money Goalies

The five best goalies of the last 20 years are probably Patrick Roy, Dominik Hasek, Martin Brodeur, Ed Belfour, and Curtis Joseph. All of them have had extensive regular season and playoff success, and have been among the league leaders over a number of years of play. Their regular season stats are readily accessible and have been looked at by many, but what I am interested in is how their playoff performances rank. Who really came up the biggest when it counted? Who had the best performances, when you factor in era, teammates, and opponents?

Quite often this discussion comes down to a single statistic: Stanley Cups won. This is unfair, because it ignores all team factors. Most of the time, the better team wins a playoff series, rather than the best goalie. A hot goalie can make a difference, but it is extremely difficult and unlikely for any goalie to steal four series in a row. In fact, many of the best playoff goaltending performances ended up just short of the Stanley Cup (e.g. Hextall in 1987, Hasek in 1999, Giguere in 2003). Goalies should be judged on their performance overall rather than simply whether their team was left standing at the end.

Hockeygoalies.org has game-by-game stats for all the goalies, which allows for a thorough breakdown of performance. I will also apply my newly introduced head-to-head wins statistic, as well as other relevant situational breakdowns.

The study is divided into 5 parts, coming over the next 5 days: clutch performance, ability to steal a game, team effects, overall performance, and my final ranking of the best playoff goalie.

Wednesday, March 14, 2007

Goalie Wins - A New Perspective

As I have demonstrated repeatedly in this space, goaltenders are often evaluated by their win record. Many fans think this is logical, since wins are the main objective in hockey, and therefore it seems that the goalie with the most wins would be the best. If the goalie's contribution determined 100% of the win or loss then this would be correct, but the problem is not the inapplicability of using wins as a measure (for example, one of baseball sabermetricians' key metrics is WARP, or wins above replacement), it is that wins are impacted very heavily by other factors. Goaltending is probably worth 20-25% of the game of hockey, and at the pro level the difference between the best and the worst goalies is marginal, something on the level of a save every two games. The skaters, therefore, have much more of an impact on the final result than the goaltenders.

Another argument made is that some goalies play to the score, and they know how to make the "big saves" when their team needs them, which shows up in their number of wins. This is a common cliche used when attempting to give credit to goalies for their high win totals. This may be a little bit true - goalies likely bear down a little more in close games than in blowouts, but I doubt the effect is large. And what about a goalie whose team never scored a single goal for him? He would never have the chance to make a "big save" to keep that one goal lead - is it therefore his fault that his team never won a game? The studies I have seen of goalie performance with one goal leads or other similar situations did not reveal certain goalies to be persistently clutch. This sentiment appears to be largely a result of selective memory bias for fans of goalies on good teams.

A problem with alternative goalie stats, like GAA or save percentage, is that one or two bad performances can have a large impact on the final season totals. An average starting goalie could let in 10 goals on a particularly bad day, which could end up dropping his seasonal save percentage .005 or more and raising his season GAA by 0.15-0.20 goals. In terms of winning or losing, however, there is little difference between letting in 5 goals and 10 goals, since both result in a very high chance of losing the game. This is a benefit of wins - not overly penalizing a goalie for the results from any particular game.

Another advantage of wins over counting stats is that wins do not (for the most part) need to corrected for era. Throughout the NHL's history, season lengths have varied, but for the last 25 years this variance has been small (between 80 and 84 games). In contrast, the level of goalscoring and the average goalie save percentage have changed drastically over the same period, making it difficult to compare most goalie statistics from the 1980s with those from the late 1990s and early 2000s. The only major change has been the introduction of a shootout in 2005, which affects only regular season results.

It would be nice to have a stat that retains the intuitive logic of wins and losses, yet focuses more closely on actual goaltending performance, without taking all of the other team factors into account. Shot totals are the most significant hidden variables in wins, since teams cannot score without shooting, and teams that take more shots have a greater chance of scoring more goals. A team outshot 50-20 is very unlikely to win, even if their goaltender plays an outstanding game.

I have developed a stat that takes out the impact of era and shot totals, and is presented in terms of wins and losses. I call it "Head-to-Head Wins". I go through game by game, and measure the save percentage of each goalie against his counterpart at the other end. The goalie with the higher save percentage gets a win, and the one with the lower save percentage gets a loss. That is the stat in its basic form. I also have an adjusted version of the statistic, based on the number of times shorthanded in the game, opponent quality, and team shot quality allowed. After these adjusting factors are applied to each goalie's save percentage, the results are again compared to determine the winner or loser. For the adjusted stat, I also count any game where the save percentages differ by .005 or less as a tie, since there really is no real difference between the two goalies' performances.

This stat is more focused on the goalie's actual performance than raw wins, yet it maintains some of the benefits of the traditional statistic. Why should a goalie be rewarded for a win, even though the guy at the other end outplayed him? The goalie who plays best gets the W, regardless of the final score. This allows us to compare goalies more directly, find out who is dominant against their peers, and find out who is winning only because of their teammates. I plan to use this statistic to look at recent regular season performance, as well as use it in a comprehensive study of playoff performance.

Monday, March 12, 2007

Stealing Games

"With a goaltender like Brodeur that can single-handedly stymie opponents and win games by himself, the Sabres know the goals won't come easy." (Game preview from Sabresfans)

It is generally assumed that when a goalie is good, they will steal games for their team. What constitutes "stealing" a game is pretty vague, and can range from being totally outshot yet winning anyway, to simply ending up on the winning team. Without detailed shot quality information, it is difficult to state with certainty that a game was stolen or not, because we don't know exactly how much each goalie was tested in terms of good scoring chances. Shot totals for and against will have to suffice to estimate the relative play of each team.

I do not believe in crediting a goalie with "stealing" a game if the shots are close. If two goalies face shots of similar quantity and quality, the winning goalie has outplayed the losing one, but he did not really "steal" the game. Stealing the game implies that the goalie's contribution overwhelmed all other contributions in the game, and defeated an opponent that would ordinarily be deserving of victory. This implies that the goalie's team must have been decisively outplayed, or the victory is merely claimed or earned, but not stolen.

Goalie's World magazine tracks a stat called "stolen wins". Their definition of a stolen win is a win in which the goalie's team was outshot by 10 shots or more. This is a reasonable definition, since there would have to be a huge gulf in shot quality for a goalie facing 25 shots to have had as difficult a task as one facing 35. Stolen wins alone, however, do not take into account the fact that some teams are outshot with much greater regularity. Getting outshot badly does not mean that a team has no chance to win - teams that are behind tend to take more shots, so teams are often outshot despite having the lead. In fact, stolen wins are relatively common; 47% of the time the team outshot by 10 shots or more ends up winning the game. We would expect, therefore, the top goalies in the league to have a winning record even when outshot by a large margin.

This season, Brodeur's Devils have been outshot by 10+ shots 8 times with him in net, which as a rate is slightly below league average. Of those 8 games, Brodeur won 3 of them, for a .375 winning percentage. Brodeur ranks in the bottom half of goalies, although there are some other big names also under .400 based on this measure, including Kiprusoff and Lundqvist. The best in the league are Cam Ward (4-0), Manny Legace (5-0-1), J.S. Giguere (3-0-1), Robert Esche (4-1-0), Chris Mason (7-2-0), and Fredrik Norrena (5-2-0).

For all goalies the sample sizes are naturally quite small. Tim Thomas has faced the most stolen win opportunities with 17, winning 6 of them. Roberto Luongo is second with 15, of which he won 10. Third is Rick DiPietro (13 chances, 6 wins), and fourth is the aforementioned Kiprusoff (12 chances, 3 wins and 3 OTL), which is somewhat surprising given the defensive reputation of the Calgary Flames.

To make any solid conclusions, we need more information. I looked at this stat for Martin Brodeur for the last three regular seasons, as well as his playoff career. For comparison's sake, I included Roberto Luongo in the regular season sample.

The results were decisive: Martin Brodeur has faced much fewer stolen win opportunities than average, but he has also been much less successful than average in winning them. Over the last four seasons (including this one), his team has been badly outshot 20 times in 282 games started, or less than half the league average. In those games his record was 8-10-2.

Luongo has been outshot by 10+ shots in 68 of 271 starts over the same time period, or one quarter of the time. In 31 games over the last four seasons, and 17 games over this season and last, his team won despite the severe shot advantage. In total, 31 of Luongo's 119 wins (26%) were stolen wins, while just 8 of Brodeur's 163 wins were (5%).

In the playoffs, the difference between Brodeur and his peers is equally pronounced. He has only been badly outshot 9 times in 153 playoff games, less than 6% of the time. Of those 9 games, New Jersey won only 3 of them. That means only 3% of Brodeur's career playoff wins were in games that he stole for his team, based on the definition being applied.

Here are some playoff stats for Brodeur's contemporaries:

P. Roy: 18 stolen wins in 33 chances, 54.5 % success, 12% of total wins
E. Belfour: 10 stolen wins in 28 chances, 35.7 % success, 11% of total wins
C. Joseph: 14 stolen wins in 28 chances, 50.0 % success, 23% of total wins
T. Barrasso: 16 stolen wins in 24 chances, 66.7 % success, 27% of total wins
D. Hasek: 11 stolen wins in 17 chances, 64.7 % success, 21% of total wins
O. Kolzig: 8 stolen wins in 13 chances, 61.5 % success, 40% of total wins
N. Khabibulin: 5 stolen wins in 14 chances, 35.7 % success, 16% of total wins

If one accepts this definition of stolen wins, then Martin Brodeur is very rarely called upon to steal a game, and is not particularly good at doing so, neither in the regular season nor in the playoffs. In the last four seasons, just 5% of his wins were stolen, and in his playoff career a mere 3% can be classified as "stolen". Compared to peers like Roy, Hasek, and Luongo, those numbers are very low.

This evidence indicates that Brodeur is a solid but not spectacular goalie. He has tended to end up on the winning side, both in the regular season and the playoffs, but this is only partly due to his excellent performance, and partly due to New Jersey's ability to consistently outshoot and outchance their opponents. He has had three Cup runs where he has played well, but has not really accomplished anything "singlehandedly," an opinion apparently shared by the voters for the Conn Smythe Trophy. His supposed game-stealing ability is just another example of media myth-making around the person of Martin Brodeur.

How Well Do Goalies Rank Goalies?

In the most recent edition of The Hockey News, a panel of former goaltenders was asked to rank the current crop in terms of their performance this season.

Here' s a link to the list.

Suffice it to say I disagree with several of the selections, since as per usual the guys with the better teams in front of them (Miller, Emery, Toskala, Fleury) were ranked ahead of other more deserving players (DiPietro, Lundqvist, Legace, Huet).

Based on anecdotal viewing of former goalie-turned-broadcasters like Glenn Healy and Greg Millen, it is likely that these rankings depended heavily on subjective viewing, as opposed to statistics. However, I attempted to find out what was impacting the former goalies most in their analysis. I ranked each of the goalies in terms of GAA, save percentage, wins, shutouts, games started, and shot-quality neutral save percentage, to see which variable seemed to be weighted the most heavily.

The completely unsurprising answer? Wins.

The coefficient between goalie rank and wins rank was 0.81. The two lists follow each other closely, as the top 6 ranked goalies are also the top 6 goalies in terms of wins, and 12 of the top 15 goalies are also in the top 15 in the league in wins.

They made one particularly egregious decision that supports the hypothesis:

Goalie A: 38 GS, 2.28 GAA, .912 Save %, 7 SO
Goalie B: 31 GS, 2.45 GAA, .905 Save %, 3 SO

It would appear that Goalie A is having a decisively better year. The panel sided with with Goalie B, putting him in 9th spot. Goalie A was ranked 27th. The reason is that Goalie B, Toskala, has a 24-9-1 record, while Nabokov (A) has gone 17-16-2. This is entirely because of a huge discrepancy in goal support, which a goalie cannot impact in any way.

The correlation with save percentage was 0.5, and the correlation with shot-quality neutral save percentage was 0.16. I guess the goalies probably printed off a list of the league leaders in wins, and made a few subjective adjustments from there.

In summary, it seems that even former goalies are blinded by team factors. Players and former players are often good at identifying the most purely talented individuals, but talent does not equal performance. Marc-Andre Fleury ranks very highly in terms of raw talent, perhaps top 10 or even top 5, but his performance this year suggests that he deserves to be ranked around 25th in terms of actual play. The panel decided to rank Fleury as 10th best in the league.

Former goalies and players are often no better than average in terms of valuing overall performance and contribution to the team. Their viewpoints are coloured by media hype, team success, and the highlight reels, just like everyone else's. Therefore, just because former goalie so-and-so said something, doesn't make it true. A thorough statistical study should be taken to ensure that the subjective viewpoint matches objective reality.

Wednesday, March 7, 2007

Goalie Consistency

Consistency is often used in a vague sort of way when it comes to sports. It can become an easy solution ("We need more consistency!"), and broadcasters like to use it as a convenient prescription for a struggling player.

Consistency would seem to be a valuable attribute in a goalie. Certainly any coach would like to have a goalie that they can count on to put forth a strong, consistent effort. This is especially true on good teams, because if they know that the goalie behind them will make the routine saves they will have a very good chance to win.

Consistency is not always a good thing, however. Consistent bad play is obviously undesirable. In addition, it is better for goalies on bad teams to be less consistent and have the capability for outstanding games. A goalie who consistently plays at an average level and never steals games is useful to a league leader, but much less valuable to a bottom feeder. It is unclear whether coaches and GMs appreciate this fact, but it is part of the goalie value equation.

To measure consistency, I calculated the variance of each goalie's save percentage from each game this year. There are some lurking variables affecting this analysis, such as quality of opposition, shot quality, and the other usual things that affect save percentage, but it is at least a starting point.

Here are the most consistent starting goalies in the league:
1. Manny Fernandez, MIN
2. Olaf Kolzig, WSH
3. Chris Mason, NSH
4. Miikka Kiprusoff, CGY
5. Marc-Andre Fleury, PIT

And the least consistent:
1. Evgeni Nabokov, SJS
2. J.S. Giguere, ANA
3. Dwayne Roloson, EDM
4. Henrik Lundqvist, NYR
5. Manny Legace, STL

Some other notables: DiPietro ranks 6th, Luongo 7th, Brodeur 11th and Hasek 22nd.

Bryzgalov, Markkanen and Weekes also posted fairly high variances, which suggests that maybe team defensive play is the reason for the variance. This is true for some on the other end as well, as McLennan, Backstrom, and Thibault also have been consistent.

More study is required to find out if consistency is predictive (i.e. the same goalies are consistent year after year), and possibly to find ways to adjust for some of the other influencing factors.

Tuesday, March 6, 2007

Stanley Cup Finals - Where the Best Team Wins

The playoffs are where goalies make their reputations, and the spotlight is never brighter than in the Stanley Cup Finals. Broadcasters and other media types ramble about the importance of goaltending, and depending on the final result assign either all the credit or all the blame to the goaltender.

If goaltending really is so important, however, then the team with the better goalie should have won a large majority of the Stanley Cup Finals. I went back and looked at all the Cup Finals since 1968 (the expansion era). For some of the seasons I have game-by-game save percentage statistics (courtesy hockeygoalies.org), which allows for an even more detailed analysis.

My goal was to identify the series where a goalie was the difference in the series. That is, either an outstanding performance by a goalie on the team that was worse during the season but still won the Cup, or a terrible performance by a goalie on a good team that ended up losing the series. It didn't really matter so much how well the goalies played, just whether or not the inferior team ended up winning.

For example, in 1996 Patrick Roy had an outstanding Cup Final, helping Colorado sweep Florida and posting a .974 save percentage while outplaying John Vanbiesbrouck in every game. But Colorado finished 12 points ahead of Florida in the standings that season, outscoring them by 72 goals with almost the same defensive record. Clearly, Colorado was a much better team, and despite Roy's performance he cannot be credited with winning the series.

I elected not to use points alone to determine which team was better, because luck can often play a role in determining a team's final position in the standings. I also considered goals for and goals against, to break the tie if teams finished with a similar amount of points.

I found only 7 instances over 39 seasons where the team I judged to be worse was triumphant in the Stanley Cup Finals. Over 80% of the time, the better team won. This is strong evidence that, even in the playoffs, the team contribution is much more valuable than the goaltender's contribution to the final result. Obviously, the goalie's contribution is also included in the team's position in the standings, but most of the time there was a clear difference between the quality of the participants.

Here are the times when the better team lost the Stanley Cup Final:
2001: New Jersey loses to Colorado
1995: Detroit loses to New Jersey
1990: Boston loses to Edmonton
1985: Philadelphia loses to Edmonton
1983: Edmonton loses to N.Y. Islanders
1980: Philadelphia loses to N.Y. Islanders
1971: Chicago loses to Montreal

I looked at the three that I had detailed information on. In 2001, Patrick Roy was outstanding (.938) while Martin Brodeur was poor (.870), and the Devils lost despite outplaying and outshooting the Avalanche. In 1995, Brodeur played well against a strong Detroit team (.913), but the decisive factor was Mike Vernon's terrible play at the other end (.846). In 1990, Edmonton's Bill Ranford stoned Boston, posting a .949 save percentage despite being outshot by nearly 5 shots a game, outplaying the tandem of Andy Moog and Reggie Lemelin (.846 combined).

For the other 4, I suspect that two of them include outstanding goaltending performances, based on the goalies receiving the Conn Smythe Trophy (Ken Dryden in 1971, Billy Smith in 1983). As I posted on before, however, Smith's performance was greatly assisted by the strength of his team, but I'll count that one just to be on the safe side. The other two were likely outliers. The Oilers were clearly a dominant team in the middle of a run of 4 Cups in 5 years, and had a bit of an off-season in 1985 when they had "only" 109 points, a season sandwiched by two years with 119. The Islanders were just beginning their Cup run, and were likely a much better team during the 1980 playoffs than they performed during the season. This is suggested by their 19 point improvement in the standings in 1980-81. It could be that there were outstanding goalie performances in either of those years, but I doubt it.

I went through the years from 1986 to 2007, and estimated that nearly half of the time the better goalie ended up losing. The better team won 17 out of the 20 (85%).

This analysis suggests that in the 39 seasons since expansion, the only goalies to steal a Stanley Cup Finals series were Patrick Roy, Bill Ranford, Ken Dryden, and possibly Billy Smith. The vast majority of the time the better team won, and on several occasions, the winning goaltender was actually outplayed by the losing goaltender. This indicates that even in the playoffs, where goaltending is supposedly so important and where players are labeled as "clutch" or "chokers", almost all of the time it is the best team that ends up on top.

Thursday, March 1, 2007

The Hall of Fame Standard for Goalies

A Poster on Battle of Alberta:
"I like Hrudey a lot, as a TV personality and as a player, but to punch his ticket to the Hall on the basis of save percentage seems a little thin...His overall career accomplishments don't seem to warrant it."

This illustrates the difference between the average fan evaluating a goalie and evaluating a player. For goalies, Cups and Vezinas are almost the entire story. Nothing else really seems to matter. For players, people look at counting stats like goals and assists, All-Star appearances and so on, and Cups are just one of the many factors that are considered.

According to this logic, the most important and valuable goalie stat (save percentage) is summarily discounted, and the goalie is not considered because he did not have the "accomplishments" (read: Cups). That is like saying a player can't go in just on the basis of points or goals, because they never won Cups. That can be held as a strike against players, but isn't nearly as decisive in determining their eligibility. This double standard has created the situation we have today, where unremarkable non-champions like Bernie Federko are in the Hall of Fame because they racked up the points, but goalies aren't allowed in unless they had good teammates around them.

This is very obvious when you consider goalies from the 1980s. Billy Smith, Grant Fuhr and Patrick Roy are in the Hall of Fame, and Mike Vernon has a good chance to join them. If he does, that means that all the goalies who won a Stanley Cups in the 1980s are in the Hall of Fame, and everyone else isn't. This despite only Roy ever leading the league in goals against average or save percentage during that time period.

Goalie play is difficult to separate from team play, but the effort needs to be made. Otherwise lazy analysis concludes that the best goalies play for the best teams, and goalies end up getting evaluating on something (Stanley Cups) which they have a relatively small amount of control over. Even if you consider a goalie to be worth as much to team defence as the rest of the team put together, you still have to factor in the offence, which the goalie can't impact at all, and therefore the goaltender becomes worth 25% of the team, max. There's a lot that can go wrong in the remaining 75%, almost no matter how good the goalie is (see Hasek, Dominik). On high-scoring teams like the 1980s Oilers and Islanders, the goalie contribution is quite insignificant to the team's success, and there is therefore no reason to give extensive credit to the Smiths and Vernons of the world solely because of the logo on the front of their sweaters.

It is much better to evaluate each goalie on their own merit, which focuses primarily on measures of performance (like save percentage) instead of "career accomplishments". Even though the task is often challenging and not completely conclusive, it is still infinitely fairer than the alternative, which unfortunately is also the status quo in the hockey establishment today.