Monday, December 31, 2007

Mike Keenan Knows Very Little About Goaltending

It is often difficult to advance contrarian positions about hockey players that contradict insider perspectives in hockey. A common retort in many of the comments and emails I receive goes something like, "But (Player X) or (General Manager Y) said Brodeur was great - how can you disagree?" General managers vote on awards like the Vezina Trophy, and their collective wisdom helps shape fan perception of certain players. However, some of these insiders have shown by their actions that they know little more about how to evaluate goalies than an average fan. One of the worst offenders has been Mike Keenan, a celebrated coach who has made his living in hockey despite being clueless about goalies for over 20 years.

In Philadelphia, Keenan built a solid defensive system that made his goalies look good most of the time. However, he chose Pelle Lindbergh as his starter over Bob Froese in 1984-85, and in 1985 until Lindbergh was killed in an accident. Lindbergh wasn't a bad goalie, but he was easily outplayed by Froese. Over the two years, the Flyers were 44-12-3 with Froese in net and 46-19-7 with Lindbergh. Froese had 2.51 GAA and .910 save percentage, Lindbergh just 3.00 and .898. It should have been obvious from the 1983-84 season who was better (Froese 28-13-7, 3.14, .887 at the age of 25; Lindbergh: 16-13-3, 4.05, .860 at the age of 24), but apparently not to Keenan. After the tragedy with Lindbergh, Keenan had to turn to Froese, but he also used the mediocre Darren Jensen and the aging Glenn "Chico" Resch. Keenan still never liked Froese, despite his excellent save percentage numbers, and handed the starting job to rookie Ron Hextall for the 1986-87 season. Midway through the year, Froese was traded to the New York Rangers.

Hextall put up worse numbers than Froese had done in 1985-86 (3.00, .902), but he won 37 games behind behind the Flyers' strong defence and received the Vezina Trophy. After this charmed season, Hextall's numbers then fell off significantly in 1987-88 (3.51, .885).

Keenan moved on to Chicago in 1988. As usual, he spent his first year there tinkering with the goaltending, challenging the incumbent starter Darren Pang with two rookies, Ed Belfour and Jimmy Waite, as well as the newly acquired Alain Chevrier. The goalie shuffle would continue in Chicago for another season, as both Jacques Cloutier and Greg Millen were brought in via trade, but both played poorly and were soon out of the league.

Dominik Hasek arrived in the NHL for the 1990-91 season, but Keenan wasn't a fan and held Hasek to just 5 games that first season, even though Hasek performed well in his first taste of NHL action (3-0-1, 2.46, .914). However, Belfour became the starter ahead of Millen and Cloutier (who was traded mid-season), playing 74 games and winning the Vezina Trophy with a 43-19-7 record, a 2.47 GAA, and a .910 save percentage. Keenan earns partial credit here for at least going with Belfour, even if he probably could have done even better with the Dominator.

Keenan moved upstairs to the GM position for the 1991-92 season, but he still didn't realize what he had with the Dominator. Hasek again put up solid stats (10-4-1, 2.60, .893), but Keenan traded him in the offseason to Buffalo for Stephane Beauregard. Iron Mike decided that he would rather have Jimmy Waite as a backup than Dominik Hasek, even though in 1991-92, Waite played 17 games with a 3.69 GAA and an .844 save percentage.

Jimmy Waite is a great example of a goalie that Keenan completely mis-evaluated. Waite played 53 games for Chicago with Keenan as coach or GM, and nearly all of them were awful - 3.72 GAA and .853 save percentage. Yet Keenan thought he was good enough to stick around, and traded one of the greatest goalies of all-time (Hasek) to move Waite up on the depth chart.

Keenan was fired in Chicago after the 1992-93 season, and found work in New York, where he made his legend by winning the Stanley Cup. With Mike Richter in net, Keenan couldn't possibly screw it up in New York. However, his term on Broadway was very short, and they were still celebrating the Cup win when he decided to move on.

Iron Mike soon found himself brought into the St. Louis Blues organization as both coach & GM. There, Keenan inherited another stellar netminder in Curtis Joseph. However, he decided to sign Shayne Corson away from Edmonton, which cost the Blues two first round picks in compensation. To get the picks back, Keenan traded Joseph and Mike Grier to Edmonton. Corson scored just 20 goals in 88 games in St. Louis, while Curtis Joseph remained one of the league's best goalies for the next decade in Edmonton, Toronto, and Detroit.

To replace Cujo, Keenan signed the aging Grant Fuhr and kept the incumbent Jon Casey, who was also getting up in years, as the backup for the next two seasons. In a low-scoring era, Fuhr was below average (.902 save percentage), and Casey was downright awful (.864). Despite a blue-line that included Al MacInnis and Chris Pronger, the Blues never won anything under Keenan, often because they were let down by their goaltending. Even after Keenan's departure, St. Louis continued to ride Fuhr to disappointing results, until they finally replaced him with Roman Turek and were rewarded with the President's Trophy in 1999-00.

Keenan moved to Vancouver, where he had Arturs Irbe and Kirk McLean. Irbe had a strong season in Vancouver in 1997-98 (2.73, .907), but he wasn't re-signed and moved on to Carolina where he had an even better one (2.22, .923). Pat Quinn was GM in Vancouver, and it is difficult to know how much input Keenan had on some of the goaltending decisions, however it is probably reasonable to assume that Quinn at least consulted with Keenan before making his moves. Midway through the season, Vancouver made a pretty good trade, swapping Martin Gelinas and Kirk McLean for Sean Burke, Geoff Sanderson, and Enrico Ciccone. However, the Canucks managed to mess it up again two months later, getting rid of Burke again in exchange for Garth Snow. Burke was admittedly not particularly outstanding in his brief stint in Vancouver, albeit on a weak team, but trading him for Snow was a blunder. Over the next 5 seasons, Sean Burke established himself as one of the best goalies in the game, posting excellent statistics (2.40, .917). During the same period, Garth Snow continued to be mediocre (2.74, .904), only outlasting Keenan in Vancouver by one season before leaving via free agency. Again, that one is probably more on Quinn than Keenan, but in any event, Iron Mike's choices for backup goalies in Vancouver, Corey Hirsch and Kevin Weekes, were terrible, combining for a 3-16-4 record with awful save statistics.

In 2000-01, Keenan coached the Bruins. He had a solid starter in Byron Dafoe (2.39 and .906 in '00-01), but throughout the course of the season Dafoe missed 35 games to injury. Keenan used four other goalies in backup and replacement duty, but they were mostly terrible, including Peter Skudra (.879), John Grahame (.867), and Kay Whitmore (.809)(!). Having such terrible goalies play big minutes was one of the major reasons the Bruins finished 9th in the East and missed the playoffs.

Keenan was fired as a result of the disappointing finish, but he found a new home in Florida, where he had an elite young goalie in Roberto Luongo. Keenan managed not to screw this situation up for three full seasons until just as he was on his way out the door, when he pulled the trigger on the now infamous trade with Vancouver (Luongo, Lukas Krajicek and a 6th round pick to Vancouver for Alex Auld, Todd Bertuzzi, and Bryan Allen). Shortly after, Keenan resigned as GM.

Keenan is now in Calgary, where he has another star goalie in Miikka Kiprusoff. Kiprusoff has been playing poorly this season, but whether that is influenced by Keenan, is because of poor team defence or just a slump is difficult to determine. It will be interesting to see how Keenan handles the situation. If he was also the GM, I would almost expect him to trade Kiprusoff for pennies on the dollar and bring in a goalie who is way past his prime (Ed Belfour, perhaps?) to take his spot.

Mike Keenan may be a good hockey coach in terms of motivation and team defensive play, but he should never be allowed to make personnel decisions about hockey goalies. His record in that department is absolutely terrible. Nearly every decision or trade he has made involving goalies has turned out bad, and some of them spectacularly so. Keenan traded Hasek, Joseph, and Luongo, and got almost nothing in return for all three of them (all the players acquired combined for just 227 games with their new teams). He also chose to play other goalies ahead of Bob Froese and Sean Burke, which probably contributed to them being traded away for little returns, and he played some brutal backups for far too many games (most notably Jimmy Waite and Jon Casey). Keenan is an extreme case, but the reality is that some scouts, GMs and coaches are not good at evaluating goalies, and whether or not they get paid by an NHL team or have a Vezina vote doesn't change that fact at all.

Friday, December 21, 2007

On Games Played

One of the differences of opinion I have with many other hockey spectators is with respect to the importance of games played for a goaltender. I maintain that in almost all cases there is no significant difference between a goalie who plays 50 games and a goalie who plays 80 games, other than the coaching and team management philosophy that led to those results. Goalies should be durable enough to play the lion's share of their team's games, but extra starts beyond that are mostly just an opportunity for certain goalies to pad their stats, simply because their coaches like to ride their starters, they have established reputations, their teams are up against the cap or because their backups are weak.

I have not seen any evidence that goalie performance deteriorates with more minutes played. Remember the talk about how Brodeur was tired in the playoffs last season? Over the last 6 seasons, Brodeur has the following line in regular season games held in April: 22-3-1, 1.87, .928. I also once did a long post about Grant Fuhr and the 1987-88 season, where he went from 44 games played to 75 games played because of the absence of Andy Moog, and put up an identical save percentage and a virtually identical winning percentage. His performance was exactly the same, just with more of it, and that won him the Vezina. Evgeni Nabokov played 56% of the minutes in San Jose last year, and did pretty well. This year he has played 98% of the minutes and his performance is very similar. Did he work on his durability over the summer? Of course not, his backup went from Vesa Toskala to a guy who put up an .888 save percentage in the AHL last season. Not surprisingly, Nabokov's coach gave him more starts.

Virtually every goalie in the NHL is capable of logging big minutes and playing well. To get to that level, they would have been the starters in minor hockey, junior or college hockey, and in most cases the minor leagues as well. I looked at the 30 current starting goalies in the NHL to see how many of them were experienced goalies (more than 5 full years in the league) without multiple good seasons with 65+ games played. Here is the complete list:

Martin Biron, Rick DiPietro, Johan Hedberg, Manny Legace, Dwayne Roloson, J.S. Giguere

I'd say there is enough evidence that Biron, DiPietro, Roloson and Giguere are capable of playing big minutes. Giguere and DiPietro have both fallen just short of my arbitrary 65 game cutoff a number of times, and Biron and Roloson have both spent most of their careers as backups or platoon goalies, although they have been quite good in the full seasons they have played (Roloson has played in 108 of 136 games since being traded to Edmonton, Biron's career best save percentage was during his 72 game season in 2001-02 and he's been great carrying the load in Philly).

So I'm not sure that Johan Hedberg and Manny Legace can handle big minutes as starting goalies, but other than that I'm pretty sure every current experienced NHL starting goalie is capable of handling a lot of games.

But Brodeur has done it year after year for over a decade, you say. Yes, he has. So have guys like Olaf Kolzig, without nearly the same level of attention. There are two main reasons why guys see their games played reduced: their backups get better, or their performance drops off. Brodeur's games played dropped when New Jersey had Mike Dunham from 1996-98. His playing time increased again when Dunham was replaced by weaker backups, even though Brodeur's performance dropped from .927 in 1997-98 to .906 in 1998-99, and then .910, .906, and .906 over the next 3 seasons. Brodeur's advantage, however, was his low shots against, so his low GAA and high win totals helped mask the fact that his performance wasn't as good. A goalie on a weaker team who had that kind of deterioration in save efficiency might have dropped below .900 in save percentage, and his coach would probably be looking to give somebody else a shot. Brodeur had an established reputation, was on a team that inflated his perceived performance level, faced fewer and easier shots than other goalies, and played with poor backups. And yes, he is very durable. So it is not surprising at all that he ended up playing a lot of games.

But surely it must be valuable to have a goalie log a lot of minutes at an above average level of play? Yes, of course, since any team would rather have above average play than mediocre play, and if the goalie is better than any other option on the team then they are better off with him in the nets as much as possible. But what about a goalie who plays fewer games, but at a higher level of performance? Is that better or worse than a Brodeur-type who is always in the net?

Well, let's take some hypotheticals here. Let's compare a .910 goalie who plays every single game in a season for his team, against a .925 goalie who, because of various personal and injury issues, isn't capable of such a demanding schedule. Let's assume that his backup goalies are terrible (.890), and that his team is about average in terms of shot prevention (30 shots per game). How many games does the second goalie have to play for his team to be ahead of the first team? The answer is just 42 games. And if his team has a quality backup, say one capable of performing at about league average (.905), the goalie needs to play in just 21 games to break even with his more durable counterpart.

Save percentages can often be misleading, because it can look like there isn't much difference between .910 and .920. In fact, a .920 goalie is much better than a .910 goalie. The higher the save percentage, the harder it is to maintain. I am of course speaking about performances over a fairly large sample size, where we can more accurately ascertain the true level of play of the goalie.

Durability is good, because you obviously want your best goaltender on the ice instead of in the press box more often than not. However, whether a goalie plays 50 games or 82 games is really not that meaningful. I'd rather have a truly excellent goalie playing in half of my games than a minutes muncher giving me league average performance in every single one of them, even if the second guy is going to end up with more wins and shutouts and Vezina Trophy votes. Alas, truly excellent goalies are rare, so teams have to make do with what they can find, so from a team management perspective it might make your job easier to find an average/above average goalie and play the heck out of him, and then cheap out on your backup. That does not, however, mean that your goalie magically becomes one of the league's best or most valuable just because he plays every night. The best goalies are the ones with the best rate stats, not the best counting stats.

Friday, December 14, 2007

Building a Defensive System

Do you need a great goalie to build a defensive system? Some would claim that whenever there is a combination of team and goalie that have a run of success that the team was able to build their defensive strength around the goalie. Does an elite defensive system require a great goalie, or is this reasoning false?

To answer this question, let's imagine a team that has a brick wall as a goalie. Yes, a 4' x 6' brick wall in net. The other team cannot score no matter how many shots it takes. What kind of strategy should that team use? Should it play a tight defensive style and try to minimize scoring chances against at the expense of offence? Or should it take every offensive risk possible, since it only needs one goal to win the game? Obviously the latter is the correct strategy, since there is no risk to giving up scoring chances and shots, and a very strong benefit for taking risks to score goals (any goal = guaranteed win). And conversely, what about a team playing with no goalie in the net? What kind of strategy should they use? In real life, teams often use an aggressive strategy with no goalie in because they are trying to tie the game, but what if there was no goalie and the score was tied? Such a team would obviously play very tight defence, because every shot on net becomes a goal. Therefore they would be very defensively focused to try to prevent shots at all costs, using a tight defensive shell and pressuring the puck carrier to block their shots or force a turnover that may allow a chance on the counterattack.

This is why claiming that a goalie is responsible for a defensive system is completely wrong. Grant Fuhr supporters may or may not be right about some things, but one thing they are definitely right about is that a great goalie is valuable for an offensive team because it allows them to take additional risks to try to score goals. If your goalie is better than the goalie on the other team, it makes sense to play a more open game with more shots on each net. If each team gets 10 shots of equal difficulty, a journeyman goalie could easily beat a Hall of Famer with a little luck. If both goalies face 60 shots, on the other hand, it is much more likely that the better goalie is going to win the game for his team.

With Dominik Hasek in net, the defencemen on the 1999 Buffalo Sabres were aggressive on their pinches, knowing that even if they gave up an odd-man rush it was likely that Hasek would make the save. In contrast, last year's Ottawa Senators played a tight defensive system and blocked as many shots as possible to try to protect their inexperienced goaltender Ray Emery. Both were the correct strategic moves, and helped them reach the Stanley Cup Final, since the trade-offs in terms of allowing/preventing scoring chances against were to their team's benefit.

Playing a defensive system does not indicate that there is a strong goalie in the net. On the contrary, a strong defensive system is most needed and most beneficial when there is a weak goalie in the net. With a great goalie, a team should take more risks to try to generate offence, since the star netminder reduces the risk of giving up extra goals from the additional scoring chances allowed.

This is also why it is so important to adjust goalie statistics for team factors. Great defensive systems can hide weak goaltenders, just as poor defensive systems can make excellent goalies look average. With the advantages of a great defensive team, a truly elite goalie should dominate, finishing at or near the top of the league in save percentage and GAA, in addition to getting their wins and shutouts. If a goalie posts an average save percentage on a great defensive team, that is a strong indicator that he is being covered by his teammates. The same thing, in reverse, can apply to goalies with a high save rate on bad teams, as their teammates may be taking a lot of risks to score and leaving their goalies open to high-quality chances against and making their performance more impressive.

Thursday, December 6, 2007

Is Brodeur....Underrated?

Jonah Keri has written an article about the 10 most underappreciated active pro athletes. Coming in first place was Martin Brodeur. Either this was Opposite Day at ESPN, or it shows how clueless ESPN has become about hockey, given that one of their columnists ranked one of the most overrated goalies of all-time as the most unappreciated athlete in sports.

I've already summarized the arguments against Brodeur, so I'm not going to do that here. The comments thread to the article, however, illustrates again how the Brodeur debate essentially falls into two camps, which, despite what Internet message board flame wars would have you believe, aren't actually divided by their personal like or dislike of Brodeur. The difference is simply in their philosophies of evaluating goaltending play, which clash head-on in the case of Martin Brodeur.

The first camp associates team success with the individual goaltender, and the second camp believes goalie play is heavily dependent on the rest of the team. Those in the first group see Brodeur as one of the greatest, if not the greatest, goalie ever, because of his wins, shutouts, and Cups. None of those things mean anything at all to those in the second camp, who look at his save percentages and the strength of his teams and conclude that Brodeur is actually quite ordinary. The reason this perception gap is so large is that Brodeur has been the beneficiary of the most favourable goaltending environment in the NHL since Ken Dryden.

So we have probably an irreconcilable debate about Brodeur, at least until everyone comes to some agreement on the evaluative criteria. Nevertheless, when you take a deeper look at goaltending numbers, it becomes quite clear that the "team dependency" side has much stronger ground to stand on. If you still need convincing, though, check out these 12 examples for starters, and since this is basically the founding philosophy for this blog there is lots more written on this topic available on this site.

Sunday, December 2, 2007

The Dead Puck Era

The period from 1995 to 2004 in the NHL is sometimes referred to as the "Dead Puck Era" in reference to the decreased scoring environment. In this period, the average goals per game decreased and the average save percentage went up. Some of this effect may have been from improved goaltending play, but a lot of it was due to better defensive play, relaxed rules on clutching and grabbing, increased goalie equipment size, and other factors.

How much did this era help the goalies who played in it? Quite a bit, as it became easier to post high save percentages and low GAA numbers. One of the major beneficiaries, of course, was Martin Brodeur. It is easy to look at his career numbers (even his save percentage numbers), and wonder how anyone could claim that several other goalies could have easily matched his performance in New Jersey. Yet that's what I'm going to try to demonstrate right now.

Roberto Luongo, probably the best goalie in the NHL, is in his eighth season in the league. I wanted to look at the goalies who entered the league between Brodeur and Luongo, i.e. roughly somewhere between 1993 and 2000, to see how playing in that particular era impacted their save statistics. For a fairer comparison, I took only their first 8 seasons, starting with the first year they played more than 1000 minutes. Both are somewhat arbitrary cutoffs, but I needed to set the yardsticks somewhere, and for most goalies 8 seasons includes a few prime years and goes up to around 29 or 30 years old. For goalies in their 8th season like Luongo, I included this year's stats. Here are the save percentage results:

1. Roberto Luongo, .919
2. Manny Fernandez, .913
2. Tomas Vokoun, .913
4. Martin Brodeur, .912
4. Mike Dunham, .912
6. Evgeni Nabokov, .911
6. Guy Hebert, .911
6. Martin Biron, .911
6. Olaf Kolzig, .911
10. Jose Theodore, .910
10. Dwayne Roloson, .910

(There are a few others who didn't make the list because they only had 7 seasons played but would also have ranked ahead of Brodeur, including Manny Legace (.916), J.S. Giguere (.916), Marty Turco (.914), and even David Aebischer (.912).)

What conclusions can we draw from this list? Simply that the effect of the era they played in was much stronger than the individual goalie effect. The difference between #2 and #10 on this list is very slight, just .003 in save percentage (1 goal every 333 shots), and that is over an 8 season span. The only guy who distinguishes himself is Luongo. Brodeur is mid-pack, despite playing on the best teams (especially defensively) of any goalie on the list. It is interesting to see him tied with Mike Dunham, who began his career as Brodeur's backup in New Jersey.

Some of the goalies played fewer minutes, because they spent times as backups, which means one or two good seasons could be skewing their results. Those who played over 20,000 minutes in their first 8 seasons include Luongo, Vokoun, Brodeur, Nabokov, Hebert, Kolzig, and Theodore. So there is a good deal of evidence that except for Luongo they were all pretty close to each other in performance over their first 350+ games in the NHL.

Olaf Kolzig and Martin Brodeur had almost the same playing time and the same save statistics (29,432 minutes and .911 for Kolzig, 30,055 and .912 for Brodeur). Pretty much the only difference between the two of them was the strength of their teams. Olaf Kolzig is certainly one of the league's most underrated goalies - I would argue that over their careers, there is little to differentiate him from Martin Brodeur. The only difference between them is quantity, but not quality.

Martin Brodeur has had a very good career, but this has been primarily because of his great defensive teams and his longevity, durability, and favourable deployment by his team (i.e. his coach sending him out to the net virtually every game). For the majority of his career, he has stopped the puck just as well as the other decent goalies in the league, even before considering how relatively easy his shots were. He just came into the league at the right time and on the right team to put up massive career numbers. The low-scoring era Brodeur played in is another major reason for his impressive career statistics, but relatively speaking they are far from great. Brodeur's save percentages are often invoked in arguments involving some of the slightly older goalies like Roy, Belfour, and Joseph, but once you factor in era there isn't much difference at all (except for Roy, who ranks well ahead of Brodeur).

Just as an aside, there was one goalie that did not make the list because his first qualifying season came in 1991-92, but I thought I'd just mention him for comparison's sake: Dominik Hasek. Despite playing in three relatively high-scoring seasons to start his career, in his first 8 seasons Hasek posted a .926 save percentage, which outclasses everybody on the above list. In the Dead Puck Era, the low-scoring environment made it look like everyone was good at stopping the puck, but the record shows that nobody was even close to Hasek.

Tuesday, November 27, 2007

Consistency

In his new book, radio host Bob McCown rates Martin Brodeur as the third best goalie of all-time. One of his arguments is Brodeur's consistency, and he cites as evidence the fact that Brodeur has only once had a GAA higher than 2.50. This, of course, is a ridiculous argument. The reason Brodeur has had low GAAs is because he has faced few shots throughout his career. In most years, he would have had to have been terrible to have a GAA even close to 2.50. Here are the save percentages Brodeur would have needed to post in each year in his career for his GAA to be 2.50:

.912, .900, .905, .902, .890, .898, .900, .898, .891, .893, .897, .914, .910, .902

For seven years in a row, Brodeur only needed to hit .900 or better to have a sub-2.50 GAA. So giving him credit for achieving that arbitrary mark is misguided.

I raise a skeptical eyebrow any time someone uses the word "consistency" (could be the Fire Joe Morgan influence). This is another example of consistency inappropriately being used to praise a player, as his performance actually shows a reasonably high degree of variance. From 1994-95 to 2003-04, the average number of shots he faced per game ranged from a low of 22.8 to a high of 26.4. That is a pretty narrow range, and was well below league average every single season. His save percentages in that span ranged from .902 to .927, i.e. from below average to excellent. Four times he was at .906 or lower, three times .917 or higher, and three times in between, showing a fairly broad range of success, and it was the stingy defence that always kept him at 2.4 goals against or lower.

Brodeur has consistently played a lot of games, but that is about the only thing he has done consistently (unless you count being a member of a great defensive team). He has had a couple of outstanding seasons, as well as several mediocre ones, which is fairly typical of goalies. His low shot totals tend to hide his off-years, and make commentators gush about his consistency, but they are wrong: The consistency truly lies in New Jersey shot prevention, not in New Jersey goaltending success.

Thursday, November 22, 2007

Count the Rings...Or Not

Here are 12 random facts to illustrate the fallacy of using Cup wins and playoff team success to evaluate individual goaltenders:

1. The last 11 Stanley Cup champions have all finished in the top 5 in the league in regular season points. None of them had the regular season save percentage leader on their team, and only one (Martin Brodeur in 2003) won the Vezina Trophy and the Stanley Cup in the same season.

2. From 1976 to 1988, only three teams won the Stanley Cup (Montreal, Edmonton, and N.Y. Islanders). Three goalies combined either a Vezina or First All-Star season with a Cup win in the same season (Fuhr, Smith, Dryden), but all three teams had regular season and playoff success with several different goalies in the net.

3. Patrick Roy is widely considered to be the greatest ever playoff goaltender who stole many games, led many upset wins, and took two mediocre Montreal teams to the Stanley Cup. In his 19-year career, however, Patrick Roy only won five playoff series in which his team did not have home ice advantage. In his four Stanley Cup wins combined, Roy's team only beat one opponent that had more wins in the regular season that year. Roy was also a member of 2 President's Trophy winners and 10 division champions.

4. Between 1993-94 and 2003-04, the New Jersey Devils outshot their opponents in 20 out of 22 playoff series. They won 14 of those 22 series. Throughout his playoff career, Martin Brodeur has faced an average of nearly 4 fewer shots per game than his goaltending opponent.

5. In Edmonton, Grant Fuhr was 226-117-54 with 5 Stanley Cups. Outside of Edmonton, he was 177-178-60 with 0 Cups. In the playoffs with Wayne Gretzky on his team, Grant Fuhr was 63-21. Without Wayne Gretzky, he was 26-29. In international competitions with Gretzky: 8-2-2. Without the Great One, Fuhr was just 1-3-1.

6. Gilles Meloche played in 788 games over 18 NHL seasons, making the All-Star Game twice. Because of the poor quality of his teams, he played in only 45 playoff games, despite having a winning record in postseason play.

7. Every playoff game in Jeff Hackett's career was against a 100+ point team with either Patrick Roy or Martin Brodeur in the other net.

8. Chico Resch's career playoff GAA with the New York Islanders was 2.49, better than Billy Smith's 2.73 mark. Resch also had much better regular season statistics than Smith. Resch was traded to the Colorado Rockies on March 10, 1981, meaning he only got one Stanley Cup ring to Smith's four.

9. Michel "Bunny" Larocque went 111-26-20 from 1974-75 to 1978-79, playing with Ken Dryden and the Montreal Canadiens, and was on four Cup-winning teams. In his post-Montreal career with Toronto, Philadelphia and St. Louis, Larocque's record was 16-41-14 with a 4.87 GAA.

10. The 2003-04 Florida Panthers scored 188 goals and allowed 221, despite Roberto Luongo's .931 save percentage. Given their scoring level, the Panthers would have needed to allow 178 goals or fewer for an expected number of points that would put them in a playoff position. To accomplish this, Luongo would have had to save an additional 43 goals over the 2,475 shots he faced that season. That would have required at least a .948 seasonal save percentage just to give his team a chance at earning the 8th playoff seed in the East and a probable first round loss against the eventual Stanley Cup winners Tampa Bay.

11. In Dominik Hasek's record-breaking 1998-99 season, when he set the single-season save percentage record of .937, his team was the #7 seed in the Eastern Conference. His team did not have home ice advantage in any rounds of the playoffs, and ended up losing in the Final to Dallas, a team that was 23 points better than Buffalo during the regular season.

12. The 1981 New York Islanders finished first during the regular season with 110 points. In the playoffs, they did not play any of the next 7 highest ranked teams. Not surprisingly, they won the Stanley Cup, outscoring their opponents 97-48 in the process. Similarly, the 2004 Tampa Bay Lightning finished second overall during the season, and then avoided meeting any of the teams ranked 1st through 7th on their way to the Stanley Cup.

Two other fortunate teams were the 1991 Pittsburgh Penguins (avoided the league's 4 best teams, 82 point average opponent), and the 1986 Montreal Canadiens (avoided the league's 5 best, opponents averaged 84 points).

An example of an unfortunate team was the 2004 Calgary Flames, who faced the top 3 seeds in the Western Conference, as well as the top seed in the Eastern Conference, before losing the Stanley Cup Final in 7 games. The average regular season record of their opponents was 105 points.

Monday, November 19, 2007

"I don't see myself as that big of a deal"

Congratulations to Martin Brodeur on achieving his latest arbitrary milestone of heavily team-influenced statistics, win #500.

I'm sure Marty was just being modest about his accomplishments in the above quote, but if he is going to say things that support my thesis then I'm probably going to end up posting them.

Friday, November 16, 2007

Marty Turco is No Longer an Elite Goaltender

One of the problems with goalie evaluation is that reputations tend to linger far too long. Because of the inherent variability with the goaltending position, as well as the changing dynamics of teams, we need to be careful to jump to conclusions. Good goalies will eventually right the ship, just as the luck will run out for those who are temporarily playing over their heads. However, when presented with sufficient evidence of an improvement or decline in a goalie's game that is independent of team factors, his ranking should be changed, regardless of how many Cups, Vezinas, or anything else he has won or done.

I think the majority of hockey fans would rate Marty Turco as an above average goaltender. The Hockey News put him in the top 5 in the league at his position. In reality, though, he is not. There is some pretty good evidence that something negative happened to Turco's game over the lockout, as the splits are quite startling:

Pre-lockout: 185 GP, 1.91 GAA, .922 save%, 24.4 SA/60

Post-lockout: 148 GP, 2.42 GAA, .904 save%, 25.3 SA/60

A lot of that is the game opening up, and some of it is Dallas getting weaker, but the reality still is that right now Marty Turco is a mediocre goalie. He ranks 23rd in the league this season according to Hockey Numbers' shot-quality neutral save percentage. However, the reputation is still there, bolstered in part by his strong playoff performance last season. It is interesting that Turco was largely underrated earlier in his career because of his playoff struggles, but now is likely overrated because of one good stretch of playoff games last year against a weak offensive team. In addition, even though he is probably the league's best puckhandling goalie, he tends to receive too much credit for that ability, much like Martin Brodeur.

Monday, November 5, 2007

Exhibit 1,567 on Why Goalie Wins Are a Poor Statistic

This season, in the games Martin Brodeur has "won", he has given up 5, 4, 1, and 2 goals. In the games that he has "lost", he gave up 2, 3, 3, 4, 2, 3, and 1. Which leads to this unusual split:

In wins: 3.00 GAA, .876 save %
In losses: 2.56 GAA, .906 save %

The other 18 skaters on the team have a very large impact on whether the team wins or loses. Usually much more so than the goaltender. This has certainly been the case this year in New Jersey, and the reason has nothing to do with goal prevention but rather what is going on at the other end of the ice:

Brodeur's goal support in wins: 5.00 goals per game
Brodeur's goal support in losses: 0.71 goals per game

Goalie wins ignore half of the inputs that determine wins and losses (goals for), and leave the other half (goals against) unadjusted for number of shots against and shot quality. With all that noise clouding the data, it is impossible to tell from his win totals if a goalie is very good or just playing for a high-scoring or great defensive team (or even both). Brodeur's season to this point is a great example that goalie wins are not very meaningful at all in terms of evaluating goalies.

Monday, October 22, 2007

The Difference Between Talent and Performance

Perhaps one of the major stumbling blocks I have encountered when trying to spread my particular message about goaltending play is that many people evaluate goalies based on talent, rather than performance. This may seem like a small semantic difference, but it is a crucial one.

There are a lot of players in the league with an abundance of talent, but lacking the performance to match. Chad Kilger of the Toronto Maple Leafs is such an example. Kilger is one of the fastest skaters on his team, and last year was clocked with the hardest shot in the league. He has size and decent hands. By all rights, Kilger should be a star player. But he is just a third liner, and the reason is that his talent does not translate into production. He's played for 6 different franchises, and has just 201 points in 661 career games heading into this season. You can watch him play a few nights and think that he is one of the best players on his team. Over the course of an 82 game season, however, he simply proves that he is not.

Performance is the difference between the player Randy Moss was in Minnesota and the Randy Moss in Oakland. It's the difference between Marc-Andre Fleury and J.S. Giguere. It is the reason that you can't trust your eyes and the highlight reels to evaluate goalies. If a goalie makes a great save and then lets in a really soft goal, he's no better than a goalie who lets in the tough one and stops the gimme, but often it is the great save that sticks in our minds and influences our perceptions. When it really comes down to it, it's not how you stop them, it's how many you stop, because hockey games are decided by goal differential, not style points. You have to look at the numerical record, because that is the only objective and comprehensive record of a goaltender's performance over the course of a season.

Martin Brodeur is a talented goalie. He is a great puckhandler, he controls his rebounds well by most measures, he is a good skater, he is athletic, he has great reflexes. Watch him, and you'll probably be impressed by something. But look at the stat sheet and divide his number of saves by the number of shots he faced, and you'll think you made a mistake, because the number is lower than perception would indicate. This is the talent/performance gap in action.

I believe the gap is sometimes very high for goalies, because of the overarching importance of positioning and technique. Cristobal Huet is one of the top goalies in the NHL, and it is not because of his talent. He has somewhat slow lateral movement, he doesn't have outstanding reflexes, he's not one of the best puckhandlers. Yet look at his save record, and it is great. This is because his positioning is usually perfect and his technique is very good. In today's high-speed NHL game, those things are more important than raw talent. Another example is Giguere, who has been frustrating opposing fans for years because he doesn't look like anything special. He is always optimally positioned and moves around in a compact block, making it very tough for shooters to pick the corners on him. Dominik Hasek was considered lucky for years because he had an unorthodox technique, but his fabulous save statistics showed that he was massively outperforming his peers who were doing things "correctly".

Talent is easier to evaluate than performance. Even professional players and scouts often fall for it, but that doesn't make it any less of a trap. Most seasoned hockey fans can tell which goalie is more fluid in his movements, who covers more net, who handles the puck better, who catches/absorbs more pucks, etc. But the difference in save percentage between top goalies is often something like .005 or less (or even .001, as last year between Luongo and Brodeur). This translates to one extra save for the better netminder every 8-10 games, which is an impossibly small margin to reliably detect based on observation alone, and that's not even taking into account team factors like shot quality against.

I place a heavy emphasis on numbers because they eliminate selective memory bias and excessive focus on talent, and look only at the most important issue: How effectively did the goaltender keep the puck out of his net, subject to the team conditions he played under? That's the bottom-line consideration that everything should be based on, because hockey goaltending is a results-oriented profession: It's the performance, not the talent, that truly counts.

Wednesday, October 10, 2007

The Curious Problem of the 1980s

Using the performance vs. backup approach described in a previous post, I wanted to see who was the best goalie in each decade. I calculated a total "goals better than backups" for each goalie, based on figuring out how much better each goalie was than their backups, and then multiplying that by the number of games played to get a total number of goals. So it was not only excellence that counted, but also longevity.

What I found was that for the most part, my answers for each ten year period followed along pretty closely with conventional wisdom, with one glaring exception.

1950s: 1. Al Rollins, 2. Jacques Plante, 3. Glenn Hall
1960s: 1. Johnny Bower, 2. Charlie Hodge, 3. Glenn Hall
1970s: 1. Tony Esposito, 2. Bernie Parent, 3. Ken Dryden
1980s: 1. Dan Bouchard, 2. Pete Peeters, 3. Chico Resch
1990s: 1. Dominik Hasek, 2. Patrick Roy, 3. Curtis Joseph
2000s: 1. Miikka Kiprusoff, 2. Dominik Hasek, 3. Roberto Luongo

For most of the decades, they go pretty much as expected. Probably the only real surprises to most would be the absences of Terry Sawchuk and Martin Brodeur, but neither would be particularly surprising to regular readers of this blog. That is until we come to the 1980s. Of the three goalies listed from the 1980s, I would not have predicted any of them to rank that highly. I think the expected names were Grant Fuhr, Billy Smith, Patrick Roy, or maybe Mike Liut, but none of them popped up.

The 1980s were in many ways a unique time for goaltenders. It was a time of dynasties, a time of expansion (both in terms of teams and a talent pool expanding to include European talent), and a time of rapid evolution in goaltending technique and equipment. I think there also was an absence of a true superstar goaltender, someone to take over the mantle from Esposito, Parent and Dryden, at least until Patrick Roy became established near the end of the decade. This is indicated by the fact that there were no repeat winners of the Vezina Trophy throughout the decade.

It unlikely that any other decade can match the 1980s for contrasting styles - Roy was leading the butterfly revolution against some of the old stand-up goalies, there were European goalies like Lindbergh coming over to add their own playing methods, athletic goalies such as Fuhr to challenge old limits of goalie capabilities, and goalies were also becoming more involved in terms of puckhandling, with Hextall leading the way. This makes it difficult to subjectively rate goalie play.

However, possibly the biggest factor was the lack of parity in the league. Some goalies played on powerhouse teams and received wide recognition for their team successes, while others were doomed to be overlooked because of their weak teammates. Looking at average GAA throughout the decade, there were 6 teams that allowed under 3.5 goals against on average: Montreal (3.14), Boston (3.30), Philadelphia (3.31), Buffalo (3.38), New York Islanders (3.40), and Washington (3.46). There was then a clear gap to the rest of the league, as the next best team was Calgary (3.73). There were 5 teams that allowed over a goal per game more than Montreal on average: Winnipeg (4.24), New Jersey (4.26), Pittsburgh (4.27), Los Angeles (4.33), and Toronto (4.43).

Is it likely that teams like Montreal and Philadelphia had better goaltenders than Toronto did? Probably, for the most part. However, that difference certainly can't be blamed entirely on the goaltending. During the decade, Toronto played 15 different goaltenders. Philadelphia used 13 different netminders, including 3 that also played in Toronto. The Leafs gave up 1.1 more goals per game. Obviously the key factor there was team defence, not goaltending.

I looked at save percentage statistics, but I soon came to the conclusion that it was a largely futile exercise. The top 6 1980s goalies in save percentage (minimum 3000 shots faced) all played on one of the 6 strong defensive teams mentioned above. At the bottom of the list were mostly the goalies from the league bottom feeders. If I used save percentage alone, I would claim that the best goalies of the decade were Patrick Roy, Kelly Hrudey, Ron Hextall, Bob Froese, Billy Smith, Andy Moog, Glen Hanlon, Reggie Lemelin, and Tom Barrasso. However, I don't believe these selections to be entirely correct.

Therefore, I turned to my method of comparing results against backup goalies. In the 1980s, platoons were quite common, so the method should be pretty accurate since sample size isn't as big of an issue. A lot of teams had goalies that virtually split time. What was very interesting was that a lot of times the goalies had very similar statistics. Andy Moog and Grant Fuhr in Edmonton, Patrick Roy and Brian Hayward in Montreal, Billy Smith and Kelly Hrudey in New York, this pattern was repeated in a number of cities, and is further evidence that the team is very influential in determining the success of a goaltender in the vast majority of cases.

Here are the top 10 goalies from the 1980s, in terms of performance against their backups (with adjusted percentage better than teammates, adjusted GAA, and adjusted teammate GAA):

1. Dan Bouchard, 15.1%, 2.86, 3.29
2. Bob Froese, 14.0%, 2.50, 2.85
3. Allan Bester, 12.6%, 3.22, 3.63
4. Kelly Hrudey, 11.3%, 2.81, 3.13
5. Mario Lessard, 11.1%, 3.17, 3.52
6. Pete Peeters, 10.8%, 2.47, 2.74
7. Chico Resch, 10.6%, 3.05, 3.38
8. Andy Moog, 9.8%, 2.84, 3.12
9. Tom Barrasso, 8.4%, 2.75, 2.98
10. Rollie Melanson, 7.4%, 2.95, 3.17

Was Dan Bouchard really the best goalie of the 1980s? Many people would probably respond to that question with, "Who?" Bouchard played 8 seasons for the Atlanta Flames, mostly in the 1970s, and then was traded to the Quebec Nordiques where he spent 5 years before finishing up his career in Winnipeg. Bouchard's save percentage from 1982-1986 was just .873, slightly below league average, but his teams were weak. Bouchard also ranks 6th out of all goalies in the 1970s, so that is more evidence that he was a good goalie. Probably a major reason why Bouchard isn't well remembered is that his career playoff win/loss record is 13-30.

Pete Peeters is another surprise. I was expecting Peeters to be revealed as a team creation, someone who was only successful because of the dominant Philadelphia, Washington and Boston defences he played behind. I think they were definitely all significant contributors to his success, but the evidence remains that Peeters outplayed the other goalies on his teams.

Allan Bester is the kind of goalie that is easy to overlook because he played for terrible teams, but he had an .885 save percentage on the 1980s Toronto Maple Leafs, which is certainly deserving of respect.

On the other side of the scale, here are some goalies who did not do significantly better than their backups did:

Grant Fuhr, 1.1%, 2.99, 3.03
Don Beaupre, 1.1%, 3.00, 3.03
Greg Millen, 1.1%, 3.17, 3.20
Bob Sauve, 1.0%, 2.81, 2.84
Richard Brodeur, 0.5%, 3.08, 3.10
Brian Hayward, 0.5%, 2.97, 2.98
Glen Hanlon, 0.2%, 2.94, 2.94
Greg Stefan, -0.1%, 3.17, 3.16
Gilles Meloche, -0.2%, 2.90, 2.89
Tony Esposito, -1.0%, 3.02, 3.00
Billy Smith, -1.4%, 2.70, 2.67
Patrick Roy, -1.6%, 2.39, 2.35
Pat Riggin, -1.7%, 2.75, 2.71
Clint Malarchuk, -4.1%, 2.81, 2.69
Don Edwards, -8.1%, 2.89, 2.66

It now becomes difficult to know where to rate someone like Roy, who put up outstanding numbers in terms of GAA, wins, and save percentage, but wasn't even able to outplay the other goalies on his team. I don't think he played poorly, but he obviously received tremendous support from his teammates, and I think it is fair to conclude that a lot of other goalies could have put up similar results in that team situation. Look at, for example, Brian Hayward. In 1985-86, with Winnipeg, Hayward was 13-28-5, 4.79, .842. In 1986-87, in Montreal, Hayward outplayed Patrick Roy and led the league in goals against average with 2.81 (Hayward's record was 19-13-4 with an .894 save percentage). That the same goaltender could have a drop of nearly 2 full goals per game in GAA just by switching teams illustrates the nature of the competitive climate of the time, and makes Roy, Smith, Fuhr, etc. appear to be more lucky than good.

Playoff results are normally important in ranking goalies, but the team factors again loom very large. Grant Fuhr, Kelly Hrudey, Billy Smith, Mike Vernon, and Patrick Roy all had very good records, but most of their playoff games were on very good teams. The Islanders and Oilers were dominant in the playoffs pretty much no matter who they had in net. For the weaker teams, the team effects were even stronger since the opponents were tougher. I see it as unfair to penalize goalies for getting shelled while their teams were being dominated by better teams like the Oilers, Islanders, Flames, or Flyers. Therefore, I'm taking playoff performance into account, but not weighting it very heavily.

In summary, there is little evidence that there were any dominant goaltenders in the 1980s. As a result, it is mostly team effects that determine which goalies endure in the public memory and which ones are forgotten. To try to identify the true standouts, I'm relying heavily on their comparative performance against teammates. The imprecise nature of the 1980s situation makes it difficult to come to a definitive ranking, and I'm not sure we'll ever arrive at one. Nevertheless, it is an interesting area of discussion, and I'll at least take a shot at it:

Rankings of 1980s goalies:
1. Dan Bouchard
2. Pete Peeters
3. Mike Liut
4. Kelly Hrudey
5. Bob Froese
6. Chico Resch
7. Reggie Lemelin
8. Andy Moog
9. Tom Barrasso
10. Allan Bester

Friday, August 31, 2007

The Insightful Serge Savard

SI's Michael Farber recently gave us his all-time NHL team.

In his article, he included this interesting paragraph on the importance of goaltending:

"The goaltending position on the historical team -- or the Ghost of Christmas Goodies Past, as I like to think of it -- vexed me the most. Recently the esteemed Serge Savard, who narrowly missed making this team as a reserve defenseman, advanced the counterintuitive argument to me that goaltending is not the most important position because, in almost every case, teams make goalies more than goalies make teams. Remembering the 1999 Buffalo Sabres and Dominik Hasek, Savard still has some convincing to do."

Savard is not completely right - goaltending is the most important position, since the potential effect of a goaltender is much greater than that of any other single player. However, I generally agree that the importance of goalies is generally overstated. The position is important, but performance is actually quite similar among most goalies at the top levels, especially after removing team effects. The difference between many good goalies is just small fractions of goals per game, which means that you often need a substantial edge in the crease to make a losing team into a winner, or vice versa. This is because the rest of the team combined is three or four times more valuable than the goalie in terms of their contribution to the final result. So it is actually fairly rare that a goalie wins or loses games because of their singular efforts. Pointing out one exception to the rule (Hasek) doesn't invalidate that entire viewpoint. In fact, it is good evidence for that particular goaltender as one of the all-time greats.

I wonder what Savard thought about his former teammate Ken Dryden. If you take his statement to be true in all cases, then Savard probably thinks Dryden was lucky to be playing on the teams he did.

Farber should have paid more attention to Savard, because he picked Terry Sawchuk as the backup on his all-time team, a goalie who is very overrated because of the strong Detroit teams he played on early in his career. Farber's top goalie ever was Roy, which isn't a bad choice, but probably not the correct one, and the reason again is the hidden factor of team strength.

It is difficult to argue with choices for all-time teams, because different observers rate things differently. For example, some focus more on career numbers and longevity, while others heavily weight a goalie's prime. Some look at playoff performance above all else, while others just see it as part of the mix.

I don't have numbers yet that I believe are conclusive in terms of ranking the best of all-time. However, I think Hasek should probably be in the top spot, and would probably lean towards Jacques Plante in a very tight decision over Glenn Hall, Tony Esposito, and Roy for the #2 position.

Friday, August 24, 2007

Testing the Teammate Theory

To test how well we can evaluate goaltenders by comparing them to their teammates, I decided to look in some more detail at the last two seasons. This allows for a direct comparison with shot quality data to see if there are parallels, and to see whether the two methods deliver similar results.

I calculated GAA compared to backups and save percentage compared to backups (with adjustments) for all the goalies who had been starters over the last two seasons. I also used shot quality information available from Alan Ryder and Ken Kryzwicki at Hockey Analytics (taking care to attempt to correct for the variance in shot distance reporting around the league recently discovered by Ryder), and used that to find out whether each goaltender's save percentage was above or below predicted levels, and by how much.

It turns out that the methods have quite similar results. The correlation coefficient between save percentage vs. predicted save percentage from shot quality and save percentage vs. teammates' save percentage was 0.73. Six of the top 10 goalies in performance vs. backups were also in the top 10 in performance against the predictions of the shot quality model (Huet, Hasek, Lundqvist, Lehtonen, Kiprusoff, and Legace). In addition, both methods agreed on the top man: Montreal's Cristobal Huet. Here are both lists:

Performance vs. Predicted (Shot Quality), 2005-07:
1. Cristobal Huet
2. Dominik Hasek
3. Tomas Vokoun
4. Henrik Lundqvist
5. Kari Lehtonen
6. Miikka Kiprusoff
7. Roberto Luongo
8. J.S. Giguere
9. Manny Legace
10. Ray Emery

Performance vs. Backups, 2005-07:
1. Cristobal Huet
2. Miikka Kiprusoff
3. Henrik Lundqvist
4. Martin Brodeur
5. Tim Thomas
6. Kari Lehtonen
7. Dominik Hasek
8. Ryan Miller
9. Manny Legace
10. Marc-Andre Fleury

Vokoun illustrates one of the problems with using backup stats: His backup is Chris Mason, a good goaltender, which makes it difficult for Vokoun to rate highly against his own teammates. Giguere is in a similar situation. Emery is knocked down by having backed up Dominik Hasek in 2005-06, while it is likely that Luongo's backups simply overachieved in the few games Roberto was given off, probably also against soft opposition.

On the other list, Brodeur and Miller were just outside the top 10 in shot-quality adjusted save percentage, so they aren't way out of place, although they both made it on because their backups did not do very well compared to their predicted save percentages. Thomas and Fleury benefitted from some very poor play by their backups (mostly Toivonen and Thibault, respectively).

Therefore, this seems to show that while using teammate data cannot replicate shot quality data, it can in many cases do a pretty good job of estimating it, as the results are correlated and the rankings similar. There will always be a few goaltenders rated too high or too low because of the strength or weakness of their teammates, even after adjustments, so this method requires some care in interpreting the results. Nevertheless, a discerning application of the method to the goalies of seasons past should yield some useful and interesting results.

Sunday, August 19, 2007

The Best Regular Season Goalie of All-Time

I have completed a comprehensive study of every goalie who has played 10,000 or more minutes in the NHL since World War II. I measured their performance in terms of GAA in each season against the other goalies on their team. This was done because goalie results are strongly impacted by the team around them. Goalies who cannot outplay their teammates are not very valuable to their teams, since their performance can be easily replaced. Therefore, the best goalies should be ones who outperform their teammates the most, after adjusting for controllable factors like era, minutes played, strength of backup goalies, etc.

All GAA results were adjusted to league average to allow comparisons across eras. I adjusted for the strength of their teammates with an adjustment factor based on their career length, performance and reputation. I ignored seasons where the goalie played less than 120 minutes, or his backups played less than 120 minutes, because those samples are too small to be meaningful yet could easily skew the results. I took the teammate goalies' combined performance and calculated how well they would have been expected to do, given the starters' minutes. I also further adjusted the starter and teammates' performances based on the number of minutes played, given that the more minutes played, the greater the statistical significance of the performance.

That left me with an adjusted starter GAA, and an adjusted backup or teammate GAA. Each goalie was then ranked by how well they performed relative to their teammates' adjusted performances. I also calculated these results for each goalie's five best consecutive seasons, i.e. their career prime. The results are described below.

There are a few limitations to this study I would like to outline here:

The adjustment factors for each goalie's backups were subjective. Changing them does not change the results substantially, but it would likely impact the rankings. I found a lot of goalies that I considered to be good based on results and reputation were indicated to be not so good by the results, and vice versa. Perhaps I will go back and revise the factors based on some of my new findings and see how the results change.

Also, it was more common in the 1940s and 1950s for a single goalie to play all of his teams games. Goalies like Glenn Hall, therefore, are underrated in this analysis because in their best seasons they did not have any backup results to compare with, so those years unfortunately could not be considered.

The way a team conducts its goalie rotation can have an impact on the results. For example, some teams only play their backup against weaker opponents, while others essentially platoon. It would be incredibly time-consuming to adjust for opponent on a game-by-game basis for every goalie, so this has been left out. I think it is common for backups to play weaker opponents, so any adjustment would likely be in a similar range for most of the goalies, but it could be that there are some goalies who were advantaged or disadvantaged because of difficulty of opposition.

Another adjustment I would have liked to have made was to introduce an age curve. Several goalies, such as for example goalies who are still active, rank much higher than they should because they do not have a decline stage, a number of games late in their career when they are no longer playing at their peak. Such an adjustment would not penalize goalies with longer careers.

On to the findings:

The best regular season goalie ever: Dominik Hasek. Hasek ranked first overall and had the best prime. Hasek was 29.9% better than his backups in terms of adjusted GAA, and 41.5% better during his prime. Given that Hasek came over to the NHL late and missed out on a number of potential seasons in his mid-20s, and is still active beyond the age of 40, for him to take first place on this list is very strong proof that he is the best goaltender to ever play NHL hockey, at least in the regular season.

Second place was a surprise: Al Rollins. Rollins took second both overall (27.4%) and prime (37.0%). Rollins toiled on weak Chicago teams in the 1950s, but was clearly a strong goaltender, as shown by his Hart Trophy award in 1954. Rollins did not have a long career (9 seasons), and is not in the Hall of Fame. However, these results indicate that his only limitation was his team, and that his performance was outstanding.

In third place was Miikka Kiprusoff (26.0%). Kiprusoff ranks high because the only results we have are from the prime of his career. He has no decline stage, and he did not play much early on. This shows how well he has played over the last few seasons, but he should definitely be ignored in terms of talk of the greatest of all-time. There were a number of other modern goalies who similarly popped up higher than they should have for the same reasons. However, Kiprusoff did rank 5th overall in terms of career prime (26.2%, basically the same as his overall career), so he is clearly an outstanding goaltender.

In fourth place was another expected name: Jacques Plante. I was not sure what to expect with Plante, whether he was a creation of some powerful Montreal teams or if he really lived up to the billing. The results indicate that he clearly was one of the greatest goalies of all-time. He had a very long career, and still easily outperformed his backups (20.9%). In terms of prime, Plante ranks third overall (31.8%). Therefore, looking at his career as a whole, Plante has a strong case to be ranked as the second best regular season goalie of all time after Dominik Hasek.

In fifth place was another well-known name, but possibly a controversial one. Ken Dryden played for some amazing teams, which has caused speculation that he was just a product of those teams. However, Dryden was far better than his backups, letting in 20.1% fewer goals. Again, Dryden had the advantage of a short career without much of a decline stage, so he ranks a little higher than he maybe should. In terms of prime, Dryden is 11th (19.8%). However, the numbers show that Dryden should probably be considered in the top 10 all-time. He played for great teams, but his performance was also apparently great.

In sixth place comes a bit of a shocker: Roman Cechmanek. Cechmanek played just four whole seasons in the NHL, most of them behind a good defence in Philadelphia, but he easily outperformed his backups (19.5%). However, there are reasons to be cautious about this result. First, Cechmanek had a very short career. Secondly, some of Cechmanek's teammates (Esche, Boucher) ranked very low, indicating that his results were probably somewhat inflated because his backups were very poor. Cechmanek's prime (which is essentially the same as his career) ranks 12th. So Cechmanek's awkward style was probably more effective than it looked during his time in the NHL, at least during the regular season.

In seventh place is Bernie Parent (18.2%). Parent also had a dominant prime, ranking 4th (30.6%). Parent is another goalie that played on some great teams, but also put up some great performances.

Eighth was another virtual unknown: Roy Edwards (17.5%). Edwards had 7 NHL seasons in the late 1960s and early 1970s. He is a classic case of a goalie that never got a chance because of the 6 team league, and also one who was underrated because he played on weak teams. Upon expansion, he won a job and proceeded to put up some relatively outstanding numbers on a weak Detroit team that missed the playoffs in 5 out of 6 seasons Edwards played for them. This was despite being 31 years old by the time he broke into the NHL. Edwards' prime ranks 6th overall (24.7%).

Ninth is another active goaltender, and a very interesting result: Manny Legace (17.0%). Legace is the most underrated goalie in the NHL today. He has had an outstanding career as a backup on Detroit, since he nearly always outperformed his more highly-rated teammates (Curtis Joseph, Chris Osgood, Dominik Hasek). He had a very good season last year on a weak St. Louis team as well. Legace ranks 21st in terms of career prime (16.9%). Legace will drop some in the years ahead as he ages, but he is definitely a top goaltender in the league.

Rounding out the top 10 is Mario Lessard. Lessard had a short career in Los Angeles in the early 1980s, but put up excellent numbers. However, his prime ranks 23rd overall, so one of the reasons he comes in so high here is because he did not have a decline stage. Nevertheless, he is another underrated goalie.

The rest of the top 10:
11. Dan Bouchard (prime: 8th)
12. Marty Turco (prime: 33rd)
13. Tony Esposito (prime: 10th)
14. Arturs Irbe (prime: 9th)
15. Bob Froese (prime: 13th)
16. Chico Resch (prime: 16th)
17. Charlie Hodge (prime: 14th)
18. Rick DiPietro (prime: 37th)
19. Fred Brathwaite (prime: 15th)
20. Daren Puppa (prime: 62nd)

OK, enough of that. Now let's go to the fun stuff: pointing out the frauds.

First, let's see how our blog's namesake did:

Martin Brodeur:

vs. backups: 4.5% better, 52nd place
career prime: 18.4% better, 18th place

So Brodeur has a pretty solid prime (1996-2000), but his career as a whole is not that impressive. Just more evidence to throw on the already huge pile that most of his success is owed to the team in front of him.

How about Grant Fuhr:

vs. backups: 2.2% worse, 115th place
career prime: 1.7% better, 93rd place

Terrible results for Grant Fuhr. He was just as good as his backups in terms of numbers. As far as a regular season goalie, it is virtually impossible to make a statistical case for Grant Fuhr as even one of the best of his era, much less all-time. However, I don't think many of his backers would even look at regular season results when arguing for Fuhr's greatness.

Most people probably noticed the absence of Patrick Roy in the top 20. So where does he come in?

vs. backups: 7.3% better, 31st place
career prime: 13.9% better, 29th place

Patrick Roy is overrated. There, I said it. Roy was a great goalie, no question, but his teams were almost always very good, and that was a big advantage throughout his career. He definitely should be considered in the greatest of all-time argument because of his unbelievable playoff record, but he is not one of the best regular season goalies ever.

How about Billy Smith:

vs. backups: 3.0% worse, 121st place
career prime: 1.1% worse, 115th place

Most people don't really see Smith as a regular season goalie, of course, because he is remembered almost entirely for playoff results. Nevertheless, I believe Smith to be one of the most overrated goalies of all time. His playoff results were only great for four seasons when the Islanders were an unstoppable dynasty, and his regular season results were very mediocre.

Terry Sawchuk:
vs. backups: 4.6% worse, 135th place
career prime: 17.5% worse, 153rd place

Terry Sawchuk, considered by many to be the greatest goalie of all-time, had the single worst career prime rating of any goalie in my study. How is this possible? Well, let's look a little closer at Sawchuk's career. Sawchuk played five seasons as the starting goalie in Detroit from 1950 to 1955. In all of those seasons he had GAAs under 2, he led the league in wins all five seasons, and he won 3 Vezinas and 4 Stanley Cups. I think pretty much everyone would agree that was the prime of his career. So was it the goalie or the team? Well, let's look at his backups. In those 5 seasons combined, Sawchuk's backups were 8-1-3 with a 1.47 GAA. This indicates that Sawchuk's Detroit teams were amazing.

Sawchuk had a long career, so that pulls his results down a lot. But over half of his career shutouts came in a five-year stretch with a Detroit team that was clearly a dominant team, and the results of his teammates show that Sawchuk clearly benefitted heavily from the defence in front of him. Sawchuk is another goalie that is clearly very overrated. His longevity, as well as his good fortune to have played for half a decade on a very dominant team, has led to great career numbers. As a whole, however, he was worse than the other goalies on his team, and is certainly not one of the best of all time.

I welcome any comments on the study's methods and results, and I plan to analyze the results in more detail as I continue to try to identify goalies with performances that do not match their reputations, both good and bad.

Monday, August 13, 2007

The "Other Teams Play the Trap" Argument

An oft-repeated argument to defend Martin Brodeur is that it cannot be claimed that the trap is the reason for his success, since if it was, everyone else would just copy it. Other teams have not been able to emulate the Devils success, so it must be someone they have on their team that is the difference-maker. The main guy that has been there the whole time has been Martin Brodeur, so he should get the credit for the Devils' run of success.

This argument is partly correct, but is nevertheless wrong with respect to Martin Brodeur. It is true that if the strategic option of playing the trap dominated all others and did not depend on a team's talent level, then every team would use it and it would even itself out. But what matters is not the strategy alone, but how well that strategy is implemented. That is, how good is the team at playing the trap? Every team in the league plays a very defensive style with a one goal lead late in the third period, because it makes strategic sense, but clearly some teams are better than others at actually pulling it off.

A similar example I can use is the power play. Every team in the league has a power play unit, but some are better than others. Some teams are very good for a number of years on the power play (a recent example is the Detroit Red Wings, top 5 in the league in power play efficiency every year from 2000-01 to 2005-06). By similar logic to that above, it could be argued that other teams should be able to copy what they do and eliminate their advantage, but what is really driving the success is not just power play tactics, but also coaching, teamwork and talent. It wouldn't matter if the Chicago Blackhawks used the exact same power play setup as the Wings, moved the puck around in a similar manner and took the same types of shots, they just wouldn't be as effective.

It is glaringly obvious both on the statsheet and in real life that no matter how many teams used the trap, nobody played it as well as New Jersey. From 2000 to 2004, the worst New Jersey finished in fewest shots against was 2nd, and the most shots they allowed per game was 24.7. This was despite the other teams in the league having a decade to copy New Jersey's style. And it wasn't just Brodeur, since his average was actually 1.5 shots per game higher than his backups (who faced just 22.3 shots per game). Alan Ryder points out that the New Jersey Devils have always led the league in his measurements of shot quality against. This is a team where the immortal Corey Schwab had a 1.27 GAA in 14 games over 2 seasons. The Dead Puck Era New Jersey Devils were absurdly good at defence.

The Devils have generally had a deep defensive unit with a number of excellent defensive defencemen, as well as excellent defensive forwards on their checking line. This is still the case in New Jersey; it is easy to label newcomers like Johnny Oduya as untalented, but Oduya was on the ice for just 37 goals against in 1110 minutes of even-strength ice time this season, a good rate that was better than the rest of the team's when he wasn't playing (source: Behind the Net). Defensive play is hard to judge, but New Jersey seems to keep finding players who are very responsible in their own end, or perhaps developing them through their strong coaching staffs at the NHL or minor league level. The Devils are also very well managed by Lou Lamoriello, who usually brings in players that fit well into the New Jersey system. These are all reasons why New Jersey has been dominant defensively for over a decade, not merely the style of play that they employ.

There are, in fact, some teams that have copied the New Jersey model with some success. The best example is probably the Minnesota Wild, with Jacques Lemaire, although there are a number of other teams playing a tight defensive style, especially in the Western Conference. After the early years after expansion, Minnesota has been quite similar to New Jersey in team defensive statistics. Over the last few seasons, their goaltending has been every bit as good as New Jersey's in terms of save percentage. The Minnesota Wild have basically been the New Jersey Devils in a tougher division and with less overall talent. It is mainly those two factors that explain why the Devils have been at the top of their conference and the Wild have been in a constant struggle just to make the playoff struggle, not merely the choice of defensive style.

No strategy on its own will be successful if executed poorly, and, similarly, there are a number of strategies that will be successful if they are well-executed on a talented team. The Devils have won with a very defensive system mainly because they have done the best job of recruiting and developing talent that complements the system. Regardless of how many copycats there have been, it has only been since the lockout that a number of other teams have begun to rival New Jersey in terms of team defensive play. This coincides with significant talent losses for New Jersey (led of course by Stevens and Niedermayer). Without talent, no system of play is likely to be successful, and it has been primarily team defensive talent that has driven New Jersey's success. Even though some teams have copied New Jersey's tactics, nobody has played it as well as they have, and that is why Brodeur has had a relatively easy job throughout his entire career.

Monday, August 6, 2007

Picking the Wrong Guy, Part 1: 1987-88

This is part one of a series on questionable Vezina voting decisions, where team factors fool NHL GMs into choosing the wrong goaltender as the league's best.

In 1987-88, Grant Fuhr won the only Vezina of his career. He also finished as the runner up to Mario Lemieux for the Hart Trophy. Fuhr outpointed Wayne Gretzky (149 points in 64 games), Steve Yzerman (50 goals, 102 points in 64 games), and Denis Savard (131 pts in 80 games). Four voters even listed Fuhr in first place ahead of Lemieux, who had scored 70 goals and added 98 assists to lead the league in scoring.

Fuhr had not done particularly well in previous years in Vezina voting. His best result was second place in 1981-82. So what was the difference? Was it perhaps an improvement in the team? Or did he merely have a career year?

Edmonton lost Paul Coffey from their blue line after the 1986-87 season, but his strengths were mainly offensive and it is doubtful they lost much on the defensive side. The other main blueliners (Lowe, Muni, Gregg, Smith, Huddy, McSorley, Beukeboom) all returned for 1987-88. The Oiler forward group was very much the same as the year before. As might be expected, then, the team played defence at a similar level. The team shots against totals were almost the same - 28.9 per game in 1986-87, 28.8 in 1987-88.

Given the continuity, therefore, one would expect Fuhr's numbers to be similar. Indeed, his performance rates were almost exactly the same in 1987-88:

1986-87: .618 win %, .881 save %, 3.44 GAA
1987-88: .610 win %, .881 save %, 3.43 GAA

So if his performance didn't improve, how then did Fuhr go from 3rd in Vezina voting with 0 Hart votes to being considered the best goalie in the league and more valuable than Wayne Gretzky?

The very simple answer: Andy Moog left to play on the Olympic team.

Andy Moog and Grant Fuhr spent six years sharing time in the Edmonton net, and were for all intents and purposes the same goalie. Most years their save percentages were very similar, and their records for Edmonton during the dynasty period were almost identical: Fuhr 107-39-14, Moog 104-37-14. No matter who was in net, Edmonton usually won.

Moog's departure in 1987 meant that Edmonton only had 21-year old Bill Ranford and 22-year old Darryl Reaugh on their team. This meant they chose to rely heavily on Fuhr. Fuhr played 75 games, leading the NHL. He also led in minutes played, wins, and shutouts. Martin Brodeur fans will probably recognize that combination. However, 14 other goalies played 2000 minutes or more and finished ahead of Fuhr in save percentage.

Was Fuhr deserving of the Vezina? Let's look at his stats again, compared to his teammates in the Edmonton net:

Fuhr's numbers: .610 win %, 3.43 GAA, .881 save %
Fuhr's backups: .714 win %, 3.95 GAA, .876 save %

Fuhr's backups' numbers are a little misleading because of the influence of one Warren Skorodenski, a rarely used career backup who saw his final bit of NHL action with Edmonton in 1987-88. In 61 minutes of play, Skorodenski was beaten 7 times for a catastrophic 6.89 GAA. Since Fuhr's backups didn't play many games in 1987-88, Skorodenski has a large impact on the stats. Here is the stat line without him included:

Ranford and Reaugh: .714 win %, 3.59 GAA, .890 save %

That compares very favourably to Fuhr, and provides evidence that his season really wasn't that special at all. As had been the case for many years with Moog, Fuhr did no better than his backups had done. Ranford was coming off of a 41 game season with an .891 save percentage in 1986-87, and he improved to .899 in 6 games in 1987-88, so he probably could have played more games at least at the same level as Fuhr. Fuhr was far from the league's best goalie, and he deserved little consideration as league MVP. Rating Fuhr as more valuable to the Oilers than Gretzky has to likely be considered one of the most curious award voting decisions in the history of the NHL.

Let's look at the way the Vezina voters ranked the rest of the top 5 after Fuhr. I have included their performance statistics, as well as that of their backups to get the team context.

2. Tom Barrasso, 2-3-3, 22

Barrasso's numbers: .569 win %, 3.31 GAA, .896 save %
Barrasso's backups: .466 win %, 4.45 GAA, .860 save %

3. Kelly Hrudey, 1-4-0, 17

Hrudey's numbers: .557 win %, 3.34 GAA, .896 save %
Hrudey's backups: .542 win %, 3.22 GAA, .893 save %

4. Brian Hayward, 2-1-1, 14

Hayward's numbers: .667 win %, 2.86 GAA, .896 save %
Hayward's backups: .625 win %, 2.97 GAA, .898 save %

5. Mike Vernon, 0-4-2, 14

Vernon's numbers: .685 win %, 3.53 GAA, .877 save %
Vernon's backups: .556 win %, 4.36 GAA, .858 save %

Barrasso's numbers are very impressive. His backups were Daren Puppa and Jacques Cloutier, both of whom were decent goalies. Barrasso's numbers were very good for any team, much less a weak Sabres team coming off of a sub-.500 campaign in 1986-87. He deserved to have won the Vezina.

Kelly Hrudey's numbers are very good, but the fact that they were pretty well matched by the 38-year old Billy Smith testifies to the Islanders' strong defensive play. It was a similar situation in Montreal, where Brian Hayward and Patrick Roy both played well, but the main reason for their success and Jennings Trophy win was their very strong defensive team.

Vernon doesn't look that great overall, except for his winning percentage. But the fact that his backups could win at a .556 rate despite a 4.36 GAA shows how good the Flames were. The Flames #2 goalie was Doug Dadswell, who played 25 of his 28 career NHL games in 1987-88, explaining the weak performance of the backup goalies. Vernon was no better than average in 1987-88, and only received notice because of his 39 wins as the primary starter on an excellent team.

Coming in sixth place in the voting was the Rangers' John Vanbiesbrouck, who might have been the second best goalie in the league in 1987-88:

Vanbiesbrouck's numbers: .545 win %, 3.38 GAA, .890 save %
Vanbiesbrouck's backups: .438 win %, 3.56 GAA, .876 save %

The backup was Bob Froese, who had a number of very good years in the 1980s, yet was well outplayed by Vanbiesbrouck.

The rest of the top 10 (Lemelin, Roy, Hanlon, and Stefan), all played for very good defensive teams, and had similar stats to the other goalies on their teams.

Were there any goalies that went unnoticed on bad teams? A couple. Daniel Berthiaume of Winnipeg had a .531 winning percentage and .882 save percentage on a weak team, much better than what his backups did, and Darren Pang stopped shots at an .891 rate behind the porous Chicago defence (35.2 shots against per game), although Pang's teammate Bob Mason also did pretty well.

The writers fell into the same trap as the GMs in their All-Star voting. Fuhr again finished first, taking 58 out of 61 first place votes. Patrick Roy was second, followed by Tom Barrasso. The rest of the list went more or less by the strength of the goalie's team, with Malarchuk, Vernon, Lemelin, Hextall, Hrudey, Peeters, Hayward, Liut and Hanlon.

In summary, then, the statistics show that Fuhr was neither outstanding nor especially valuable in 1987-88, and his per-game performance was almost exactly the same as what he had done the previous year. This means that the voters got it completely wrong. Fuhr's Vezina win was the result of the Oilers' loss of Andy Moog, which led to more games played and therefore more wins and shutouts. This attracted the attention of the voters, who, as they often do, overrated durability and gave too much credit for team success. They even went so far as to claim that Fuhr was more valuable than Wayne Gretzky, a completely laughable assertion. The 1987-88 Vezina should have instead gone to Tom Barrasso.

Friday, August 3, 2007

The Critical Importance of Team Context

There are so many synergies in hockey that the team context always needs to be taken into account for all players, not just for goaltenders. Edmonton just acquired Dustin Penner from Anaheim, giving up 3 high draft picks and committing to $21.5 million over 5 years. I doubt that Edmonton's front office staff uses any kind of advanced statistical metrics, or they likely would never have made such a move.

There were probably no more than a handful of players in the entire league who benefitted as much from their linemates and from soft minutes played as much as Dustin Penner. He might be one of the last players in the league I would have rolled the dice on, not because I don't think he is good, just because he is completely and utterly unproven, and I don't care how many goals he scored last season, I still can't be sure that every one of them wasn't because of Corey Perry and Ryan Getzlaf, or because of weak opposition, or both. Not to mention Penner playing almost 3 minutes per game on one of the league's top power play units.

Getzlaf and Perry were #664 and #665 out of #676 players in terms of easiest minutes played according to Behind the Net. Over the course of the season, Penner's minutes were slightly harder (he ranked #572), but a lot of his scoring came together with those two guys playing the same kind of soft minutes. Looking at the scoring rates, the impression you get is that if anything Penner was dragging the other two down. Here are the per game goals for and goals against on the ice stats for Getzlaf and Perry, with and without Penner, from David Johnson's Hockey Analysis site:

Getzlaf with Penner: +2.19, -1.97
Perry with Penner: +2.24, -2.24

Getzlaf without Penner: +3.40, -1.59
Perry without Penner: +3.25, -1.44

Second line scorers on good teams can often be like goalies playing on strong defensive teams - their teammates make them look way better than they actually are. I don't think this signing will end well for Oiler fans, and all because their management (which should have learned this lesson by now, see Lupul, Joffrey) apparently didn't stop to take the time to consider team effects in their decision making.

Wednesday, August 1, 2007

Evaluating Puckhandling

One of Martin Brodeur's strengths is puckhandling. It is difficult to evaluate puckhandling since there are very few stats to go on, meaning we have to mostly rely on subjective judgment. There is no consensus on the subject, as shown in this HF Boards thread, where a number of Brodeur fans claim he is the greatest puckhandler ever, while others disagree, arguing that Brodeur is not even the best in the game today, surpassed by others like Marty Turco and Rick DiPietro. Given that there seems to be substantial agreement on those three as the best in the league, I will focus mainly on those three in the following analysis.

How can we try to analyze puckhandling? Probably the most obvious place to start is with the scoring numbers, goals and assists. Brodeur is famous for scoring a goal in the playoffs, but goalie goals are obviously very rare, as are goalie assists. This year, Turco had 4 assists to lead all goalies. DiPietro had 2, while Brodeur had just one. However, one season can be a bit fluky - some of the goalies who beat out both DiPietro and Brodeur in terms of assists include Vesa Toskala, John Grahame, Dwayne Roloson, Olaf Kolzig, Marc-Andre Fleury, and Johan Holmqvist, none of whom are particularly well-known for their puckhandling.

Looking back a few years to compare Turco and Brodeur can give perhaps a better comparison. In the last 4 seasons, Marty Turco has 10 assists to Brodeur's four. There were a number of goalies who also outscored Brodeur, many of them not known for puckhandling, like, for example, Roberto Luongo. Consistently piling up assists may be evidence of some puckhandling ability, but I think it is a poor measure because of the luck factor. Sometimes a goalie will get a second assist just for setting it up for one of their defencemen, because a teammate makes a superb play. Also, assists do not measure how well a goalie makes the easy plays, just how likely they are to connect on a home-run pass.

A statistic that can perhaps shed more light on the situation would be giveaways. The problem with giveaways, as with all of the NHL's real-time stats, are that scorers around the league vary substantially in how generous they are with awarding them. Therefore, we need to keep that in mind when interpreting the numbers.

Martin Brodeur's numbers came out as pretty solid - 33 giveaways in 78 games. However, his opponents had just 27 giveaways. There could be several reasons for this, such as the Devils having a less aggressive forecheck, but probably the main reason is that the scorers covering Brodeur seem to credit fewer giveaways. In the 4 games involving backup Scott Clemmensen, for example, there were just 2 giveaways for both teams (both charged to Clemmensen). Brodeur almost certainly handled the puck more often than most of his opponents, but his higher giveaway rate means he would have had to have made substantially more good plays just to break even with the goalies playing against him. It's certainly not unreasonable to suggest that he did, but the giveaway numbers don't seem to indicate that Brodeur is head and shoulders above his peers in this regard, as some observers claim.

Marty Turco had 31 giveaways in 63 games. His opponents had 44 giveaways, indicating that the scorers were more generous with the giveaways. This is reinforced by the games involving backup Mike Smith, as Smith had 12 giveaways and his opponents had 15 in 19 games. So although Turco had a slightly higher rate of giveaways per game, he did much better than his opponents, and when taking that into account it is likely that he outperformed Brodeur, at least in terms of taking care of the puck.

DiPietro is a very different story. He had an awful 75 giveaways in 61 games, compared to just 39 for his direct opponents. The scorers seemed to be pretty hard on the Islanders (backup Dunham had 12 giveaways in 15 games), but DiPietro's numbers are still very poor. His giveaways per game rate is almost triple Brodeur's, and he was charged with almost twice as many giveaways as his goaltending opponents. DiPietro may be able to shoot it hard, but he clearly needs to improve in terms of taking care of the puck, and it is doubtful that the positives make up for his very high error rate.

Martin Brodeur has the established reputation, but I believe the best puckhandling goaltender in the NHL is his successor on the competition committee, Marty Turco. This is from a combination of subjective observation and the above number-crunching. Turco is very aggressive with the puck, and his team's defensive style relies heavily on him playing the puck. Nevertheless, he still commits substantially fewer giveaways than his opposing goalies, and he led the NHL in assists. Brodeur was not charged with many giveaways either, but his opponents had even fewer, indicating that the official scorers watching the games were a little trigger-shy in handing out giveaways. Rick DiPietro is not close to as good as either of those two. He makes far too many mistakes, and isn't scoring enough points to make up for it.

This was a very superficial look at puckhandling, but objective analysis is very limited by the numbers available. I would need to collect more numbers to look at other goalies, as well as to get more of a league average benchmark to more properly analyze giveaway rates and possibly adjust for scorer bias in various cities. One other point that I hope to look at some time is whether some goalies reduce the number of hits taken by their teammates when they are in the game. For the time being, combining giveaways and assists with a subjective analysis of the goalie's play will have to suffice.

Monday, July 30, 2007

It's All About the Team

Over the last few weeks, I haven't posted much, as I have been working on a stats project, developing a new metric to evaluate historical NHL goalies. In recent years, there has been a big improvement in our evaluation of goaltenders, first as a result of a movement away from team-dependent stats like wins and GAA and towards save percentage as the key metric, and then further refining that by adjusting for era or workload. This is definitely a step forward, but even that ignores probably the most important single factor in a goaltender's success: The team in front of them.

Perhaps the biggest breakthrough has been shot quality measurement (read up on that at Hockey Analytics), which has allowed us to remove the impact of the defence in front of a goaltender, subject of course to the limitations of the NHL's RTSS data. However, these data are only available for the past 5 years or so. How then can we attempt to correct for team factors for goalies throughout the history of the NHL?

We can't, really, at least not with very great precision. However, there are ways to estimate shot quality. I looked at correlations with various variables and team shot quality, and found that the variables that were most closely correlated were overall goals against, and team save percentage. Therefore, to estimate shot quality, we can use the statistics of a goalie's teammates, and by using that as a benchmark, evaluate whether the goalie's performance is exceptional or team-driven.

There are several issues with this, primarily sample size. The elite starting goalies play 70-75 games a year, meaning sometimes there are only a few games played with someone else in the net. Also, I don't have save percentage information from before 1982, so I used goals against average. GAA is very team dependent, but since we are comparing teammates here I consider it to be reasonable to use. I also am not sure how much I can trust shot totals over the years, and according to some theories certain goaltenders are responsible for reducing the shots against themselves through puckhandling, rebound control, etc., so using GAA removes those potential issues. Backups also tend to be weaker goalies, but they also tend to player weaker opponents as well. To adjust for this, I have a subjective method for adjusting backup performance, based on their ability and career record.

In short, then, if a goalie gives up fewer goals than the other goalies on his team do, he is better than them, and is contributing to the team's defensive success. If a goalie has similar stats to the other goalies on his team, then it is likely the team that can be credited with the defensive success. In general, the greater the gap between the goalie and his teammates, the better the goalie.

My main area of interest is to determine who were the best performing goalies of the 1980s - was it Fuhr, Smith, Roy, or Vernon, i.e. the goalies with the rings, or were there other unsung heroes who never received their due? This project is almost completed, so I will be posting some of my findings, especially the cases of goalies who appear to be substantially under- or overrated, and hopefully expand this blog's dialogue to include many other goalies beyond Martin Brodeur.

Thursday, July 12, 2007

Defend Martin Brodeur Here

Thus far, I've made my case that Brodeur is an overrated goalie with an excellent defence in front of him. This is shown by, among other things, the strong performances of his mediocre backups and what other goalies have done in similar defensive systems. He has faced few shots against, relatively few difficult chances, and few opposing power plays. Despite these advantages, he has put up unimpressive save percentage numbers throughout his career.

Brodeur's record in clutch situations in the playoffs is not outstanding, and his overall playoff performance is weaker than the best of his peers. The Devils have had playoff success at times even when they did not receive strong play from the goaltending position.

More than anything, Brodeur's success appears to stem from his durability and his teammates, neither of which are particularly good reasons to rank him as the best goalie in the league, much less one of the best of all time. I have demonstrated that all three of his Vezina Trophy wins were undeserved, and that even despite being favored by voters ahead of statistically similar peers throughout his career, he does not have the award recognition to match the all-time greats.

However, judging by reader feedback, not everyone is convinced by the evidence presented. Here's your chance to defend Brodeur by pointing out errors in my position, or contributing alternative viewpoints and hopefully evidence, statistical or otherwise, in support of Marty.

What am I missing with Martin Brodeur? Does his puckhandling save so many goals per year to make up for any other deficits? His intangibles ("inspires his teammates", "leadership", consistency") are often cited - can they be quantified? If they are so effective, why haven't they shown up clearly in his goaltending record? What else does Brodeur bring to the table that other star goalies do not? Or is it all about the career wins, Cups, and gold medals, and who cares how he got them?

Thanks for the continued feedback, positive and negative, as we continue to search for better ways to evaluate the position of hockey goaltender.