Estimating Brodeur's Shot Prevention

There has been an ongoing debate at this blog over how much goalies can influence the number of shots they face in the course of a hockey game. To clarify my position, I always thought it was reasonable that someone like Martin Brodeur would face fewer shots than an overmatched ECHL-level goalie with no puckhandling skills, terrible rebound control and poor positioning who just stood deep in his net quaking and looking small and leaving his teammates to take care of the defensive zone. But even in that best-possible vs. worst-possible scenario, I'd estimate a maximum difference of a few shots per game. From anywhere near the net, NHLers will likely decide to shoot no matter who is in net, and from a long distance they probably will elect to pass or stickhandle even with a weaker goalie between the pipes. Missed shots don't seem to have much of an effect, and rebound shots make up only a couple of shots against per game on average, so those things are unlikely to have a huge contribution to shots against. A top goalie might be able to intercept a pass or two that could lead to a shot, might be able to help his team exit the zone a few times with his stickhandling to avoid subsequent scoring chances against, might be able to hold onto an extra puck or two rather than allow a dangerous rebound chance, and might dissuade a few opposing shooters from shooting from moderate-level scoring positions, at least compared to our replacement level example.

When looking at NHL goalies only, however, it seems to me unlikely that anyone with very poorly developed "soft skills" would make it all the way to the best league in the world. Any differences between them would likely be substantially smaller than compared to the Brodeur/ECHL goalie example.

Can we test this hypothesis? I have repeatedly used comparisons to backup goalies on this blog, and that seems like a good tactic to address the shot prevention issue. Most of Martin Brodeur's backups were with the team only briefly and did not play many games in New Jersey, but there are a couple of backup goalies, Mike Dunham and Chris Terreri, that played a fair number of minutes with Brodeur and should provide an approximate context. I will also use results for Dunham and Terreri outside of New Jersey to estimate their own shot prevention skills for comparison.

First off, Mike Dunham. Dunham broke in with the Devils, and in two seasons in New Jersey faced 26.5 SA/60, compared to Brodeur's 24.1. Looks like Brodeur was doing well, until we realize that Dunham apparently always gives up a lot of shots. In his next six seasons, Dunham went +1.8, +1.9, +2.6, -0.9, +2.4, +1.3 compared to his backups. So he looks to be a guy who faces about 1.5 shots per game more than average. Brodeur beat him by 2.5, so that gives us an estimate of Brodeur being about a shot per game better than average.

Chris Terreri has a few years as a starter before Brodeur broke in. During those years, he was +1.3 and -0.8 playing with Sean Burke, and +0.4 and +0.1 playing with Craig Billington. Both Burke and Billington were very similar to their teammates in terms of shots against, with an average of about a half-shot per game difference between them and their teammates over their careers. Since Terreri was virtually equal with them, it implies that Terreri was about average in terms of shots against.

With Brodeur, Terreri went +1.0 in 1993-94 while sharing time, and +0.4 in 1994-95 in a backup role which may have had some effect on reducing his shots. That supports the estimate from Dunham's stats of Brodeur having a potential effect of something near a shot per game.

Interestingly, as his career went on Terreri seemed to get better at shot prevention. In 1995-96, 1996-97 and 1997-98, he played on a few bad teams and significantly outperformed his teammates in shots against (-2.8, -1.7, -1.2). When we went back to New Jersey for the 1999-2001 seasons, he was basically Brodeur's equal (24.9 SA/60 for Terreri over those 3 seasons, 24.7 SA/60 for Brodeur). Terreri may have again benefitted from weak opponents, but if he was outperforming by 1 - 1.5 shots per game in other places and then was equal with Brodeur in New Jersey, that is more evidence for a "Brodeur effect" of about a shot per game compared to the average guy.

In conclusion, there appears to be some evidence that Brodeur may prevent about a shot per game compared to average, based on comparing the results of the two backup goalies with the most games played, although the sample sizes are pretty small. Since many observers would consider Brodeur to be the best in the league at the skills that may contribute to preventing shots, I'd guess if we isolate his effect that could be considered pretty close to the maximum possible positive effect. From looking at the results of Dunham and others like him, it might be possible that the gap could be a little bigger on the other end (i.e. some goalies are substantially below average), but I think in general it is probably reasonable that goalie shots against results may vary within the range of about +/- 1 shot per game on average.

If this is the case, it raises a number of subsequent questions, such as: Are the shots being prevented/created more or less dangerous than average? Do goalies who are good at preventing shots tend to have higher save percentages? Can we isolate the specific skill that most contributes to this shot gap? Does shooter choice come into the picture at all, e.g. do shooters choose to shoot more or less often against certain goalies? Does the shot differential result primarily from even-strength play or on special teams? Is shot prevention consistent year to year? Can goalies substantially improve the number of shots they face per game? More research is certainly required in this area.

Adjusted Shutouts

In an earlier post, I calculated how well goalies did against league-adjusted save percentage. Some of the feedback from that post was that save percentage doesn't show everything. Fair enough, I will be looking at a couple of other traditional goalie statistics in a series of upcoming posts. With Martin Brodeur chasing down the career shutout record, I decided to focus first on shutout statistics. Anyone can look up the raw stats and see the all-time and active leaders, but as we have seen many times, dealing with unadjusted goalie stats is almost always a bad idea. As such, we need to add some context to the stats to be able to isolate who was actually outstanding and who was merely average or worse. I came up with two different ways of adjusting shutout totals. The first is adjusting shutouts for era, which is described in this post. The second, adjusting shutouts for the number of shots faced, will be described in a future post.

If you look at the career shutout ranking, you notice right away that shutouts are heavily impacted by era effects. The best periods for shutouts were the pre-World War II era, the 1950s, and the so-called "Dead Puck Era" (1997-2004). Goalies who played during these periods dominate the career list. The most difficult times to record shutouts were during World War II and the entire 1980s decade.

To take this into account, I have calculated an era-adjusted shutouts statistic, done by adjusting each goalie's number of shutouts based on the league average frequency of shutouts on a season-by-season basis. If shutouts were common around the league that season then each one received less of a weighting, but if shutouts were scarce goalies received bonus points for each one they recorded.

Here are the top 25 goalies of all-time in adjusted shutouts, from 1917-18 through 2007-08. I have included total shutouts, adjusted shutouts, as well as adjusted shutouts per 1000 games played to compare goalies with different career lengths.

Rank	Goalie	SO	Adj SO	AdjSO/1000 GP
1.	Clint Benedict	57	107.7	297.5
2.	Martin Brodeur	96	94.7	97.8
3.	Tony Esposito	76	94.0	106.1
4.	Ed Belfour	76	89.7	93.1
5.	Terry Sawchuk	103	87.8	90.4
6.	Patrick Roy	66	87.4	84.9
7.	Glenn Hall	84	84.9	93.7
8.	Dominik Hasek	81	83.8	114.0
9.	Jacques Plante	82	80.9	96.7
10.	Turk Broda	62	70.3	111.8
11.	Tiny Thompson	81	68.3	123.5
12.	Mike Liut	25	64.6	97.3
13.	George Hainsworth	94	64.5	138.7
14.	Harry Lumley	71	63.6	79.2
15.	Rogie Vachon	51	63.2	79.5
16.	Bernie Parent	54	62.3	102.5
17.	Tom Barrasso	38	61.9	79.7
18.	Alec Connell	81	61.3	147.0
19.	John Vanbiesbrouck	40	59.1	67.0
20.	Ed Giacomin	54	58.5	96.1
21.	Ken Dryden	46	55.9	140.8
22.	Lorne Chabot	71	55.1	133.7
23.	Curtis Joseph	51	50.9	55.2
24.	Andy Moog	28	49.6	69.6
25.	Frank Brimsek	40	49.5	96.3

Martin Brodeur does well by this metric, hanging onto the same #2 spot he holds in actual career shutouts. Tony Esposito, Ed Belfour and Patrick Roy move up the adjusted rankings while Terry Sawchuk and George Hainsworth drop down. Clint Benedict is the new leader, but his adjusted shutouts per game rate makes it obvious that his numbers are somewhat inflated. For most of his career he had the advantage of playing on the best team in a 4 team league, which makes it much easier to beat the average. Benedict has a good shutout record, but I'm not sure it is quite that good.

Probably the most interesting result on the list is Mike Liut, who had only 25 career shutouts but ends up 12th all-time in adjusted total. Over the course of Liut's career, his adjusted shutout per games mark has been virtually equivalent to Martin Brodeur's. Other goalies who suffered from playing in the wide-open 1980s but still posted very strong shutout totals include Tom Barrasso, John Vanbiesbrouck, and Andy Moog.

2 Major Records ≠ Destroying the Record Book

Who does journalist Damien Cox think is the most dominant goaltender of the modern era?

If you guessed that it was the goalie who is likely to break the career wins and shutouts records this season, and whose autobiography Cox co-wrote, well, you don't win a prize.

Cox goes on to refer to Brodeur's alleged "destruction of the NHL record books" and argues that 6 early regular season games prove Brodeur deserves the starting job at the 2010 Olympics. If I was someone like Cox who placed a heavy emphasis on team stats like wins and shutouts, I might be more concerned by Brodeur's 13-16 record in the Olympics and NHL playoffs combined since the lockout. I might even suggest that to be a more important sample than shutouts against Atlanta in October, but I'm not a mainstream journalist so what do I know?

My standard response to the "OMG, Brodeur is the Gretzky of goaltending and is going to smash every single goalie record eva!!1!!1!" school of thought continues to be this: How come Martin Brodeur doesn't hold a single record for GAA or save percentage, including career, single season, career playoff, or single playoff marks, as well as times leading the league in either?

With respect to that last one, times leading the league, something that I think most reasonable people would consider to be important evidence when considering who has been the most dominant, Brodeur has only ever led the league once in GAA and has never led the league in save percentage. Let's compare that to some of his contemporaries: Roy led 3 times in GAA and 4 times in save percentage, Hasek led twice in GAA and 6 times in save percentage, and even Belfour led twice in GAA and twice in save percentage. You can make a case for Brodeur as an all-time great, but then you should be using the words "longevity", "reliable", "durable", "valuable" and "team success". And unless you want to remind everyone of this guy, both in terms of his name and his play, you should probably stay away altogether from the word "dominant".

Goaltender Value

I just ran across this excellent post from this past summer by Five Hole Fanatics (hat tip: Battle of Alberta) on the topic of drafting goalies. I think it does a better job of explaining goalie value than anything I have written on the subject. In short, goaltending is a paradox: it is the most valuable position but most goaltenders have little individual value, and the reason is simply that the supply (decent pro goalies) exceeds the demand (available starting jobs in the league). A lot of teams think they need to sign their backstop to big bucks rather than risk losing him, but unless your goalie is one of the top handful of guys you are probably better off signing somebody else who will give you similar performance for a fraction of the price.

As Matt Fenwick at BoA writes: "A goalie who reliably stops pucks at a rate well above average is worth his weight in gold (proverbially; literally, he's worth much more). One who stops them at a below average rate, however, is worthless."

All the value in goaltending is at the top end. Subpar goaltending has no value at all, and there isn't much value in league average performance either. Just look the league's starting goalies who aren't earning huge paycheques, and then account for the fact that some teams have 1 or 2 quality backup goalies just waiting for their shot, and the fact that there are always guys dominating the AHL or KHL or Swedish or Finnish leagues but either haven't got a look in North America or are blocked by the guys above them in their NHL organizations. Studies of minor-league goalie performance (like this one at Oil Droppings) repeatedly imply that there are a number of minor league late-bloomers who go on to have some NHL success, and these guys can usually be had pretty cheaply. A lot of goalie selection and development is probably luck, and teams are somewhat limited in the guys they can develop because of the number of spots available in their systems. However, all an NHL team needs is two goaltenders, so a team with a good scouting and coaching should be able to find two guys capable of playing at league average or better. And once a team has two of them, then a guy playing 70 games at an average level is not actually helping his team much at all.

To make a long story short, and to give a general piece of advice concerning roster management, teams should just do what the Detroit Red Wings do. As GM and former goalie Ken Holland puts it (via Mc79hockey):

"My feeling is if you can get one of the five or six best goalies in the league you can spend the money. We can’t get into those guys, and the difference between the eighth goalie in the league and the 15th goalie, it’s a big difference in money. It’s not a big difference in performance."

If you can get a true difference-maker (i.e. top-5 goalie, which in my estimation appears to currently be Luongo, Giguere, Brodeur, Lundqvist, or Vokoun) then do it, otherwise save your money because you can probably get real close to replacing your performance at a bargain price.

Predicted 2008-09 Save Percentages

As requested by one of the commenters in response to my post-lockout goalie ranking, here are some predicted save percentage numbers for 2008-09, based on a weighting of each goalie's results over the last 3 seasons (the weights were 50% for 2007-08, 33% for 2006-07, and 17% for 2005-06). I have included both a predicted shot-quality neutral save percentage number, as well as an expected raw save percentage number based on predicted team shot quality (the team numbers were weighted and calculated in the same way as the individual save percentage numbers). There is no subjectivity in these results, I expect several of these goalies to do better or worse than predicted, but it gives a sense of how some of these guys are trending and how their results could be impacted by the teams around them. Here is the list (sorted by predicted save percentage):

Rank	Goalie	SQNSV%	Save %
1.	Backstrom	.916	.923
2.	Vokoun	.919	.919
3.	Giguere	.921	.918
4.	Brodeur	.913	.917
5.	Luongo	.913	.915
6.	Bryzgalov	.912	.915
7.	Thomas	.912	.914
8.	Lundqvist	.914	.914
9.	Huet	.917	.912
10.	Kiprusoff	.904	.911
11.	Lehtonen	.913	.911
12.	Fleury	.911	.911
13.	Garon	.911	.911
14.	Biron	.909	.909
15.	Leclaire	.902	.909
16.	Legace	.914	.908
17.	Miller	.905	.908
18.	Osgood	.907	.908
19.	Turco	.909	.906
20.	DiPietro	.906	.906
21.	Gerber	.907	.906
22.	Nabokov	.899	.905
23.	Roloson	.904	.904
24.	Theodore	.898	.903
25.	Budaj	.905	.902
26.	Mason	.908	.902
27.	Khabibulin	.907	.902
28.	Toskala	.901	.899
29.	Ward	.895	.896
30.	Kolzig	.893	.891

It's Early, Folks

Quite a few of the upper-tier goalies are off to relatively slow starts (Luongo, Kiprusoff, Turco, Giguere, Vokoun, among others). The league average save percentage is at .900, compared to last year's average of .909, so across the board goalies have been struggling.

Just a reminder, though, that there are plenty of games left to be played. Before anybody starts making any premature conclusions about goalies this season, keep in mind how the last 3 Vezina Trophy winners started their seasons, and beware the folly of trying to hand out any trophies before Hallowe'en:

Brodeur, 2007-08: 2-6-0, 3.27, .877
Brodeur, 2006-07: 4-4-0, 3.41, .889
Kiprusoff, 2005-06: 3-4-1, 2.99, .899

The Best Goalie Since the Lockout

In a recent post, knowledgeable goalie observer and frequent commenter Bruce did a ranking of the best goalies since the lockout. I think his list is pretty solid, although I do have a couple of issues with his method, which I am going to briefly point out, and then propose my own criteria and ranking.

First of all, I prefer not to use league rank, but rather the statistics themselves to generate a ranking since that better recognizes dominance. Otherwise somebody can destroy the field by .015 in save percentage and then finish 4th in wins with a couple less than the league leader, and end up ranked behind a guy who finishes 2nd in both categories. Secondly, I don't like the haphazard use of a bunch of related categories, because it ends up double-counting or triple-counting two major statistics: games played (which more or less determines all the counting stats, since obviously more playing time equals more wins, shutouts, saves, etc.), and save percentage (a goalie's ability to stop the puck impacts goals against, and contributes to shutouts and wins).

I think it is fair to include some sort of measure outside of save percentage, and to look at a combination of quantity and quality. I also would prefer to use team context-adjusted stats, rather than simply raw totals.

As such, I decided to use two different measures, both of which reward quality as well as quantity and adjust for the goalie's team context. The first is wins above teammates, calculated by finding the difference between a goalie's winning percentage and the total winning percentage of his teammates, and then multiplying by the number of decisions and dividing by 2 to get a win total. Since we expect that most of the goalie teammates are weaker backup goalies, I also included a .060 winning percentage adjustment for the backup numbers, based on overall league winning percentages for goalies who play backup minutes.

The second is shot quality neutral wins above average, calculated by taking a goalie's shot quality neutral save percentage, subtracting league average, multiplying the difference by the number of shots faced to get a goals above average figure, and dividing that number by 5 since an increase of 5 in goal differential roughly translates to one extra win in the standings based on the Pythagorean expected points theorem. My shot quality numbers were taken from Alan Ryder's Hockey Analytics site. It has been observed that certain rinks around the league vary in the way they record shot distances and types, so I used Ryder's road shot quality results for 2007 and 2008 to reduce rink bias, and I adjusted the 2006 numbers for a few teams that have unusual reporting tendencies. I didn't have the numbers for every individual goalie for each season so I used overall team numbers.

The result is that for each goalie I have two "wins above average" numbers, using two completely different methods. I think the shot-quality neutral method is a superior method, so I decided to give it 2/3 of the weighting, with 1/3 decided by record against backups. If you disagree with the weights or would look at the raw averages I have included all the relevant numbers, but here are the results based on my chosen scoring system:

Rank	Goalie	SQNSV%	Win%vsTm	SQN Wins	Wins over Tm	Score
1.	Roberto Luongo	.914	+.165	12.0	11.8	11.9
2.	J.S. Giguere	.919	+.143	12.9	7.0	10.9
3.	Martin Brodeur	.912	+.164	8.9	11.8	9.9
3.	Henrik Lundqvist	.916	+.131	11.5	6.7	9.9
5.	Miikka Kiprusoff	.909	+.196	5.0	15.0	8.3
6.	Tomas Vokoun	.920	-.041	16.5	-8.6	8.1
7.	Cristobal Huet	.920	+.051	11.5	-0.6	7.5
8.	Manny Legace	.915	+.134	8.1	5.6	7.3
9.	Niklas Backstrom	.916	+.171	5.8	5.1	5.6
10.	Tim Thomas	.912	+.087	6.9	2.1	5.3
11.	Rick DiPietro	.908	+.153	3.3	8.4	5.0
12.	Kari Lehtonen	.915	-.001	9.5	-4.4	4.9
13.	Dominik Hasek	.914	+.075	6.1	1.0	4.9
14.	Mathieu Garon	.910	+.136	5.1	4.5	4.3
15.	Martin Biron	.910	+.042	3.9	-1.1	2.2

There are 10 goalies that make both Bruce's list and mine. The top 6 are the same on both lists, just reordered, so I think it is pretty clear that those half-dozen guys were the league's best over the given time frame. I don't think there is a clear #1 goalie these days - I think Luongo is a good contender for the title, especially since he has really only had one strong season by his standards in the last three years yet still ranks at or near the top among goalies over that same period, but one could probably make a case for any of Luongo, Brodeur, Lundqvist or Giguere. My estimate is that the difference between any of them is less than one win per season.

Just as a postscript, Mathieu Garon does pretty well by this measure, so I might have been selling him a bit short in my earlier post on him. However, the main reason for his ranking was his success in 2007-08, which as I pointed out was in large part driven by outstanding results on the penalty kill, something which is probably not likely to be duplicated in 2008-09.

Adjusted vs. Unadjusted Stats

Which of these two lists do you think is better at ranking the best goalies since the lockout?

List A:
1. Kiprusoff
2. Backstrom
3. Luongo
4. Brodeur
5. DiPietro
6. Giguere
7. Garon
8. Toskala
9. Legace
10. Lundqvist

List B:
1. Hasek
2. Osgood
3. Backstrom
4. Giguere
5. Gerber
6. Toskala
7. Brodeur
7. Turco
9. Miller
10. Legace

List A is winning percentage compared to teammates. List B is straight-up winning percentage.

I appreciate the criticisms I get with my repeated use of backup statistics, and know that it is not always perfect to compare goalies to their teammates. Some guys get lucky with goal support or have strong/weak backups or play most of the games so their backups play a small sample size against weak opposition. Regardless, I still prefer winning percentage against teammates because I think it does a much better job than the unadjusted number. For every guy who has a very strong or a very weak backup impacting their numbers, there are a couple of Osgoods or Gerbers who are racking up wins because they play on strong teams. Since both stats are imperfect I'm going to pick the one that is less flawed, and to me that is clearly the backup-adjusted one.

There will be special situations that arise (say, for example, Dominik Hasek was your backup goalie), but in that case we just need to take notice and adjust for it, like we often do with hockey statistics (e.g. point totals of wingers playing with Crosby, plus/minus numbers on Detroit, save percentages on the Minnesota Wild, etc.). I don't think that is so dangerous.

This is a bit of a pre-emptive post, I have been working on an evaluation of post-lockout goalies in response to Bruce's study which is coming shortly, and as per usual I am partially relying on teammate-based statistics.

Pat Burns: A Goalie's Best Friend

What is the impact of a coach in hockey? That is one of the most difficult analytical questions in hockey, as indeed it is in all sports. Certainly there are many coaches who have reputations either built or ruined because of the talent of their teams. For the most part, I think most observers would agree that talent is the most important factor in determining which teams are successful. However, when it comes to evaluating goalie performance we are not only concerned with overall results (i.e. wins and losses), but also how well the goalie stops the puck and how many goals he allows. This a coach can certainly impact through establishing a team's style of play.

Jacques Lemaire is well-known as a defensive guru, the statistics of his goalies have been consistently excellent in both New Jersey and Minnesota (see this post for some relevant numbers). However, it is difficult to evaluate Lemaire because except for a short stint in Montreal he has worked for only two teams, New Jersey and Minnesota, and as the only coach the Wild has ever employed it is very difficult to estimate how well anyone else would have done in that position. We can guess based on how goalies did on other teams compared to how they did on a Lemaire-coached team, but we don't really know how good the other teams are and the sample sizes are mostly small. Ideally, we would prefer to look at a coach who coached on multiple teams, and evaluate how the team did before he arrived, during his tenure, and after he left. This would allow us to evaluate the overall shot prevention effect and shot quality/goaltending effect of his defensive system.

Pat Burns fits the criteria of a defensive coach with multiple stops, so I decided to focus on him. My method was to look at the two seasons before he was hired, and the two seasons after he was fired, and compare those results to how he did as coach. Since Burns was fired very early in the 2000-01 season, I decided to make the judgment call of counting that campaign as the first season after Burns, rather than his final season.

I looked at how Burns did in Montreal, Toronto, Boston and New Jersey. In the years prior to Burns showing up, his teams allowed 3.05 goals per game, 27.6 shots per game, and posted an .889 save percentage. In the years after Burns left, his teams allowed 2.78 goals per game, 28.7 shots per game, and his goalies stopped shots at a .903 rate. With Burns as coach, his teams alowed 2.57 goals per game, faced 27.5 shots per game and had their goalies post a .907 save percentage. If we look at shutouts, something that often influences Vezina voting, the goalies had .047 shutouts per 60 minutes before Burns, .074 shutouts per 60 with Burns, and .060 shutouts per 60 after Burns, so they were about 25-30% more likely to record a shutout with Burns behind the bench.

Just from the raw numbers, it looks pretty clear that Pat Burns has a positive effect on team defence. The problem here is obviously that most of the years prior to Burns' arrival came in the late 1980s or early 1990s, when scoring levels were higher, whereas his "after" years came when you would expect reduced scoring and improved goalie performance. We therefore need to normalize the numbers to league average. I took the average goals per game, shots against per game, and save percentages for Burns' career (3.08, 29.0, .897) and used them as adjustment factors to remove league effects for each individual season.

Prior: .517, 2.95, .893, 27.5 SA/60
Burns: .567, 2.57, .907, 27.6 SA/60
After: .542, 2.96, .899, 28.9 SA/60

With adjustments, the numbers are clear: Pat Burns helps his goaltenders out to a substantial degree, something in the neighbourhood of a .010 boost in save percentage and a .40 reduction in GAA.

The argument could be made that Burns had better goalies in net. Felix Potvin and Byron Dafoe both became starters in Burns' first year as coach for their respective teams, and they were better goalies than their predecessors. However, both were still around after Burns had left, and we see how the numbers fell back down to earth. I think the small discrepancy between the before and after results can be partly explained by goalie quality, but for the most part the same goalies were employed both during Burns' tenure and in the seasons after he left, so talent does not account for the performance improvement under Burns.

To try to avoid the goalie talent issue altogether, we can look at just the Montreal and New Jersey results, since in both cases Patrick Roy and Martin Brodeur were in net before, during, and after Pat Burns was coach (again the numbers are adjusted to league averages):

Prior: .608, 2.51, 26.0, .905
Burns: .610, 2.37, 26.2, .911
After: .611, 2.75, 28.2, .906

The results are similar, although less striking (which is partly because Montreal was already a strong team and had less room to improve than some of the other teams Burns worked for). Of course player talent is another consideration - for example, the New Jersey Devils would have been expected to suffer some dropoff in defensive play with the loss of Stevens and Niedermayer. However the persistence of the defensive improvement across four different teams makes it pretty likely that Burns' defensive philosophy was responsible for improved play.

Team defence is made up of shot prevention, shot quality against, and goaltending. Pat Burns' defensive effect seems to be primarily based on reducing opposing shot quality. His teams gave up almost exactly the same relative number of shots per game as they did previously, yet the goalies saw their numbers skyrocket under Burns' watch. Pat Burns is known for wearing out his welcome in a city after a few seasons, but I doubt he ever was that unpopular with the goalies. This is more evidence that goalies can receive a significant advantage by playing under the right coach in the right defensive system.

Don't Be Fooled by Cinderella Playoff Runs

Battle of California links to ESPN's expert predictions for this upcoming season, and points out that the Dallas Stars were picked by 5 out of 6 writers to win their division, and 2 of them predict them to win the Cup. The Philadelphia Flyers were picked by two writers to win their division, and one of them to win the Eastern Conference.

What do those two teams have in common? They were playoff overachievers, rising from the #5 seed in their conference to make it to the Conference Finals. "It's amazing how much public perception can change with a little playoff success," wrote jamestobrien at BoC, and he is certainly right.

But is there any justification for those picks? Does a Cinderella playoff run mean great things the next season? I decided to look at the issue in more detail, and the answer is, in nearly every case: No, it does not.

In the last 20 years, 23 teams have reached the Conference Finals despite finishing out of the top 4 in their Conference in points. Those 23 teams averaged 87 points in the year they made their playoff run. The next year, the average points for the same teams was just 81.

Twelve of them (over half) did not even qualify for the playoffs the next season. Nine of them had major declines the next season (drops of 10+ points) while only three of them improved by at least 10 points. The teams that improved were Anaheim in 2005-06 (who traded for Chris Pronger and won the 2007 Cup), Boston in 1991-92 (just had an off-year sandwiched between 100 point seasons), and Chicago in 1988-89 (which was the only instance of a team coming together during a playoff run and significantly improving the next year without major additions).

Of the teams that did manage to make the playoffs again, most of them went out in the first round. Only 4 of the teams returned to the Conference Finals the next year ('07 Ducks, '98 Sabres, '91 Oilers, '89 Hawks), only 2 of those teams made it to the Finals (Ducks and Sabres), and only the '07 Ducks won the Stanley Cup.

Unless they added an elite player or had a Hall of Famer in goal, every single lower seed that overachieved in the playoffs did not make it as far the next time around. Most of them didn't even make it back to the playoffs in the first place, and nearly all of those who did qualify simply failed to duplicate the postseason magic.

If we look closer at both Philadelphia and Dallas, I think they are likely to continue the same trend. The Flyers were outshot 2604-2359 last season. They held their heads above water because of a 10.4% shooting percentage, and a .915 save percentage from their goalies. They have some young talent on the team so they could have some internal improvement, but I highly doubt they match either their shooting percentage or save percentage next season, which will result in a decline in the standings.

The Stars had a +37 goal differential, which was the third-best mark in the entire NHL after Detroit and Montreal, which indicates they weren't an ordinary #5 seed. However, Dallas led the NHL in shooting percentage (10.8%), which is why they outscored their opponents despite barely outshooting them (2187-2128) and only getting league-average goaltending (.908). Even with some regression to the mean in terms of scoring the Stars should be competitive in a tough division, but I don't think they will repeat either their 2008 goal differential or 2008 playoff success.

Save Percentage vs. League Average

One of my frequent commenters did an analysis of save percentage vs. league average recently, and I wanted to revisit that metric for the goalies of the save percentage era. This statistic has been referred by some as "Goals Saved", so I'll use that name. For each goalie, I calculated a weighted league average save percentage, based on the league average and the number of shots they faced each season. This allows us to compare their actual save percentage to league average, and we can convert it into goals by multiplying by the number of shots faced. I did this for both the regular season and the playoffs to arrive at a combined total number of goals saved compared to average.

Remember that these are unadjusted save percentages, so there are hidden team effects for some of these goalies. That is why a straight 1-2-3-4-5-etc. ranking isn't really called for here, so I divided them up into tiers, based on a few very obvious breaks. Longevity is also important in this ranking, so someone like Kiprusoff, whose rate numbers are right up with the Tier 2 guys, still drops into Tier 3 because he hasn't faced that many shots.

I think if there is one thing you need to take away from these numbers, it is the gap between the two guys at the top and everyone else. A lot of people want to bring Brodeur, Belfour, Fuhr or others into that top group, but it is really not even close. Also note, however, that Roberto Luongo is only 29 years old, and has a good chance to surpass the 300+ goals saved mark and move into the top group before his career is over.

I also threw in a few well-known guys at the end for interest's sake, because tearing down reputations is what we like to do around here.

Tier 1 (All-Time Greats):

Name	Sv%	LgSv%	G Sv	Pl Sv%	Pl LgSv%	Pl G Sv	Tot G Sv
Patrick Roy	.910	.895	425.3	.918	.905	92.9	518.2
Dominik Hasek	.922	.903	384.2	.925	.914	36.1	420.3

Tier 2 (Top starting goalies):

Name	Sv%	LgSv%	G Sv	Pl Sv%	Pl LgSv%	Pl G Sv	Tot G Sv
John Vanbiesbrouck	.899	.891	197.4	.915	.903	24.9	222.3
Martin Brodeur	.913	.905	194.0	.919	.913	25.6	219.6
Curtis Joseph	.907	.900	184.9	.917	.911	23.6	208.5
Ed Belfour	.906	.900	148.5	.920	.909	49.2	197.7
Roberto Luongo	.919	.907	178.7	.941	.919	9.4	188.1

Tier 3 (Decent starters):

Name	Sv%	LgSv%	G Sv	Pl Sv%	Pl LgSv%	Pl G Sv	Tot G Sv
Kelly Hrudey	.893	.887	122.0	.891	.893	-5.2	116.8
Mike Richter	.904	.899	96.1	.909	.905	8.9	105.0
J.S. Giguere	.915	.907	90.5	.925	.918	10.3	100.8
Guy Hebert	.909	.902	100.8	.913	.918	-1.9	98.9
Andy Moog	.892	.886	114.9	.890	.897	-18.9	96.0
Tom Barrasso	.892	.888	88.4	.902	.900	7.1	95.5
Tomas Vokoun	.914	.907	89.6	.922	.921	0.4	90.0
Sean Burke	.902	.898	93.2	.888	.892	-4.2	89.0
Miikka Kiprusoff	.915	.906	74.7	.925	.919	8.7	83.4
Ron Hextall	.895	.890	81.8	.897	.897	0.0	81.8

Tier 4 (Overrated):

Chris Osgood	.907	.905	33.1	.914	.915	-2.5	30.6
Grant Fuhr	.887	.887	0.0	.900	.895	20.1	20.1
Mike Vernon	.890	.892	-39.8	.896	.900	-14.1	-53.9