Team Effects in the Original Six

Original Six goalies are very difficult to evaluate, because of the extreme team effects. The best talent was usually concentrated on a couple of the best teams, and the goalies on those teams not only had the advantage of playing behind a great team but they also never had to face their own team's elite goal scorers.

It is very difficult to estimate these effects, however, because most teams gave all or nearly all of their minutes to a single starting goalie. Glenn Hall, the most extreme example, played 503 games in a row at one point, so we don't even have a single game's worth of results for any other goalies on his team for a full 7 year stretch. That makes it impossible to use the method of comparing results to backup goalies. There was also less freedom of movement, so goalies didn't change teams as often.

These limitations mean that any method used is going to be less than ideal, but focusing on the goalies who changed teams and trying to estimate the team impact seemed like the best option. I decided to look at all the goalies that changed teams in the post-WWII Original Six era (1946-47 to 1966-67). The goalies were: Glenn Hall, Johnny Bower, Jacques Plante, Gump Worsley, Emile Francis, Hank Bassen, Frank Brimsek, Terry Sawchuk, Al Rollins, Jim Henry, Don Simmons, Bruce Gamble, Cesare Maniago and Harry Lumley.

I figured out the cumulative winning percentage and GAA for each goalie on all of the different teams they played for, and then tried to estimate each team's relative rank based on the differences.

Of course it is unrealistic to assume that every team had the same relative strength over a 20 year period. Montreal and Toronto were consistently good, while Boston and the New York Rangers were pretty mediocre, but Detroit was a powerhouse in the 1950s and terrible in the 1960s, while Chicago was exactly the reverse. Another problem is career arc - someone like Glenn Hall spent his entire prime in one place, and the only points of comparison we have of him somewhere else are either as a very young goalie or as an old one. The final problem was sample size: there were a few team pairings that didn't have a single goalie play on both of them. It was particularly difficult to evaluate Montreal goalies, since there were really only two goalies who played a lot of games in both Montreal as well as somewhere else, and both of them happened to also play for the Rangers (Plante and Worsley). There were also a few well-travelled goalies (like Harry Lumley, who played on 4 out of the 6 teams) that ended up having a larger effect on the sample.

However, despite these limitations, the numbers seemed to validate the method through a reasonable degree of consensus. For example, if you compare the goalies that played in both Boston and Toronto, they had a GAA in Toronto that was 0.42 better and a winning percentage .084 higher compared to Boston. If you used the Chicago results to verify this (by looking at the goalies who played in both Chicago and Boston, and comparing those results to the goalies who played in both Chicago and Toronto), the estimate was that the Leafs were 0.34 better in terms of GAA and .092 in winning percentage. Using the Detroit comparisons, it came out to 0.51 and .037. We can therefore ballpark the expected effect of getting traded from Boston to Toronto as being something like 0.40 - 0.50 in GAA and .070 - .090 in winning percentage.

I took averages from several of these comparisons, and came up with a relative set of rankings:

1. Montreal: 0.00 GAA, 0.000 win %
2. Toronto: 0.00 GAA, -0.040 win %
3. Detroit: +0.15 GAA, -0.025 win %
4. Boston: +0.40 GAA, -0.115 win %
5. Rangers: +0.70 GAA, -0.185 win %
6. Chicago: +1.00 GAA, -0.245 win %

If we compare these numbers to the actual results, we can both verify them and see which teams apparently had strong or weak goaltending:

1. Montreal: 2.36 GAA, .600 win %
2. Toronto: 2.51 GAA, .535 win % (+0.15 GAA, -0.065 win %)
3. Detroit: 2.54 GAA, .561 win % (+0.18 GAA, -0.039 win %)
4. Chicago: 3.04 GAA, .441 win % (+0.68 GAA, -0.159 win %)
5. Rangers: 3.08 GAA, .430 win % (+0.72 GAA, -0.170 win %)
6. Boston: 3.09 GAA, .432 win % (+0.73 GAA, -0.168 win %)

The total results confirm that Montreal, Toronto and Detroit were the three front-runners, with similar GAA totals. Montreal likely did have somewhat better goaltending than the Leafs or Wings, but the main reason the Canadiens had more team success was probably not goaltending but superior offensive play. The model predicts the Rangers quite well relative to the Canadiens, which suggests that despite often being a bottom-feeding team the Rangers got decent performances from the goalie position. Boston, on the other hand, appears to have had weak play in net, since they allowed the most goals of any team but apparently had a better defensive environment than either New York or Chicago.

Chicago's comparative results are exaggeratedly poor because, as previously noted, Glenn Hall was the only guy in their net during the early to mid-1960s. If we take Chicago's results prior to Hall's arrival in 1957-58, the Blackhawks' cumulative goalie stats were 3.46, .345, which means they were 1.10 and .255 worse than the Canadiens, numbers that are very close to my team effect estimate. There seemed to be more goaltender movement in the post-war years than in the early 1960s, which means that these estimates are probably more representative of the league competitive balance in the late 1940s and 1950s, a period where Chicago was consistently the worst team in the league.

These are just ballpark estimates to keep in mind when looking at older goalie statistics. This is also evidence of the dependence of goaltending statistics on team play, since the variance of team effects is much larger than the variance of goaltending play. The model predicts the results reasonably well for 4 out of the 6 teams, as well as for Chicago up until the 1960s. This would most likely not have been the case if there were drastic differences in goalie quality across the league. The difference between, say, a Gump Worsley and a Terry Sawchuk was certainly much, much smaller than the difference between the Red Wings and the Rangers. After looking at these numbers, I'm not sure there was much difference at all between many of the longtime starting goalies of the time period.

This is why evaluating goalies based on on wins and shutouts from that era is pretty pointless - instead of finding the best goalies, you will merely end up finding the goalies that spent the most time playing on the Canadiens, Maple Leafs, or Red Wings.

From High to Low: What Happens when Goalies Get Traded?

To continue the discussion on shots against per game and their effects on goalie play, I went through Hockey Reference's data banks to look at goalies who had been traded mid-season to see what the effects were on their statistics with their new team. I took every goalie in the save percentage era (since 1983-84) who was traded during the season and had played at least 500 minutes with both teams.

Here is the breakdown of how they did, broken down into three roughly equal groups by shot differential between the two teams:

	Low	Shot	Team	High	Shot	Team
Diff	Win%	GAA	Sv%	Win%	GAA	Sv%
0-2	.471	2.78	.902	.456	3.02	.896
2-5	.542	2.78	.900	.459	3.27	.893
5+	.510	2.93	.887	.375	3.63	.890

What does this tell us? The GAA differential increased as the shot differential did, which is to be expected since more shots against means more chances to allow goals. There was also a distinct relationship in terms of winning percentage for goalies going from low shot to high shot teams. Clearly it is much easier to win games on teams that are good at preventing shots against.

There was not, however, a direct relationship between save percentage and shots. The goalies with the largest shot differentials actually had similar save percentages on both teams, whereas goalies with a smaller differential did slightly better on the lower shot team. This is evidence that shot quality is not directly related to shots against, but varies on a team-by-team basis.

It is obvious that goalie results are very team-dependent. Save percentage is not perfect, but it is by far the best measuring stick for goalies, since the variance in save percentage was much lower than the variance in GAA or especially winning percentage. If we take the sample of goalies who faced a difference of 7 shots against or more after being traded, we can see this quite clearly:

Low Shot Team: .886, 2.94, .531 win%
High Shot Team: .886, 3.98, .320 win%

The exact same goalies stopped pucks at exactly the same average rate both before and after the trade, but they allowed over one goal more per game with one of the teams and had a difference in winning percentage of .211, which would be the equivalent of 35 points in the standings over an 82 game schedule. The moral of the story is that using straight GAA or winning percentage statistics to rate goalies is not a good idea. If you have to use unadjusted stats then use save percentage, but all goalie stats are dependent on the rest of the team.

There was one result that was a bit of a surprise, and that was the lack of variance in shutout results. For the group as a whole, the shutout rate on the lower shot teams was .048 per 60 minutes, and for the higher shot group it was .041. That is a difference of about half a shutout per season for a starter with a typical 60 game workload. That was not too surprising, but the underlying distribution was a little unusual. For goalies with either a low shot differential (0-2 shots difference) or a high shot differential (5+) there was very little difference in shutout rate. The goalies with the large differentials were actually very slightly more likely to record a shutout on the higher shot team. Only in the 2-5 shots per game range was there a large discrepancy, and it was in the expected direction (.048 on lower shot teams, .032 on higher). Sample size is an issue here, however, because shutouts are so infrequent. One or two goalies who managed to get lucky for a half-season in terms of shutouts could easily have skewed the overall totals.

My main beef with shutouts is that they are an arbitrary stat. Why do shutouts get all the hype, but one-goal games get no credit even though both of them nearly always result in a win? However, I am starting to think that there may be some useful information in shutouts. Certainly shutouts are still team-dependent to a large degree, but they are a dominance indicator. Great goalies on bad teams (e.g. Luongo in Florida) will still usually post a relatively high number of shutouts, especially compared to the other goalies around them. My expected shutouts ranking, which was designed to remove some of the team factors, seemed to pass the common sense test for the most part. I'd still prefer another more inclusive metric to shutouts (for example, the number of times a goalie allowed 2 goals or fewer, given that team winning percentages are very high across the board when teams gives up 2 or less, and also since 1 and 2 goal games are more frequent than shutouts they would likely be less subject to variance and more consistent from year to year), but that would require a lot of additional work to compile.

Sundin and Leadership

Is he the kind of of player, like Mark Messier, who can lead the Canucks over the hump?

Ha.

The only connection between Messier and Sundin is the "leadership" award Messier bestowed on Sundin during last season's playoffs. Talk about shams.

Scott Burnside of ESPN, proving that writers are just as incapable of separating team performance from individual star players as they are from goalies.

Burnside sounds like he buys into the Messier leadership mythology, and trashes Sundin for being a selfish choker. What is leadership, though, really? Carrying your team, correct? Bearing the scoring burden, driving results while on the ice? I'd guess most people would talk about inspiring your teammates and making great locker room speeches and all that as well, but I think most people would agree the best place to be a leader is on the ice, and when it comes down to it they would take the guy out there dominating the game over a mediocre player who happens to be a great motivational speaker and everyone's best friend.

I'm going to use Messier's results from his first stint as a New York Ranger for comparison purposes, since that was when his legend as the "Greatest leader in sports" really grew. Look at the playoff results, and Sundin was carrying the Leafs just as much as Messier was carrying the Rangers. Probably even more so. Sundin scored 15% of Toronto's playoff goals, Messier just 12% of the New York's. Messier had an edge in points, having a hand in 34% of the Rangers' goals compared to 32% for Sundin, but Sundin missed 17 games due to injury while Messier missed just 2. Messier's overall PPG rate was higher (1.14 to 0.92), but he did it in a higher-scoring era with better teammates and probably more ice time.

If we look at plus/minus, Sundin destroys Messier. Sundin was +7 in the playoffs for Toronto, when the team as a whole was -88. Messier was -9 on Ranger teams that combined for a total of -1. If we figure that an elite forward plays 1/3 of the game, we would expect their plus/minus total to account for roughly one-third of their teams total. That total includes the minutes played by Sundin and Messier, so we can do a quick and dirty estimate of the "off-ice" plus/minus for each player by multiplying their plus/minus by 5 (since there are usually 5 players on the ice for an even-strength goal) and subtracting that from the overall team total, then dividing by 2/3 to factor out their ice time. I thereby estimate that the Leafs were around -30 goals at even-strength without Mats Sundin on the ice, and +7 with him on it, and that the Rangers were around +4 goals without Mark Messier on the ice and -9 with him on it. That is a very quick and dirty method, the assumptions aren't really completely correct and it doesn't account for the special teams factors that are one of the biggest problems with plus/minus, but it appears that the Moose isn't even in the ballpark compared to Mats.

Based on the results, then, someone like Burnside would have to argue that Messier inspired everyone else around him to be much better, while Sundin just couldn't get the rest of his team going. Maybe that's true, I don't know (isn't that the coach's job?), but when it came down to leadership by example Sundin looks pretty elite.

Leadership is supposed to mean, "Player who has great intangibles, carried his team and helped his teammates", but I think a lot of times it ends up really meaning, "Player who had a lot of team success." That is even more likely to be the case in the opposite scenario, i.e. for a player like Sundin who did not have a lot of team success. Apparently it is pretty much out of the question to call them a great leader, no matter how well they actually perform, just because their teammates weren't very good.

I don't think the Canucks are up there with the Wings or Sharks in terms of Cup favourites, but they were my pick to win the division before they got Sundin and they could be a dangerous team in the postseason. Is there any reason to expect that someone like Mats Sundin would not succeed if they were placed in a playoff pressure situation? Of course not. As the above numbers show, he did a pretty good job carrying the Leafs through a few playoff series. His record in international best-on-best games is 18 goals, 21 assists, 39 points in 30 games, which prorated to an NHL season would be 106 points. Every season in every team sport star players without a history of team success pull a Peyton Manning and finally win a championship, yet all the non-thinkers who were writing about their lack of leadership and lack of winning attitude and poor clutch play right up until the champagne celebration never seem to learn their lesson.

Tim Thomas: How Long Will the Fairy Tale Last?

Tim Thomas has a .938 save percentage through his first 18 games. I am not one of those people who likes to trash Thomas and call him a fluke, because I think goaltending is a results business and style is more or less irrelevant (unless a goalie has a single, exploitable weakness or something like that). Breaking down a prospect's style will give you clues as to how likely they are to make it at the NHL level, but in my view a record of success at the pro level over multiple seasons trumps any negative scouting reports. Thomas may be a bit clumsy and awkward, but if he stops the puck then he stops the puck, and a career .916 save percentage suggests that he is no mediocre netminder. He also passes my standard litmus test, which is results compared to backups (2005-06 to present):

Thomas: 82-64-23, 2.70, .916
Backups: 44-48-15, 3.09, .897

Having said that, I don't think Thomas is a reincarnation of Dominik Hasek either, so how sustainable is that .938?

First off all, let's look at the backup goalie. Manny Fernandez is 9-2-1, 2.08, .924. I doubt Manny Fernandez is much better than average, so that suggests Boston has been very good in their own end this season. If we assume that Fernandez is really an average goalie (say, .910 or so) disguised by Boston's stalwart defence, then we can estimate the Bruins' shot quality against at about 15% better than average. Adjust Thomas' career average based on those results and you get .929.

Based on that estimate, it does seem reasonable that a strong year by Thomas could keep him up near .930 for the season. However, there are some warning signs:

1. The Bruins are leading a pretty charmed life right now, with all the percentages in their favour (11.5% shooting percentage, .933 team save percentage). Unfortunately, history has taught us that regression to the mean appears to be an unavoidable fact of life for hockey teams, and the smart money is that the Bruins will not continue either their scoring or save rates for too much longer.

2. Tim Thomas' penalty killing save percentage so far is .938. Needless to say that is leading the league. Thomas' ES save percentage is also very good (.941), but nobody can stop pucks at that rate forever when down a man. League average on the PK is usually around .870, and Thomas' PK save percentages for the last two seasons were .871 and .846. Maybe Boston has a particularly strong penalty kill this year, but I bet Thomas does no better than .900 on the PK, if even that, over the rest of the season, which would take some of the air out of his overall save percentage.

3. I don't think that 15% shot quality estimate accurately reflects Boston's defensive play. Julien is a good defensive coach, and the Bruins play a good team system, but that kind of shot quality would be elite. Fernandez has probably been either outperforming or lucky this season as well. Something like 5-10% better than average would probably be a more realistic estimate, which would suggest that Thomas should be closer to last year's .921 mark than somewhere above .930.

4. The Bruins are playing a soft schedule (Sagarin has them ranked 25th in schedule strength) in a weak division in the weaker conference. That's probably not likely to change much, but I just felt like pointing that out.

5. Thomas is facing 31.5 shots per game, compared to Fernandez' 27.3. I don't know if that is a statistical fluke or represents a real difference in on-ice play. Thomas has faced about 2 shots per game more than his backup goalies since the lockout, so he seems to be one of those types who have to make an extra save or two every game compared to average. If those shots represent more dangerous than normal chances (e.g. rebounds, turnovers, etc.) then he will be less likely to sustain a high save rate. For what it's worth, Hockey Numbers has Thomas facing more difficult shots than Fernandez so far.

Tim Thomas is not a fluke, but his season so far appears to be. I think it is probably fair to say Thomas is a good NHL starting goalie, but he looks like he is playing way over his head so far. I don't see his numbers going anywhere but down over the rest of the season.

Even if we expect him to cool off in the coming months, Tim Thomas does have that .938 and 1.96 GAA "in the bank" already, so to speak. If he regresses considerably over the next 52 games, he will still likely end up with some very good numbers this season. Let's say he plays 60% of Boston's minutes the rest of the way, faces the same number of shots per game (31.5), and stops 91.5% of them, which is probably pretty achievable given his career numbers and the team around him. His season numbers would then be 2.40, .924, which when combined with 30-35 wins and the injuries to Brodeur and Luongo could still put Thomas in contention for the postseason awards.

Thomas is an interesting case of a well-travelled guy who apparently had the light come on at the age of 29. In 2002-03 he was a journeyman AHLer (35 GP, .906), although he did show enough to get called up to play in 4 NHL games. Then out of nowhere came a huge AHL season in 2003-04 (.941 save percentage), a big year in Finland during the lockout (1.58 GAA), and then a season split between the AHL (.923) and NHL (.917) that finally solidified him at the NHL level. The late bloomer phenomenon is well-documented, and anybody who can find out just what makes these goalies suddenly put it all together will be able to command a very large salary from some grateful NHL organization.

Grant Fuhr and Clutch Play

The Hockey Summary Project is up and running, and it is a very interesting resource for hockey history, with box scores from many historical regular season and playoff games.

One thing I have wanted to check for quite some time now is how Grant Fuhr played in the third period of hockey games in Edmonton, to test whether his reputation of being "the guy who will never let in the next goal" matches reality (see here for round 1 of the debate). Unfortunately, not all the playoff seasons are posted yet (they are still missing 1988-1993), but we have 1983-1987 to at least begin the analysis.

For every Edmonton Oilers playoff game from 1983-1987, I took the goals for/against and shots/for against from the first two periods, and compared it to the goals and shots during the third period and OT. Here are the results:

Andy Moog: 2.88, .896 in first two periods, 3.38, .875 in 3rd & OT
Grant Fuhr: 3.00, .901 in first two periods, 2.74, .899 in 3rd & OT

So Fuhr did indeed allow fewer goals in the third period, but his save percentage also went down slightly. That suggests that the Edmonton Oilers, even in their most high-flying years, did actually play some defence when the game situation required it.

I broke down Edmonton's third period numbers with Fuhr in net, based on the score after 2 periods, to see how much of an effect the game score had on the results.

Score	P 1 & 2				Per 3
	SF	SA	SH%	SV%	SF	SA	SH%	SV%
Lead by 3+	39.8	29.7	20.1%	.944	31.0	26.3	20.4%	.861
Lead by 2	35.1	32.1	16.5%	.913	31.7	28.3	12.2%	.909
Lead by 1	34.4	29.1	11.0%	.921	33.0	28.2	15.0%	.919
Tie Game	30.5	28.1	9.8%	.893	35.1	26.1	14.6%	.926
Trail by 1	30.0	35.4	8.1%	.889	39.4	29.3	7.6%	.872
Trail by 2	31.5	35.5	6.3%	.859	31.0	26.0	12.9%	.846

In my view, it is incorrect to attribute the differences to "clutch play", as I think they are more reflective of the shot quality of both shots taken and shots allowed. The results when leading, tied and trailing were similar when Andy Moog was in net, suggesting that the numbers are being primarily driven by the rest of the team, rather than the goalie. When the game was tied or the Oilers were ahead, Fuhr had a save percentage of .909 in the third period. When the Oilers were behind, Fuhr had a save percentage of .862 in the third period. Part of this is a reflection of strength of opposition: the Winnipeg Jets were almost never ahead of the Oilers in the third period, for example. However, the first two periods were played against the same opposition, and yet there is a clear difference in the third period for games when the Oilers were tied or trailing by one goal.

The shots for and against numbers support these observations. When the Oilers were well ahead, they shut down their offence a bit in the third period. When the Oilers were behind, they far outshot the opposition. This is probably the combination of the opposing team taking fewer risks (shots against were down when the Oilers were behind) and the Oilers taking more discretionary shots (Edmonton averaged almost 40 shots per 60 minutes of play in third periods they entered trailing by a goal, yet their shooting percentage was just 8.2%). How about this for a surprising fact: the highest-scoring team of all-time was just 4-15 over the sample period in games which they trailed entering the third period.

Having said all that, there does appear to be some evidence that Grant Fuhr did well in important situations. His save percentage was highest when the game was tied. That may be somewhat expected, since that is when teams would generally play the most cautious, but that is still an impressive save percentage for 1980s hockey. His save percentage also was quite low when the Oilers had the game well in hand, just .861 with Edmonton up by 3 or more goals, which supports the perception that Fuhr let in softies when it didn't matter. He wasn't quite so good when the Oilers were behind, but this is likely when the shot quality against was at its highest, so he probably has a bit of an excuse for that. I still don't think there is any reason to call Grant Fuhr a great goalie or one of the best ever, but it looks like he did help Edmonton in the playoffs in the 1980s. Still, from the above table it looks pretty clear that Edmonton's shooting percentage and outshooting results were the main drivers of their success, rather than goaltending.

This is just a quick study, a better one would break down the results in more detail, looking at Fuhr's actual performance with Edmonton up a goal in the third period, not just how he did in third periods that Edmonton entered up by one. With their propensity to score quickly, there were some third periods that the Oilers blew open in the first few minutes, despite entering tied or holding a narrow lead.

One interesting bit of trivia: Between 1983 and 1987, the Edmonton Oilers apparently won 6 overtime games on the very first shot of OT. I'm not sure if there are mistakes in the box scores, but all of those games ended in less than a minute and a half so it is certainly possible. There was also one game when Edmonton lost on the first shot of OT. Looks like it wasn't a good idea to try to grab a hot dog before overtime started in Edmonton in the 1980s.

The Replacements

In the last couple of weeks, I've noticed a distinct lack of gleeful comments posted by Bruce giving New Jersey's goalie stats since Martin Brodeur went down. I decided to look into it myself to see just how New Jersey's replacement goalies are holding up.

Scott Clemmensen: 7-3-0, 2.19, .926
Kevin Weekes: 2-3-0, 2.86, .903
Backups Combined: 9-6-0, 2.42, .918

Martin Brodeur: 6-2-2, 2.16, .916

So far I'd say they are doing pretty well. It is a very small sample size to be sure, and it is certainly much too early to discount the possibility that Scott Clemmensen may just be playing way over his head for a couple of weeks. However, I am just as interested in the number and type of shots Brodeur's replacements are facing, rather than simply whether they are stopping them or not. I cruised over to Hockey Numbers to see what his shot-quality calculations are telling us about New Jersey's goalies. SQN% is shot-quality neutral save percentage, and SQI is shot quality index (1.00 is average, below 1.00 means a team allows easier than average shots).

Brodeur: .914 SQN%, 0.98 SQI, 25.7 SA/60
Clemmensen: .918 SQN%, 0.90 SQI, 29.5 SA/60
Weekes: .892 SQN%, 0.90 SQI, 29.6 SA/60

Clemmensen and Weekes are very similar in their underlying numbers, facing almost identical shot quality and quantity. The difference is that Clemmensen is making more saves.

What really stands out, however, is that Brodeur has faced 4 fewer shots per game than his backups goalies have. Is this finally evidence of his soft goaltending skills as a third defenceman out on the ice, or are there other factors at play? In hockey, there are pretty much always other factors at play. If we look at the shot quality numbers, Clemmsen and Weekes have faced shots that were estimated as being 10% easier than average. Brodeur's shots were only 2% easier than average. This means that while New Jersey has allowed more shots without Brodeur in net, the extra shots faced have been apparently much easier to stop. If the primary reason for the difference was Brodeur's impact on puck possession, I'm not sure that we would expect to see a difference in shot quality.

Obviously we need to track shots and shot quality over a larger number of games to see if these differences are fluke or reality (and there are a number of factors that make shot quality less than perfectly reliable, such as reporting bias, failure to consider shot angle, etc.), but there are two possible explanations that do come to mind that could explain these results (maybe New Jersey fans can weigh in if either of them seems reasonable). The numbers suggest that either opposing teams are shooting from everywhere to test Clemmensen and Weekes, or New Jersey has changed its defensive style of play to protect their goalies which has resulted in allowing a higher number of lower quality shots.

We can try to quantify the goal prevention effect from the difference in shots allowed, by estimating the expected goals against by an average goalie facing Brodeur's shot distribution and then comparing that to his backups. The league average so far is .907, so let's go with that as our baseline number. We can adjust that for the shot quality for each goalie, and then multiply that by number of shots actually faced to get an expected GAA.

Here are the results:
Brodeur: 2.33
Clemmensen: 2.48
Weekes: 2.47

Through an expected goals approach, Brodeur's 4 fewer shots per game translate into a GAA effect of -0.15 goals per game. If we want to try to express that gap in terms of save percentage, it would be the equivalent of +.005 in save percentage for a goalie with a league average save percentage facing league average shots. We don't know at this point whether Clemmensen and Weekes are better or worse than average in terms of shot prevention, so it isn't necessarily correct to attribute the entire gap to Brodeur.

I don't know the typical starter/backup split in terms of shot quality, especially in this type of situation where lightly regarded backups replace an All-Star. However, there is a very similar situation going on in Vancouver, so I'll bring that in as a point of comparison. Here is how Luongo has done compared to his replacements by all the same metrics as above (SQN% = shot-quality neutral save percentage, SQI=shot quality index, expGAA = expected GAA for a league average goalie facing the same shots):

Luongo: 2.17, .928, .930 SQN%, 30.2 SA, 1.03 SQI, 2.89 expGAA
Sanford: 2.85, .905, .897 SQN%, 30.0 SA, 0.92 SQI, 2.57 expGAA
Schneider: 2.80, .896, .878 SQN%, 26.9 SA, 0.85 SQI, 2.13 expGAA

Backups: 2.83, .902, .891 SQN%, 28.9 SA, 0.90 SQI, 2.41 expGAA

The combined shot quantity and quality for Luongo's backups is very similar to Brodeur's (same shot quality against and a difference of less than one shot against per game). Just like Brodeur, Luongo has apparently faced more difficult shots, but Luongo has also faced more of them (1.3 extra shots per game than his backups). Luongo's expected GAA is actually 0.48 goals above that of his backups because of these factors, yet his actual GAA is 0.66 lower. According to the numbers, Vancouver has been hurt a lot more by goalie injuries than New Jersey has.

What about evidence that Brodeur affects his teammates, or that his puckhandling contributes to his team's offence? I went to Time on Ice to check out the possession statistics while each goalie was in the game (using Behind the Net's numbers to estimate minutes played at 5 on 5):

Brodeur: +8.4 Shot Diff/60, +7.9 Corsi/60, +12.1 Fenwick/60
Backups: +1.8 Shot Diff/60, +4.9 Corsi/60, +5.4 Fenwick/60

Luongo: -2.6 Shot Diff/60, -3.4 Corsi/60, -4.7 Fenwick/60
Backups: -2.9 Shot Diff/60, -4.2 Corsi/60, -7.5 Fenwick/60

Both goalies have better outshooting results than their backups do. New Jersey is obviously a better outshooting team than Vancouver, but Brodeur outperforms his backups by a larger margin than Luongo. Vancouver blocks a similar ratio of shots no matter who their goalie is, while New Jersey's shot block to total shot attempts against rate is 3% higher with Clemmensen or Weekes in net than with Brodeur.

One factor at work with the possession stats could be that Luongo has been in the lead much more often than his backup goalies. That may have led to more shots against, since there is some evidence to suggest that trailing teams are likely to shoot more. Vancouver tends to reduce their offensive pressure when they are leading, focusing instead on preserving the lead. New Jersey likely usually uses a similar strategy, but New Jersey's backups have a better record than Vancouver's so this may not have affected them as much.

It's pretty early to conclude anything, but there are a few observations to be made. New Jersey is a good team, and both the shot quality numbers and Scott Clemmensen's stats suggest they are still a very good defensive team. Playing in New Jersey helps a goalie's statistics, of that there is little doubt. However, the interesting numbers are the ones that relate to Martin Brodeur's impact on his team's play beyond stopping pucks (e.g. shots against and puck possession stats), and that is certainly something to follow along with as the season goes on.

Durability Matters...Except When It Doesn't

Wendel Clark: 793 games played
Wendel Clark's teams: 1,186 games played
Participation rate: 66.9%

Dominik Hasek, from the time he became a starting goalie in 1993-94 to the end of his career: 682 games played
Dominik Hasek's teams over the same time period (excluding retirement season in 2002-03): 1034 games played
Participation rate: 66.0%

Ask your average Leaf fan about those two players, and one of them will be the biggest injury risk who ever played hockey while the other descended directly from the heavens via Kelvington, Saskatchewan.

NHL vs. CHL

When doing a bit of research for my last post about Canadian world junior team goalies, I took a quick look into CHL stats across the three major junior hockey leagues. For me, it was a bit of a reminder that the NHL results may not necessarily be representative of all levels of hockey.

Take, for example, shots against vs. save percentage, a common theme in this space. There is a vocal group that repeatedly maintains that those two things are positively related, and that as shots against go up save percentage necessarily also goes up. This is supposedly either because shots against the run of play are more dangerous, or because it is easier for a goalie to concentrate when he faces more shots. There have been some seasons at the NHL level where the data seem to suggest this is a possible relationship, including this season so far where James Mirtle calculated a 0.49 correlation coefficient between shots against per game and save percentage.

However, if these arguments are correct they should be generally true for all levels of hockey, including junior. Here are the correlation coefficients between shots against per game and save percentage for the past 3 seasons:

Ontario Hockey League:
2005-06: -.269
2006-07: +.078
2007-08: -.278

Western Hockey League:
2005-06: -.219
2006-07: -.347
2007-08: -.214

Quebec Major Junior Hockey League:
2005-06: -.112
2006-07: -.180
2007-08: +.266

Not much support for the "more chances = better save %" theory. The Western Hockey League is the league that tends to produce the best defensive teams with the strictest defensive systems, and that league has the strongest relationship between save percentage and shots against. The numbers indicate that playing on a strong defensive team in junior hockey probably helps your save percentage. This suggests that goalie prospects who had a big advantage in juniors because of their defensive teammates (like Justin Pogge, Leland Irving, Jeff Glass and Tyson Sexsmith) might turn out more like Kelly Guard than Carey Price.

Tyler Dellow posted recently on save percentage with certain players on the ice, and concluded that it appears to be mostly driven by randomness. Players with unusually high or low save percentages will regress to the mean as the season goes on. His conclusions make sense for the NHL level, but I would like to see similar numbers for junior hockey players, to see if the dominant players at that level had an effect on their own team's save percentage. It would also be interesting to see the numbers from an era of the NHL with a different level of competitive balance (the 1970s, for instance). In the 1970s, the save percentage-to-shots-against relationship was very similar to the junior results above, i.e. a weak negative correlation. This relationship inspired hockey analysts Klein and Reif to come up with the goalie perseverance rating, a rating that penalized goalies on low-shot teams and gave bonus points to goalies who faced more rubber, in their influential Hockey Compendium (published in 1986). Their weighting system did not stand the test of time, however, as hockey's competitive landscape changed (see the comments to this post for a discussion of how Klein and Reif's results no longer hold in the current NHL).

I think that bad hockey teams generally give up high shot quality against, and good hockey teams generally give up low shot quality against. It is only in a league with a good competitive balance (such as the NHL) that other factors come into play and affect the result, for example because the outshooting team takes longer shots or more of their shots come from their third- or fourth-line players, or possibly the players are playing a specific offensive or defensive system, or some other similar reason(s).

My overall point is that I'm not sure whether all of the statistics-based conclusions that are being made these days represent essential hockey truths, or whether we are merely collecting evidence of the high degree of parity in today's NHL. It might be wise now and then to test out conclusions with results from a different league or era to see if the findings still hold. Unfortunately, the lack of data available does not always make this possible.

Fire the World Junior Goalie Scouts

I simply cannot believe Tyson Sexsmith was invited to Team Canada's world juniors selection camp. I can't understand how a goalie who is so obviously a product of his team has the potential to be representing the best hockey country in the world in an international tournament. Here are three stat lines: Sexsmith over the last 3 seasons, his backup goalies over the last 3 seasons, and the numbers of the Vancouver Giants starting goalie who preceded Sexsmith from 2005-06. See if you can spot the resemblance:

1.89 GAA, .911 save %, .742 win %
1.89 GAA, .910 save %, .731 win %
1.90 GAA, .912 save %, .713 win %

Team Canada often seems to do this for these tournaments, picking goalies off the best defensive teams at the expense of guys that are better but have inferior teammates. Chet Pickard and Dustin Tokarski are by all accounts pretty decent goaltenders, but doing the same exercise indicates that they also have the advantage of playing on great defensive teams. Here is the combined stat line this season for the backup goalies of the 3 WHL goalies invited to Canada's selection camp:

18 GP, 14-0-2, 1.63, .933, 3 SO

Let's just say I don't particularly trust our nation's junior scouts in terms of separating the goalie from the team.

But worst of all, Sexsmith's invitation to the camp was sent at the expense of the best junior goalie in Canada. For some reason Belleville's Mike Murphy never got a call. Let's look at how Murphy compares to his goalie teammates over the last two years, as well as compared to his predecessor in the net in Belleville:

Murphy: 2.15 GAA, .935 save %, 53-10-8, 5 SO
Backups: 3.35 GAA, .895 save %, 15-11-1, 2 SO
Previous: 3.01 GAA, .919 save %, 27-17-3, 3 SO

It is pretty hard to look at those numbers and not conclude that this guy is a difference-maker. Murphy led the OHL in save percentage last year (.929) and is leading it again this year by a wide margin (.944). Despite facing over 35 shots per game, Murphy is still leading the OHL in GAA, which is remarkable. Meanwhile, Tyson Sexsmith faces less than 22 shots per game and only ranks 3rd in the WHL in GAA (and just 15th in save percentage).

Apparently last year Sexsmith was rated the most overrated player in the WHL by a wide margin, which makes sense. However, I guess none of those people were involved in picking this year's squad. Hopefully Canada's coaching staff decides to go with some combination of Tokarski, Pickard or fourth option Jake Allen on the basis that they are far more deserving of the honour than Sexsmith.

The Luck of the Draw

I was looking at some historical results, and found something interesting about the career shutout record held by Terry Sawchuk. The main reason that Sawchuk holds the record (and that he held the career wins record up until Patrick Roy broke it) appears to be that Jack Adams was a better goalie evaluator than Dick Irvin.

Terry Sawchuk and Jacques Plante were both born in 1929. Sawchuk became a Red Wings prospect, while Plante's rights were held by Montreal. Sawchuk had Harry Lumley ahead of him in the organization, but Jack Adams realized that both of them were good goalies and that he could maximize value by trading Lumley, who was coming off of a Stanley Cup winning season. As a result, Terry Sawchuk found himself starting in the NHL at the tender age of 21 for the best team in the league.

Jacques Plante, on the other hand, found himself stuck behind Gerry McNeil, a middling goaltender who rose to the starting job because Bill Durnan retired. McNeil looked good for a few seasons behind the Montreal Canadiens' strong defence, but after Plante took his job McNeil would find himself essentially out of the NHL at the age of 28. Plante first got NHL playing time during the 1952-53 season, but Montreal continued to give McNeil the vast majority of the starts even though Plante put up outstanding numbers: 2-0-1, 1.33 in 1952-53 and 7-5-5, 1.59 in 1953-54. Not only that, but Plante was stealing playoff starts from McNeil as well and dominating with his opportunities. Plante went 3-1 with a 1.75 GAA in the 1953 playoffs, and 5-3, 1.88 in 1954 where he finally won the starting job from McNeil for good. However, by the time he had taken over the reins, Plante was already 26 years old.

Before the age of 26, Terry Sawchuk already had 199 wins and 57 shutouts in the regular season, as well as 28 playoff wins and 3 Stanley Cups. It was this starting advantage that allowed him to edge out Plante in most of the regular season career categories, as over the rest of their careers most observers would agree that Plante was the better goaltender. From age 26 on, Terry Sawchuk's career stats were 248-249-107, 2.82, 46 SO (and 26-33, 3.05, 3 SO in the playoffs). If Sawchuk had those results as his career numbers, he'd probably be remembered together with guys like Harry Lumley or Gump Worsley, rather than Plante and Glenn Hall.

Montreal should have followed Detroit's lead by dumping McNeil and starting Plante in 1951-52. Even if Plante had simply matched McNeil's numbers from 1951 to 1954, he would have far surpassed Sawchuk with 524 wins and would have equalled Sawchuk's mark of 103 shutouts. In all likelihood Plante would have beaten the shutout record as well, and would probably be unanimously rated ahead of Sawchuk in the all-time debates.

This is one of the problems with the frequent focus on career totals to evaluate goalies - some goalies break in earlier than others, often through no special abilities of their own but merely by the luck of the draw in terms of who was above them in the organization, whether or not the coach or GM recognized their talent, or whether or not they were on a hot streak or a slump in training camp when the team was looking to evaluate its goaltenders. In general better goalies will break in earlier, but this is not always the case. There is some luck involved in career length too, based on a goalie's health, league expansion, the strength of other goalies on the team in the pre-free agency era, etc. These reasons suggest we should place more weighting on a goalie's peak than on his entire career, as this helps us avoid being overly influenced by some of those extraneous factors that can have a big effect on a goalie's legacy despite reflecting little of their actual skill or abilities.

My Definition of a "Money Goalie"

Patrick Roy's number 33 was recently retired by the Montreal Canadiens, prompting journalists everywhere to find a hundred different ways to say, "Patrick Roy was a clutch goalie". One of the most common cliches in this context, of course, is "money goalie".

I think nearly all talk about goalies who are supposedly "money goalies" or are considered "clutch" is a result of the most basic goalie observation bias - the goalie who is in front looks better because every time he makes a save it looks like he is saving the game. Most games simply come down to whatever team had the better scoring chances, but in any close game you can always use hindsight and selective memory to find a save that looks like it "won" the game, and of course this is what gets focused on by the team's fans and by broadcasters and newspaper writers in the game summary. "He made the key saves at key times in the game," they say, or some other similar cliche.

However, this is not unusual. All goalies, including the bad ones, stop at least 85% of the shots against them, even in the most high-leverage situations like late in the third period. If there are 5 minutes to go in a game and your team is up by a goal, the odds are strongly in your favour even with a terrible goalie like Dan Cloutier in net. The team has to just hold the opposition to a few shots against the rest of the way and it is likely they will win based on probability alone. It is even more likely if they manage to add an insurance goal or two. That's why teams have a combined 213-19-32 record this season when leading after two periods. And then of course Cloutier gets named the game's first star for "making the big save", the save that "preserved the win"...

Of course you can also find a number of saves by the losing goalie that kept his team in it, but they aren't as likely to make the highlight reels or create a reputation for the goalie of being the guy who "makes the key saves to keep his team only a single goal down".

Playing goal is a percentages game. The difference between a Hall of Famer and an OK goalie is that the great netminder stops an extra 2-3% of shots, or about 1 out of every 50. It is easy to say that someone like Grant Fuhr stopped the vast majority of the shots against him while his team was up by a goal in the third period, but all goalies stop the vast majority of shots against them in all situations. The investigative question becomes whether Fuhr was actually playing better or whether he was just in that game situation so often that a few timely saves linger in the memory banks long after hundreds of similar scoring chances have been forgotten.

Name anyone often commonly referred to as a "money goalie", and I bet you they played on a good team. In my view, the term "money goalie" is a term that gets applied almost exclusively to goalies who play on very good teams and therefore spend a lot of time playing with the lead. Obviously goalies have some impact on the game, so a great goalie will make it more likely that his team is in the lead, but the rest of the teammates combined have a much greater impact than the goaltender alone.

This mythology usually extends further. People tell stories about Billy Smith and Gerry Cheevers and say things like, "All they ever cared about was winning. They would give up meaningless goals in regular season games, but when the chips were down they were unbeatable." Fine, I don't necessarily buy it, but that is at least plausible that somebody would increase their level of focus and effort when it mattered most. However, here is my problem with this line of reasoning: If you accept the premise that a goaltender is not completely responsible for his team's win/loss record, which I believe any reasonable person would, then it is quite likely that a goalie with such a competitive mindset would nevertheless lose a game or a playoff series, despite his best efforts. After all, goalies like Roy and Fuhr still had more early playoff exits than Stanley Cup victories, even with outstanding teammates around them. How can we be sure that goalies like Gilles Meloche, Cesare Maniago, Gary Smith, Dan Bouchard and Mike Liut didn't have the same focus on winning that Billy Smith and Gerry Cheevers allegedly did? Was it merely because they were never dealt the good fortune of playing on teams stacked with Hall of Famers?

I want to know which goalies "only cared about winning" and "came up big in the clutch" yet played on a mediocre team and had little team success. If we cannot identify anyone who meets that description, then that tells me that clutch play is difficult to identify objectively and that many of the goalies on great championship teams are almost certainly getting too much credit for their contributions to that team success.

Brian Boucher Is Exposing The NHL's Most Overrated Goalie

Q: How do you win 10 out of 12 games despite a mediocre .891 save percentage?

A: Play for the San Jose Sharks.

Evgeni Nabokov is probably the most overrated goalie in the league. He didn't deserve his Vezina runner-up finish last season, and he doesn't deserve his status as one of the league's elite goalies (for example, Rotowire.com calls him "arguably the best goalie in the NHL"). This has become even more clear over the last couple of weeks, as Nabokov has missed several games with a knee injury and the Sharks haven't missed a beat with backup goalie Brian Boucher between the pipes.

Boucher has posted outstanding results since San Jose acquired him last season. He has gone 10-2-2 with a 1.85 GAA, .929 save percentage, and 3 shutouts. Unless Boucher has morphed into an elite goalie on the wrong side of 30, he has either been on a very lucky streak or he is playing on a strong team defence. It is probably true that Boucher has been at least somewhat lucky, and he may have been playing mostly weak opposition, but it would still be very unlikely for him to post similar results over 14 decisions on nearly every other team in the league.

Even without Boucher's success, there is still lots of evidence to argue that Nabokov is overrated. Nabokov's shot-quality adjusted save percentage numbers are very ordinary. Only one of his post-lockout seasons has been above league average (2006-07), and even that one was by a small margin.

Another problem with Nabokov is that he is erratic. His month-by-month save percentages are all over the place (e.g. last year his month-by-month line went .916, .929, .915, .890, .867, .941, .909). He almost seems to alternate between good and bad years throughout his career, and some of his seasons have been quite poor (2002-03 and 2005-06).

Nabokov makes some flashy saves and when he is on his game he can be pretty good. I can understand that there are people who caught the right sample of Sharks games could think Nabokov is a terrific goalie. However, every game counts, and over the long haul Nabokov's results are not elite.

What is interesting to me is that San Jose fans, who are not likely to be biased from a small viewing sample, seem to have a very different perspective of Nabokov than the numbers do. The opinions I have read may not be representative of all San Jose fans, but most of them that post or comment online appear to have the perception that their defence is weak and giveaway-prone, and that Nabokov has to make a lot of difficult stops. That conflicts with the shot quality numbers (San Jose has been consistently 5-7% better than average in shot quality against over the last 3 years), it conflicts with the giveaway numbers (looking at road giveaways only to remove scorer bias, San Jose ranked 3rd in the league in fewest giveaways in 2007-08), it conflicts with the results of Boucher and other Sharks goalies, and it conflicts with the shot prevention numbers (San Jose ranked 2nd in fewest shots allowed last year, and currently rank 2nd again this year). In this type of situation I think an objective analyst should at least be open to the possibility that the numbers aren't telling the whole story, but Nabokov is hardly the first high win total/low save percentage goalie to get lots of love from his local fanbase so for now I'll trust the evidence more than the hometown fans.

Nabokov is currently sidelined with an injury but is expected to return soon. If he gets nearly all of the starts the rest of the way and the Sharks continue to dominate, Nabokov could once again lead the league in wins. That may be difficult, since Boucher's success will likely result in San Jose throwing a few more starts his way, but it would be interesting to see a goalie win 45+ games with a sub-.900 save percentage, if nothing else as an interesting test case for Vezina voters.

In somewhat related news, Toronto fans seem to be souring on Vesa Toskala. This is not that surprising, since Toskala hasn't been anything special at all since the lockout, either in San Jose or Toronto. Here is a comparison of Toskala and Nabokov in San Jose:

Toskala, SJS: 65-28-10 (.680), 2.34, .914
Nabokov, SJS: 218-152-48 (.579), 2.38, .910

Now, here is Toskala so far in Toronto:

Toskala, TOR: 40-31-10 (.556), 2.86, .898

Makes one wonder how well Nabokov would do on a weaker team. I'm not sure he would do a whole let better than Toskala has. San Jose gets a lot of credit for their work in developing goalies, but I think much of it may simply be a result of the strong team and goalie-friendly climate in which they play.

Goalies Always Get the Blame

Are NHL teams good at evaluating goalies independent of their team context? I think there is a lot of evidence that they certainly aren't perfect.

Take, for example, this information that I collected on the number of goalies each team has used in the last 10 years (since 1998-99). What stat is the best predictor of the number of goalies? Turns out it is GAA. Most observers would agree that GAA is strongly team-influenced, especially for goalies who play on very weak teams. Of the 10 best teams in GAA, only Ottawa and Philadelphia used more than 11 goalies. Of the 15 worst teams, everyone except Columbus used more than 11 goalies, and some of the teams used 20 or more.

Some of this effect is legitimately a result of goalie quality, but the fact that so many different goalies failed to make an impact on the league's weakest teams shows how tough it was to play there. It also shows how much more stable the goaltending situation tends to be on a top defensive team, and why goalies who come into the league on one of those teams have a much better chance at staying in that position for a while.

Here is the list of teams with GAA and number of goalies they have employed in the last decade:

Rank	Team	GAA	#	Rank	Team	GAA	#
1.	DAL	2.23	11	16.	VAN	2.62	19
2.	NJD	2.25	7	17.	CAR	2.68	13
3.	DET	2.33	11	18.	BOS	2.69	17
4.	MIN	2.40	8	19.	TOR	2.70	14
5.	SJS	2.40	10	20.	PHX	2.73	20
6.	COL	2.42	11	21.	LAK	2.74	24
7.	OTT	2.42	13	22.	FLA	2.75	15
8.	ANA	2.46	10	23.	NYR	2.77	17
9.	PHI	2.48	14	24.	WSH	2.78	13
10.	BUF	2.49	8	25.	CHI	2.84	17
11.	STL	2.52	18	26.	NYI	2.85	15
12.	CGY	2.59	18	27.	CBJ	2.88	11
13.	MTL	2.59	13	28.	TBL	2.96	21
14.	NSH	2.62	10	29.	PIT	2.98	14
15.	EDM	2.62	14	30.	ATL	3.21	16

(Post edited to correct errors)

Goalie Hyperbole, Part 1,542

Damien Cox, who has been known in the past to massively overrate the impact of goalies in hockey (which may explain his enthusiastic support of Marty Brodeur as the best ever), is at it again.

Here is the money quote:

"There have been many who have suggested the sport shouldn't be called hockey, but rather, goalie. The identity of the player who fills the crease, after all, is the major determing factor when it comes to the success or failure of a hockey club."

That must be great news, then, for the Tampa Bay Lightning, who currently sit 4th in the NHL in save percentage behind Mike Smith's goaltending. With that kind of production at such an overwhelmingly important position, they must be having a great season. Oh, wait.

Other top-10 save percentage teams languishing at the bottom of the standings include Phoenix, Ottawa and Florida. Meanwhile, the two best teams in the league are San Jose and Detroit, with their scintillating .906 and .892 save percentages respectively. I guess there must be other factors at work in NHL games other than goaltending! Who knew?

Correlation of goals for and winning percentage in 2008: .611

Correlation of save percentage and winning percentage in 2008: .374

If we are going to rename hockey, I think "scoring" should rank a lot higher on the list than "goalie".

Shots Attempts Against

With the issue of shots against featuring prominently in recent discussion here and elsewhere in the blogosphere, I thought to take a slightly different look at the issue. Generally attempts to define the shots against/save percentage relationship look at total numbers by season (like this one, for example). I was interested in how teams performed over multiple seasons, and how similar shot preventing teams did in terms of save percentage.

Based on the results from the even-strength save percentage while tied numbers, I was also interested in exploring the results of blocked shots and missed shots against. The frequency of blocked or missed shots is quite high. Over the last 8 seasons, there were 555,531 shots, 215,508 blocked shots, and 208,889 missed shots, meaning that 43% of shot attempts never make it on net. It would therefore seem that a team's commitment to shot blocking could have a big impact on the number of shots actually faced.

The "fewer shots = lower save percentage" argument is usually based on one of the following two premises: 1. Goalies who face infrequent shots lose focus and are less physically prepared for each shot that comes than those who are facing more frequent shots, or 2. Goalies who face infrequent shots are facing higher quality chances. The first one could possibly be true, but based on my personal experience I do not think it is a major factor. It is, however, a difficult one to test (it would require analysis of the play-by-play records to do it properly), so this post is directed at premise 2.

Before I continue, just a caveat: with RTSS data there is always the possibility of systematic errors. It is, in fact, quite likely that teams differ in their reporting criteria, based on what we have discovered from past shot and shot distance reporting results. I will present the data as is, but if there is reason to believe that the numbers aren't quite correct please point it out.

I took the last 8 seasons (1999-00 to 2007-08), and collected the total minutes, shots against, saves, blocked shots and missed shots for each team. I then divided the teams up into 5 groups, based on where they rank in shot attempts against (Att), defined as saves plus goals allowed plus blocked shots plus missed shots. I have also included shots against (SA) and save percentage numbers for each team.

Low Shot Group:
Anaheim: 46.4 Att, 28.2 SA, .913 Sv%
Detroit: 46.5 Att, 25.9 SA, .910 Sv%
Chicago: 46.7 Att, 28.2 SA, .899 Sv%
Dallas: 46.9 Att, 25.0 SA, .910 Sv%
San Jose: 47.1 Att, 27.0 SA, .910 Sv%
New Jersey: 47.5 Att, 25.5 SA, .912 Sv%

If a low shots against total was the only handicap preventing Martin Brodeur from posting elite save percentage numbers then we would expect the Devils' goaltending to outperform the rest of this group. The fact that Anaheim, Detroit, Dallas and San Jose have virtually identical save percentages while facing even fewer shot attempts than New Jersey seems to be strong evidence against that viewpoint.

Anaheim and Chicago do not block many shots, at least according to NHL scorers, so even with average shot totals they both move up into the top group here. Chicago is definitely the outlier, finishing far behind the other teams. Could the Chicago scorers possibly be underreporting blocks and missed shots? Or do the Hawks simply suffer from bad goaltending and/or high shot quality against?

Moderately Low Shot Group:
Boston: 48.4 Att, 28.5 SA, .903 Sv%
Nashville: 48.7 Att, 29.3 SA, .912 Sv%
Ottawa: 48.8 Att, 26.8 SA, .908 Sv%
Vancouver: 48.8 Att, 27.2 SA, .906 Sv%
St. Louis: 48.9 Att, 25.4 SA, .901 Sv%
Calgary: 48.9 Att, 27.5 SA, .907 Sv%

St. Louis ranks near the very best in shots against, but the reason seems to be that they block so many shots. Given the goaltending they have had that is maybe not too surprising, but lumping in St. Louis with the top possession teams during the study period appears to be incorrect. Ottawa is similar to the Blues. On the other hand, Nashville apparently lets a lot more shots through, and they had the best save percentage in this group.

Average Shot Group:
Colorado: 49.3 Att, 27.4 SA, .912 Sv%
Tampa Bay: 49.5 Att, 28.3 SA, .898 Sv%
Philadelphia: 49.8 Att, 27.1 SA, .908 Sv%
Toronto: 49.9 Att, 27.9 SA, .903 Sv%
Buffalo: 50.0 Att, 27.9 SA, .909 Sv%
Minnesota: 50.0 Att, 28.5 SA, .916 Sv%
Phoenix: 50.0 Att, 29.4 SA, .905 Sv%

The average group had the widest range of save percentage results, from Minnesota (best in the league) to Tampa Bay (worst in the league). Even though Minnesota blocked a normal number of shots and allowed an average number of shots on goal, their goalies had very high save percentages, which suggests that Jacques Lemaire knows how to make life easier for his goalies. Both Colorado and Buffalo had good results even after Roy and Hasek left town, which suggests that they were good defensive teams.

Moderately High Shot Group:
Columbus: 50.3 Att, 30.3 SA, .905 Sv%
Carolina: 50.4 Att, 27.5 SA, .901 Sv%
Los Angeles: 50.8 Att, 27.7 SA, .900 Sv%
Washington: 50.9 Att, 29.9 SA, .905 Sv%
Edmonton: 50.9 Att, 27.2 SA, .904 Sv%

The save percentages are very similar for all 5 teams and none of these teams had elite goalies in the period (other than maybe a couple of Kolzig seasons), so the results likely generally reflect team shot quality against. Edmonton had the second highest total of blocked and missed shots in the league while Columbus had one of the lowest, creating a difference of over 3 shots on goal per game, but the two teams had almost identical save percentages.

High Shot Group:
NY Islanders: 52.2 Att, 29.6 SA, .904 Sv%
Pittsburgh: 52.2 Att, 30.4 SA, .900 Sv%
Florida: 52.9 Att, 31.9 SA, .913 Sv%
NY Rangers: 52.9 Att, 29.1 SA, .904 Sv%
Montreal: 53.3 Att, 30.1 SA, .914 Sv%
Atlanta: 53.3 Att, 31.6 SA, .898 Sv%

For teams that give up a lot of chances, the norm appears to be mediocre save percentages rather than higher ones. Florida and Montreal are the only above-average teams and they rank far ahead of everyone else.

Here are the overall averages for each group:

Low Shots: 46.9 Att, 26.6 SA, .909 Sv%
Mod. Low: 48.8 Att, 27.5 SA, .906 Sv%
Average: 49.8 Att, 28.1 SA, .907 Sv%
Mod. High: 50.7 Att, 28.5 SA, .903 Sv%
High Shots: 52.8 Att, 30.5 SA, .906 Sv%

There is no real evidence of a pattern in terms of goaltending success. There are a few teams that have outlier results based on their groupings, either underperforming (Chicago, St. Louis, Tampa Bay) or overperforming (Nashville, Florida, Montreal, Minnesota) their expected save percentages. If we remove the best and the worst team in each group to deal with potential outliers, the save percentages by group go .911 - .906 - .907 - .903 - .905. This evidence certainly doesn't show that playing on a top defensive club hurts one's save percentage; if anything it suggests the opposite. However, save percentage is not a perfect proxy for shot quality - the goalies themselves obviously have an impact. If top teams tend to have better goaltending, for instance, then we would expect this result.

These numbers are polluted by a few variables, like suspect RTSS data and special teams play. The topic of possession, outshooting, and scoring percentages is still being investigated. However, I remain unconvinced that playing on a strong defensive team makes it tougher to put up high save percentages. I did not really evaluate possession effects here, merely shot prevention, so it could be that there is an effect for outshooting teams. In any event, even if a general relationship is established, this post is evidence that there is still substantial variability from team to team even within similar shots against groupings. This variability means that we cannot necessarily go from an established general result to the specific case for individual goaltenders or individual teams without additional supporting evidence.

These findings also show significant differences in shot-blocking tactics, which support the use of Fenwick or Corsi numbers rather than raw shot totals to evaluate team puck possession. Finally, there is some evidence of a relationship between blocked shots and goalie save efficiency (-0.30 correlation between blocked shots per game and save percentage). It remains to be determined whether this may be because shot blocking makes it harder to stop pucks, or because teams with bad goaltending simply try to make more saves themselves.

ES Save % While Tied

There has been a very insightful investigation of the effects of possession on scoring success going on at Mc79hockey (check it out here and here). The key takeaway is that teams play to the score, and in-game tactics appear to have a large effect on both outshooting and scoring percentages.

To remove game score as a factor, Vic Ferrari wrote an app at his website timeonice.com to look at even-strength play with the score tied. I ran the numbers for each team in the league during the 2007-08 season. The full table is below, but here are a couple of comments:

Correlation between number of shots faced and save percentage: .097. Correlation between shot differential and save percentage: .063. Based on this sample, outshooting appears to have little effect on save percentage at even strength in a tie game.

Correlation between blocked shots and save percentage: -0.239. I'd like to see more data on this, but it seems to suggest there could be a weak negative relationship between blocked shots and save percentages (i.e. it is tougher for a goalie to play on a team that blocks more shots).

These results are more sensitive to luck/randomness since the sample size is smaller. There appears to be a clear shot quality effect, however, since most of the elite defensive teams show up near the top with weaker teams near the bottom. There is also a gap of .041 between the best and worst teams.

I compared the teams' results with Alan Ryder's shot quality numbers. Most teams had similar rankings, but there were a few teams with large discrepancies. Either those teams had very good or very bad goaltending, or they were very good or bad at protecting leads or playing from behind, or there was something else going on.

Better than expected:
Anaheim: 29th in shot quality, 1st in ES SV% while tied
N.Y. Rangers: 25th in shot quality, 6th in ES SV% while tied
Dallas: 27th in shot quality, 17th in ES SV% while tied
Montreal: 17th in shot quality, 8th in ES SV% while tied

Worse than expected:
Columbus: 1st in shot quality, 20th in ES SV% while tied
Washington: 4th in shot quality, 22nd in ES SV% while tied
Buffalo: 13th in shot quality, 30th in ES SV% while tied
Calgary: 3rd in shot quality, 18th in ES SV% while tied
Tampa Bay: 14th in shot quality, 29th in ES SV% while tied

Anaheim and Buffalo are the most interesting ones here. I don't think that either Giguere is that good or Miller is that bad. It could be that both teams have extreme results in the opposite direction when they are leading or trailing. However, I suspect that for both of those teams shot quality simply may not accurately measure the true difficulty of scoring chances against.

Assuming the RTSS shot distance figures that make up the shot quality rating are more or less correct (and these are road numbers only, to reduce rink reporting bias), the difference would therefore have to be either because of the average shot angle, the average defensive pressure on the shooter, or the quality of the average opposing shooter. Behind the Net has a shot quality measure that takes into account shot angle. According to those numbers, Giguere faced shots of about average difficulty, while Miller faced slightly easier than average shots against. It looks like the Ducks probably kept more of their shots away from the middle of the ice than normal, but that doesn't explain the entire difference.

I would suspect that as an offensive team Buffalo might be giving up more rush chances and more space to opposing forwards, which allows them more time to evaluate their options and make their shots. Anaheim's defence-first philosophy likely creates more pressured shots, which are easier to stop. Another possible explanation is that Anaheim has an elite checking line which may help their goalies by reducing shots against from the opposition's best players.

I would love to see these numbers for when teams were leading and trailing as well, to see both the overall save percentage effects and which teams were great at locking it down or opening it up.

Here are the full numbers for 2007-08 at even-strength, game tied:

Rank	Team	SA	GA	Sv%
1.	Anaheim	648	36	.944
2.	Boston	738	43	.942
3.	New Jersey	675	41	.939
4.	N.Y. Islanders	745	46	.938
5.	Vancouver	708	46	.935
6.	N.Y. Rangers	599	40	.933
7.	Nashville	694	47	.932
8.	Montreal	689	47	.932
9.	Minnesota	724	50	.931
10.	Phoenix	723	50	.931
11.	Atlanta	881	62	.930
12.	Detroit	517	38	.926
13.	Pittsburgh	702	53	.925
14.	Florida	788	60	.924
15.	Ottawa	604	46	.924
16.	San Jose	588	45	.923
17.	Dallas	633	49	.923
18.	Calgary	669	53	.921
19.	Carolina	629	50	.921
20.	Columbus	661	53	.920
21.	Edmonton	823	67	.919
22.	Washington	675	56	.917
23.	St. Louis	704	59	.916
24.	Colorado	692	58	.916
25.	Toronto	691	58	.916
26.	Philadelphia	689	62	.910
27.	Chicago	621	56	.910
28.	Los Angeles	708	65	.908
29.	Tampa Bay	713	69	.903
30.	Buffalo	636	62	.903

New Jersey's Forgotten Star

Many hockey observers, especially New Jersey fans themselves, claim that the current New Jersey Devils squad is a motley crew of castoffs and journeymen, held singlehandedly together by Martin Brodeur. I must admit I don't really understand why such a large portion of a fan base seemingly comes down so hard on their own players like Devils fans do, although it probably has a lot to do with high expectations and standards resulting from New Jersey's recent history of championship teams. I especially don't understand it since those fans are wrong. New Jersey still has at least one other star player remaining on its team, a player that has been one of their most critical pieces for the last decade. That player is Patrik Elias.

Since 1999-00, Patrik Elias has scored 231 goals and 314 assists for 545 points in 593 games, with a +130 rating. The points may not seem all that impressive, but he put up those numbers playing on a team with a primary focus on defence. In the one season the reins were loosened a bit, 2000-01, Elias scored 40 goals and 96 points to finish 3rd in the league in scoring.

Elias contributes in many more ways than the scoresheet. He combines his offensive strengths with excellent defensive play, and his plus/minus rating reflects his two-way dominance. A recent development in hockey analysis has been the use of the Corsi number, which is the difference between the number of shots directed at the opposition net and the number of shots directed at the player's own net while he is on the ice. Last season, Detroit Red Wing players dominated the Corsi numbers as they were the most dominant outshooting team in the league. The best player in the league in terms of Corsi numbers who did not play for Detroit? Patrik Elias.

But didn't Elias have a bad year last year? Not really, he was just unlucky. With Elias on the ice, New Jersey took 32 shots per 60 minutes of 5-on-5 play, and allowed just 21. Elias' only problem was that when he was on the ice this season the goaltending just happened to be much better at the wrong end of the ice: the save percentage on shots by him and his teammates was .930, while it was just .904 for shots against. Elias' shooting percentage has dropped in both of the last two seasons so he may be losing his scoring touch, but this season it afflicted his linemates as well. Some of this may be attributable to a difference in shot quality, but it seems unlikely Elias and his linemates were giving up large numbers of dangerous scoring chances against. When Elias is playing, the puck is usually in the other end of the ice and that is a tremendous advantage for his team as well as for his goaltender.

Perhaps the best way to express Elias' impact on the team is to show New Jersey's record with him in the lineup compared to without. Last season, they were 3-4-1 with Elias out of the lineup, and 43-25-6 with him. In 2006-07, 4-3-0 without Elias, 45-21-9 with him. Two seasons ago, when Elias missed a substantial amount of time, the Devils were 19-18-7 without him, and 27-9-2 with him. Overall for those three seasons, the Devils were a .508 team (26-25-8) without Patrik Elias, and a .660 team (115-55-17) with him in the lineup.

New Jersey's goals for and against splits were substantially different with and without Elias:

with Elias:

2007-08: 2.47 GF, 2.28 GA
2006-07: 2.53 GF, 2.31 GA
2005-06: 2.92 GF, 2.50 GA

without Elias:

2007-08: 1.88 GF, 3.00 GA
2006-07: 2.29 GF, 2.86 GA
2005-06: 2.77 GF, 2.95 GA

How did the Devils do without Martin Brodeur over that same time period? New Jersey was 6-7-5 in games without Martin Brodeur for a .472 winning percentage, and 135-73-20 with MB30 between the pipes (.636). That's a gap of .164, compared to Elias' difference of .152 (although the "without Brodeur" sample is much smaller). New Jersey allowed a lot fewer goals with Brodeur in net (2.30 per game) than with his backups (3.20).

Interestingly, though, the defensive impact of Elias was not too far behind that of his goaltender. With Elias the team allowed 2.34 goals per game, and without him it was 2.95. Most of the games the Devils played without Elias came in 2005-06, a season where the team played poorly during the first half and got on a roll in the second half. Elias missed the entire first half and played most of the second half, which was the ideal scenario to put up positive with/without win/loss splits. It is likely that Elias' contribution explains some of the improvement, but there were probably many other factors at play. However, even if we exclude 2005-06 entirely and look at just 2006-07 and 2007-08 the results are almost exactly the same. In those two seasons, the Devils were 7-7-1 without Elias (.500) and 88-46-15 with him (.641). There was also a combined average of 2.30 goals against per game with Elias in the lineup compared to 2.93 goals per game without him. The numbers certainly suggest that Elias has a very strong impact on New Jersey team success and goal prevention. They are especially impressive when you consider that when Elias was not available, the Devils would most likely elevate an established NHLer from the second or third lines to take his spot. Because of this it would have been reasonable to expect the Elias differential to be much smaller than the gap between Brodeur and his replacements (primarily infrequently-used backup/minor-leaguer Scott Clemmensen), yet the effect on team success by Elias and Brodeur might not be that different at all.

Vic Ferrari did a post on this topic last year, investigating the difference between when a star player is playing compared to when they are out. Often teams do much worse without their best player, since it creates a ripple effect throughout the lineup - the great player benefits his teammates not only directly by playing with them, but also indirectly makes things easier for the rest of the team by taking on extra minutes and tougher opponents, as well as often drawing penalties or ending their shift in the offensive zone. Earl Sleek made a similar "trickle-down" argument for explaining the importance of Scott Niedermayer to the Ducks in the second half of '07-08.

These types of effects are a lot more difficult to see. Everybody knows when the backup goalie is in the game, but not everyone notices when a second-pairing defencemen has to suddenly start playing 25 minutes a game, or when a second-liner accustomed to playing against checking forwards finds himself moving up to play against the opposing top line, or when a rarely used forward has to be pressed into service as a replacement on the penalty kill unit. And this subtlety is another reason why the importance of goaltending tends to be overrated compared to other positions. Valuable, of course, but still generally overrated.

Is Patrik Elias more valuable to the Devils than Martin Brodeur? I don't think so, but Elias is nevertheless underrated player. Led by Elias, the Devils have a strong group of forwards that usually wins the territorial battle and outshoots the opposition. This talent is recognized by informed hockey fans - over 80% of the people who voted in a recent poll on this site predicted New Jersey would still make the playoffs despite losing Brodeur to injury. Unless Elias or several other key forwards join Marty on the injury list then I agree with that assessment.

More Musings on Career vs. Peak

I think one of the irreconcilable differences when it comes to doing all-time rankings is an observer's personal perspective on the weighting of peak vs. career. I dislike the expression "agree to disagree", because I think it is usually a retreat tactic employed by someone who has run out of arguments, but I think it sometimes legitimately comes into play during all-time debates when an impasse has been reached because of a fundamental difference in evaluation criteria.

Conceptually, I think there is one group of people that approaches the issue as if they were a general manager looking to build a hockey team in an expansion draft. The draft eligible group consists of every 18 year old kid that has ever played hockey, and their goal is to pick the one that provides the highest contribution to that team over their entire career. In this scenario, things like durability, consistency, and longevity are important considerations.

The other group comes at it from a totally different angle. They tend to think of themselves as the coach of a one game playoff. They can pick anybody who has ever played to be on their team, and each player will participate during the prime of their careers, at the absolute height of their skills and abilities. In this scenario, durability and longevity are very much secondary considerations.

This is an oversimplification, because I don't think you will find many people that will take either case to their extremes (e.g. take Mark Recchi over Guy Lafleur, or Jose Theodore over Ed Belfour). Both peak and longevity should have some weighting, but at the end of the day everyone is going to favour one or the other.

Anyone who has read more than a few posts on this blog will be aware that I fall into the second group. For that matter, anyone who has just read this website's title could probably guess that correctly. I've said this many times before, but I am far more interested in how good somebody was than how valuable they were. When a player has retired, and a few decades have passed, and the memories have mostly faded away, the only thing that sticks in the collective recollection of hockey fans is the talent and skills that player displayed when they were at their best. Bobby Hull unleashing a wicked slapshot, Rocket Richard cutting towards the net, Bobby Orr wheeling up ice, and so on. For anyone who is over 25, picture Wayne Gretzky as a player. Is he wearing the jersey of either St. Louis or New York? I didn't think so. When your grandchild asks you, "How good was Dominik Hasek?", he is going to be a lot more interested in how Hasek played against the Canadians and Russians in Nagano than how he did as a 40 year old platooning with Chris Osgood.

If you had a time travelling machine and you went back in time, collected every goalie who has been a top starter in the NHL at their peak, gave them all access to the same nutrition, training, equipment, etc. (to minimize era effects, as otherwise you would just take whoever is the best in the league today), and then lined them up against a wall to pick sides, I might not even glance Brodeur's way until a dozen guys are already off the board. But if I am a GM looking over the 18 year old versions of every goalie who has ever played NHL hockey and trying to figure out who I want in net for my team for the next 2 decades, I would give Brodeur a much closer look.

In summary, when I rank hockey players, I tend to divide them into tiers. Within each tier, things like longevity and durability and so on become important, because if you had to choose between two guys of similar abilities then you would rather have the guy you can count on to deliver every single night. But a flaky superstar always beats a consistent very good player because he is on a different level, and all the consistent 30 goal or 80 point seasons in the world don't bridge the gap, in my estimation. Which is why I don't consider career records to be that important, whether they are career wins, shutouts, goals, assists, passing yards, strikeouts, whatever.

Here's a discussion question, for those who like the career value approach: Let's say Chris Osgood, through some experimental genetic engineering or possibly a fortunate archaeological find, is able to stay eternally youthful and play goal in the NHL for as long as he wants. Let's also say that Osgood stays on a perennially strong team in Detroit, continues to put up his customary slightly-above-average save percentages, and averages 30 wins and 5 shutouts per season and one Stanley Cup win every 10 seasons. How many more years would Ozzy have to play before you would rate him ahead of Dominik Hasek in your all-time goalie rankings?

(In the unlikely event that there is someone reading this that doesn't like goalies, I'd suggest substituting Rod Brind'Amour, 70 points, and Mario Lemieux into the previous paragraph to get a more-or-less equivalent scenario for skaters).

Brodeur, Roy, and the Career Wins Record

The question was recently raised by someone in the Yahoo Hockey Analysis Group about whether Martin Brodeur's career win total is more impressive than Patrick Roy's, once you take into account the changes in league tiebreaking procedures (e.g. 4-on-4 OT, shootouts), the number of games played in a season, and the cyclical nature of how many games goalies play in a season. I decided to take a closer look at this question.

First of all, wins is not a very informative stat, unless you take into account the team a goalie is playing on. Goal support and the number and type of shots against all have a big impact on winning and losing. I will not be adjusting for any of those things in this post, so take the numbers with the standard critical eye reserved for goalie win totals. The objective here is to compare wins across different eras rather than to evaluate goaltenders.

First of all, for each season since 1917-18 I figured out the percentage of games that ended up in a win for one of the goalies, and used it as an adjustment factor in my analysis. In today's NHL, that figure is 100% because of the shootout. In 2003-04, before the shootout was introduced, 14% of games ended up as ties, so just 86% of the time a goalie was awarded with a win. In lower scoring eras early in the NHL's history, it was not uncommon for the percentage of games with a win to be at 80% or lower.

Another consideration was the number of games in a season. Everything was normalized to the current 82 game schedule.

The final thing I looked at was how goalies were utilized. I wanted to use a similar measurement for all the years, so I took the average of the top 6 goalies with the highest minutes played in each season. I then figured out the percentage of minutes played by the top 6, and used this as an adjustment factor. In seasons where the starters played nearly every game, such as the 1930s, the number was very close to 100%. Last season the figure was 89%. For much of the 1970s and 1980s, however, this number was much lower. For example, in 1986-87 the top 6 goalies played just 67% of the minutes.

The problem with the minutes adjustment was that it was quite variable. In the Original Six era, if one or two teams decided to platoon goaltenders for a season it would drastically reduce the overall total. I therefore decided to use a three-year average for each season, consisting of the previous season, the current season, and the following season. This smoothed the curve and allowed for, in my opinion, a better reflection of the era effect, rather than putting an excessive weighting on an individual team's roster management choices.

This allows us to calculate a season-by-season adjusted win figure, based on the win frequency, schedule length, and average level of minutes played for top goalies. Here are the all-time top 30 (up to the end of 2007-08):

Rank	Goalie	Wins	Adj Wins
1.	Patrick Roy	551	610
2.	Terry Sawchuk	449	558
3.	Jacques Plante	437	555
4.	Martin Brodeur	538	543
5.	Glenn Hall	407	536
6.	Tony Esposito	423	529
7.	Ed Belfour	484	515
8.	Curtis Joseph	449	474
9.	Grant Fuhr	403	462
10.	Rogie Vachon	355	447
11.	Mike Vernon	389	443
12.	Tiny Thompson	284	442
13.	Gump Worsley	335	437
14.	Andy Moog	372	434
15.	Clint Benedict	190	430
16.	Turk Broda	302	429
17.	John Vanbiesbrouck	374	426
18.	Harry Lumley	330	421
19.	Tom Barrasso	369	415
20.	Dominik Hasek	389	399
21.	John Ross Roach	219	397
22.	George Hainsworth	246	384
23.	Ed Giacomin	289	381
24.	Chris Osgood	363	373
25.	Billy Smith	305	373
26.	Frank Brimsek	252	364
27.	Mike Liut	294	361
28.	Sean Burke	324	352
29.	Dan Bouchard	286	351
30.	Bernie Parent	271	341

By this measure, Patrick Roy is still well ahead of everyone. Martin Brodeur not only has a lot more ground to make up on Roy, but he also has to first pass Plante and Sawchuk. Based on these figures and his current health situation, Brodeur probably won't pass Roy in career adjusted wins until some time in 2011.

These results show that wins were much easier to come by in the late 1990s and early 2000s. For example, coming into this season Roberto Luongo had a total of 197 wins but only 182 adjusted wins, which shows how win totals are being inflated in the current era where top goalies play nearly every game and somebody wins every single night. Of course Luongo is a textbook case of a goalie who is very underrated by wins totals because he has played most of his career on weak teams, but that is just a reminder of the limitations of the statistic.

Roy's wins record is certainly more impressive when you take into account the adjusting factors. There are many who make the mistake of comparing Brodeur's numbers directly with Roy's, but that is something that simply cannot be done without understanding the league contexts. Once you adjust for that, Patrick Roy comes out ahead in almost everything. There is no question that both Roy and Brodeur played most of their careers on outstanding teams, but at this point Roy still has to be rated the better goaltender, and that is even without taking into account their playoff records (where Roy has a large advantage over Brodeur).

This list also doesn't adjust for career length. This is a disadvantage for older goalies who tended to have shorter careers, both because they were more likely to get injured and also because there was a lot more competition for the few starting jobs that were available. That is likely the main reason that 7 of the top 10 goalies on the list played most of their careers post-1967. It is also why the longevity of Sawchuk, Plante and Hall is impressive, and why they are usually rated very highly on all-time goalie lists.

Expected Shutouts - Original Six

There is limited shot data available prior to NHL officially began tracking shots in the 1980s, but shot data has been compiled for the period for 1955-1967. I thought to take a look at that period, which covers most of the careers of three of the general consensus best goalies ever (Hall, Plante, Sawchuk), and calculate expected shutout numbers for those three goalies, as well as contemporaries Johnny Bower and Gump Worsley.

Glenn Hall: 67 SO, 42.9 Exp SO, +24.1, +56%

Terry Sawchuk: 55 SO, 38.6 Exp SO, +16.4, +42%

Jacques Plante: 58 SO, 42.1 Exp SO, +15.9, +38%

Johnny Bower: 26 SO, 19.6 Exp SO, +6.4, +33%

Gump Worsley: 27 SO, 25.5 Exp SO, +1.5, +6%

I am a bit suprised that Sawchuk beats out Plante, especially since this sample includes nearly all of Plante's Montreal career but does not include Sawchuk's seasons from 1950-54 in Detroit where Sawchuk put up 45 shutouts. However, it is very close between the two of them and Bower for second place behind Hall, and the numbers show why all four of them are considered all-time greats.