Team Effects in the Original Six

Original Six goalies are very difficult to evaluate, because of the extreme team effects. The best talent was usually concentrated on a couple of the best teams, and the goalies on those teams not only had the advantage of playing behind a great team but they also never had to face their own team's elite goal scorers.

It is very difficult to estimate these effects, however, because most teams gave all or nearly all of their minutes to a single starting goalie. Glenn Hall, the most extreme example, played 503 games in a row at one point, so we don't even have a single game's worth of results for any other goalies on his team for a full 7 year stretch. That makes it impossible to use the method of comparing results to backup goalies. There was also less freedom of movement, so goalies didn't change teams as often.

These limitations mean that any method used is going to be less than ideal, but focusing on the goalies who changed teams and trying to estimate the team impact seemed like the best option. I decided to look at all the goalies that changed teams in the post-WWII Original Six era (1946-47 to 1966-67). The goalies were: Glenn Hall, Johnny Bower, Jacques Plante, Gump Worsley, Emile Francis, Hank Bassen, Frank Brimsek, Terry Sawchuk, Al Rollins, Jim Henry, Don Simmons, Bruce Gamble, Cesare Maniago and Harry Lumley.

I figured out the cumulative winning percentage and GAA for each goalie on all of the different teams they played for, and then tried to estimate each team's relative rank based on the differences.

Of course it is unrealistic to assume that every team had the same relative strength over a 20 year period. Montreal and Toronto were consistently good, while Boston and the New York Rangers were pretty mediocre, but Detroit was a powerhouse in the 1950s and terrible in the 1960s, while Chicago was exactly the reverse. Another problem is career arc - someone like Glenn Hall spent his entire prime in one place, and the only points of comparison we have of him somewhere else are either as a very young goalie or as an old one. The final problem was sample size: there were a few team pairings that didn't have a single goalie play on both of them. It was particularly difficult to evaluate Montreal goalies, since there were really only two goalies who played a lot of games in both Montreal as well as somewhere else, and both of them happened to also play for the Rangers (Plante and Worsley). There were also a few well-travelled goalies (like Harry Lumley, who played on 4 out of the 6 teams) that ended up having a larger effect on the sample.

However, despite these limitations, the numbers seemed to validate the method through a reasonable degree of consensus. For example, if you compare the goalies that played in both Boston and Toronto, they had a GAA in Toronto that was 0.42 better and a winning percentage .084 higher compared to Boston. If you used the Chicago results to verify this (by looking at the goalies who played in both Chicago and Boston, and comparing those results to the goalies who played in both Chicago and Toronto), the estimate was that the Leafs were 0.34 better in terms of GAA and .092 in winning percentage. Using the Detroit comparisons, it came out to 0.51 and .037. We can therefore ballpark the expected effect of getting traded from Boston to Toronto as being something like 0.40 - 0.50 in GAA and .070 - .090 in winning percentage.

I took averages from several of these comparisons, and came up with a relative set of rankings:

1. Montreal: 0.00 GAA, 0.000 win %
2. Toronto: 0.00 GAA, -0.040 win %
3. Detroit: +0.15 GAA, -0.025 win %
4. Boston: +0.40 GAA, -0.115 win %
5. Rangers: +0.70 GAA, -0.185 win %
6. Chicago: +1.00 GAA, -0.245 win %

If we compare these numbers to the actual results, we can both verify them and see which teams apparently had strong or weak goaltending:

1. Montreal: 2.36 GAA, .600 win %
2. Toronto: 2.51 GAA, .535 win % (+0.15 GAA, -0.065 win %)
3. Detroit: 2.54 GAA, .561 win % (+0.18 GAA, -0.039 win %)
4. Chicago: 3.04 GAA, .441 win % (+0.68 GAA, -0.159 win %)
5. Rangers: 3.08 GAA, .430 win % (+0.72 GAA, -0.170 win %)
6. Boston: 3.09 GAA, .432 win % (+0.73 GAA, -0.168 win %)

The total results confirm that Montreal, Toronto and Detroit were the three front-runners, with similar GAA totals. Montreal likely did have somewhat better goaltending than the Leafs or Wings, but the main reason the Canadiens had more team success was probably not goaltending but superior offensive play. The model predicts the Rangers quite well relative to the Canadiens, which suggests that despite often being a bottom-feeding team the Rangers got decent performances from the goalie position. Boston, on the other hand, appears to have had weak play in net, since they allowed the most goals of any team but apparently had a better defensive environment than either New York or Chicago.

Chicago's comparative results are exaggeratedly poor because, as previously noted, Glenn Hall was the only guy in their net during the early to mid-1960s. If we take Chicago's results prior to Hall's arrival in 1957-58, the Blackhawks' cumulative goalie stats were 3.46, .345, which means they were 1.10 and .255 worse than the Canadiens, numbers that are very close to my team effect estimate. There seemed to be more goaltender movement in the post-war years than in the early 1960s, which means that these estimates are probably more representative of the league competitive balance in the late 1940s and 1950s, a period where Chicago was consistently the worst team in the league.

These are just ballpark estimates to keep in mind when looking at older goalie statistics. This is also evidence of the dependence of goaltending statistics on team play, since the variance of team effects is much larger than the variance of goaltending play. The model predicts the results reasonably well for 4 out of the 6 teams, as well as for Chicago up until the 1960s. This would most likely not have been the case if there were drastic differences in goalie quality across the league. The difference between, say, a Gump Worsley and a Terry Sawchuk was certainly much, much smaller than the difference between the Red Wings and the Rangers. After looking at these numbers, I'm not sure there was much difference at all between many of the longtime starting goalies of the time period.

This is why evaluating goalies based on on wins and shutouts from that era is pretty pointless - instead of finding the best goalies, you will merely end up finding the goalies that spent the most time playing on the Canadiens, Maple Leafs, or Red Wings.

From High to Low: What Happens when Goalies Get Traded?

To continue the discussion on shots against per game and their effects on goalie play, I went through Hockey Reference's data banks to look at goalies who had been traded mid-season to see what the effects were on their statistics with their new team. I took every goalie in the save percentage era (since 1983-84) who was traded during the season and had played at least 500 minutes with both teams.

Here is the breakdown of how they did, broken down into three roughly equal groups by shot differential between the two teams:

	Low	Shot	Team	High	Shot	Team
Diff	Win%	GAA	Sv%	Win%	GAA	Sv%
0-2	.471	2.78	.902	.456	3.02	.896
2-5	.542	2.78	.900	.459	3.27	.893
5+	.510	2.93	.887	.375	3.63	.890

What does this tell us? The GAA differential increased as the shot differential did, which is to be expected since more shots against means more chances to allow goals. There was also a distinct relationship in terms of winning percentage for goalies going from low shot to high shot teams. Clearly it is much easier to win games on teams that are good at preventing shots against.

There was not, however, a direct relationship between save percentage and shots. The goalies with the largest shot differentials actually had similar save percentages on both teams, whereas goalies with a smaller differential did slightly better on the lower shot team. This is evidence that shot quality is not directly related to shots against, but varies on a team-by-team basis.

It is obvious that goalie results are very team-dependent. Save percentage is not perfect, but it is by far the best measuring stick for goalies, since the variance in save percentage was much lower than the variance in GAA or especially winning percentage. If we take the sample of goalies who faced a difference of 7 shots against or more after being traded, we can see this quite clearly:

Low Shot Team: .886, 2.94, .531 win%
High Shot Team: .886, 3.98, .320 win%

The exact same goalies stopped pucks at exactly the same average rate both before and after the trade, but they allowed over one goal more per game with one of the teams and had a difference in winning percentage of .211, which would be the equivalent of 35 points in the standings over an 82 game schedule. The moral of the story is that using straight GAA or winning percentage statistics to rate goalies is not a good idea. If you have to use unadjusted stats then use save percentage, but all goalie stats are dependent on the rest of the team.

There was one result that was a bit of a surprise, and that was the lack of variance in shutout results. For the group as a whole, the shutout rate on the lower shot teams was .048 per 60 minutes, and for the higher shot group it was .041. That is a difference of about half a shutout per season for a starter with a typical 60 game workload. That was not too surprising, but the underlying distribution was a little unusual. For goalies with either a low shot differential (0-2 shots difference) or a high shot differential (5+) there was very little difference in shutout rate. The goalies with the large differentials were actually very slightly more likely to record a shutout on the higher shot team. Only in the 2-5 shots per game range was there a large discrepancy, and it was in the expected direction (.048 on lower shot teams, .032 on higher). Sample size is an issue here, however, because shutouts are so infrequent. One or two goalies who managed to get lucky for a half-season in terms of shutouts could easily have skewed the overall totals.

My main beef with shutouts is that they are an arbitrary stat. Why do shutouts get all the hype, but one-goal games get no credit even though both of them nearly always result in a win? However, I am starting to think that there may be some useful information in shutouts. Certainly shutouts are still team-dependent to a large degree, but they are a dominance indicator. Great goalies on bad teams (e.g. Luongo in Florida) will still usually post a relatively high number of shutouts, especially compared to the other goalies around them. My expected shutouts ranking, which was designed to remove some of the team factors, seemed to pass the common sense test for the most part. I'd still prefer another more inclusive metric to shutouts (for example, the number of times a goalie allowed 2 goals or fewer, given that team winning percentages are very high across the board when teams gives up 2 or less, and also since 1 and 2 goal games are more frequent than shutouts they would likely be less subject to variance and more consistent from year to year), but that would require a lot of additional work to compile.

Sundin and Leadership

Is he the kind of of player, like Mark Messier, who can lead the Canucks over the hump?

Ha.

The only connection between Messier and Sundin is the "leadership" award Messier bestowed on Sundin during last season's playoffs. Talk about shams.

Scott Burnside of ESPN, proving that writers are just as incapable of separating team performance from individual star players as they are from goalies.

Burnside sounds like he buys into the Messier leadership mythology, and trashes Sundin for being a selfish choker. What is leadership, though, really? Carrying your team, correct? Bearing the scoring burden, driving results while on the ice? I'd guess most people would talk about inspiring your teammates and making great locker room speeches and all that as well, but I think most people would agree the best place to be a leader is on the ice, and when it comes down to it they would take the guy out there dominating the game over a mediocre player who happens to be a great motivational speaker and everyone's best friend.

I'm going to use Messier's results from his first stint as a New York Ranger for comparison purposes, since that was when his legend as the "Greatest leader in sports" really grew. Look at the playoff results, and Sundin was carrying the Leafs just as much as Messier was carrying the Rangers. Probably even more so. Sundin scored 15% of Toronto's playoff goals, Messier just 12% of the New York's. Messier had an edge in points, having a hand in 34% of the Rangers' goals compared to 32% for Sundin, but Sundin missed 17 games due to injury while Messier missed just 2. Messier's overall PPG rate was higher (1.14 to 0.92), but he did it in a higher-scoring era with better teammates and probably more ice time.

If we look at plus/minus, Sundin destroys Messier. Sundin was +7 in the playoffs for Toronto, when the team as a whole was -88. Messier was -9 on Ranger teams that combined for a total of -1. If we figure that an elite forward plays 1/3 of the game, we would expect their plus/minus total to account for roughly one-third of their teams total. That total includes the minutes played by Sundin and Messier, so we can do a quick and dirty estimate of the "off-ice" plus/minus for each player by multiplying their plus/minus by 5 (since there are usually 5 players on the ice for an even-strength goal) and subtracting that from the overall team total, then dividing by 2/3 to factor out their ice time. I thereby estimate that the Leafs were around -30 goals at even-strength without Mats Sundin on the ice, and +7 with him on it, and that the Rangers were around +4 goals without Mark Messier on the ice and -9 with him on it. That is a very quick and dirty method, the assumptions aren't really completely correct and it doesn't account for the special teams factors that are one of the biggest problems with plus/minus, but it appears that the Moose isn't even in the ballpark compared to Mats.

Based on the results, then, someone like Burnside would have to argue that Messier inspired everyone else around him to be much better, while Sundin just couldn't get the rest of his team going. Maybe that's true, I don't know (isn't that the coach's job?), but when it came down to leadership by example Sundin looks pretty elite.

Leadership is supposed to mean, "Player who has great intangibles, carried his team and helped his teammates", but I think a lot of times it ends up really meaning, "Player who had a lot of team success." That is even more likely to be the case in the opposite scenario, i.e. for a player like Sundin who did not have a lot of team success. Apparently it is pretty much out of the question to call them a great leader, no matter how well they actually perform, just because their teammates weren't very good.

I don't think the Canucks are up there with the Wings or Sharks in terms of Cup favourites, but they were my pick to win the division before they got Sundin and they could be a dangerous team in the postseason. Is there any reason to expect that someone like Mats Sundin would not succeed if they were placed in a playoff pressure situation? Of course not. As the above numbers show, he did a pretty good job carrying the Leafs through a few playoff series. His record in international best-on-best games is 18 goals, 21 assists, 39 points in 30 games, which prorated to an NHL season would be 106 points. Every season in every team sport star players without a history of team success pull a Peyton Manning and finally win a championship, yet all the non-thinkers who were writing about their lack of leadership and lack of winning attitude and poor clutch play right up until the champagne celebration never seem to learn their lesson.

Tim Thomas: How Long Will the Fairy Tale Last?

Tim Thomas has a .938 save percentage through his first 18 games. I am not one of those people who likes to trash Thomas and call him a fluke, because I think goaltending is a results business and style is more or less irrelevant (unless a goalie has a single, exploitable weakness or something like that). Breaking down a prospect's style will give you clues as to how likely they are to make it at the NHL level, but in my view a record of success at the pro level over multiple seasons trumps any negative scouting reports. Thomas may be a bit clumsy and awkward, but if he stops the puck then he stops the puck, and a career .916 save percentage suggests that he is no mediocre netminder. He also passes my standard litmus test, which is results compared to backups (2005-06 to present):

Thomas: 82-64-23, 2.70, .916
Backups: 44-48-15, 3.09, .897

Having said that, I don't think Thomas is a reincarnation of Dominik Hasek either, so how sustainable is that .938?

First off all, let's look at the backup goalie. Manny Fernandez is 9-2-1, 2.08, .924. I doubt Manny Fernandez is much better than average, so that suggests Boston has been very good in their own end this season. If we assume that Fernandez is really an average goalie (say, .910 or so) disguised by Boston's stalwart defence, then we can estimate the Bruins' shot quality against at about 15% better than average. Adjust Thomas' career average based on those results and you get .929.

Based on that estimate, it does seem reasonable that a strong year by Thomas could keep him up near .930 for the season. However, there are some warning signs:

1. The Bruins are leading a pretty charmed life right now, with all the percentages in their favour (11.5% shooting percentage, .933 team save percentage). Unfortunately, history has taught us that regression to the mean appears to be an unavoidable fact of life for hockey teams, and the smart money is that the Bruins will not continue either their scoring or save rates for too much longer.

2. Tim Thomas' penalty killing save percentage so far is .938. Needless to say that is leading the league. Thomas' ES save percentage is also very good (.941), but nobody can stop pucks at that rate forever when down a man. League average on the PK is usually around .870, and Thomas' PK save percentages for the last two seasons were .871 and .846. Maybe Boston has a particularly strong penalty kill this year, but I bet Thomas does no better than .900 on the PK, if even that, over the rest of the season, which would take some of the air out of his overall save percentage.

3. I don't think that 15% shot quality estimate accurately reflects Boston's defensive play. Julien is a good defensive coach, and the Bruins play a good team system, but that kind of shot quality would be elite. Fernandez has probably been either outperforming or lucky this season as well. Something like 5-10% better than average would probably be a more realistic estimate, which would suggest that Thomas should be closer to last year's .921 mark than somewhere above .930.

4. The Bruins are playing a soft schedule (Sagarin has them ranked 25th in schedule strength) in a weak division in the weaker conference. That's probably not likely to change much, but I just felt like pointing that out.

5. Thomas is facing 31.5 shots per game, compared to Fernandez' 27.3. I don't know if that is a statistical fluke or represents a real difference in on-ice play. Thomas has faced about 2 shots per game more than his backup goalies since the lockout, so he seems to be one of those types who have to make an extra save or two every game compared to average. If those shots represent more dangerous than normal chances (e.g. rebounds, turnovers, etc.) then he will be less likely to sustain a high save rate. For what it's worth, Hockey Numbers has Thomas facing more difficult shots than Fernandez so far.

Tim Thomas is not a fluke, but his season so far appears to be. I think it is probably fair to say Thomas is a good NHL starting goalie, but he looks like he is playing way over his head so far. I don't see his numbers going anywhere but down over the rest of the season.

Even if we expect him to cool off in the coming months, Tim Thomas does have that .938 and 1.96 GAA "in the bank" already, so to speak. If he regresses considerably over the next 52 games, he will still likely end up with some very good numbers this season. Let's say he plays 60% of Boston's minutes the rest of the way, faces the same number of shots per game (31.5), and stops 91.5% of them, which is probably pretty achievable given his career numbers and the team around him. His season numbers would then be 2.40, .924, which when combined with 30-35 wins and the injuries to Brodeur and Luongo could still put Thomas in contention for the postseason awards.

Thomas is an interesting case of a well-travelled guy who apparently had the light come on at the age of 29. In 2002-03 he was a journeyman AHLer (35 GP, .906), although he did show enough to get called up to play in 4 NHL games. Then out of nowhere came a huge AHL season in 2003-04 (.941 save percentage), a big year in Finland during the lockout (1.58 GAA), and then a season split between the AHL (.923) and NHL (.917) that finally solidified him at the NHL level. The late bloomer phenomenon is well-documented, and anybody who can find out just what makes these goalies suddenly put it all together will be able to command a very large salary from some grateful NHL organization.

Grant Fuhr and Clutch Play

The Hockey Summary Project is up and running, and it is a very interesting resource for hockey history, with box scores from many historical regular season and playoff games.

One thing I have wanted to check for quite some time now is how Grant Fuhr played in the third period of hockey games in Edmonton, to test whether his reputation of being "the guy who will never let in the next goal" matches reality (see here for round 1 of the debate). Unfortunately, not all the playoff seasons are posted yet (they are still missing 1988-1993), but we have 1983-1987 to at least begin the analysis.

For every Edmonton Oilers playoff game from 1983-1987, I took the goals for/against and shots/for against from the first two periods, and compared it to the goals and shots during the third period and OT. Here are the results:

Andy Moog: 2.88, .896 in first two periods, 3.38, .875 in 3rd & OT
Grant Fuhr: 3.00, .901 in first two periods, 2.74, .899 in 3rd & OT

So Fuhr did indeed allow fewer goals in the third period, but his save percentage also went down slightly. That suggests that the Edmonton Oilers, even in their most high-flying years, did actually play some defence when the game situation required it.

I broke down Edmonton's third period numbers with Fuhr in net, based on the score after 2 periods, to see how much of an effect the game score had on the results.

Score	P 1 & 2				Per 3
	SF	SA	SH%	SV%	SF	SA	SH%	SV%
Lead by 3+	39.8	29.7	20.1%	.944	31.0	26.3	20.4%	.861
Lead by 2	35.1	32.1	16.5%	.913	31.7	28.3	12.2%	.909
Lead by 1	34.4	29.1	11.0%	.921	33.0	28.2	15.0%	.919
Tie Game	30.5	28.1	9.8%	.893	35.1	26.1	14.6%	.926
Trail by 1	30.0	35.4	8.1%	.889	39.4	29.3	7.6%	.872
Trail by 2	31.5	35.5	6.3%	.859	31.0	26.0	12.9%	.846

In my view, it is incorrect to attribute the differences to "clutch play", as I think they are more reflective of the shot quality of both shots taken and shots allowed. The results when leading, tied and trailing were similar when Andy Moog was in net, suggesting that the numbers are being primarily driven by the rest of the team, rather than the goalie. When the game was tied or the Oilers were ahead, Fuhr had a save percentage of .909 in the third period. When the Oilers were behind, Fuhr had a save percentage of .862 in the third period. Part of this is a reflection of strength of opposition: the Winnipeg Jets were almost never ahead of the Oilers in the third period, for example. However, the first two periods were played against the same opposition, and yet there is a clear difference in the third period for games when the Oilers were tied or trailing by one goal.

The shots for and against numbers support these observations. When the Oilers were well ahead, they shut down their offence a bit in the third period. When the Oilers were behind, they far outshot the opposition. This is probably the combination of the opposing team taking fewer risks (shots against were down when the Oilers were behind) and the Oilers taking more discretionary shots (Edmonton averaged almost 40 shots per 60 minutes of play in third periods they entered trailing by a goal, yet their shooting percentage was just 8.2%). How about this for a surprising fact: the highest-scoring team of all-time was just 4-15 over the sample period in games which they trailed entering the third period.

Having said all that, there does appear to be some evidence that Grant Fuhr did well in important situations. His save percentage was highest when the game was tied. That may be somewhat expected, since that is when teams would generally play the most cautious, but that is still an impressive save percentage for 1980s hockey. His save percentage also was quite low when the Oilers had the game well in hand, just .861 with Edmonton up by 3 or more goals, which supports the perception that Fuhr let in softies when it didn't matter. He wasn't quite so good when the Oilers were behind, but this is likely when the shot quality against was at its highest, so he probably has a bit of an excuse for that. I still don't think there is any reason to call Grant Fuhr a great goalie or one of the best ever, but it looks like he did help Edmonton in the playoffs in the 1980s. Still, from the above table it looks pretty clear that Edmonton's shooting percentage and outshooting results were the main drivers of their success, rather than goaltending.

This is just a quick study, a better one would break down the results in more detail, looking at Fuhr's actual performance with Edmonton up a goal in the third period, not just how he did in third periods that Edmonton entered up by one. With their propensity to score quickly, there were some third periods that the Oilers blew open in the first few minutes, despite entering tied or holding a narrow lead.

One interesting bit of trivia: Between 1983 and 1987, the Edmonton Oilers apparently won 6 overtime games on the very first shot of OT. I'm not sure if there are mistakes in the box scores, but all of those games ended in less than a minute and a half so it is certainly possible. There was also one game when Edmonton lost on the first shot of OT. Looks like it wasn't a good idea to try to grab a hot dog before overtime started in Edmonton in the 1980s.

The Replacements

In the last couple of weeks, I've noticed a distinct lack of gleeful comments posted by Bruce giving New Jersey's goalie stats since Martin Brodeur went down. I decided to look into it myself to see just how New Jersey's replacement goalies are holding up.

Scott Clemmensen: 7-3-0, 2.19, .926
Kevin Weekes: 2-3-0, 2.86, .903
Backups Combined: 9-6-0, 2.42, .918

Martin Brodeur: 6-2-2, 2.16, .916

So far I'd say they are doing pretty well. It is a very small sample size to be sure, and it is certainly much too early to discount the possibility that Scott Clemmensen may just be playing way over his head for a couple of weeks. However, I am just as interested in the number and type of shots Brodeur's replacements are facing, rather than simply whether they are stopping them or not. I cruised over to Hockey Numbers to see what his shot-quality calculations are telling us about New Jersey's goalies. SQN% is shot-quality neutral save percentage, and SQI is shot quality index (1.00 is average, below 1.00 means a team allows easier than average shots).

Brodeur: .914 SQN%, 0.98 SQI, 25.7 SA/60
Clemmensen: .918 SQN%, 0.90 SQI, 29.5 SA/60
Weekes: .892 SQN%, 0.90 SQI, 29.6 SA/60

Clemmensen and Weekes are very similar in their underlying numbers, facing almost identical shot quality and quantity. The difference is that Clemmensen is making more saves.

What really stands out, however, is that Brodeur has faced 4 fewer shots per game than his backups goalies have. Is this finally evidence of his soft goaltending skills as a third defenceman out on the ice, or are there other factors at play? In hockey, there are pretty much always other factors at play. If we look at the shot quality numbers, Clemmsen and Weekes have faced shots that were estimated as being 10% easier than average. Brodeur's shots were only 2% easier than average. This means that while New Jersey has allowed more shots without Brodeur in net, the extra shots faced have been apparently much easier to stop. If the primary reason for the difference was Brodeur's impact on puck possession, I'm not sure that we would expect to see a difference in shot quality.

Obviously we need to track shots and shot quality over a larger number of games to see if these differences are fluke or reality (and there are a number of factors that make shot quality less than perfectly reliable, such as reporting bias, failure to consider shot angle, etc.), but there are two possible explanations that do come to mind that could explain these results (maybe New Jersey fans can weigh in if either of them seems reasonable). The numbers suggest that either opposing teams are shooting from everywhere to test Clemmensen and Weekes, or New Jersey has changed its defensive style of play to protect their goalies which has resulted in allowing a higher number of lower quality shots.

We can try to quantify the goal prevention effect from the difference in shots allowed, by estimating the expected goals against by an average goalie facing Brodeur's shot distribution and then comparing that to his backups. The league average so far is .907, so let's go with that as our baseline number. We can adjust that for the shot quality for each goalie, and then multiply that by number of shots actually faced to get an expected GAA.

Here are the results:
Brodeur: 2.33
Clemmensen: 2.48
Weekes: 2.47

Through an expected goals approach, Brodeur's 4 fewer shots per game translate into a GAA effect of -0.15 goals per game. If we want to try to express that gap in terms of save percentage, it would be the equivalent of +.005 in save percentage for a goalie with a league average save percentage facing league average shots. We don't know at this point whether Clemmensen and Weekes are better or worse than average in terms of shot prevention, so it isn't necessarily correct to attribute the entire gap to Brodeur.

I don't know the typical starter/backup split in terms of shot quality, especially in this type of situation where lightly regarded backups replace an All-Star. However, there is a very similar situation going on in Vancouver, so I'll bring that in as a point of comparison. Here is how Luongo has done compared to his replacements by all the same metrics as above (SQN% = shot-quality neutral save percentage, SQI=shot quality index, expGAA = expected GAA for a league average goalie facing the same shots):

Luongo: 2.17, .928, .930 SQN%, 30.2 SA, 1.03 SQI, 2.89 expGAA
Sanford: 2.85, .905, .897 SQN%, 30.0 SA, 0.92 SQI, 2.57 expGAA
Schneider: 2.80, .896, .878 SQN%, 26.9 SA, 0.85 SQI, 2.13 expGAA

Backups: 2.83, .902, .891 SQN%, 28.9 SA, 0.90 SQI, 2.41 expGAA

The combined shot quantity and quality for Luongo's backups is very similar to Brodeur's (same shot quality against and a difference of less than one shot against per game). Just like Brodeur, Luongo has apparently faced more difficult shots, but Luongo has also faced more of them (1.3 extra shots per game than his backups). Luongo's expected GAA is actually 0.48 goals above that of his backups because of these factors, yet his actual GAA is 0.66 lower. According to the numbers, Vancouver has been hurt a lot more by goalie injuries than New Jersey has.

What about evidence that Brodeur affects his teammates, or that his puckhandling contributes to his team's offence? I went to Time on Ice to check out the possession statistics while each goalie was in the game (using Behind the Net's numbers to estimate minutes played at 5 on 5):

Brodeur: +8.4 Shot Diff/60, +7.9 Corsi/60, +12.1 Fenwick/60
Backups: +1.8 Shot Diff/60, +4.9 Corsi/60, +5.4 Fenwick/60

Luongo: -2.6 Shot Diff/60, -3.4 Corsi/60, -4.7 Fenwick/60
Backups: -2.9 Shot Diff/60, -4.2 Corsi/60, -7.5 Fenwick/60

Both goalies have better outshooting results than their backups do. New Jersey is obviously a better outshooting team than Vancouver, but Brodeur outperforms his backups by a larger margin than Luongo. Vancouver blocks a similar ratio of shots no matter who their goalie is, while New Jersey's shot block to total shot attempts against rate is 3% higher with Clemmensen or Weekes in net than with Brodeur.

One factor at work with the possession stats could be that Luongo has been in the lead much more often than his backup goalies. That may have led to more shots against, since there is some evidence to suggest that trailing teams are likely to shoot more. Vancouver tends to reduce their offensive pressure when they are leading, focusing instead on preserving the lead. New Jersey likely usually uses a similar strategy, but New Jersey's backups have a better record than Vancouver's so this may not have affected them as much.

It's pretty early to conclude anything, but there are a few observations to be made. New Jersey is a good team, and both the shot quality numbers and Scott Clemmensen's stats suggest they are still a very good defensive team. Playing in New Jersey helps a goalie's statistics, of that there is little doubt. However, the interesting numbers are the ones that relate to Martin Brodeur's impact on his team's play beyond stopping pucks (e.g. shots against and puck possession stats), and that is certainly something to follow along with as the season goes on.

Durability Matters...Except When It Doesn't

Wendel Clark: 793 games played
Wendel Clark's teams: 1,186 games played
Participation rate: 66.9%

Dominik Hasek, from the time he became a starting goalie in 1993-94 to the end of his career: 682 games played
Dominik Hasek's teams over the same time period (excluding retirement season in 2002-03): 1034 games played
Participation rate: 66.0%

Ask your average Leaf fan about those two players, and one of them will be the biggest injury risk who ever played hockey while the other descended directly from the heavens via Kelvington, Saskatchewan.

NHL vs. CHL

When doing a bit of research for my last post about Canadian world junior team goalies, I took a quick look into CHL stats across the three major junior hockey leagues. For me, it was a bit of a reminder that the NHL results may not necessarily be representative of all levels of hockey.

Take, for example, shots against vs. save percentage, a common theme in this space. There is a vocal group that repeatedly maintains that those two things are positively related, and that as shots against go up save percentage necessarily also goes up. This is supposedly either because shots against the run of play are more dangerous, or because it is easier for a goalie to concentrate when he faces more shots. There have been some seasons at the NHL level where the data seem to suggest this is a possible relationship, including this season so far where James Mirtle calculated a 0.49 correlation coefficient between shots against per game and save percentage.

However, if these arguments are correct they should be generally true for all levels of hockey, including junior. Here are the correlation coefficients between shots against per game and save percentage for the past 3 seasons:

Ontario Hockey League:
2005-06: -.269
2006-07: +.078
2007-08: -.278

Western Hockey League:
2005-06: -.219
2006-07: -.347
2007-08: -.214

Quebec Major Junior Hockey League:
2005-06: -.112
2006-07: -.180
2007-08: +.266

Not much support for the "more chances = better save %" theory. The Western Hockey League is the league that tends to produce the best defensive teams with the strictest defensive systems, and that league has the strongest relationship between save percentage and shots against. The numbers indicate that playing on a strong defensive team in junior hockey probably helps your save percentage. This suggests that goalie prospects who had a big advantage in juniors because of their defensive teammates (like Justin Pogge, Leland Irving, Jeff Glass and Tyson Sexsmith) might turn out more like Kelly Guard than Carey Price.

Tyler Dellow posted recently on save percentage with certain players on the ice, and concluded that it appears to be mostly driven by randomness. Players with unusually high or low save percentages will regress to the mean as the season goes on. His conclusions make sense for the NHL level, but I would like to see similar numbers for junior hockey players, to see if the dominant players at that level had an effect on their own team's save percentage. It would also be interesting to see the numbers from an era of the NHL with a different level of competitive balance (the 1970s, for instance). In the 1970s, the save percentage-to-shots-against relationship was very similar to the junior results above, i.e. a weak negative correlation. This relationship inspired hockey analysts Klein and Reif to come up with the goalie perseverance rating, a rating that penalized goalies on low-shot teams and gave bonus points to goalies who faced more rubber, in their influential Hockey Compendium (published in 1986). Their weighting system did not stand the test of time, however, as hockey's competitive landscape changed (see the comments to this post for a discussion of how Klein and Reif's results no longer hold in the current NHL).

I think that bad hockey teams generally give up high shot quality against, and good hockey teams generally give up low shot quality against. It is only in a league with a good competitive balance (such as the NHL) that other factors come into play and affect the result, for example because the outshooting team takes longer shots or more of their shots come from their third- or fourth-line players, or possibly the players are playing a specific offensive or defensive system, or some other similar reason(s).

My overall point is that I'm not sure whether all of the statistics-based conclusions that are being made these days represent essential hockey truths, or whether we are merely collecting evidence of the high degree of parity in today's NHL. It might be wise now and then to test out conclusions with results from a different league or era to see if the findings still hold. Unfortunately, the lack of data available does not always make this possible.

Fire the World Junior Goalie Scouts

I simply cannot believe Tyson Sexsmith was invited to Team Canada's world juniors selection camp. I can't understand how a goalie who is so obviously a product of his team has the potential to be representing the best hockey country in the world in an international tournament. Here are three stat lines: Sexsmith over the last 3 seasons, his backup goalies over the last 3 seasons, and the numbers of the Vancouver Giants starting goalie who preceded Sexsmith from 2005-06. See if you can spot the resemblance:

1.89 GAA, .911 save %, .742 win %
1.89 GAA, .910 save %, .731 win %
1.90 GAA, .912 save %, .713 win %

Team Canada often seems to do this for these tournaments, picking goalies off the best defensive teams at the expense of guys that are better but have inferior teammates. Chet Pickard and Dustin Tokarski are by all accounts pretty decent goaltenders, but doing the same exercise indicates that they also have the advantage of playing on great defensive teams. Here is the combined stat line this season for the backup goalies of the 3 WHL goalies invited to Canada's selection camp:

18 GP, 14-0-2, 1.63, .933, 3 SO

Let's just say I don't particularly trust our nation's junior scouts in terms of separating the goalie from the team.

But worst of all, Sexsmith's invitation to the camp was sent at the expense of the best junior goalie in Canada. For some reason Belleville's Mike Murphy never got a call. Let's look at how Murphy compares to his goalie teammates over the last two years, as well as compared to his predecessor in the net in Belleville:

Murphy: 2.15 GAA, .935 save %, 53-10-8, 5 SO
Backups: 3.35 GAA, .895 save %, 15-11-1, 2 SO
Previous: 3.01 GAA, .919 save %, 27-17-3, 3 SO

It is pretty hard to look at those numbers and not conclude that this guy is a difference-maker. Murphy led the OHL in save percentage last year (.929) and is leading it again this year by a wide margin (.944). Despite facing over 35 shots per game, Murphy is still leading the OHL in GAA, which is remarkable. Meanwhile, Tyson Sexsmith faces less than 22 shots per game and only ranks 3rd in the WHL in GAA (and just 15th in save percentage).

Apparently last year Sexsmith was rated the most overrated player in the WHL by a wide margin, which makes sense. However, I guess none of those people were involved in picking this year's squad. Hopefully Canada's coaching staff decides to go with some combination of Tokarski, Pickard or fourth option Jake Allen on the basis that they are far more deserving of the honour than Sexsmith.

The Luck of the Draw

I was looking at some historical results, and found something interesting about the career shutout record held by Terry Sawchuk. The main reason that Sawchuk holds the record (and that he held the career wins record up until Patrick Roy broke it) appears to be that Jack Adams was a better goalie evaluator than Dick Irvin.

Terry Sawchuk and Jacques Plante were both born in 1929. Sawchuk became a Red Wings prospect, while Plante's rights were held by Montreal. Sawchuk had Harry Lumley ahead of him in the organization, but Jack Adams realized that both of them were good goalies and that he could maximize value by trading Lumley, who was coming off of a Stanley Cup winning season. As a result, Terry Sawchuk found himself starting in the NHL at the tender age of 21 for the best team in the league.

Jacques Plante, on the other hand, found himself stuck behind Gerry McNeil, a middling goaltender who rose to the starting job because Bill Durnan retired. McNeil looked good for a few seasons behind the Montreal Canadiens' strong defence, but after Plante took his job McNeil would find himself essentially out of the NHL at the age of 28. Plante first got NHL playing time during the 1952-53 season, but Montreal continued to give McNeil the vast majority of the starts even though Plante put up outstanding numbers: 2-0-1, 1.33 in 1952-53 and 7-5-5, 1.59 in 1953-54. Not only that, but Plante was stealing playoff starts from McNeil as well and dominating with his opportunities. Plante went 3-1 with a 1.75 GAA in the 1953 playoffs, and 5-3, 1.88 in 1954 where he finally won the starting job from McNeil for good. However, by the time he had taken over the reins, Plante was already 26 years old.

Before the age of 26, Terry Sawchuk already had 199 wins and 57 shutouts in the regular season, as well as 28 playoff wins and 3 Stanley Cups. It was this starting advantage that allowed him to edge out Plante in most of the regular season career categories, as over the rest of their careers most observers would agree that Plante was the better goaltender. From age 26 on, Terry Sawchuk's career stats were 248-249-107, 2.82, 46 SO (and 26-33, 3.05, 3 SO in the playoffs). If Sawchuk had those results as his career numbers, he'd probably be remembered together with guys like Harry Lumley or Gump Worsley, rather than Plante and Glenn Hall.

Montreal should have followed Detroit's lead by dumping McNeil and starting Plante in 1951-52. Even if Plante had simply matched McNeil's numbers from 1951 to 1954, he would have far surpassed Sawchuk with 524 wins and would have equalled Sawchuk's mark of 103 shutouts. In all likelihood Plante would have beaten the shutout record as well, and would probably be unanimously rated ahead of Sawchuk in the all-time debates.

This is one of the problems with the frequent focus on career totals to evaluate goalies - some goalies break in earlier than others, often through no special abilities of their own but merely by the luck of the draw in terms of who was above them in the organization, whether or not the coach or GM recognized their talent, or whether or not they were on a hot streak or a slump in training camp when the team was looking to evaluate its goaltenders. In general better goalies will break in earlier, but this is not always the case. There is some luck involved in career length too, based on a goalie's health, league expansion, the strength of other goalies on the team in the pre-free agency era, etc. These reasons suggest we should place more weighting on a goalie's peak than on his entire career, as this helps us avoid being overly influenced by some of those extraneous factors that can have a big effect on a goalie's legacy despite reflecting little of their actual skill or abilities.