Tuesday, December 30, 2008

Team Effects in the Original Six

Original Six goalies are very difficult to evaluate, because of the extreme team effects. The best talent was usually concentrated on a couple of the best teams, and the goalies on those teams not only had the advantage of playing behind a great team but they also never had to face their own team's elite goal scorers.

It is very difficult to estimate these effects, however, because most teams gave all or nearly all of their minutes to a single starting goalie. Glenn Hall, the most extreme example, played 503 games in a row at one point, so we don't even have a single game's worth of results for any other goalies on his team for a full 7 year stretch. That makes it impossible to use the method of comparing results to backup goalies. There was also less freedom of movement, so goalies didn't change teams as often.

These limitations mean that any method used is going to be less than ideal, but focusing on the goalies who changed teams and trying to estimate the team impact seemed like the best option. I decided to look at all the goalies that changed teams in the post-WWII Original Six era (1946-47 to 1966-67). The goalies were: Glenn Hall, Johnny Bower, Jacques Plante, Gump Worsley, Emile Francis, Hank Bassen, Frank Brimsek, Terry Sawchuk, Al Rollins, Jim Henry, Don Simmons, Bruce Gamble, Cesare Maniago and Harry Lumley.

I figured out the cumulative winning percentage and GAA for each goalie on all of the different teams they played for, and then tried to estimate each team's relative rank based on the differences.

Of course it is unrealistic to assume that every team had the same relative strength over a 20 year period. Montreal and Toronto were consistently good, while Boston and the New York Rangers were pretty mediocre, but Detroit was a powerhouse in the 1950s and terrible in the 1960s, while Chicago was exactly the reverse. Another problem is career arc - someone like Glenn Hall spent his entire prime in one place, and the only points of comparison we have of him somewhere else are either as a very young goalie or as an old one. The final problem was sample size: there were a few team pairings that didn't have a single goalie play on both of them. It was particularly difficult to evaluate Montreal goalies, since there were really only two goalies who played a lot of games in both Montreal as well as somewhere else, and both of them happened to also play for the Rangers (Plante and Worsley). There were also a few well-travelled goalies (like Harry Lumley, who played on 4 out of the 6 teams) that ended up having a larger effect on the sample.

However, despite these limitations, the numbers seemed to validate the method through a reasonable degree of consensus. For example, if you compare the goalies that played in both Boston and Toronto, they had a GAA in Toronto that was 0.42 better and a winning percentage .084 higher compared to Boston. If you used the Chicago results to verify this (by looking at the goalies who played in both Chicago and Boston, and comparing those results to the goalies who played in both Chicago and Toronto), the estimate was that the Leafs were 0.34 better in terms of GAA and .092 in winning percentage. Using the Detroit comparisons, it came out to 0.51 and .037. We can therefore ballpark the expected effect of getting traded from Boston to Toronto as being something like 0.40 - 0.50 in GAA and .070 - .090 in winning percentage.

I took averages from several of these comparisons, and came up with a relative set of rankings:

1. Montreal: 0.00 GAA, 0.000 win %
2. Toronto: 0.00 GAA, -0.040 win %
3. Detroit: +0.15 GAA, -0.025 win %
4. Boston: +0.40 GAA, -0.115 win %
5. Rangers: +0.70 GAA, -0.185 win %
6. Chicago: +1.00 GAA, -0.245 win %

If we compare these numbers to the actual results, we can both verify them and see which teams apparently had strong or weak goaltending:

1. Montreal: 2.36 GAA, .600 win %
2. Toronto: 2.51 GAA, .535 win % (+0.15 GAA, -0.065 win %)
3. Detroit: 2.54 GAA, .561 win % (+0.18 GAA, -0.039 win %)
4. Chicago: 3.04 GAA, .441 win % (+0.68 GAA, -0.159 win %)
5. Rangers: 3.08 GAA, .430 win % (+0.72 GAA, -0.170 win %)
6. Boston: 3.09 GAA, .432 win % (+0.73 GAA, -0.168 win %)

The total results confirm that Montreal, Toronto and Detroit were the three front-runners, with similar GAA totals. Montreal likely did have somewhat better goaltending than the Leafs or Wings, but the main reason the Canadiens had more team success was probably not goaltending but superior offensive play. The model predicts the Rangers quite well relative to the Canadiens, which suggests that despite often being a bottom-feeding team the Rangers got decent performances from the goalie position. Boston, on the other hand, appears to have had weak play in net, since they allowed the most goals of any team but apparently had a better defensive environment than either New York or Chicago.

Chicago's comparative results are exaggeratedly poor because, as previously noted, Glenn Hall was the only guy in their net during the early to mid-1960s. If we take Chicago's results prior to Hall's arrival in 1957-58, the Blackhawks' cumulative goalie stats were 3.46, .345, which means they were 1.10 and .255 worse than the Canadiens, numbers that are very close to my team effect estimate. There seemed to be more goaltender movement in the post-war years than in the early 1960s, which means that these estimates are probably more representative of the league competitive balance in the late 1940s and 1950s, a period where Chicago was consistently the worst team in the league.

These are just ballpark estimates to keep in mind when looking at older goalie statistics. This is also evidence of the dependence of goaltending statistics on team play, since the variance of team effects is much larger than the variance of goaltending play. The model predicts the results reasonably well for 4 out of the 6 teams, as well as for Chicago up until the 1960s. This would most likely not have been the case if there were drastic differences in goalie quality across the league. The difference between, say, a Gump Worsley and a Terry Sawchuk was certainly much, much smaller than the difference between the Red Wings and the Rangers. After looking at these numbers, I'm not sure there was much difference at all between many of the longtime starting goalies of the time period.

This is why evaluating goalies based on on wins and shutouts from that era is pretty pointless - instead of finding the best goalies, you will merely end up finding the goalies that spent the most time playing on the Canadiens, Maple Leafs, or Red Wings.

19 comments:

Bruce said...

Interesting stuff, CG. That was a goaltenders' league, with many more great goalies than weak ones. The Plante-for-Worsley trade would be an interesting one to analyze to support your case. Worsley got a lot "better" after he donned the bleu, blanc et rouge, while Plante's stats plummeted.

Detroit was a powerhouse in the 1950s and terrible in the 1960s, while Chicago was exactly the reverse.

Agreed on Chicago, but the Wings went to the Stanley Cup Finals in '61, '63, '64 and '66 and won the Prince of Wales Trophy in '65. They certainly weren't a powerhouse, but "terrible" seems a strong word. Are you referring specifically to their defensive record?

1. Montreal: 2.36 GAA, .600 win %
2. Toronto: 2.51 GAA, .535 win % (+0.15 GAA, -0.065 win %)
3. Detroit: 2.54 GAA, .561 win % (+0.18 GAA, -0.039 win %)
4. Chicago: 3.04 GAA, .441 win % (+0.68 GAA, -0.159 win %)
5. Rangers: 3.08 GAA, .430 win % (+0.72 GAA, -0.170 win %)
6. Boston: 3.09 GAA, .432 win % (+0.73 GAA, -0.168 win %)


The division between top three and bottom three is pretty stark, isn't it? Looking at the distribution of Stanley Cups during those 21 years, we have:

Montreal 8
Toronto 8
Detroit 4
Chicago 1
New York 0
Boston 0

... with Toronto tied for first by virtue of the years you selected; the Leafs won the Cup in the first and last years of the sample, with Montreal winning just before and just after. The same three dominated the Prince of Wales Trophy during the same era, in fact of the 25 years of the "Original" 6 one of those three took first place every season until the very last one, when Chicago finally beat the Curse of Muldoon.

James Benesh said...

As always, I applaud your work. Nothing too surprising here. There are sv% and shots against totals out there for 1954-1967. Wouldn't those tell us a lot more about the performances of the goalies of this era than this exercise?

The Contrarian Goaltender said...

James: Maybe, but not neccessarily. Save percentages, like all goalie stats, are affected by the defensive play in front of the goalie and in the Original 6 the difference in defensive strength was apparently quite large. The main goal here was really to try to assess the team strengths rather than rate the goalies.

But that is a good idea to combine this with the save percentage numbers. Here are the cumulative save percentages by franchise from 1954-1967:
1. Montreal .916
2. Toronto .915
3. Chicago .913
4. Detroit .908
5. Rangers .908
6. Boston .899

I'd say that confirms that Boston was lagging behind everyone else, but for the rest of the teams the results are close enough that strong team factors could easily change the order.

If we take the shots per game numbers for the same period, and then combine that with my expected GAA numbers, we can calculate an "expected save percentage".

1. Toronto .918
2. Montreal .917
3. Detroit .914
4. Rangers .910
5. Boston .905
6. Chicago .903

Now this won't be totally accurate because I am mixing a 1947-1967 sample with a 1955-1967 sample, but I think it gives an idea of how the team effects would also impact save percentage. We can't just look at Jacques Plante or Johnny Bower leading the league in save percentage and declare them to be the best goalies - we need to take into account the fact that they were playing on Toronto and Montreal and therefore should have been at or near the top of the league in save percentage.

I can attempt to deal with the non-matching sample problem by estimating save percentage and shots for the early 1950s. I used 1955-1958 to project backwards, and came up with these adjusted results:

1. Montreal: .915 exp, .917 act
2. Toronto: .916 exp, .912 act
3. Detroit: .913 exp, .912 act
4. Rangers: .911 exp, .911 act
5. Chicago: .904 exp, .910 act
6. Boston: .902 exp, .903 act

No matter what way you look at it, the numbers seem to show the same thing - goaltending likely wasn't a huge difference maker in the O6 era. This was not because the goalies weren't very good - they were. It was simply the case that every team had a top goalie. As a result there was little advantage to be had, and team defence was the main thing that drove results.

overpass said...

Thanks for this, it's very interesting.

1. Montreal: .915 exp, .917 act
2. Toronto: .916 exp, .912 act
3. Detroit: .913 exp, .912 act
4. Rangers: .911 exp, .911 act
5. Chicago: .904 exp, .910 act
6. Boston: .902 exp, .903 act


It looks like Chicago is the one team that significantly outperformed expected SV%. That suggests that Glenn Hall was the best regular season goalie of the era - which agrees with his being voted a first-team all-star seven times.

It also suggests that Terry Sawchuk was not as good as his early career numbers on a Detroit dynasty, and Johnny Bower can thank Punch Imlach and a terrific Toronto defence for his (retroactive) SV% titles.

The Contrarian Goaltender said...

I'm still a bit hesitant to make too many conclusions based on this evidence, especially regarding Chicago. I think Chicago's expected save percentage is likely a little too low, because most of the inputs into my relative rankings were from earlier on, i.e. pre-1960, when the Hawks were a lot weaker.

There are definitely some error bars that should be put around those numbers, and it wouldn't require too much variation on some of the estimates for the rankings to switch around. At this point, I think we can only conclude that it was easier to play in Montreal/Toronto/Detroit and tougher to play in New York/Chicago/Boston, and that the variability in goaltending from team to team was pretty low, since it closely tracks the expected numbers. If the Hockey Summary Project gets that far I'd definitely like to update the numbers with the actual save percentages.

The Manic Ranger said...

When do you think Lundqvist is finally going to get some credit and win a Vezina? Fourth times a charm

Bill Morran said...

I was looking through your post about "the Curious Case of the 1980s" and was wondering if your 1990s rankings took into account that Ed Belfour's backups turned into Dominik Hasek, Jeff Hackett, Evgeni Nabokov, Manny Fernandez, and Marty Turco. Every one of them turned into an NHL starter, and in the case of Turco and Hasek, very good ones. I was wondering is that sort of thing will ever be taken into account when using your vs backup stats. Not that I dislike the backup stats, in general, it makes sense, but Ed Belfour has a very noted, unfortunate history with his backups.

The Contrarian Goaltender said...

Bill: Yes, there was an adjustment factor for backup quality, but I'm not completely happy with it. I also noticed that Ed Belfour played with a lot of good goalies. He even got pushed out of town by two of them (Hackett and Turco), which is not something that is likely to happen to somebody who was playing with replacement level backups. I think it is probably reasonable to assume that Belfour may be at least somewhat underrated by the backup comparison method.

Bill Morran said...

I think it's safe to say that there's no fair system, but that's why we don't look at one statistic blankly and judge a player. Several other posts you've made, mostly including playoff numbers, have Ed Belfour shining next to his contemporaries.


Just another thing you may be interest in. I am in the middle of writing a book about statistic based hockey. I've really been laying into Grant Fuhr, who's entire legacy really bothers me, because there's no statistical evidence he was even really the caliber of an NHL starter, let alone a Hall of Famer.

To prove that Grant Fuhr had nothing to do with the Stanley Cup wins, using the playoff year in which Fuhr faced the most shots, if you took the worst save percentage, and the best save percentage in the regular season in 1985, on the Oilers in the playoffs, the leader (based on a minimum of 25 games) and the last place goaltender (again, minimum of 25 games), the worst goalie would only allow 1.5 more goals per game. In games the Oilers won that playoff year, their average margin of victory was more than three times that.

Really, no matter when those goals were let in, the Oilers had the Cup won.

Anonymous said...

i dont think you mentioned how badthe rangers were enough

Christopher said...

i think 2 underrated factors at work when comparing backup to starter are also:

A) Backups are more likely to be deployed vs. weaker teams.

B) a team may play a more conservative game knowing that a weaker goaltender is in net.

Both of these would serve to make the #'s a bit closer dont you think?

Bill Morran said...

@ Chris: But don't you think most goaltenders would have the same disadvantage?

Christopher said...

Bill, excellent point. I'm more referring to how often it'll be referred that certain goaltenders arent that great due to being "not much better than their backups" when they probably were. Of course, the best in this case would seem even better in comparison. I guess one thing is clear, that we may be seeing the best case for this site in the Scott Clemmensen era haha

Bruce said...

there's no statistical evidence he was even really the caliber of an NHL starter, let alone a Hall of Famer.

Bill: That's just ridiculous.

To prove that Grant Fuhr had nothing to do with the Stanley Cup wins, using the playoff year in which Fuhr faced the most shots, if you took the worst save percentage, and the best save percentage in the regular season in 1985, on the Oilers in the playoffs, the leader (based on a minimum of 25 games) and the last place goaltender (again, minimum of 25 games), the worst goalie would only allow 1.5 more goals per game. In games the Oilers won that playoff year, their average margin of victory was more than three times that.

Are you saying that the Oilers' average margin of victory was four and a half goals??? You'll be needing to improve either your statistical analysis skills and/or your writing skills before finishing that book.

Really, no matter when those goals were let in, the Oilers had the Cup won.

It sure as hell DID matter. For the record, Grant Fuhr was chosen the first star of the first four playoff games in 1985, including the entire first series against the Kings. I attended those (home) games and intently watched the road games on the tube and can tell you that I agreed with those selections; Grant played brilliantly as the Oilers gained traction in those playoffs. Later they started to fill the net and won a bunch of 8-3 games, but they struggled out of the gate, allowing lots of shots (107 in 3 games against the Kings) and not putting a lot of goals on the board (discounting an empty netter, just 8 in regulation, far below their usual levels). Fuhr held them in when they needed him most, and was mentioned right to the end of those playoffs as a Smythe candidate along with his teammates Gretzky and Coffey (who both set records that year that still stand). In fact he was a strong Smythe candidate in each of the four Cups that Oilers won when he was a starting goalie, and likely deserved to win at least one of them over those years (1987 IMO). Ultimately Smythe discussion is just opinion too, but it was the opinion of people who actually watched the games at the time, which it is painfully apparent you didn't.

Remind me not to buy your book.

Anonymous said...

The 1984-85 Kings finished 13th out of 21 teams, with 82 pts in 80 games. (When adjusted for quality of opposition, they had an "SRS" of 0.23, which was 11th overall.)

The '85 Kings weren't exactly an elite team.

Bruce said...

Anonymous: So what's your point?

Mine is that goaltending won that series for the Oilers. And if they hadn't won that series, they probably :) wouldn't have won the Stanley Cup.

Inferior clubs win in the playoffs all the time, as the Kings had proven in their previous meeting with the overdog Oilers. In '85 Grant Fuhr was the main man responsible for that not happening again. Even though the Oilers "had the Cup won", they still needed a key performance from their Hall of Fame goaltender. And they got it.


(Word verification: "ablest")

Anonymous said...

My pt is that keeping your team in the game(s) against a team that is barely above .500 is probably not as significant as doing so against, e.g., the 75-76 Canadiens (Tretiak, New Yrs Eve).

Agent Orange said...

Bruce

Do you have boxscore data for the playoff games?

I want to confirm for myself the shots against Fuhr in those Kings games.

Based on you comment of 107 shots in 3 games that is an average of just under 34/60. This is pretty close to Fuhr regular season but would mean in the rest of the playoffs teams averaged ~28 shots against.

These would be better teams than the kings that got 3 wins.

I did a quick check and it looks like the oilers margin of victory was ~3.0 gpg in wins only. This was inflated by some big wins against chicago.

Please let me know if you have a source for playoff box scores.

Host PPH said...

There are talent and teamwork. I have to admit that Teamwork can exceed the first one.