Sunday, August 19, 2007

The Best Regular Season Goalie of All-Time

I have completed a comprehensive study of every goalie who has played 10,000 or more minutes in the NHL since World War II. I measured their performance in terms of GAA in each season against the other goalies on their team. This was done because goalie results are strongly impacted by the team around them. Goalies who cannot outplay their teammates are not very valuable to their teams, since their performance can be easily replaced. Therefore, the best goalies should be ones who outperform their teammates the most, after adjusting for controllable factors like era, minutes played, strength of backup goalies, etc.

All GAA results were adjusted to league average to allow comparisons across eras. I adjusted for the strength of their teammates with an adjustment factor based on their career length, performance and reputation. I ignored seasons where the goalie played less than 120 minutes, or his backups played less than 120 minutes, because those samples are too small to be meaningful yet could easily skew the results. I took the teammate goalies' combined performance and calculated how well they would have been expected to do, given the starters' minutes. I also further adjusted the starter and teammates' performances based on the number of minutes played, given that the more minutes played, the greater the statistical significance of the performance.

That left me with an adjusted starter GAA, and an adjusted backup or teammate GAA. Each goalie was then ranked by how well they performed relative to their teammates' adjusted performances. I also calculated these results for each goalie's five best consecutive seasons, i.e. their career prime. The results are described below.

There are a few limitations to this study I would like to outline here:

The adjustment factors for each goalie's backups were subjective. Changing them does not change the results substantially, but it would likely impact the rankings. I found a lot of goalies that I considered to be good based on results and reputation were indicated to be not so good by the results, and vice versa. Perhaps I will go back and revise the factors based on some of my new findings and see how the results change.

Also, it was more common in the 1940s and 1950s for a single goalie to play all of his teams games. Goalies like Glenn Hall, therefore, are underrated in this analysis because in their best seasons they did not have any backup results to compare with, so those years unfortunately could not be considered.

The way a team conducts its goalie rotation can have an impact on the results. For example, some teams only play their backup against weaker opponents, while others essentially platoon. It would be incredibly time-consuming to adjust for opponent on a game-by-game basis for every goalie, so this has been left out. I think it is common for backups to play weaker opponents, so any adjustment would likely be in a similar range for most of the goalies, but it could be that there are some goalies who were advantaged or disadvantaged because of difficulty of opposition.

Another adjustment I would have liked to have made was to introduce an age curve. Several goalies, such as for example goalies who are still active, rank much higher than they should because they do not have a decline stage, a number of games late in their career when they are no longer playing at their peak. Such an adjustment would not penalize goalies with longer careers.

On to the findings:

The best regular season goalie ever: Dominik Hasek. Hasek ranked first overall and had the best prime. Hasek was 29.9% better than his backups in terms of adjusted GAA, and 41.5% better during his prime. Given that Hasek came over to the NHL late and missed out on a number of potential seasons in his mid-20s, and is still active beyond the age of 40, for him to take first place on this list is very strong proof that he is the best goaltender to ever play NHL hockey, at least in the regular season.

Second place was a surprise: Al Rollins. Rollins took second both overall (27.4%) and prime (37.0%). Rollins toiled on weak Chicago teams in the 1950s, but was clearly a strong goaltender, as shown by his Hart Trophy award in 1954. Rollins did not have a long career (9 seasons), and is not in the Hall of Fame. However, these results indicate that his only limitation was his team, and that his performance was outstanding.

In third place was Miikka Kiprusoff (26.0%). Kiprusoff ranks high because the only results we have are from the prime of his career. He has no decline stage, and he did not play much early on. This shows how well he has played over the last few seasons, but he should definitely be ignored in terms of talk of the greatest of all-time. There were a number of other modern goalies who similarly popped up higher than they should have for the same reasons. However, Kiprusoff did rank 5th overall in terms of career prime (26.2%, basically the same as his overall career), so he is clearly an outstanding goaltender.

In fourth place was another expected name: Jacques Plante. I was not sure what to expect with Plante, whether he was a creation of some powerful Montreal teams or if he really lived up to the billing. The results indicate that he clearly was one of the greatest goalies of all-time. He had a very long career, and still easily outperformed his backups (20.9%). In terms of prime, Plante ranks third overall (31.8%). Therefore, looking at his career as a whole, Plante has a strong case to be ranked as the second best regular season goalie of all time after Dominik Hasek.

In fifth place was another well-known name, but possibly a controversial one. Ken Dryden played for some amazing teams, which has caused speculation that he was just a product of those teams. However, Dryden was far better than his backups, letting in 20.1% fewer goals. Again, Dryden had the advantage of a short career without much of a decline stage, so he ranks a little higher than he maybe should. In terms of prime, Dryden is 11th (19.8%). However, the numbers show that Dryden should probably be considered in the top 10 all-time. He played for great teams, but his performance was also apparently great.

In sixth place comes a bit of a shocker: Roman Cechmanek. Cechmanek played just four whole seasons in the NHL, most of them behind a good defence in Philadelphia, but he easily outperformed his backups (19.5%). However, there are reasons to be cautious about this result. First, Cechmanek had a very short career. Secondly, some of Cechmanek's teammates (Esche, Boucher) ranked very low, indicating that his results were probably somewhat inflated because his backups were very poor. Cechmanek's prime (which is essentially the same as his career) ranks 12th. So Cechmanek's awkward style was probably more effective than it looked during his time in the NHL, at least during the regular season.

In seventh place is Bernie Parent (18.2%). Parent also had a dominant prime, ranking 4th (30.6%). Parent is another goalie that played on some great teams, but also put up some great performances.

Eighth was another virtual unknown: Roy Edwards (17.5%). Edwards had 7 NHL seasons in the late 1960s and early 1970s. He is a classic case of a goalie that never got a chance because of the 6 team league, and also one who was underrated because he played on weak teams. Upon expansion, he won a job and proceeded to put up some relatively outstanding numbers on a weak Detroit team that missed the playoffs in 5 out of 6 seasons Edwards played for them. This was despite being 31 years old by the time he broke into the NHL. Edwards' prime ranks 6th overall (24.7%).

Ninth is another active goaltender, and a very interesting result: Manny Legace (17.0%). Legace is the most underrated goalie in the NHL today. He has had an outstanding career as a backup on Detroit, since he nearly always outperformed his more highly-rated teammates (Curtis Joseph, Chris Osgood, Dominik Hasek). He had a very good season last year on a weak St. Louis team as well. Legace ranks 21st in terms of career prime (16.9%). Legace will drop some in the years ahead as he ages, but he is definitely a top goaltender in the league.

Rounding out the top 10 is Mario Lessard. Lessard had a short career in Los Angeles in the early 1980s, but put up excellent numbers. However, his prime ranks 23rd overall, so one of the reasons he comes in so high here is because he did not have a decline stage. Nevertheless, he is another underrated goalie.

The rest of the top 10:
11. Dan Bouchard (prime: 8th)
12. Marty Turco (prime: 33rd)
13. Tony Esposito (prime: 10th)
14. Arturs Irbe (prime: 9th)
15. Bob Froese (prime: 13th)
16. Chico Resch (prime: 16th)
17. Charlie Hodge (prime: 14th)
18. Rick DiPietro (prime: 37th)
19. Fred Brathwaite (prime: 15th)
20. Daren Puppa (prime: 62nd)

OK, enough of that. Now let's go to the fun stuff: pointing out the frauds.

First, let's see how our blog's namesake did:

Martin Brodeur:

vs. backups: 4.5% better, 52nd place
career prime: 18.4% better, 18th place

So Brodeur has a pretty solid prime (1996-2000), but his career as a whole is not that impressive. Just more evidence to throw on the already huge pile that most of his success is owed to the team in front of him.

How about Grant Fuhr:

vs. backups: 2.2% worse, 115th place
career prime: 1.7% better, 93rd place

Terrible results for Grant Fuhr. He was just as good as his backups in terms of numbers. As far as a regular season goalie, it is virtually impossible to make a statistical case for Grant Fuhr as even one of the best of his era, much less all-time. However, I don't think many of his backers would even look at regular season results when arguing for Fuhr's greatness.

Most people probably noticed the absence of Patrick Roy in the top 20. So where does he come in?

vs. backups: 7.3% better, 31st place
career prime: 13.9% better, 29th place

Patrick Roy is overrated. There, I said it. Roy was a great goalie, no question, but his teams were almost always very good, and that was a big advantage throughout his career. He definitely should be considered in the greatest of all-time argument because of his unbelievable playoff record, but he is not one of the best regular season goalies ever.

How about Billy Smith:

vs. backups: 3.0% worse, 121st place
career prime: 1.1% worse, 115th place

Most people don't really see Smith as a regular season goalie, of course, because he is remembered almost entirely for playoff results. Nevertheless, I believe Smith to be one of the most overrated goalies of all time. His playoff results were only great for four seasons when the Islanders were an unstoppable dynasty, and his regular season results were very mediocre.

Terry Sawchuk:
vs. backups: 4.6% worse, 135th place
career prime: 17.5% worse, 153rd place

Terry Sawchuk, considered by many to be the greatest goalie of all-time, had the single worst career prime rating of any goalie in my study. How is this possible? Well, let's look a little closer at Sawchuk's career. Sawchuk played five seasons as the starting goalie in Detroit from 1950 to 1955. In all of those seasons he had GAAs under 2, he led the league in wins all five seasons, and he won 3 Vezinas and 4 Stanley Cups. I think pretty much everyone would agree that was the prime of his career. So was it the goalie or the team? Well, let's look at his backups. In those 5 seasons combined, Sawchuk's backups were 8-1-3 with a 1.47 GAA. This indicates that Sawchuk's Detroit teams were amazing.

Sawchuk had a long career, so that pulls his results down a lot. But over half of his career shutouts came in a five-year stretch with a Detroit team that was clearly a dominant team, and the results of his teammates show that Sawchuk clearly benefitted heavily from the defence in front of him. Sawchuk is another goalie that is clearly very overrated. His longevity, as well as his good fortune to have played for half a decade on a very dominant team, has led to great career numbers. As a whole, however, he was worse than the other goalies on his team, and is certainly not one of the best of all time.

I welcome any comments on the study's methods and results, and I plan to analyze the results in more detail as I continue to try to identify goalies with performances that do not match their reputations, both good and bad.


Anonymous said...

how about gerry cheevers?

The Contrarian Goaltender said...

Gerry Cheevers:
3.5% better than backups overall
4.6% better in his prime

57th overall, 76th prime

Good results, but not spectacular. Cheevers was helped by some strong Bruins teams most years, and didn't outperform his playing partners by much.

Joe said...

I could've saved you a lot of work and just told you that Legace is pretty underrated. :-( I always felt he really got shafted here in Detroit, the first round loss to the Oil was soooooo not his fault, and far more an indictment of the team's style of play, given that that upset was done in almost the same exact way the previous upsets by Anaheim and Calgary were done. Such a shame.

I think another thing to consider when analyzing your Original Six results is the quality of the backup goaltenders. Given that at the time, only 6 goalies could be employed at any one time as a starter, its certainly not unreasonable to think that there were more than 6 good goaltenders within the NHL and the minor leagues, and obviously some of them would have to be backups. This creates the illusion that the starting goaltender may not have been any better than his backup, when it is entirely possible that while the starter may not have been much better than his backup, both were top-flight goalies. While this may mean the goalie was not as valuable to his team, because he could be replaced by a similar caliber player, it doesn't make the starter any less good. A modern day comparison would be if you put Kiprusoff and Luongo on the same team. It makes both of them less valuable, as they are immediately replaceable, though it doesn't mean that either is any worse. Given the small size of the league, these situations were much more common than they are today. And of course, thats in addition to the aforementioned problem of small backup sample sizes. I'm not sure of any possible way to deal with this issue, but combined with lack of detailed statistics, it would seem to make accurate analysis of Original Six goaltenders virtually impossible.

Any chance of you putting up the entire list, or at least a top 50 or 100? I'm really curious to see how this all shook out. Excellent work, this is seriously one of the coolest blogs I've seen.

An interesting idea: if you did this sort of thing with current NHL minor league affiliates over the last couple of seasons, you could predict which goaltenders should succeed in the NHL, given the chance. Go get a job at some pro team doing this stuff, I bet it pays a lot better!

The Contrarian Goaltender said...

Thanks for the kind words. Predicting NHL performance from minor league results is indeed one of the things I hope to investigate at some point, I'm just trying to build up a good set of analytical tools to measure goalie performance that I'm going to need before I move on to that particular challenge.

I realized the potential problems with evaluating Original Six backup goalies when I started getting a bit deeper into the project. There were a number of goalies that took a long time to break into the league and then played very well when they got their chance (like Johnny Bower), or only got to play in the expansion era but were then outstanding (like Roy Edwards). I continued with the same methodology, but I am debating going back and editing the backup adjustment factors for the Original Six era, since as you say all backup goalies were likely to be quite good, even those that had brief careers in the NHL.

I don't think the 6-team league makes accurate analysis impossible, but it might in certain scenarios. Most goalies play on a couple of different teams, and play on a few good teams and a few weaker ones, so we have a reasonable amount of information to evaluate them on. The problem is if we have a starter and a backup who play nearly their entire careers on one team, then it is very difficult to determine if the starter was good or the backup was weak. One example is Ken Dryden and Michel Larocque - Dryden spent his entire career with excellent Montreal teams, and Larocque has just a few seasons elsewhere when he was past his prime. It makes it tough to really gauge how good Larocque was, and that is obviously a very important question for evaluating a guy like Dryden. That is why I wouldn't rely only on these results to make a judgment, but would combine them with other statistics for a more complete analysis.

I'm still tweaking a few things and I still need to audit some of the data, but I do plan to post the complete list, so stay tuned.

Joe said...

Arg, last try didn't go through right.

Basically, the best way to figure the Original Six might be to figure out the strength of schedule faced by the backups, versus the starters. There was a big difference between the 50's Wings and Canadiens, and the Bruins and Blackhawks. Obviously, in the modern era, the number of teams and games make this very hard to do, but the fewer teams and games in the Original Six could make it easier. Figure out how good/bad each of the 6 teams were, then just look at the games that each backup played (which is a small number), and figure out what their strength of schedule looked like. That's the most accurate way I can think of to handle the Original Six era. The modern era has enough games and teams and backups get enough starts that it shouldn't make as big of a deal in that time period.

Anonymous said...

what about Vadislav Trediak? what if he played

Anonymous said...

The whole foundation for your measure of greateness really hurst Brodeur. You are using backup statistcs as pretty much the #1 weighted facotr. But Brideur backups barely play. Marty plays 75 games every single year. The sample size of his backups is just too very small to accurately ascertain a comparison. Since the trap rules among other have changed, Brodeurs numbers have remained identical. A solid proof that he was no fluke during the trap years.

midi haytham said...

شركة تنظيف بيارات بالرياض
شركة رش مبيدات بالرياض
شركة مكافحة الحشرات بالرياض
شركة عزل خزانات بالرياض

تسليك مجارى
شركة تنظيف بالرياض

شركة تنظيف بالرياض
شركة تنظيف بيوت الشعر بالرياض
شركة شفط بيارات بالرياض

Mike M. said...

Hard to accurately assess with plain stats.

When Roger Crozier played for Buffalo, he was backed up by Dave Dryden and Joe Daley. Dryden played the fewest games and his GAA was the best of the three. Daley had a nearly identical GAA to Crozier's.

The difference is that Crozier played most of the games when the Sabres had to face powerhouses like Boston, NY, Montreal, Chicago and St. Louis. When Crozier had a night off, it was usually when the team was playing against clubs like LA, Pittsburg, California and Vancouver.