Monday, October 22, 2007

The Difference Between Talent and Performance

Perhaps one of the major stumbling blocks I have encountered when trying to spread my particular message about goaltending play is that many people evaluate goalies based on talent, rather than performance. This may seem like a small semantic difference, but it is a crucial one.

There are a lot of players in the league with an abundance of talent, but lacking the performance to match. Chad Kilger of the Toronto Maple Leafs is such an example. Kilger is one of the fastest skaters on his team, and last year was clocked with the hardest shot in the league. He has size and decent hands. By all rights, Kilger should be a star player. But he is just a third liner, and the reason is that his talent does not translate into production. He's played for 6 different franchises, and has just 201 points in 661 career games heading into this season. You can watch him play a few nights and think that he is one of the best players on his team. Over the course of an 82 game season, however, he simply proves that he is not.

Performance is the difference between the player Randy Moss was in Minnesota and the Randy Moss in Oakland. It's the difference between Marc-Andre Fleury and J.S. Giguere. It is the reason that you can't trust your eyes and the highlight reels to evaluate goalies. If a goalie makes a great save and then lets in a really soft goal, he's no better than a goalie who lets in the tough one and stops the gimme, but often it is the great save that sticks in our minds and influences our perceptions. When it really comes down to it, it's not how you stop them, it's how many you stop, because hockey games are decided by goal differential, not style points. You have to look at the numerical record, because that is the only objective and comprehensive record of a goaltender's performance over the course of a season.

Martin Brodeur is a talented goalie. He is a great puckhandler, he controls his rebounds well by most measures, he is a good skater, he is athletic, he has great reflexes. Watch him, and you'll probably be impressed by something. But look at the stat sheet and divide his number of saves by the number of shots he faced, and you'll think you made a mistake, because the number is lower than perception would indicate. This is the talent/performance gap in action.

I believe the gap is sometimes very high for goalies, because of the overarching importance of positioning and technique. Cristobal Huet is one of the top goalies in the NHL, and it is not because of his talent. He has somewhat slow lateral movement, he doesn't have outstanding reflexes, he's not one of the best puckhandlers. Yet look at his save record, and it is great. This is because his positioning is usually perfect and his technique is very good. In today's high-speed NHL game, those things are more important than raw talent. Another example is Giguere, who has been frustrating opposing fans for years because he doesn't look like anything special. He is always optimally positioned and moves around in a compact block, making it very tough for shooters to pick the corners on him. Dominik Hasek was considered lucky for years because he had an unorthodox technique, but his fabulous save statistics showed that he was massively outperforming his peers who were doing things "correctly".

Talent is easier to evaluate than performance. Even professional players and scouts often fall for it, but that doesn't make it any less of a trap. Most seasoned hockey fans can tell which goalie is more fluid in his movements, who covers more net, who handles the puck better, who catches/absorbs more pucks, etc. But the difference in save percentage between top goalies is often something like .005 or less (or even .001, as last year between Luongo and Brodeur). This translates to one extra save for the better netminder every 8-10 games, which is an impossibly small margin to reliably detect based on observation alone, and that's not even taking into account team factors like shot quality against.

I place a heavy emphasis on numbers because they eliminate selective memory bias and excessive focus on talent, and look only at the most important issue: How effectively did the goaltender keep the puck out of his net, subject to the team conditions he played under? That's the bottom-line consideration that everything should be based on, because hockey goaltending is a results-oriented profession: It's the performance, not the talent, that truly counts.

Wednesday, October 10, 2007

The Curious Problem of the 1980s

Using the performance vs. backup approach described in a previous post, I wanted to see who was the best goalie in each decade. I calculated a total "goals better than backups" for each goalie, based on figuring out how much better each goalie was than their backups, and then multiplying that by the number of games played to get a total number of goals. So it was not only excellence that counted, but also longevity.

What I found was that for the most part, my answers for each ten year period followed along pretty closely with conventional wisdom, with one glaring exception.

1950s: 1. Al Rollins, 2. Jacques Plante, 3. Glenn Hall
1960s: 1. Johnny Bower, 2. Charlie Hodge, 3. Glenn Hall
1970s: 1. Tony Esposito, 2. Bernie Parent, 3. Ken Dryden
1980s: 1. Dan Bouchard, 2. Pete Peeters, 3. Chico Resch
1990s: 1. Dominik Hasek, 2. Patrick Roy, 3. Curtis Joseph
2000s: 1. Miikka Kiprusoff, 2. Dominik Hasek, 3. Roberto Luongo

For most of the decades, they go pretty much as expected. Probably the only real surprises to most would be the absences of Terry Sawchuk and Martin Brodeur, but neither would be particularly surprising to regular readers of this blog. That is until we come to the 1980s. Of the three goalies listed from the 1980s, I would not have predicted any of them to rank that highly. I think the expected names were Grant Fuhr, Billy Smith, Patrick Roy, or maybe Mike Liut, but none of them popped up.

The 1980s were in many ways a unique time for goaltenders. It was a time of dynasties, a time of expansion (both in terms of teams and a talent pool expanding to include European talent), and a time of rapid evolution in goaltending technique and equipment. I think there also was an absence of a true superstar goaltender, someone to take over the mantle from Esposito, Parent and Dryden, at least until Patrick Roy became established near the end of the decade. This is indicated by the fact that there were no repeat winners of the Vezina Trophy throughout the decade.

It unlikely that any other decade can match the 1980s for contrasting styles - Roy was leading the butterfly revolution against some of the old stand-up goalies, there were European goalies like Lindbergh coming over to add their own playing methods, athletic goalies such as Fuhr to challenge old limits of goalie capabilities, and goalies were also becoming more involved in terms of puckhandling, with Hextall leading the way. This makes it difficult to subjectively rate goalie play.

However, possibly the biggest factor was the lack of parity in the league. Some goalies played on powerhouse teams and received wide recognition for their team successes, while others were doomed to be overlooked because of their weak teammates. Looking at average GAA throughout the decade, there were 6 teams that allowed under 3.5 goals against on average: Montreal (3.14), Boston (3.30), Philadelphia (3.31), Buffalo (3.38), New York Islanders (3.40), and Washington (3.46). There was then a clear gap to the rest of the league, as the next best team was Calgary (3.73). There were 5 teams that allowed over a goal per game more than Montreal on average: Winnipeg (4.24), New Jersey (4.26), Pittsburgh (4.27), Los Angeles (4.33), and Toronto (4.43).

Is it likely that teams like Montreal and Philadelphia had better goaltenders than Toronto did? Probably, for the most part. However, that difference certainly can't be blamed entirely on the goaltending. During the decade, Toronto played 15 different goaltenders. Philadelphia used 13 different netminders, including 3 that also played in Toronto. The Leafs gave up 1.1 more goals per game. Obviously the key factor there was team defence, not goaltending.

I looked at save percentage statistics, but I soon came to the conclusion that it was a largely futile exercise. The top 6 1980s goalies in save percentage (minimum 3000 shots faced) all played on one of the 6 strong defensive teams mentioned above. At the bottom of the list were mostly the goalies from the league bottom feeders. If I used save percentage alone, I would claim that the best goalies of the decade were Patrick Roy, Kelly Hrudey, Ron Hextall, Bob Froese, Billy Smith, Andy Moog, Glen Hanlon, Reggie Lemelin, and Tom Barrasso. However, I don't believe these selections to be entirely correct.

Therefore, I turned to my method of comparing results against backup goalies. In the 1980s, platoons were quite common, so the method should be pretty accurate since sample size isn't as big of an issue. A lot of teams had goalies that virtually split time. What was very interesting was that a lot of times the goalies had very similar statistics. Andy Moog and Grant Fuhr in Edmonton, Patrick Roy and Brian Hayward in Montreal, Billy Smith and Kelly Hrudey in New York, this pattern was repeated in a number of cities, and is further evidence that the team is very influential in determining the success of a goaltender in the vast majority of cases.

Here are the top 10 goalies from the 1980s, in terms of performance against their backups (with adjusted percentage better than teammates, adjusted GAA, and adjusted teammate GAA):

1. Dan Bouchard, 15.1%, 2.86, 3.29
2. Bob Froese, 14.0%, 2.50, 2.85
3. Allan Bester, 12.6%, 3.22, 3.63
4. Kelly Hrudey, 11.3%, 2.81, 3.13
5. Mario Lessard, 11.1%, 3.17, 3.52
6. Pete Peeters, 10.8%, 2.47, 2.74
7. Chico Resch, 10.6%, 3.05, 3.38
8. Andy Moog, 9.8%, 2.84, 3.12
9. Tom Barrasso, 8.4%, 2.75, 2.98
10. Rollie Melanson, 7.4%, 2.95, 3.17

Was Dan Bouchard really the best goalie of the 1980s? Many people would probably respond to that question with, "Who?" Bouchard played 8 seasons for the Atlanta Flames, mostly in the 1970s, and then was traded to the Quebec Nordiques where he spent 5 years before finishing up his career in Winnipeg. Bouchard's save percentage from 1982-1986 was just .873, slightly below league average, but his teams were weak. Bouchard also ranks 6th out of all goalies in the 1970s, so that is more evidence that he was a good goalie. Probably a major reason why Bouchard isn't well remembered is that his career playoff win/loss record is 13-30.

Pete Peeters is another surprise. I was expecting Peeters to be revealed as a team creation, someone who was only successful because of the dominant Philadelphia, Washington and Boston defences he played behind. I think they were definitely all significant contributors to his success, but the evidence remains that Peeters outplayed the other goalies on his teams.

Allan Bester is the kind of goalie that is easy to overlook because he played for terrible teams, but he had an .885 save percentage on the 1980s Toronto Maple Leafs, which is certainly deserving of respect.

On the other side of the scale, here are some goalies who did not do significantly better than their backups did:

Grant Fuhr, 1.1%, 2.99, 3.03
Don Beaupre, 1.1%, 3.00, 3.03
Greg Millen, 1.1%, 3.17, 3.20
Bob Sauve, 1.0%, 2.81, 2.84
Richard Brodeur, 0.5%, 3.08, 3.10
Brian Hayward, 0.5%, 2.97, 2.98
Glen Hanlon, 0.2%, 2.94, 2.94
Greg Stefan, -0.1%, 3.17, 3.16
Gilles Meloche, -0.2%, 2.90, 2.89
Tony Esposito, -1.0%, 3.02, 3.00
Billy Smith, -1.4%, 2.70, 2.67
Patrick Roy, -1.6%, 2.39, 2.35
Pat Riggin, -1.7%, 2.75, 2.71
Clint Malarchuk, -4.1%, 2.81, 2.69
Don Edwards, -8.1%, 2.89, 2.66

It now becomes difficult to know where to rate someone like Roy, who put up outstanding numbers in terms of GAA, wins, and save percentage, but wasn't even able to outplay the other goalies on his team. I don't think he played poorly, but he obviously received tremendous support from his teammates, and I think it is fair to conclude that a lot of other goalies could have put up similar results in that team situation. Look at, for example, Brian Hayward. In 1985-86, with Winnipeg, Hayward was 13-28-5, 4.79, .842. In 1986-87, in Montreal, Hayward outplayed Patrick Roy and led the league in goals against average with 2.81 (Hayward's record was 19-13-4 with an .894 save percentage). That the same goaltender could have a drop of nearly 2 full goals per game in GAA just by switching teams illustrates the nature of the competitive climate of the time, and makes Roy, Smith, Fuhr, etc. appear to be more lucky than good.

Playoff results are normally important in ranking goalies, but the team factors again loom very large. Grant Fuhr, Kelly Hrudey, Billy Smith, Mike Vernon, and Patrick Roy all had very good records, but most of their playoff games were on very good teams. The Islanders and Oilers were dominant in the playoffs pretty much no matter who they had in net. For the weaker teams, the team effects were even stronger since the opponents were tougher. I see it as unfair to penalize goalies for getting shelled while their teams were being dominated by better teams like the Oilers, Islanders, Flames, or Flyers. Therefore, I'm taking playoff performance into account, but not weighting it very heavily.

In summary, there is little evidence that there were any dominant goaltenders in the 1980s. As a result, it is mostly team effects that determine which goalies endure in the public memory and which ones are forgotten. To try to identify the true standouts, I'm relying heavily on their comparative performance against teammates. The imprecise nature of the 1980s situation makes it difficult to come to a definitive ranking, and I'm not sure we'll ever arrive at one. Nevertheless, it is an interesting area of discussion, and I'll at least take a shot at it:

Rankings of 1980s goalies:
1. Dan Bouchard
2. Pete Peeters
3. Mike Liut
4. Kelly Hrudey
5. Bob Froese
6. Chico Resch
7. Reggie Lemelin
8. Andy Moog
9. Tom Barrasso
10. Allan Bester