The Wrong Numbers

I had been thinking about putting up a post on the problems of evaluating hockey players by observation, but Kent Wilson saved me the trouble with his recent comprehensive post on that very topic. I fully agree with his conclusion that subjective observation alone is inadequate in judging hockey talent. I also agree that the stats vs. scouting debate is often presented as a false dichotomy and tends to verge into strawman territory, because nobody really uses only one method or the other.

You won't find many objective analysts who claim to never watch hockey. You will, on the other hand, find many journalists and fans that will tell you that they ignore stats and rely only on their terrific scouting abilities and crystal clear memories. I find that interesting, because I'd argue that in almost every case they are simply wrong about that. They are being affected by many different variables, biases, groupthink, etc., and the evidence is pretty good that one of the strongest factors is indeed statistical performance.

I've already pointed out, for example, how GAA leaders are almost always voted to the year-end All-Star team. Similarly, Art Ross Trophy winners are almost always Hart finalists. Since the lockout year of 1994-95, every Selke Trophy winner has scored at least 20 goals, and Rod Brind'Amour was the only Selke winner with a plus/minus rating below +17. If the stat sheets weren't affecting the votes, then those are some remarkable coincidences.

The truth is that everyone relies on some numbers, whether they want to admit it or not. What drives stat guys the most crazy is when people argue that the newer advanced metrics are flawed and wrong, and then go ahead and base their judgments, often unwittingly, on traditional stats that are far worse.

One of the best examples of how people were misled by a single number is Joe Nieuwendyk's Conn Smythe in 1999. There is pretty much one reason that Nieuwendyk was named playoff MVP, and it was that he scored a record-setting 6 game-winning goals in the '99 postseason.

Was Nieuwendyk really that clutch? He did have a knack for scoring goals late when the game was tied. However, goals that break the tie aren't the only important ones. For example, if your team is trailing late then it's impossible to get the game-winner without first knotting up the score. That makes the tying goal a pretty important goal as well, even though it does not appear anywhere on the stat sheet.

One of the reasons Dallas won the Cup that year was that they were great at coming from behind, a rarity for the Dead Puck Era. The Stars were 4-4 in games where they trailed after two periods, a phenomenal record given that the other 15 playoff teams combined to go just 9-42 in the same situation. In all, Dallas scored 10 goals that tied the game in the third period in that playoff season. Somewhat strangely, Nieuwendyk's clutchness didn't seem to manifest itself when his team was losing late. He didn't score any of the goals, and only assisted on one of them.

The Stars also scored 5 goals that gave the team a two-goal lead in the third period (not including empty netters), goals that effectively sealed the victory. Again, none of those goals were scored by Nieuwendyk, and he only assisted on one.

Let's drop game-winning goals and look at a different definition of clutch scoring that takes into account both of the above situations as well. Counting all points on goals scored in the third period or overtime that either tied the game, gave Dallas the lead, or gave Dallas a two-goal lead (empty-netters excluded) gives the following scoring totals in the '99 playoffs:

Modano: 3 goals, 7 assists, 10 points
Nieuwendyk: 5 goals, 3 assists, 8 points
Langenbrunner: 5 goals, 3 assists, 8 points
Lehtinen: 5 goals, 1 assist, 6 points

Nieuwendyk wasn't any more of a clutch scorer than the other guys, he just got the recognition because of the arbitrary nature of game-winning goals. His goals helped Dallas win games, of course, but so did the goals that Modano and Lehtinen were scoring to tie the game up in the first place.

Three of Nieuwendyk's GWG and both of his OT goals came in the first six games, all of them won by Dallas against significantly inferior opponents (Edmonton and St. Louis). In the finals Nieuwendyk had just 2 goals and 1 assist while Modano's line made the difference (Modano had 7 assists in the Finals). I'm not necessarily against that, I would prefer the trophy to be awarded to the best player throughout the playoffs rather than simply the best player in the last series, but that is atypical for a Conn Smythe Trophy winner.

The final reason Nieuwendyk never should have won it over Modano is because Modano played a way tougher role. We don't have play-by-play records or shift charts from the '99 playoffs, but I'm sure they would have shown Ken Hitchcock matching Mike Modano or Guy Carbonneau up against the opposition's best players. I bet the majority of Nieuwendyk's goals and points in those playoffs came against the other team's second, third or fourth lines.

Modano was a big part of the Stars' 90.5% penalty kill, a PK that ran two forward pairings almost exclusively (Carbonneau/Keane and Modano/Lehtinen). Modano averaged 2:59 per game on the PK, compared to Nieuwendyk's 0:04. At even strength, Modano played 17:29 per game while Nieuwendyk played just 14:43. On the power play the two were closer (4:11 for Modano compared to 3:38 for Nieuwendyk), which again reflects how Nieuwendyk was used in an offensive role.

Modano played more minutes, played tougher minutes, played better defence, scored more points on clutch goals, scored more overall and carried the team in the Finals, yet somehow Nieuwendyk was the MVP? That does not compute. Ask anybody who voted on it and they'll tell you how Nieuwendyk brought leadership and was clutch, but I'd bet that what was really shaping their perceptions was the 6 GWG. The Conn Smythe should have gone to either Modano or Buffalo's Dominik Hasek. With Ed Belfour in the mix as well, I think Nieuwendyk would have been, at best, a distant #4 on my MVP list.

The moral of the story is that, no matter how much of an anti-stats hard line you profess, the numbers are going to affect your perceptions anyway, either directly or indirectly. After all, it's pretty tough to watch a hockey game on TV without being fed a bunch of numbers, or hearing the broadcasters talking about so-and-so's scoring totals and making claims about players that are largely based on their stats to date. Given all that, you might as well be aware of the right stats, rather than being misled by traditional numbers.