Wednesday, December 30, 2009

Book Review: HockeyNomics

Stats analysis has slowly but steadily crept its way into the world of professional sports over the last two decades, and hockey has been no exception. I'm not someone who needs to be convinced of the value of statistical analysis (obviously), but the numbers work done in the hockey mainstream continues to be pretty simplistic.

HockeyNomics is Darcy Norman's attempt to popularize and spread the idea of "study[ing] NHL stats based on science, not just opinion". I found the book to be a useful introduction to the world of statistical analysis in hockey. When I say "statistical analysis", I really mean "proper statistical analysis", because there are lots of fantasy hockey GMs out there who can quote lots of numbers that really say very little at all.

Norman quotes hockey analyst Alan Ryder in the book as saying, "Hockey is awash with meaningless and, even worse, misleading statistics" (p. 34). As a goalie analyst, I've run into lots of sportswriters who write screeds against judging goalies based on stats, and then fill out their Vezina ballot based on the league leaders in wins and shutouts. Similarly, points make up such a huge part of a skater's rating, even though the player's usage (situational ice time, defensive vs. offensive zone faceoffs) and situation (strength of linemates and opposition) can have a big impact on those numbers. And that's not even starting to discuss the player's defensive play. Many people seem to evaluate a player's penalty killing skill, for example, by the number of shorthanded goals they score, which is at best an awfully incomplete analysis.

Nobody watches every game, so even the most hard-core anti-numbers crowd is probably going to be influenced by at least some numerical evidence. The trick is convincing fans to focus on the good numbers (e.g. possession metrics, save %, rate stats) instead of the bad (scoring numbers, wins, career stats).

The book does not get into any really heavy mathematical work (Norman says a few times that doing so would be outside of his scope). The limited scope means that some parts had to be simplified. For example, the Ovechkin-Crosby debate was limited to a discussion of which player was likely to be more high-scoring over the course of their careers. A full analysis could have tried to factor in additional variables like linemates, ice time, or defensive play. Those who follow the likes of Vic Ferrari, Tyler Dellow, Gabe Desjardins or Puck Prospectus on a regular basis might want to see some deeper analysis or think that the book didn't go far enough in a few spots.

Norman discusses a number of interesting topics. I thought the Poisson modeling to identify the best goalscoring season was an interesting method. The typical way to adjust is to use the average goals scored per game to create an adjusted goals figure, but that doesn't seem to work in all cases since the scoring totals of the top forwards and the league average goals scored have not always followed each other in perfect lockstep. In some seasons in the 1980s league scoring was very high, yet other than Gretzky the top forwards in the league were only scoring around 110 points. In the early 1990s the league scoring level had dropped, but for a few seasons in 1992-93 and 1993-94 many of the top forwards put up terrific numbers.

The book also gets into various measures of drafting success. I personally feel that player development is more important than drafting, and that how teams develop their 18, 19 and 20 year old players should be considered as well. There is a high degree of consensus on high picks, so it seems very unlikely that we would see the observed divergence in terms of player performances based simply on what names get called out on draft day. Part of it may just be a matter of overall team-building strategy: Some teams rely heavily on drafted talent and are content to develop players at the NHL level, while others either have much less turnover or fill many of their roster spots with players that were developed elsewhere.

The last chapter is of particular interest, since it deals with the question of whether Martin Brodeur is overrated. I don't want to give too much away, but Norman quotes me a few times and that should tell you which side he ends up on. To be honest, a regular reader of this blog would be familiar with all the arguments presented in the chapter. It is important once again to point out that "overrated" simply means that a player is rated more highly than he deserves to be. Martin Brodeur is a unique athlete in terms of his durability, his playing style and his ability to contribute to his team in ways other than simply making saves. Despite this many fans consider him to be the best goalie ever, even though Brodeur is most likely the third best goalie of the last 20 years.

HockeyNomics gives a quick summary of the most relevant arguments, but I think there are a couple of issues that still need to be resolved to properly evaluate Brodeur. They include a more nuanced analysis of his overall non-save impact, properly assessing any scorer bias effects from playing in New Jersey, and improving shot quality metrics. My confidence in shot quality measures has been shaken somewhat over the last few months, but I'm still pretty sure that there is some signal in the noise. It may be only useful for analyzing outlier teams, but if anybody was an outlier in terms of defensive skill it surely was the New Jersey Devils.

HockeyNomics is a decent read for anyone interested in hockey, but if you want deep analysis or heavy number-crunching you might find it a bit light in spots. It might be a good gift idea for that friend or relative who either buys into just a bit too much of the conventional wisdom in hockey, or perhaps knows the numbers but tends to focus on the wrong ones.


Anonymous said...

Hi CG. Great blog. More appreciation for the Huet / Brodeur "duel" tonight because of it.

Visited here yesterday the first time while surfing to find out what the heck the "trapezoid rule / goalie penalty" was after Turco was called for it agains the Blackhawks. Disclosure: I started as a hockey fan with the Blackhawks first year of Tony-O in the late 60s and slowly waned during the Bob Pulford Or-gan-i-za-tion Era and pretty much gave up on them after Hasek, Belfour (wow - two of the greatest of all time going into their prime and just gave them away!), Keenan and Roenick were gone...until one moring about 2 years ago turned on the radio and heard Mike North, a local sports radio host, exclaim, "Great news Blackhawk fans - Bill Wirtz is dead !" Well, it pretty much has turned out that way and I have gotten back into the game since - pretty easy now with home games televised.

Several of your posts brought me back down memory lane from 30 to 40years ago - remembering goalies from the Original Six and First Expansion Eras. Saddest day in Chicago sports for me was the Jacques Lemaire red-line shot in 1971 finals. Henri Richard had a big one from the circle, too.

Loved your and Bruce's analysis about Tony-O's career and playoffs - and pretty much agree. Maybe it would have been better for his career in the long-run if they had kept Meloche, too. The Blackhawks of the early - mid 70s had an unconventional team which had alot of two-way forwards, one really good defenseman in Bill White, but not an offensive power by any means even though they went something like 300 games without being shutout themselves.

You guys are all over the stats, so I am only going by what I remember seeing as a kid: best goalie of that time on "winning team" was by far the Flyers' Bernie Parent; best among not winning teams seemed to be Rogie Vachon of the Kings and Maniago of the North Stars and Rutherford of Red Wings. Ed Giacomon was indeed overrated and the biggest challenge to solving Ken Dryden was getting over the center line in games against the Habs. Never thought too much about Cheevers for Boston, as the action was mostly at the other end, a Phil somebody with the Bruins always knew to hit the top shelf stick side over Tony.

Anyway, thanks for the blog and all of the astute commentors here, too.

Derick said...

I bought that book the night before your post because I wanted a hockey book and most hockey books seem to be nothing but cliches and a list of famous people the writer has met.

It was interesting because of the subject matter but it seemed like it was written in a real fast, manufactured way. There was way too much time showing examples of specific formulas in proportion to defending/explaining that way of looking at hockey or analyzing specific players and notions.

Also, there were a few mistakes. The glaring one off the top of my head is the reference to the influence Malkin's playoffs may have had on the Hart trophy votes in 2008. The Hart Trophy is voted on before the playoffs.

The Contrarian Goaltender said...

Anonymous: It's interesting that you point out Maniago, there have been some new save percentage numbers recently published and he's one of the guys that jumped out at me as a decent and probably underrated goalie.

Anonymous said...

Suggesting Brodeur as the third-best of the past twenty years is very generous of you and I imagine you realise that.

He wouldn't even crack my top twenty of the past twenty years. I wonder if he would even make my top forty.

Homme de Sept-Iles

Derick said...

What's a hockey book you would recommend?

The Contrarian Goaltender said...

I'd certainly recommend "The Game" by Ken Dryden, if you haven't read it. Otherwise, it really depends what you are looking for.

If you want to read something stats heavy you won't find too much in book format as there isn't a mass market for it, and you'd probably be better off just reading through the archives of some of the best hockey blogs, like Irreverent Oilers Fans or something like that. Beyond that, good sports journalists usually do biographies pretty well and there are some good ones out there especially on some of the older players like for example Bobby Orr and Jacques Plante.