Tuesday, November 30, 2010

Occam's Razor

Roberto Luongo is 16-2-2 with a .926 save percentage while representing Canada in senior men's international competitions. Before he turned pro, he also played great in two world junior tournaments and helped two different teams qualify for the Memorial Cup.

Luongo's career professional playoff numbers against every team other than the Chicago Blackhawks are 16-12, 2.15, .931.

Combine that with the international numbers, and Luongo is a sparkling 32-14-2, 2.07, .929 in postseason and international games as a pro that did not involve the Blackhawks.

Against Chicago in the playoffs, Luongo is 4-8, 3.52, .888.

Occam's Razor says the simplest solution is more likely to be correct. So, which solution seems simpler and better suited to the facts?

1. Luongo chokes in pressure situations, and just happened to either play behind powerful defences or get lucky every single time he played in big games, other than when he played against Chicago in the playoffs where his true nature was revealed.

2. Luongo is just fine in pressure situations, and he and his teammates matched up poorly against Chicago the past two playoff seasons.

Friday, November 19, 2010

The Wrong Numbers, Part 2

(With about thee-quarters of this post already written, I noticed that my prior post on the wrong numbers in hockey had been linked to from the Leafs Central message boards, and in an interesting coincidence it was in a post where occasional commenter here Seventieslord was arguing exactly the same thing that I am about to claim in this post. I'll credit him where appropriate for a couple of things that I added on to strengthen my own argument).

Point totals in hockey seem to be magic numbers. The players who score the most are generally considered to be the best players, and players are routinely described to be playing well if they are racking up the points and not playing well if they aren't.

The problem is that isn't always true. There may be lots of reasons why a player is scoring or not scoring points other than his level of play, such as the type of situations his coach is putting him in, the shooting luck of his teammates, the play of the opponents he is matched up against, and just general puck luck. It is often the case that a player who outscored another in the playoffs was the better player, but this is far from always true, and that means it is a mistake to assume that a higher point total trumps all.

In my opinion, the 2009 Conn Smythe Trophy went to the wrong guy. I don't want to take anything away from Evgeni Malkin (well, other than the trophy they gave him, I guess), but I'd argue that Sidney Crosby was the Penguins' best player. There have been many Malkin vs. Crosby arguments debating which player stepped up in the Finals, which one the Red Wings focused their defensive attention on, which one carried the team in the key games in the earlier rounds, which one was the bigger leader, and so on. At the end of the day, I think there were really only two numbers that mattered: 31 and 36, the respective point totals of Crosby and Malkin. Like it or not, those numbers coloured the rest of the debate, and since 36 > 31, Malkin was the popular choice as Conn Smythe Trophy winner.

The way I see it, Crosby was the better player and the point totals are misleading. Crosby scored more goals and more even strength points than Malkin. With Malkin on the ice, the Penguins scored 41 goals and gave up 21. With Crosby on the ice, the team scored 40 and allowed just 14, in a similar amount of ice time, which indicates he and his linemates may have had a better two-way effort (which also came against tougher opposition). From those numbers and from watching the games, I don't think Malkin was outplaying Crosby, just outpointing him.

The reason that Malkin won the points race was power play scoring. Malkin scored 16 power play points to Crosby's 10. Not surprisingly, however, both stars were on the ice for the majority of the Penguins goals, given that they both played heavily on the team's first PP unit. There were 16 PPG that both were on the ice for, and Crosby and Malkin each had one additional goal where they were on the ice but the other was not.

During the regular season, both players got points on 75% of the team power play goals they were on the ice for. In the playoffs Malkin had points on 94% of his on-ice goals, while Crosby was at just 59%. It was certainly not the case that somebody else replaced Crosby's contribution (both Crosby and Malkin each scored more PPP than the rest of Pittsburgh's forwards combined). I think that the simple explanation is that puck luck worked to the benefit of Malkin and the expense of Crosby.

I watched all the power play goals Pittsburgh scored with either player on the ice on the NHL game highlights on Youtube, and noted how each goal was scored. Here is the breakdown of how they got their points:

Solo effort or good play to create the goal: Malkin 4, Crosby 3
Converting a routine rebound: Malkin 0, Crosby 2
Routine pass to a teammate who created the goal: Malkin 7, Crosby 4
Fortunate bounce: Malkin 5, Crosby 1
Third assists: Malkin 0, Crosby 3

The first Penguins' power play goal of the playoffs I counted as a fortunate bounce for both players, as Malkin's attempted pass across the crease was deflected by Martin Biron off of Crosby's skate and into the net. The other four goals counted as lucky for Malkin included: a point shot that bounced in off of Malkin's knee, a Malkin pass that was batted by an opposing defender right to Mark Eaton who promptly scored, an attempted pass to Crosby on a 2-on-1 that was deflected into the net by a defender for an overtime game-winner, and Brad Stuart knocking the rebound into his own net after Chris Osgood made the initial save on Malkin.

In all likelihood, Malkin and Crosby played at a similar level on the power play in the '09 playoffs, Malkin just ended up on the scoresheet more often. Even if Malkin did create a few extra scoring plays compared to Crosby 5-on-4, I don't think it makes up for Crosby's better overall performance.

Much was also made of the Stanley Cup Final scoring differential, but I'm still not sure that Crosby was any worse than Malkin in the Finals, even discounting the fact that the Red Wings obviously targeted Crosby as their #1 defensive priority. Crosby and his linemates were simply snakebitten that entire series. Bill Guerin and Chris Kunitz, Crosby's most frequent linemates, combined to score 0 goals on 32 shots. Guerin missed several point blank chances, and Crosby himself hit several posts and was robbed repeatedly by either Chris Osgood or Henrik Zetterberg. Guerin and Kunitz had almost the same shot rate against Detroit as they did in the other three rounds, yet nothing was going in. By my eyes, that certainly wasn't Crosby's fault. I think the hockey gods deserve at least as much credit for "shutting down" Crosby as Lidstrom and Zetterberg.

In contrast, Malkin's most frequent linemates Ruslan Fedotenko and Max Talbot combined to shoot 17% in the Finals. Malkin himself scored 8 points, but again it was a case of him being on the right side of some puck luck. Two of Malkin's routine power play assists came in that series on goals by Letang and Gonchar, and two more assists came on lucky breaks, one a horrible rebound by Osgood left for Fedotenko to bang in, and the other a bounce off of a forechecking Malkin's skate right to Talbot, who promptly scored. Malkin also got credit for the Brad Stuart own goal mentioned above.

There are two additional arguments for Crosby that I'm stealing from Seventieslord at Leafs Central. First, Crosby shouldered a heavy load with faceoffs, leading all players in the playoffs by taking a 37.7% share of his team's draws while winning a respectable 53% of them. Secondly, Malkin cost his team quite a bit of time spent on the penalty kill by taking 18 minor penalties, nearly double the number of any other player in the playoffs, while Crosby was only whistled for 7 minors.

While I think Crosby was superior, Malkin did play very well. Malkin also hit his share of posts and created chances where the puck luck wasn't on his side, which means it perhaps wasn't so unrighteous that he got some of the bounces along the way. It is also incorrect to completely discount routine plays, as the ability to consistently make routine plays under pressure is part of what differentiates a great player. Not every goal is an amazing end-to-end rush, after all. That said, the rate that those routine plays get turned into goals can certainly vary quite a bit in the short term.

In nearly every other playoff season, Malkin would have been a very deserving Conn Smythe winner. I just don't think he was in 2009, as the evidence supports Sidney Crosby. Unfortunately, many hockey observers have a tendency of looking at the wrong or the simple numbers and ignoring the importance of context. Even to self-professed stats-hating hockey observers, the power of a single number can be very strong indeed, and at the end of the day 36 points were just too much to ignore.

Thursday, November 11, 2010

The Wrong Numbers

I had been thinking about putting up a post on the problems of evaluating hockey players by observation, but Kent Wilson saved me the trouble with his recent comprehensive post on that very topic. I fully agree with his conclusion that subjective observation alone is inadequate in judging hockey talent. I also agree that the stats vs. scouting debate is often presented as a false dichotomy and tends to verge into strawman territory, because nobody really uses only one method or the other.

You won't find many objective analysts who claim to never watch hockey. You will, on the other hand, find many journalists and fans that will tell you that they ignore stats and rely only on their terrific scouting abilities and crystal clear memories. I find that interesting, because I'd argue that in almost every case they are simply wrong about that. They are being affected by many different variables, biases, groupthink, etc., and the evidence is pretty good that one of the strongest factors is indeed statistical performance.

I've already pointed out, for example, how GAA leaders are almost always voted to the year-end All-Star team. Similarly, Art Ross Trophy winners are almost always Hart finalists. Since the lockout year of 1994-95, every Selke Trophy winner has scored at least 20 goals, and Rod Brind'Amour was the only Selke winner with a plus/minus rating below +17. If the stat sheets weren't affecting the votes, then those are some remarkable coincidences.

The truth is that everyone relies on some numbers, whether they want to admit it or not. What drives stat guys the most crazy is when people argue that the newer advanced metrics are flawed and wrong, and then go ahead and base their judgments, often unwittingly, on traditional stats that are far worse.

One of the best examples of how people were misled by a single number is Joe Nieuwendyk's Conn Smythe in 1999. There is pretty much one reason that Nieuwendyk was named playoff MVP, and it was that he scored a record-setting 6 game-winning goals in the '99 postseason.

Was Nieuwendyk really that clutch? He did have a knack for scoring goals late when the game was tied. However, goals that break the tie aren't the only important ones. For example, if your team is trailing late then it's impossible to get the game-winner without first knotting up the score. That makes the tying goal a pretty important goal as well, even though it does not appear anywhere on the stat sheet.

One of the reasons Dallas won the Cup that year was that they were great at coming from behind, a rarity for the Dead Puck Era. The Stars were 4-4 in games where they trailed after two periods, a phenomenal record given that the other 15 playoff teams combined to go just 9-42 in the same situation. In all, Dallas scored 10 goals that tied the game in the third period in that playoff season. Somewhat strangely, Nieuwendyk's clutchness didn't seem to manifest itself when his team was losing late. He didn't score any of the goals, and only assisted on one of them.

The Stars also scored 5 goals that gave the team a two-goal lead in the third period (not including empty netters), goals that effectively sealed the victory. Again, none of those goals were scored by Nieuwendyk, and he only assisted on one.

Let's drop game-winning goals and look at a different definition of clutch scoring that takes into account both of the above situations as well. Counting all points on goals scored in the third period or overtime that either tied the game, gave Dallas the lead, or gave Dallas a two-goal lead (empty-netters excluded) gives the following scoring totals in the '99 playoffs:

Modano: 3 goals, 7 assists, 10 points
Nieuwendyk: 5 goals, 3 assists, 8 points
Langenbrunner: 5 goals, 3 assists, 8 points
Lehtinen: 5 goals, 1 assist, 6 points

Nieuwendyk wasn't any more of a clutch scorer than the other guys, he just got the recognition because of the arbitrary nature of game-winning goals. His goals helped Dallas win games, of course, but so did the goals that Modano and Lehtinen were scoring to tie the game up in the first place.

Three of Nieuwendyk's GWG and both of his OT goals came in the first six games, all of them won by Dallas against significantly inferior opponents (Edmonton and St. Louis). In the finals Nieuwendyk had just 2 goals and 1 assist while Modano's line made the difference (Modano had 7 assists in the Finals). I'm not necessarily against that, I would prefer the trophy to be awarded to the best player throughout the playoffs rather than simply the best player in the last series, but that is atypical for a Conn Smythe Trophy winner.

The final reason Nieuwendyk never should have won it over Modano is because Modano played a way tougher role. We don't have play-by-play records or shift charts from the '99 playoffs, but I'm sure they would have shown Ken Hitchcock matching Mike Modano or Guy Carbonneau up against the opposition's best players. I bet the majority of Nieuwendyk's goals and points in those playoffs came against the other team's second, third or fourth lines.

Modano was a big part of the Stars' 90.5% penalty kill, a PK that ran two forward pairings almost exclusively (Carbonneau/Keane and Modano/Lehtinen). Modano averaged 2:59 per game on the PK, compared to Nieuwendyk's 0:04. At even strength, Modano played 17:29 per game while Nieuwendyk played just 14:43. On the power play the two were closer (4:11 for Modano compared to 3:38 for Nieuwendyk), which again reflects how Nieuwendyk was used in an offensive role.

Modano played more minutes, played tougher minutes, played better defence, scored more points on clutch goals, scored more overall and carried the team in the Finals, yet somehow Nieuwendyk was the MVP? That does not compute. Ask anybody who voted on it and they'll tell you how Nieuwendyk brought leadership and was clutch, but I'd bet that what was really shaping their perceptions was the 6 GWG. The Conn Smythe should have gone to either Modano or Buffalo's Dominik Hasek. With Ed Belfour in the mix as well, I think Nieuwendyk would have been, at best, a distant #4 on my MVP list.

The moral of the story is that, no matter how much of an anti-stats hard line you profess, the numbers are going to affect your perceptions anyway, either directly or indirectly. After all, it's pretty tough to watch a hockey game on TV without being fed a bunch of numbers, or hearing the broadcasters talking about so-and-so's scoring totals and making claims about players that are largely based on their stats to date. Given all that, you might as well be aware of the right stats, rather than being misled by traditional numbers.