Thursday, November 11, 2010

The Wrong Numbers

I had been thinking about putting up a post on the problems of evaluating hockey players by observation, but Kent Wilson saved me the trouble with his recent comprehensive post on that very topic. I fully agree with his conclusion that subjective observation alone is inadequate in judging hockey talent. I also agree that the stats vs. scouting debate is often presented as a false dichotomy and tends to verge into strawman territory, because nobody really uses only one method or the other.

You won't find many objective analysts who claim to never watch hockey. You will, on the other hand, find many journalists and fans that will tell you that they ignore stats and rely only on their terrific scouting abilities and crystal clear memories. I find that interesting, because I'd argue that in almost every case they are simply wrong about that. They are being affected by many different variables, biases, groupthink, etc., and the evidence is pretty good that one of the strongest factors is indeed statistical performance.

I've already pointed out, for example, how GAA leaders are almost always voted to the year-end All-Star team. Similarly, Art Ross Trophy winners are almost always Hart finalists. Since the lockout year of 1994-95, every Selke Trophy winner has scored at least 20 goals, and Rod Brind'Amour was the only Selke winner with a plus/minus rating below +17. If the stat sheets weren't affecting the votes, then those are some remarkable coincidences.

The truth is that everyone relies on some numbers, whether they want to admit it or not. What drives stat guys the most crazy is when people argue that the newer advanced metrics are flawed and wrong, and then go ahead and base their judgments, often unwittingly, on traditional stats that are far worse.

One of the best examples of how people were misled by a single number is Joe Nieuwendyk's Conn Smythe in 1999. There is pretty much one reason that Nieuwendyk was named playoff MVP, and it was that he scored a record-setting 6 game-winning goals in the '99 postseason.

Was Nieuwendyk really that clutch? He did have a knack for scoring goals late when the game was tied. However, goals that break the tie aren't the only important ones. For example, if your team is trailing late then it's impossible to get the game-winner without first knotting up the score. That makes the tying goal a pretty important goal as well, even though it does not appear anywhere on the stat sheet.

One of the reasons Dallas won the Cup that year was that they were great at coming from behind, a rarity for the Dead Puck Era. The Stars were 4-4 in games where they trailed after two periods, a phenomenal record given that the other 15 playoff teams combined to go just 9-42 in the same situation. In all, Dallas scored 10 goals that tied the game in the third period in that playoff season. Somewhat strangely, Nieuwendyk's clutchness didn't seem to manifest itself when his team was losing late. He didn't score any of the goals, and only assisted on one of them.

The Stars also scored 5 goals that gave the team a two-goal lead in the third period (not including empty netters), goals that effectively sealed the victory. Again, none of those goals were scored by Nieuwendyk, and he only assisted on one.

Let's drop game-winning goals and look at a different definition of clutch scoring that takes into account both of the above situations as well. Counting all points on goals scored in the third period or overtime that either tied the game, gave Dallas the lead, or gave Dallas a two-goal lead (empty-netters excluded) gives the following scoring totals in the '99 playoffs:

Modano: 3 goals, 7 assists, 10 points
Nieuwendyk: 5 goals, 3 assists, 8 points
Langenbrunner: 5 goals, 3 assists, 8 points
Lehtinen: 5 goals, 1 assist, 6 points

Nieuwendyk wasn't any more of a clutch scorer than the other guys, he just got the recognition because of the arbitrary nature of game-winning goals. His goals helped Dallas win games, of course, but so did the goals that Modano and Lehtinen were scoring to tie the game up in the first place.

Three of Nieuwendyk's GWG and both of his OT goals came in the first six games, all of them won by Dallas against significantly inferior opponents (Edmonton and St. Louis). In the finals Nieuwendyk had just 2 goals and 1 assist while Modano's line made the difference (Modano had 7 assists in the Finals). I'm not necessarily against that, I would prefer the trophy to be awarded to the best player throughout the playoffs rather than simply the best player in the last series, but that is atypical for a Conn Smythe Trophy winner.

The final reason Nieuwendyk never should have won it over Modano is because Modano played a way tougher role. We don't have play-by-play records or shift charts from the '99 playoffs, but I'm sure they would have shown Ken Hitchcock matching Mike Modano or Guy Carbonneau up against the opposition's best players. I bet the majority of Nieuwendyk's goals and points in those playoffs came against the other team's second, third or fourth lines.

Modano was a big part of the Stars' 90.5% penalty kill, a PK that ran two forward pairings almost exclusively (Carbonneau/Keane and Modano/Lehtinen). Modano averaged 2:59 per game on the PK, compared to Nieuwendyk's 0:04. At even strength, Modano played 17:29 per game while Nieuwendyk played just 14:43. On the power play the two were closer (4:11 for Modano compared to 3:38 for Nieuwendyk), which again reflects how Nieuwendyk was used in an offensive role.

Modano played more minutes, played tougher minutes, played better defence, scored more points on clutch goals, scored more overall and carried the team in the Finals, yet somehow Nieuwendyk was the MVP? That does not compute. Ask anybody who voted on it and they'll tell you how Nieuwendyk brought leadership and was clutch, but I'd bet that what was really shaping their perceptions was the 6 GWG. The Conn Smythe should have gone to either Modano or Buffalo's Dominik Hasek. With Ed Belfour in the mix as well, I think Nieuwendyk would have been, at best, a distant #4 on my MVP list.

The moral of the story is that, no matter how much of an anti-stats hard line you profess, the numbers are going to affect your perceptions anyway, either directly or indirectly. After all, it's pretty tough to watch a hockey game on TV without being fed a bunch of numbers, or hearing the broadcasters talking about so-and-so's scoring totals and making claims about players that are largely based on their stats to date. Given all that, you might as well be aware of the right stats, rather than being misled by traditional numbers.

11 comments:

Anonymous said...

The Conn Smythe should have gone to Dominik Hasek. Without Hasek, the Sabres wouldn't have made the playoffs, let alone come to within two wins of the Cup. Hasek was more than the heart and soul of that Sabres team, he was the team. He likely carried the Sabres even more singlehandedly than Giguere did in 2003, and Giggy quite rightfully won the Conn Smythe.

I agree with you that Modano carried a harder load than Nieuwendyk, but Modano was still one of several star players on that team that each played a big role. He did not carry the Stars on his back.

The Contrarian Goaltender said...

I would have been perfectly fine with the Smythe going to Hasek, and he may indeed have been the most deserving, but I am also aware that there is a voting bias in favour of the winning team. That's why I think the voters should at least have gone with Modano rather than Nieuwendyk, given the assumption that they were going to pick one of the Dallas Stars for the honour.

I wouldn't say that Modano carried the Stars on his back, because that's almost never even close to true in a team sport, but I would argue that he was clearly Dallas' best player and that his importance to that team is largely underrated. Nieuwendyk hit a hot streak in the '99 playoffs that I think camouflaged the true difference in ability for many (aided by the Smythe voters of course).

Look at the scoring and plus/minus numbers for the late '90s and early '00s, and Modano is consistently at the top of a very good group of players. And in the 2000 playoffs, Modano and Hull absolutely carried the Stars on their backs offensively. Both scored over twice as many points as Nieuwendyk (the #3 scorer) as Dallas once again went to the Finals before losing.

Anonymous said...

I agree that Modano should have gotten it over Nieuwendyk, but if you had your way, who would the Conn Smythe have gone to if being on the winning team was not an issue?

J. Gryphon said...

As a diehard '99 Stars fan, I agree that Hasek probably did more good for his team than any other single player did for their team in the 1998-99 Stanley Cup playoffs.

Eddie the Eagle was great; he outplayed Hasek in the last game of the series, but he wasn't responsible for the Stars' success in quite the same way that Hasek was for Buffalo's success. The only series where the Stars, as a team, obviously had trouble against the other team, and relied on their goalie to save the day, was against the St. Louis Blues (look at the stats -- the Blues outshot them; the only edge I noticed for the Stars was in save percentage). The underrated, underestimated Stars were an amazing team and would have been great even if their goaltending wasn't quite as hot as it was.

On the other hand, the Sabres were average, or maybe slightly above average, as a team and heavily depended on possibly the best goalie of all time to push them into the threshold of greatness. Don't get me wrong, the Sabres would have been an all right team without Hasek...

...but you wouldn't have been talking about the Sabres as Stanley Cup Finalists without him, and that's what really matters in the end.

Anonymous said...

"Eddie the Eagle was great; he outplayed Hasek in the last game of the series"

Not sure that I or TCG would agree. The SOG were close, and compare the vast disparity in QUALITY of shots that the Stars were getting through as opposed to the Sabres. I think quality of shots/good scoring chances matters just as much as quantity of shots, particularly if dealing with very unevenly matched teams. Same applies to the St. Louis series, Eddie the Eagle faced much easier shots than Roman Turek. I believe TCG has run the numbers comparing desperation shots taken from everywhere by a desperate inferior team to shots taken with more purpose and intention by the superior team, from a position of superior comfort and control.

Matteau the Magic Wrap-Around said...

What's more is that Modano ALWAYS played the toughest minutes on the Stars' teams in that era. The non-Smythe (haha) is just par for the course on a career of underappreciation. It's very likely he'll finish his career without having won any major trophy.

J. Gryphon said...

Remember that Roman Turek was the Stars backup in '99; the starter for the Blues in the 1999 playoffs was Grant Fuhr.

He might've been overrated before, and he almost surely was by then, with a save percentage of about 89%.

James Benesh said...

Boy, am I glad to read some anti-Nieuwendyk sentiments. Not that I have anything against this classy individual, nor do I imply that you do, TCG, but the media lately has been elevating this guy to godlike status and a surefire first ballot HHOFer. (I was happy that didn't happen but no happier with Ciccarelli) nieuwendyk is not a God!

No one ever sent him votes for the Selke trophy, now suddenly he's described as a "shutdown forward" by THN.

Nieuwendyk scored at a 60-point clip in the playoffs, but suddenly he's described as "clutch".

The guy is called a "winner" because he won three cups with three teams, but he was a second-year player who was, at best, 5th in importance to Calgary, 9th at best to New Jersey (which is generous because that would place him ahead of four players who got more ice time than him, plus he never played in the finals), and 3rd to Dallas (I've thought all along that his Smythe was a dirty ripoff of either Modano or Hasek)

Because he's such a nice and classy guy, the media wants to think he was some sort of hero. Why can't they just be satisfied to remember him for what he was - an elite 2nd line center who could score goals but wasn't much of a playmaker, who once scored a few game winning goals in a six-game stretch in rounds 1 and 2, who contributed to three cup-winning teams, who three times was one of his team's two most-used forwards, (never the most), was decent defensively and amazing on faceoffs?

The Contrarian Goaltender said...

It's definitely interesting how people seem to adjust their memories when they see the career numbers somebody put up, often while simultaneously ripping stats and people who "go off of box scores" and pointing to all kinds of intangibles or making claims that are demonstrably false, e.g. your example that Nieuwendyk was a shutdown forward.

Kent W. said...

Well, I'm glad someone made it through that entire post.

It's funny, due to my psychology background people will occasionally ask me about the psych factors that might effect players - anxiety, team togetherness, having an overbearing coach, blah blah blah...the gossip stuff.

The reality is, my degree in psych did a lot more to highlight to me the errors in thinking, biases in perception in analysis and the complete ignorance of stats in general when it comes to player evaluation.

If I learned anything in psych it was to test your premises and assumptions against actual data, because a lot of plausible sounding things turn out to be inaccurate or completely backward. It's a tough thing to do in human behavior because you cal almost never have a truly controlled experiment, but we do the best with what we have (ie: stats).

Host PPH said...

oh I would be the first one that would be reading your post on the problems of evaluating hockey players by observation, that would be great!