Monday, May 28, 2012

Shootout Records Don't Predict Playoff Success

There was an interesting statistical pattern this year in the first round of the playoffs:  Only one of the goalies on a winning team had a better shootout win/loss record this year than his counterpart on the losing team.

Rangers vs. Ottawa:  Lundqvist 4-3, Anderson 6-1, Rangers win
Boston vs. Washington:  Thomas 7-1, Holtby 0-1, Caps win
Florida vs. New Jersey:  Theodore 4-7, Brodeur 7-2, Devils win
Pittsburgh vs. Philadelphia:  Fleury 9-2, Bryzgalov 3-5, Flyers win

Vancouver vs. Los Angeles:  Luongo 6-6 (Schneider 2-1), Quick 6-8, Kings win
St. Louis vs. San Jose:  Halak 3-7 (Eliott 1-3), Niemi 8-4, Blues win
Phoenix vs. Chicago:  Smith 6-8, Crawford 6-4, Coyotes win
Nashville vs. Detroit:  Rinne 4-5, Howard 7-2, Predators win

Combined totals:
Winning goalies:  34-42 in shootouts this year
Losing goalies:  55-28 in shootouts this year

This result was surely largely influenced by randomness, especially given that the trend somewhat reversed itself in subsequent rounds with Brodeur and the Devils continuing to win.  However, some matchups probably looked closer than they were in the standings because the weaker team had the benefit of a regular season shootout edge.  The Flyers, for example, would have had a better record than the Penguins if all shootouts were counted as ties (although the Penguins' goal differential advantage indicates that Pittsburgh probably still should have been a slight favourite in that series).

The shootout looks like it is here to stay in the NHL, but it does still seem unfair that it has an impact on the regular season standings that are used to determine playoff seedings.  When comparing two teams to make predictions for a playoff series, shootout results should obviously be disregarded.  That said, regular season records aren't even the best way to predict which playoff teams to bet on anyway (as the Kings and Devils are currently demonstrating), with metrics such as score-tied Fenwick and goal differential showing more predictive power anyway.

Tuesday, May 22, 2012

Mike Smith: Good Season or Good Goalie?

Case A:  2011-12 Was A Fluke Season

The fact that goalie results are heavily influenced by randomness and have a lot of variance from year-to-year has been clearly established by statistical analysis.  It would be very unlikely for an ordinary starting goalie to hit .930 just by luck, but it is not beyond the realm of possibility.  If the goalie is perhaps slightly above-average and aided by some relevant team factors then that explanation becomes a lot more compelling.  The sudden massive improvement in numbers for Mike Smith makes it reasonable to question where his true talent lies.  After all, it is not very often that a 30-year old goalie is able to bump up his career save percentage by .008 in the course of a single season.

The most commonly advanced argument by those seeking to downgrade Mike Smith is the team argument that claims that Dave Tippett is a defensive genius that helps all goaltenders succeed in Phoenix.  It's generally a good idea to be skeptical of shot quality arguments, but on the other hand there are three goalies now that have superior numbers under Tippett then they managed everywhere else since the lockout (regular season and playoffs combined):

Ilya Bryzgalov:
Without Tippett:  7517 SA, .911
With Tippett:  4481 SA, .919

Jason LaBarbera:
Without Tippett:  2703 SA, .905
With Tippett:  1474 SA, .916

Mike Smith:
Without Tippett:  4430 SA, .907
With Tippett:  2617 SA, .934

It is probably also worth noting that Smith played in the Pacific Division, a division that had a higher average save percentage than any other division in the league.  Part of that is surely because the Pacific has some great, top-flight goalies, but the league-leading number was also likely influenced at least to some degree by the tight style of play that is more typical for teams on the West Coast.

In the playoffs and regular season combined this year, Smith has faced just 15% of his shots against on the penalty kill.  Throughout the rest of his career his average was 19% (although part of the decrease was because penalties were down in 2011-12).  Smith also posted a .909 save percentage on the PK, which vastly surpasses his previous career PK rate of .874.  This year was only the second time in his career that Smith's PK rate even went above .880.

If Smith had faced 19% of his shots against on the PK with a PK save percentage of .880, his seasonal save percentage would have dropped .006 to .924.  Assuming that Phoenix has some additional shot quality effect at even strength relative to the rest of the league, he may only have played at a level of around .920 in a neutral team situation.  Obviously Smith had a terrific campaign, but perhaps those potential team factors caused some skeptical GMs around the league to drop him out of the top three in Vezina voting, even though his numbers suggest he probably deserved that ranking (Smith led the league in GVT, a measure that historically tracks very well with Vezina winners).

One large negative indicator for Mike Smith heading into this season is that he had an extreme home/road split.  Prior to this year he had a career save percentage of .918 at home compared to just .896 on the road.  That's usually not a good sign, given that road save percentages are subject to less potential scorer bias since they are spread across a number of different arenas.  This year, Smith's degree of improvement on the road (+.030 to .926) was almost twice as much as the increase in his home save percentage (+.016 to .934).  It is perhaps interesting to note that both LaBarbera (.922 on the road) and Bryzgalov (.921) have excellent road numbers under Tippett in Phoenix (both actually did better away from home than they did in Glendale). Phoenix has also finished in the top eight in road record in each of the past three seasons, despite finishing 16th in points at home this season for the second year in a row.

To summarize, Smith's track record doesn't support anything close to a season of 67 GP at .930, and there is evidence that suggests Phoenix may be a place that helps its goalies out a bit.  As a result, Smith is probably headed for a major regression in his numbers for 2012-13.


Case B:  Mike Smith Is A Good Goalie

Smith has been very impressive in the playoffs, causing some to argue that his game has developed and he is in the process of joining the game's goaltending elite.  The caveat is that subjective arguments are affected by all kinds of observer biases, and sometimes it is hard to differentiate between a good goalie and a lucky goalie or a hot goalie, but Smith's numbers have been so outstanding that it is likely they do represent a significant improvement, even taking into account some of the other factors that may be in play.

One of the biggest question marks in evaluating Mike Smith is determining how much of an impact the concussion he suffered late in 2008 had on his career development.  It's certainly possible to make the numbers fit a narrative that portrays Smith as a promising goalie early in his career before he was brought low by a concussion, and that it shouldn't be that surprising that Smith has blossomed into a quality NHL starter now that he has fully recovered.

Smith was concussed in December 2008.  His career save percentage as of December 31, 2008 was .913, which is very strong taking into account the league average of .907 from 2007 to 2009.  Smith's 2008-09 season on a really bad Tampa team looks quite impressive in retrospect.  The Lightning fired Barry Melrose in mid-season, ended up second last overall, and went 10-22-9 with a 3.62 GAA with their backup goalies in the net (Smith himself was 14-18-9 and 2.62).  Unfortunately Smith's campaign was cut short by his concussion symptoms.

How much did the effects of his head injury affect Smith in 2009-10 and 2010-11?  It's difficult to tell, particularly because he faced a relatively small sample size of 1600 shots.  In 2010-11 he would likely have been much closer to full recovery, but rarely played in a backup role behind Dwayne Roloson.  Smith's combined numbers for those two seasons, together with January 2009 when he was playing with concussion symptoms before getting shut down for the year, come to .899 on 2096 SA, which is well below where he was prior to the injury.

Smith have simply have been somewhat lucky early in his career and then unlucky later on, we've seen that happen many times over a couple of 2000 shot samples.  However, it is also entirely possible that his injury had a major impact on his results and that he has had above-average talent since the start of his career.

It has been widely reported that Phoenix goalie coach Sean Burke wanted to bring in Mike Smith because Burke saw potential there.  Maybe Burke was correct that Smith was a star in waiting, although a cautionary note is that those are the types of things that are easy to say in hindsight if you want to make a specific talent evaluator or coach look great, mainly because they don't account for any of the misses (if Smith didn't work out then nobody would be writing stories about how Sean Burke got it wrong).

Smith's two-year, $4 million deal doesn't exactly suggest that Phoenix thought they were going to be getting Vezina-calibre goaltending, but at the same time with LaBarbera in the backup role and no real other options on the farm the Coyotes were clearly betting their season on Smith.  If they didn't think he was going to be at least average it is questionable whether they would have made that move.  Goalies don't always follow linear career paths, and maybe the change of scenery combined with coaching and his personal development helped Smith make the jump and join the games' elite.

Conclusion

It seems to be an absolute rock-solid bet that Mike Smith's numbers are going to regress next season, and that his true talent level doesn't quite measure up to the level of performance he has displayed this year.  On the other hand, it seems clear that Smith's concussion had a negative impact on his career, and as a result his career numbers are at least somewhat understated.

There is a chance that Smith is now one of the game's best goalies, and there is a chance that this was his career season.  The most likely case is that Smith is not an elite goalie who can be counted on to remain near the top of the league year after year, but he may be an above-average starter who still has a number of useful years ahead of him.  Smith is probably still a good candidate to put up another season of around .920 or maybe even a little better next year, given Phoenix's team discipline and the fact that he outperformed league average by .006 over a three season stretch earlier in his career when he was likely not as good as he is right now.

Thursday, May 10, 2012

Understanding When to Make A Save

"Goaltending in the playoffs so much is about when you give up goals, giving your team a chance to stay in a game when the momentum's going against you or when you've had it.  Brian Elliott simply didn't do that...Conversely, Jonathan Quick, he was timely, he was strong on his game.  The St Louis Blues scored six goals in this series on Jonathan Quick, largely due in part to him not only being a talented goaltender, but to being dialed in to the time of the game and understanding that a save needed to be made, he was there all the way through the series."  (Craig Button, TSN panel)

I'm pretty sure that Jonathan Quick probably does understand when a save needs to be made a lot better than Craig Button does.  If any goalie is in a position to realize the importance of every goal against, it is the backstop for a Los Angeles team that ranked 29th in the league in scoring and played in nine 1-0 games this season.  Even with the Kings' improved playoff offence, it seems very unlikely that Quick was taking anything for granted.  Obviously any goalie who allows just six goals against in four games is making a lot of saves, not just a lot of "timely" saves, and Quick has been great so far in these playoffs because he has stopped nearly everything that has been thrown at him.

If you had to make a case for anything about the Kings being timely, it would be their scoring and possession game more so than their goaltending.  Against St. Louis Quick was pretty great regardless of score, posting a save percentage of .947 or better during each of the key score differentials (down by 1, tied, leading by 1).  The Kings' offence scored 1/4 (25%) while trailing by one, 6/33 (18.2%) while tied, and 3/35 (8.6%) when leading by one.

Los Angeles was also dominant on the shot clock over the Blues with the score tied.  In game one shots were 16-16 and goals were 1-1 with the game tied, and a strong effort from Quick was a big factor in the result.  In games two through four, however, the Kings outshot St. Louis 17-6 and outscored them 5-0 with the score tied.  To score almost as many goals as shots allowed is amazing.  In addition, the average distance on those half-dozen shots against was 42.2 feet and probably only a couple of them could even be marked down as scoring chances.

The biggest problem with Button's logic, though, is the implication that there is choice involved in goaltending.  His absurd premise is that any goaltender can choose to stop any puck, if only they have the necessary clutchness or understanding of clutch play to know that it would be best for them to make that save, and that the difference between a goalie who performs well in a pressure situation and one that doesn't is merely a matter of knowledge or understanding.  Anyone making that claim obviously doesn't understand how much making saves requires a netminder to play the percentages, particularly in today's NHL where key goals are often scored through screens or from deflections or on pucks ping-ponging around the crease or slot area.

Even if goalies can increase their focus or energy level and actually boost their results (which is debatable), that still doesn't mean they have any chance at all at stopping a screened double-deflection into the top corner no matter how well they understand the delicate balance of momentum at that exact point in the game.  It really doesn't take a mathematical background or a detailed knowledge of expected win probabilities to understand that goals against are a bad thing in close hockey games.  That simple knowledge is surely the most basic of prerequisites to tend goal in the NHL, and it seems preposterous that it actually has an impact on the outcome of any games.