Friday, July 3, 2009

Why Is Josh Harding 19-27-4?

Bruce from Oil Droppings asked me if I had any thoughts on Josh Harding's poor win/loss record. Given that there are trade rumours circling around Harding, an RFA, I thought it might be interesting to take a closer look. Over the last two seasons, Harding has put up some very nice save stats, but the team has had much more success with Niklas Backstrom in net.

Backstrom: 70-37-16, 2.32, .921
Harding: 14-24-3, 2.68, .916

At first glance, it doesn't make much sense that Harding's record would be that poor and Backstrom's would be that strong given that they were just .005 apart in save percentage.

The first thing I like to look at when evaluating backups is how they did in games they started vs. relief appearances. Apparently Jacques Lemaire has a quick hook, as Harding had quite a few of those over the last two seasons. A lot of goalies do worse when they come into games but not Josh Harding, who made the most of his relief opportunities:

Starts: 13-20-3, 2.83, .912, 32.0 SA/60
Relief: 1-4-0, 1.68, .944, 29.9 SA/60

This shows that Harding's relief appearances were skewing his numbers a bit, both by making his overall numbers look better and his win/loss numbers look worse. Harding faced fewer and likely easier shots when he came in as a substitute, and yet was tagged with 4 losses. In two of those losses, Harding only let in 1 goal and in the other 2 he was only beaten twice, so it's difficult to blame him too much for those games since the main reason he "lost" them was because Backstrom had already dug the team into a hole. In Harding's relief win, he came in against the Detroit Red Wings after Backstrom gave up 4 goals in 28 minutes, and stopped 26 of 27 shots as the Wild won in a shootout. Because of Harding's relief appearances, Backstrom likely got away with 5 fewer losses on his record than he would have otherwise had if he was made to finish what he started.

The second thing to look at with backup goalies is strength of opposition. Often backups play easier opponents. That was not the case for Josh Harding, however. His average opponent as a starter had 93 points, and 22 of his 36 starts were against playoff teams. As might be expected, Harding's record was much worse against stronger opponents, 7-13-2 against playoff qualifiers compared to 6-7-1 against non-playoff teams.

Harding's goal support was 2.48 per 60 minutes. Minnesota's overall goalscoring rate was 2.62 goals per 60, meaning that Backstrom had the advantage of about 2.66 goals per 60. This difference likely partially reflects the opposition each goalie faced. Once we take out Harding's relief appearances and adjust for his relatively difficult schedule, his record makes a lot more sense relative to Backstrom's.

We can estimate what each goalie's record should be by using the Pythagorean expected points formula, which is GF^2/(GF^2+GA^2) and then modified to reflect the loser points available in the new NHL. I calculated an expected winning percentage of .625 for Backstrom and .478 for Harding, compared to their actuals of .634 and .403 respectively. We see that Backstrom's record is not unusual, especially if we take into account the losses that Harding saved him. In contrast, Harding falls well short of his projection.

This means that either Harding was unlucky with the distribution of goals for and against while he was in net, or he played poorly at key times in the game.

Part of it was the distribution. Minnesota scored 49 goals in Harding's 13 wins, and just 30 goals in his 20 regulation losses. Harding had a record of 4-8-3 in one-goal games. That can likely be partially attributed to bad luck, as winning close games is something Minnesota usually does pretty well (Backstrom was 38-13-16 in one-goal games).

Minnesota is a team that is good at locking it down when leading after 2, and struggles to come back when trailing after 2, so their record after 2 periods is likely to be a good indicator. Here is Harding's record compared to Backstrom's record when leading/trailing after 2:

Leading after 2: Harding 10-0-1 (.955), Backstrom 49-1-4 (.944)
Trailing after 2: Harding 1-13-2 (.125), Backstrom 6-33-5 (.193)
Tied after 2: Harding 2-7-0 (.222), Backstrom 14-7-7 (.625)

There are two main differences: With Backstrom in net, Minnesota was in the lead a lot more often after 2 periods, which I think reflects the difference in performance between the two goalies. Secondly, Minnesota was way more likely to come out on top when the game was tied after two periods.

Those records are so disparate that I simply had to look into how both goalies and their teammates did in the third periods and overtimes when the game was tied:

Backstrom: 21 GF, 20 GA, 8.2% SH, .930 SV%
Harding: 4 GF, 12 GA, 4.7% SH, .874 SV%

Backstrom clearly outplayed Harding here, although he did also get more support. There were quite a few games where Minnesota didn't score, yet Backstrom kept it at zeroes to earn at least a loser point. Harding nearly always let in at least one third period goal, and most of the time his team didn't bail him out.

I wouldn't be too worried about Harding's record, if I was pursuing him for my team. This is such a small sample size that I doubt it tells us much about Harding's ability to play in close games. He was likely just unfortunate to have some poor results over a few high-leverage minutes while his teammates were simultaneously shooting blanks, resulting in a poor record. There were several times when Harding was terrific late in the third, holding down the fort as Minnesota was getting heavily outshot. If Harding had managed to extend a few more of those games to overtime or a shootout, and ended up with a record of something like 3-3-3 in third period ties, then his overall win/loss record as a starter would have been about right given the goals for and against in his games.

I would downgrade Harding a bit after taking into account his performance in games he entered the game as a backup. His career stats in starts are a bit worse than his overall stats, and are likely a better indicator of what can be expected from him in the future:

Harding, career: 2.49, .920
Harding, starter only: 2.64, .917

That means Niklas Backstrom is also probably a bit better than a quick comparison vs. backups would suggest.

Wednesday, July 1, 2009

A Look at UFA Goalies: Is Craig Anderson a Starter?

One of the most interesting free agent goaltenders this year is Craig Anderson, the Florida understudy who has put up some eye-popping stats as a backup over the last 3 seasons (24-14-7, 2.52, .928).

I don't think Anderson will continue to put up those numbers if he was given a shot at a #1 job, because they are likely at least to some degree a result of a smaller sample size. However, the evidence does suggest that he is a legitimate NHL goalie.

Even if you don't trust his most recent couple of years, his .920 save percentage over his last 3,000 shots against at the AHL level is evidence of a goalie that is very good at the minor-league level. Florida must also hold Anderson in fairly high regard, since he managed to take 27 starts away from one of the league's best goalies in Tomas Vokoun.

Two areas we have to usually be concerned about when evaluating backup goalies are whether their numbers have been impacted by their performance in relief appearances, and whether they are playing easier opponents. The first one is not an issue for Anderson as his numbers are very similar both in games he starts and games he enters. Opposition hasn't been a concern either, over the last 2 years 21 of Anderson's 40 starts have come against playoff teams, which is a decent ratio considering that only 3 of Florida's 8 divisional opponents over that time span ended up making the playoffs.

Last year Anderson actually did much better against playoff teams (4-2-0, 1.82, .949). This year he fattened his stats a bit on the weaker teams (8-1-2, 2.08, .940), and had a few rough outings against some elite opponents (7-6-2, 3.18, .913 against playoff teams). I think a team that signed him to a bargain price would be pretty happy even if he could just hold onto that .913.

Anderson just turned 28, so age is not an issue. I think I'd place my bets on him if I had a team that was up against the cap and needed goaltending help. I think Anderson should be at least capable of putting up league average numbers over an entire season. It's difficult to fully assess his ability at this point until we get more information, but what is there looks pretty promising.

Here are some of the other key UFA goalies and their salaries from last year, as well as their even-strength save percentages and number of even-strength shots faced since 1998-99:

Wade Dubielewicz ($0.500): .929, 817 SA
Ty Conklin ($0.750): .924, 2930 SA
Manny Fernandez ($4.333): .923, 6724
Martin Biron ($3.500): .921, 9184 SA
Craig Anderson ($0.575): .921, 2290 SA
Nikolai Khabibulin ($6.750): .920, 9734
Dwayne Roloson ($3.000): .920, 8506 SA
Martin Gerber ($3.700): .920, 4726 SA
Scott Clemmensen ($0.500): .920, 1364 SA
Manny Legace ($2.500): .919, 6343 SA
Mathieu Garon ($1.000): .918, 4192 SA
Kevin Weekes ($0.700): .916, 6763 SA
Brent Johnson ($0.825): .916, 4619 SA
Curtis Sanford ($0.650): .916, 1895 SA
Andrew Raycroft ($0.800): .914, 4644 SA
Jason Labarbera ($0.800): .912, 2104 SA
Brian Boucher ($0.650): .910, 4703 SA
Antero Niittymaki ($1.375): .910, 3315 SA
Joey MacDonald ($0.500): .908, 1592 SA

(Source: UFAs and salaries from HFBoards, ES SV% stats from NHL.com)

League average over the period has been .917, so none of the guys are close to the elite level, but a few of them are serviceable starters. I wouldn't mind too much having Roloson, Khabibulin, or Biron in my net next season, although I wouldn't sign any of them to long-term deals or pay them anything more than $2-3 million. Fernandez should also be able to provide somebody with average play.

The goalies teams might want to roll the dice on would be Dubielewicz, Conklin, Anderson, and Clemmensen (or Jonas Gustafsson, of course, but that's another discussion). Of those 4, I'd probably rate Anderson as the best bet at being able to fill a starter's role, although I wouldn't hesitate at all to sign Conklin as a backup. The other two likely got somewhat lucky over a small sample size. Dubiewelicz is intriguing, he has a .920 career AHL save percentage and has been pretty decent in some NHL spot duty, although I'd like to see more. Clemmensen was pretty ordinary in the AHL last year and doesn't have the supporting minor league numbers; I'd be very leery of him as a small sample size fluke last year playing on a team that has typically been a potential shot quality outlier.

Monday, June 29, 2009

Estimating Even Strength Save Percentages

I've been doing a bit of work lately looking at even-strength save percentages, since the evidence suggests that they do a better job of estimating team-independent performance than overall save percentages, which can be strongly influenced by special teams play and the number of special teams opportunities.

We have official even-strength performances available from NHL.com since 1998-99, and one of the things I have been tinkering with is estimating even-strength save percentages for prior years based on goalie overall save percentages and special team statistics like power plays against and power play goals against.

I compiled the even-strength numbers for the consensus top 5 goalies of the official save percentage era (Roy, Hasek, Brodeur, Belfour, Joseph), and filled them out by estimating their even-strength records for the remaining years of their careers. I'm still working on getting the best fit, so these numbers will be further refined, but I believe they give a good approximation. Once again, what many people see as a "Top 3 and Lesser 2" is more probably a "Top 2 and Lesser 3", perhaps even a "1-1-3" ranking:

Dominik Hasek: .935 even-strength sv%, .914 league average, +.021
Patrick Roy: .921 even-strength sv%, .906 league average, +.015
Martin Brodeur: .922 even-strength sv%, .916 league average, +.006
Curtis Joseph: .917 even-strength sv%, .911 league average, +.006
Ed Belfour: .916 even-strength sv%, .911 league average, +.005

I also thought it would be interesting to post a breakdown of seasons by "poor", "average", "good", and "great". I defined poor as a save percentage of below 99.5% of league average, an average season as from 99.5% - 100.5% of league average, good as 100.5% - 101.5% of league average, and great above 101.5% of league average. Here is how each goalie ended up doing in each season they faced at least 500 shots against:

Hasek: 0 bad, 0 average, 5 good, 8 great
Roy: 1 bad, 1 average, 6 good, 10 great
Brodeur: 0 bad, 7 average, 6 good, 2 great
Joseph: 3 bad, 7 average, 3 good, 3 great
Belfour: 3 bad, 6 average, 6 good, 2 great

There is a consistency argument that can be made for Brodeur over Belfour and Joseph, although some of the bad seasons for the latter two guys were late-career seasons when they were clearly on the decline. Belfour also had one average season effectively spiked by getting traded to the expansion-era San Jose Sharks, who were a legitimate shot quality outlier. In addition, Brodeur almost always tended to play more minutes than the other guys did, so as such we would expect his numbers to less affected by random variations.

If we further restrict it to seasons where the goalies faced at least 1000 shots against, and were between the ages of 21 and 36 (to match Brodeur's career so far), we get the following:

Hasek: 0-0-1-6
Roy: 0-1-6-9
Brodeur: 0-6-5-2
Joseph: 0-5-2-3
Belfour: 2-4-3-2

That shows much less of a difference between the bottom three guys, particularly for Joseph who really had his game desert him in his later years.

I still find it hard to see a significant difference between Brodeur and Belfour. There's not much between them other than in the games played column, and one could argue that Belfour closes some of that gap with superior playoff results. Over a 20 year career Brodeur likely gives a team more total value, but if you had to choose one of them to play a key game for you it's pretty much a tossup.

There was also probably little to no difference between Brodeur and Joseph in terms of stopping the puck, once you put the two of them on a level playing field. Joseph didn't age gracefully, but at his peak he was probably better than Brodeur at making the first save. When taking everything into consideration, however, Brodeur has much better non-save skills and that breaks the tie.

In an all-time perspective, if one was to rate goalies based on how they actually played, not based on how well their teams did or how good sportswriters thought they were, then Brodeur should be ranked much closer to Belfour and Joseph than to Hasek or Roy. As I've said many times before, I believe it's a mistake to rank Brodeur up with the latter two, as both of them had 6-7 years of prime seasons at a level well above anything Brodeur ever reached.

I hope to post some more numbers for other goalies when I get a chance. I did manage to run the numbers for Roberto Luongo, since he has done well in similar performance measures. I know it doesn't matter anymore now that he let in 7 goals in one playoff game, thereby rendering all previous results obsolete, but Luongo has a career .929 even-strength save percentage, compared to a league average of .918, for a +.011 difference, with 5 good seasons and 3 great seasons. That puts him on a trajectory to end up somewhere around the Belfour/Brodeur/Joseph range or even a bit above, keeping in mind that Belfour and Brodeur close the gap some when you factor in their apparent shot prevention effects.

Tuesday, June 23, 2009

How 2 Months Make You a Hero

Playoffs are a funny thing. A couple of games have huge significance. Someone like Max Talbot can show up and score two big goals in a game 7 win and make a name for themselves in the hockey community, and someone who plays poorly can damage his reputation for decades to come (I was listening to the radio the other day as old-timer after old-timer called in, reminiscing about the 1971 Stanley Cup Finals and how it turned when Tony Esposito whiffed on a long shot from center ice).

Chris Osgood had a horrific regular season, and then followed it somewhat surprisingly with two excellent months in the playoffs. This disparity has led to many journalistic screeds about Osgood's clutch ability and his ability to "focus when the playoffs come around" and all kinds of similar thoughts, but having a terrific stretch in the middle of average or below average play is not nearly as unusual as people apparently think.

There is a long list of goalies who have played at Osgood's current level for two consecutive months at some point this season. Unfortunately for those guys, they weren't lucky enough to either make the playoffs, or, if they did play in the postseason, to time their hot streak to coincide with their playoff play (with a couple of exceptions, most notably Jonas Hiller).

Osgood finished the playoffs at .926. Given that the average save percentage usually rises in the playoffs, we can get an equivalent regular season amount by adjusting for the difference between a .915 postseason average and a .907 regular season average. Part of the first figure might be that playoff teams have better goalies on average, but I'm not really convinced of that this year.

Let's say .920 is an equivalent regular season rate. I also set 6 games played in each month as a minimum cutoff. This season there were 22 goalies that played 2 straight months at .920 or better. Fourteen of them had a better save rate over their best 2 months than Osgood has in the last 8 weeks.

The complete list is as follows, with the goalie's best 2 month save percentage in parentheses:

Yann Danis (.945)
Tomas Vokoun (.941)
Tim Thomas (.940)
Chris Mason (.940)
Craig Anderson (.938)
Steve Mason (.938)
Jonas Hiller (.936)
Pekka Rinne (.934)
Mike Smith (.932)
Martin Biron (.931)
Ryan Miller (.929)
Cristobal Huet (.929)
Niklas Backstrom (.928)
Roberto Luongo (.928)
Henrik Lundqvist (.925)
Nikolai Khabibulin (.923)
Scott Clemmensen (.923)
Carey Price (.923)
Evgeni Nabokov (.921)
Cam Ward (.920)
Ty Conklin (.920)
Miikka Kiprusoff (.920)

There were a number of others who were much worse in the rest of the season than they were in their two best months, a la Osgood. The best example is the leader of the pack here, the Islanders' Yann Danis.

Danis is a great example of the variance of goaltending. In 14 games over 2 straight months, he stopped 481 of 509 shots for a .945 save percentage. The rest of the season, he was at just .868. I suppose according to the cliches he must have been more mentally tough or he just really bore down or something in January and February than during the rest of the season. I think he probably just wasn't all that good and got hot and lucky for a bunch of games in a row. Fourteen games is two playoff rounds, so Danis could have become a legend if he happened to time that streak to begin with the start of the playoffs on a playoff team. Unfortunately for him he's probably a career minor-leaguer, but he'll always be able to look back fondly on that brief glorious stretch where nearly everything sent his way just seemed to hit him.

The hot streaks were all spread out, some had them right at the start of the season, some had them at the end of the season. I find it hard to believe that any of them were voluntary, or why wouldn't the goalie simply do the same thing the entire way?

All athletic performance is variable to some degree. Goaltending is based on angles and percentages, which makes it even more variable than other athletic endeavours. Hot and cold streaks happen to everybody and they can be quite extreme over short periods of time. Most of the starting goalies in the league had a 2 month stretch like Osgood has had in the playoffs. Osgood was the only one fortunate enough to have his hot streak come when the games started becoming meaningful and with the Detroit Red Wings playing in front of him.

In my view, if Osgood was unusually clutch we'd see more evidence of it in his past playoff career, but his results are decent but pretty close to average. If he is able to turn the switch off and on at will, then he must have curiously left it off for quite a few playoff seasons, which doesn't make a whole lot of sense. That leaves us to conclude that he either learned how to be "clutch" at some point in 2008, or he perhaps more likely just got hot at the right time.

Having said all that, Osgood has been unusually hot over the last two playoff seasons, especially at even-strength. Here are his even-strength save percentages compared to league average for the last two seasons, as well as during the preceding decade. Which one does not belong with the others?

1998-2007:
Season: Osgood .913, League .916
Playoffs: Osgood .920, League .927

2008-2009:
Season: Osgood .910, League .919
Playoffs: Osgood .948, League .927

If we assume that Osgood's actual skill level in the playoffs is .920, the probably that he plays at .948 over 853 shots, assuming an equal team context, is about 0.1%. In other words, extremely unlikely. Part of that is likely the team, though. I think everyone who watched the games would concede that Osgood got a lot of help from his team last season, but this year he has had to do more on his own.

If Osgood has somehow learned how to be a terrific even-strength goalie in his mid-thirties and only decides to play at that level in the playoffs, then he would certainly be a most unique case. To me that explanation seems both illogical and unlikely.

People seem to get all hot and bothered for some reason when they hear the word "lucky" used to describe the result of a game or an athlete's success, so I'll be charitable and won't go there, but let's just say that Osgood's run of strong play in 2009 was certainly well-timed. Streaks happen, most of the time they defy explanation, all you can do is hope they come at the right time and ride them as long as you can. Good for Ozzy, he was a big contributor to his team in the 2009 postseason, but that still doesn't mean the smart money is on him repeating the feat the next time around.

Monday, June 22, 2009

Is Marc-Andre Fleury Elite?

In the over-simplified world of sports journalism or Internet hockey message board-fandom, Marc-Andre Fleury is now an elite goalie. If Chris Osgood had stopped both of Maxime Talbot's shots in game 7, however, then Fleury would not be elite (and by similar logic, Osgood would then be deserving of being immortalized as one of the 30 or so best goalies in the history of hockey). Such is the narrow margin of what constitutes "eliteness".

Is Marc-Andre Fleury actually one of the very few best goalies in the NHL today? I don't think so. He played very well in games 6 and 7 of the Stanley Cup Finals, but I don't think he was that terrific in the postseason as a whole. In fact, Fleury quite likely had the worst overall playoff season of any Stanley Cup winning goalie in the last decade (.908 save percentage), so it seems a bit curious to me to suddenly claim him to be one of the league's best, no matter how spectacular he was stoning Lidstrom with seconds left to play in game 7.

Since the lockout, Fleury's save percentage has been .908 overall and .921 at even-strength. League average has been about .906 and .918 respectively. I don't think the Penguins are or have been a shot quality outlier (except maybe 2005-06), so those numbers probably give a fairly good indication of what Fleury is: An above-average goalie. On a good day, Fleury's athleticism gives him a higher peak than other goalies of his performance range, but he doesn't appear to be a Lundqvist or a Luongo, despite his more impressive jewellery collection.

Fleury is 24 years old, so it is probably reasonable to still expect some improvement. But goalies generally peak earlier than conventional wisdom suggests, and Fleury is likely either just beginning or already in his prime years. He's good enough that most teams would lock him up long-term if they had him, and he will likely give Pittsburgh good goaltending for the next decade or so, but I'm not expecting any Vezina calibre performances (although with Crosby and Malkin up front, the possibility is there for a 45-50 win season that might end up being recognized with some not-necessarily-deserved hardware).

Thursday, June 18, 2009

Vezina '09: Who Should Have Been Runner-Up?

I think the consensus is pretty clear that Tim Thomas is going to win the 2009 Vezina. One of the reasons I have little doubt is that I don't think either Niklas Backstrom or Steve Mason were one of the top 3 goalies in the league last year, which makes the disparity between them and Thomas even more glaring in my eyes.

Both Mason and Backstrom played on strong defensive teams and had high save percentages on the penalty kill. Backstrom led the league at .918, which is just an off-the-charts figure, and Mason was in the top 10 among starting goalies at .885. The evidence seems to suggest that penalty kill results are more team-dependent than even-strength results, and the smaller sample size means that those results are more variable, so it is tough to tell whether they were playing great on special teams or benefitting from some lucky bounces and/or excellent defenders in front of them.

Backstrom's save percentage at 4 on 5 was almost identical to league average at 5 on 5, which either suggests that Backstrom was absolutely phenomenal when the Wild were down a man or that Minnesota is doing something special on the PK. I haven't seen much of the Wild this season, so I'm not really sure how the credit should be split up. Hockey Numbers has the Wild ranked 3rd in the league in shot quality against at even strength and 6th in the league in shot quality against on the penalty kill, so I'm pretty sure Backstrom is getting some quality help.

Mason, as I've observed before, only really had one spectacular month, and has been pretty ordinary in the 2009 calendar year.

Even-Strength Save Percentage Leaders (min. 500 SA at ES):
1. Tim Thomas: 1320 SA, .940
2. Roberto Luongo: 1156 SA, .936
3. Tomas Vokoun: 1514 SA, .935
4. Jonas Hiller: 899 SA, .934
5. Nik Khabibulin: 912 SA, .933
6. Martin Brodeur: 667 SA, .933
7. Mike Smith: 961 SA, .931
8. Scott Clemmensen: 913 SA, .929
9. Craig Anderson: 753 SA, .928
10. Manny Fernandez: 636 SA, .928
...
18. Steve Mason: 1266 SA, .925
19. Niklas Backstrom: 1633 SA, .923
(League Average: .919)

What is interesting is that there are 2 Bruins, 2 Devils, and 2 Panthers in the top 10 in even-strength save percentage. That seems to me to be pretty unlikely to happen in a 30 team league if there is little to no variation in terms of shot quality against across teams at even-strength. That might raise a few question marks about the results for Thomas and Vokoun, but I think both Fernandez and Anderson are above-average backups. Thomas probably had an easier job than average in Boston, but his performance is still likely far enough ahead of everyone else's that he remains in front, all things considered. It is possible to put up a great goaltending performance on a good team, and that's what Thomas did this season.

Who should have finished 2nd and 3rd in the voting? I don't think it really matters all that much, because there is a clear separation between the winner and the runners-up, but I think I would have picked Vokoun and Luongo as the #2 and #3 guys this year. Hockey Numbers has Vokoun ranked 5th and Luongo ranked 7th in shot-quality neutral save percentage, ahead of Mason (8th) and Backstrom (14th), and both faced several hundred more shots against than anyone else ahead of them other than leader Thomas. Behind the Net also has shot quality ratings for 5 on 5 play, and Thomas is ranked 1st, Luongo 3rd, and Vokoun 5th. In 2nd is Khabibulin, and in 4th in Lundqvist, who likely benefits from MSG's biased scorers. I think Khabibulin is another guy who should get consideration as one of the year's top goalies, as he quite possibly outperformed Backstrom and Mason.

Mason vs. Luongo is a pretty close call, since they had pretty similar seasons. Both of them had a terrific first half, with one month each where they were almost unbeatable, then had some health issues and weren't able to play up to the same level late in the season. They also both significantly outperformed their backups, and their teams did much better with them in net than in their absence. Their overall numbers are pretty similar as well. There's not much between them, and I don't think it is necessarily wrong to put Mason first, but given their track records it's a lot more likely than Mason got lucky than that Luongo did, and that Luongo was better 5 on 5 is enough of a tiebreaker for me.

I think the top 5 were probably Thomas, Vokoun, Luongo, Khabibulin and Mason, with the latter 4 close enough together that it really doesn't make a huge difference how you rank them.

I think 2008-09 will mostly be remembered for goalie injuries. Over 75 starts, which is what he was pretty much assured of hitting without getting injured, Luongo's numbers project to 45 wins and 12 shutouts. That probably would have taken the Vezina over Thomas, especially since one of these years Luongo is going to benefit from the "I can't believe he's never won a Vezina yet!" narrative (see Brodeur in '02-03). I think Martin Brodeur would have put up some gaudy team stats as well, considering his numbers and how well Scott Clemmensen managed in his absence, so it likely would have been similar to 2007 with Brodeur and Luongo facing off for the hardware if fate didn't intervene. Oh well, luck's a part of hockey, as we all know, and this time the guy who benefits the most is Tim Thomas.

Wednesday, June 17, 2009

Vezina '09: Why Thomas' GP Doesn't Matter

Anyone who has followed this blog for a while knows that I dislike arguments based on the number of games played by a goalie. If two goalies have a similar level of performance and one has played substantially more games, then it makes sense to take the guy who has more games played because it less likely that his performance was simply a hot streak or a good run of luck. However, often a goalie who has easily outplayed another goalie is considered inferior because he played less games, even though he would have had to play horribly in the remaining games to drop down to the other goalie's level.

Is there a big difference between a goalie who plays 50 games and a goalie who plays 65? Most people probably would say yes, and I know that most award voters would agree with that statement. The problem is that I doubt any of them ever tried to quantify the difference. What if the 50 game guy has a .935 save percentage, and the 65 game guy has a .905 save percentage? What about .925/.915?

I haven't seen any evidence that there is a statistical difference between playing 65 and playing 50 in terms of fatigue affecting your play. Fifty also represents the bulk of the season, which means that the additional games played do not have a major effect on the average.

To demonstrate this, let's take a goalie who has played 50 games at a pretty average level on an average team (say, .908 save percentage), and consider two scenarios, Scenario A: He plays the last 15 games like the best goalie in the league (.940), and Scenario B: He plays the last 15 games like the worst goalie in the league (.880). What is the effect on his seasonal numbers?

Scenario A: .915
Scenario B: .901

In both cases, his seasonal save percentage moves just .007, even though the extra 15 games played were either fantastic or horrific. It is pretty unlikely that NHL goalies will put together results more extreme than either of those two over a 15 game stretch.

Do the same thing comparing 50 games to 75 games (adjusting the save percentage assumptions a little closer to say, .930 for A and .890 for B since the games played sample is larger and extreme results are less likely to occur), and we get a split of about +/- .010 in save percentage.

I think that is a safe general rule of thumb to use, that if one goalie is .010 or better than another in save percentage, after adjusting for the team context they play in, and if both goalies have played the majority of games for their teams and you don't suspect there is any large differential in shot prevention between the two, then you can pretty safely say that the goalie with the higher save percentage is better. Even if one has played 70 games and the other has played 50.

Let's do a similar calculation for Tim Thomas this season to show that he was demonstrably better than the two other nominees, even though they played more minutes. Thomas played 829 fewer minutes than Backstrom, and 405 fewer minutes than Mason. If we assume that Thomas would face the same shot rate against if we has to play those extra minutes, we can figure out what stats he would need in that extra playing time to match Backstrom's and Mason's numbers.

To match Backstrom: 3.26 GAA, .884 save %
To match Mason: 3.85 GAA, .778 save %

Tim Thomas this season: 2.10 GAA, .933 save %
Tim Thomas, career: 2.62 GAA, .918 save %

How's this for a stat: If Tim Thomas played in 11 extra games and faced his usual shot rate, he could have allowed 5 goals against and lost every single one of them, and he still would have a better winning percentage and save percentage than Steve Mason.

Decide for yourself how likely it is that fatigue or any other factor involved would drop Thomas below those other guys.

Those are unadjusted numbers, of course, so take that into account. Shot quality measures suggest that Thomas faced tougher shots than Backstrom, but easier shots than Mason. We can calculate the shot quality factors that would be necessary for Thomas to have equivalent performance:

To match Backstrom: Bruins 13% easier SQA than Wild
To match Mason: Bruins 20% easier SQA than Blue Jackets

The typical spread from best to worst in the entire league is about 20%, so to argue that Mason's team-adjusted performance was better than Thomas' you would have to demonstrate that Boston was the best team in the league at shutting down opposing scoring chances, while Columbus was the worst. Even then the two of them would be virtually tied, so it would be a coinflip as to who wins.

Tim Thomas should win the 2009 Vezina Trophy. He should also be the First Team All-Star. I'm pretty confident Thomas takes the Vezina, as I don't even see the argument for either Backstrom or Mason to finish ahead of him, but I'll be interested to see what the writers do. They tend to put more weighting on things like wins and games played, so they might throw everyone for a loop and go with Mason, or even Evgeni Nabokov again.