Thursday, January 6, 2011

Goalie Performance on the Road

It has been shown pretty convincingly that the definition of a shot on goal can vary from rink to rink around the NHL. The problem is that there is still a fair bit of guesswork involved in trying to tease out exactly which rinks may be padding the stats and which ones may be shorting them, and even once we can prove that there has been a track record of bias in one city or another it could still change instantly with a single hiring decision.

In response to this uncertainty, I thought to take an ultra-conservative route in this post and assume that every official scorer in the NHL is so biased that they can't be trusted for anything at all related to the home team. Under that assumption we would have to throw out all home save percentage data and rely on road numbers only, hoping that any potential undercounting or overcounting will roughly balance out for goalies who play in a variety of rinks on the road.

In addition to the issue of scorer bias, it seems more likely that a team could control their style of play while in their home rink. To quote Vic Ferrari, "When a team wants to play a low tempo game, the opponent is more likely to oblige in your barn than in front of their own fans." The home team has last change and can decide whether it wants to shut down the opposition's best players with a defensive line or whether to slug it out power vs. power. It seems that the level of parity in today's NHL is such that there is little difference in shot quality from team to team, but if there are any persistent team effects on a goalie's numbers from style of play it seems more likely that they would appear in the home sample than in the road sample.

I'd like to break these down by game situation, ideally, but that special teams goaltending data only goes back about a decade or so whereas detailed home/road splits for all goaltending stats are available going back to 1988 on Hockey Reference. For each season, all home games were thrown out and the road numbers were adjusted based on the league average that year. Current season results were not included.

Nothing changes much in terms of rankings at the top of the list, but there are a few goalies with unusual results that either indicate that there may have been something going on with the shot counting or with the team's style of play at home. On the other hand, perhaps they really enjoyed that home cooking, or maybe it is no more than a statistical quirk arising from cutting the sample size in half. Nothing can be proven with any certainty by this type of surface analysis, but there are some team situations that definitely seem to warrant a closer look.

The goalies are sorted by road goals saved over average, calculated by subtracting the league average save percentage from the goalie's road save percentage for each season and then multiplying by the number of shots faced on the road. Note that since goalies typically have better numbers at home than on the road, comparing to league average means that this definition of average is a slightly higher standard than usual. Current goalies with fewer than 450 career games played were not included for now, but will be discussed in a future post. Keep in mind also that these numbers represent partial career results for goalies who played in the NHL prior to 1987-88.

Tier 1: The Elite

1. Patrick Roy: +.015 save %, +180 goals
2. Dominik Hasek: +.017 save %, +163 goals
3. Martin Brodeur: +.010 save %, +145 goals
4. Ed Belfour: +.009 save %, +110 goals
5. Roberto Luongo: +.010 save %, +92 goals

Tier 2: The Good

6. Marty Turco: +.006 save %, +40 goals
7. Curtis Joseph: +.003 save %, +36 goals
8. J.S. Giguere: +.005 save %, +35 goals
9. Felix Potvin: +.003 save %, +27 goals
10. Dwayne Roloson: +.003 save %, +25 goals
11. John Vanbiesbrouck: +.003 save %, +25 goals

Tier 3: The Average

12. Evgeni Nabokov: +.002 save %, +14 goals
13. Chris Osgood: +.001 save %, +12 goals
14. Tomas Vokoun: +.001 save %, +11 goals
15. Sean Burke: +.001 save %, +10 goals
16. Nik Khabibulin: +.001 save %, +5 goals
17. Guy Hebert: +.001 save %, +4 goals
18. Jose Theodore: +.001 save %, +4 goals
19. Ron Hextall: .000 save %, +3 goals
20. Mike Richter: .000 save %, +2 goals
21. Martin Biron: .000 save %, -1 goal
22. Andy Moog: .000 save %, -2 goals
23. Tom Barrasso: .000 save %, -2 goals
24. Miikka Kiprusoff: -.001 save %, -8 goals
25. Jocelyn Thibault: -.001 save %, -10 goals
26. Jeff Hackett: -.002 save %, -12 goals
27. Arturs Irbe: -.002 save %, -16 goals

Tier 4: The Mediocre

28. Ron Tugnutt: -.003 save %, -19 goals
29. Kelly Hrudey: -.003 save %, -26 goals
30. Olaf Kolzig: -.003 save %, -32 goals
31. Tommy Salo: -.005 save %, -33 goals

Tier 5: The Dinosaurs

32. Ken Wregget: -.006 save %, -39 goals
33. Grant Fuhr: -.006 save %, -46 goals
34. Mike Vernon: -.006 save %, -57 goals
35. Kirk McLean: -.007 save %, -58 goals
36. Bill Ranford: -.007 save %, -65 goals

Some points of discussion:

- The top five really separate themselves from the field by this metric. All are 50+ goals better than the rest. Both Brodeur and Belfour had higher save percentages on the road than at home over the course of their careers, and both benefit quite a bit from going on away numbers only. In fact, assuming the road numbers accurately reflect the overall level of performance and taking into account shot prevention while also recognizing both Roy's team advantages in Montreal as well as the weaker pool of goaltending talent that Patrick was competing against in the late '80s, plus the fact that Roy retired younger than the other two and skipped his decline phase, there may not actually be that much separation between the three of them in terms of regular season results.

For example, if we add one shot prevented to Belfour's save percentage and assume that Brodeur's shot prevention is balanced out by his special teams advantages, then both are around 145 goals saved. All it would take to drop Roy to roughly the same career number is to assume that either he created one extra shot per game against on average or that his expected save percentage was understated by .003 because of weaker competition or Montreal's defensive play and team discipline. Having said that, this doesn't take into account playoff play and the 1988 cutoff means Roy isn't getting credit for two of his seasons. Brodeur is also on pace to give back around 15 goals compared to average this season if he can't fix his struggles in the second half.

- In his first 12 seasons in the NHL, Curtis Joseph saved his teams 68 goals compared to league average on the road. In his final 7 seasons, he gave 32 of them back. Joseph's early career peak would have put him near the elite group, but he did not have anything close to the staying power of the top 4.

- Sean Burke is the anti-Cujo, with -41 in his first dozen seasons and +50 in his final six. If you want further evidence of my argument that it was technique not equipment that was driving the changes in goaltending through the '90s, you'd be hard-pressed to find better examples than Burke and Joseph. Joseph's athleticism allowed him to excel early on, but when that faded as he aged the game rapidly passed him by. In contrast, Burke remodeled himself into a butterfly blocker and put up his best performances in the twilight of his career.

- If you define a goalie's peak as his best five consecutive seasons with a significant number of games played (and including numbers from light workload seasons that fell within the same stretch), Burke ranks an impressive 7th in peak road goals above average per game, trailing only Hasek, Roy, Joseph, Belfour, Luongo and Brodeur.

- Other surprise goalies in the top 15 for peak road results include Jeff Hackett (9th), Dwayne Roloson (10th), Felix Potvin (11th) and Arturs Irbe (15th).

- Arturs Irbe had an eight season stretch where he was 37 goals better than average on the road, playing three of them behind the fledgling San Jose Sharks. His overall numbers nosedived when he stuck around too long after his game fell off a cliff in Carolina, but that was a pretty impressive run without much team support at all.

- To show the kind of impact playing on the Sharks in the early '90s had, Jeff Hackett was -44 goals in San Jose in the first two seasons of that expansion franchise and +32 everywhere else.

- Trivia question: From 1992-93 to 1997-98, which goalie ranked third behind Hasek and Roy in most goals above average on the road? Not Brodeur, not Belfour, not Joseph, not Vanbiesbrouck. Would you believe Felix Potvin? Potvin was a lot like Cujo in that he didn't age well, but there was a lot to like early in his career, even though a lot of people underrate him because he played for mostly mediocre teams.

- Chris Osgood ranks 13th in career total, but just 25th in peak. Osgood has a close comparable in Evgeni Nabokov, who ranked 12th and 29th respectively. Nabokov had a few more peaks and valleys than the fairly steady Osgood, but the end result is that neither was able to put together a five season stretch on the road that was any better than slightly above average for a starting-calibre goalie.

- The biggest surprise on this list has to be Miikka Kiprusoff coming out as slightly below average. Throughout his career, Kiprusoff has a .920 save percentage at home compared to just .906 on the road. Even in his 2006 Vezina year Kiprusoff did not have great results on the road (a mere .904 in 35 games played). The confusing thing is that his shots against split is 27.0 per 60 at home compared to 29.6 per 60 on the road, which really doesn't seem to suggest a generous home scorer.

If there is a logical explanation for this, I'm not aware of it. Was Calgary more disciplined at home, did they change their style of play, did they benefit from lots of back-to-back games against teams that had just played the Oilers? I really have no idea.

- Olaf Kolzig is another guy who had underwhelming results relative to his reputation on the road both overall and in a Vezina winning season (.903 in 1999-00).

- Tomas Vokoun is another interesting case. His save numbers have been very good, especially post-lockout, but he has a Kiprusoffian home/road split of .923/.910. Vokoun's road numbers in Nashville were actually below average, which makes it seem even more clear that there is something going on in Nashville. Vokoun's numbers are better in Florida, although some remain skeptical about his performance there as well.

- I strongly suspect that Guy Hebert got his stats padded in Anaheim. He is a guy that I've noted before as often doing well in various save percentage rankings, but that's probably a bit misleading as Hebert was very average on the road and has another extreme home/road split (.916/.902).

- I still have no idea where to rank Tom Barrasso. Like the rest of his stats, his road numbers are all over the place throughout his career, although he'd presumably rate above average if his extra seasons from the mid-eighties were included.

22 comments:

overpass said...

Pretty cool. The idea behind it seems sound.

Have you run any tests to see if the distribution of home/road differences is different than a random distribution, and if so, how different?

Regarding the results, it's interesting how tight the top of the goalie list is, at least in regular season career value. These numbers definitely boost Brodeur. Do they flatten his upside down career curve?

The Contrarian Goaltender said...

Haven't run any distribution tests yet but I probably should.

Here are Brodeur's road goals saved above average numbers by year:

1994: 14.7
1995: -7.5
1996: 16.9
1997: 13.4
1998: 14.0
1999: 7.3
2000: 3.5
2001: 3.1
2002: 0.1
2003: 11.6
2004: 15.9
2006: 9.6
2007: 25.4
2008: 13.1
2009: -2.3
2010: 7.8

The upside-down career curve is still there, the difference is that the "down" part of it is shortened somewhat by Brodeur's improved results in 2002-03 and 2003-04. If there was a major scorer bias issue in those seasons in New Jersey than he may have a stronger claim to his Vezinas from those two seasons than the overall numbers suggest.

Anonymous said...

Very well thought-out post. Naturally the team that comes to mind most in terms of "cooking the books" for shot totals is the Rangers, but your analysis has shown that many teams may be doing this sort of thing.

Question for you: what do you think has happened to Cristobal Huet's career? He was one of the league's better minders for several years, then became average and below-average quite suddenly.

JLikens said...

Good stuff here.

I agree with you that, while the assertion of recording bias with respect to shots on goal is uncontroversial, it's difficult to actually identify the specific rinks that undercount or overcount shots, aside from the obvious cases.

For that reason, I think that simply looking at road totals is the best approach for an exercise such as this, rather than making an adjustment to overall totals (the latter being the approach I've used in the past).

James Benesh said...

I wonder how much of this has to do with New Jersey's infamous "road game" and how simple they keep things. They've famously been the best road team for a long time, haven't they? Also, is there a way to check if they give the opposition fewer PP opportunities while on the road? That's a sv% boosting tactic for sure.

The Contrarian Goaltender said...

Anonymous: If you run this same analysis on Cristobal Huet, he comes out at almost exactly average on the road. I'll talk about him in a bit more detail in my next post.

The Contrarian Goaltender said...

James Benesh:

It is true the Devils have been a top road team. Compare the road win/loss records of the top 5:

Hasek: 152-136-52
Roy: 186-159-54
Brodeur: 272-177-75
Belfour: 206-172-55
Luongo: 108-134-32

Obviously New Jersey is the outlier there.

Looking at road numbers should remove any scorer bias, but it doesn't remove all team effects. A deeper analysis of things like PP opportunities against and the performance of backup goalies could shed additional light on each goalie's team context.

There is a way to check PP opportunities on the road, it requires sorting through the penalty record on the Hockey Summary Project, removing coincidental penalties and misconducts and counting the rest. I'm fine-tuning a spreadsheet that takes care of that for me, I definitely think there would be value in having those numbers as from there I could estimate a road even-strength save percentage that would put everyone on even more equal terms in this analysis.

overpass said...

You can also check nhl.com for road times shorthanded since 1997-98 (go to team stats, penalty kill). I glanced over the numbers there, and it appears that the Devils are equally disciplined at home and on the road over this time. If the Devils haven't had a persistent skill in this area, it's unlikely that any team has.

If you're running a look at penalties from the HSP, it would be cool to see what types of penalties the Devils take, and in what way they have been disciplined. But only if you have the data easily available.

James Benesh said...

that would be interesting. Prior to such analysis being done, I'm going to guess that it's a combination of both obstruction and aggression penalties. Reason being, I think "not taking a penalty" is a huge and very underappreciated/underrecognized part of New Jersey's "system". Players must be taught in New Jersey that penalties are bad, and that as a rule, you shouldn't take them.

The Contrarian Goaltender said...

I'll see if I can find a way to quickly aggregate the penalties data. I could definitely do a specific look at the Devils, but the thing is that it's kind of meaningless to run the numbers just for them with no context to evaluate the relative frequencies of obstruction/aggression/stick fouls/whatever else. With all the numbers though then that would be interesting to see how teams differed in what they were called for.

The Contrarian Goaltender said...

Overpass: Having a roughly equal number of PPOA at home and on the road means that a team was actually quite disciplined on the road, given that the road team typically gets whistled for more penalties.

I looked at NHL.com's numbers from 2004-2010 (I could have looked at more years but I had a specific interest in Calgary and Kiprusoff), and over that period road teams had 9% more power plays against. The Devils had a roughly equal split over that time as well, and they ranked second to only the Ottawa Senators in terms of the highest percentage of PPOA that came at home (OTT 49.6%, NJD 49.5%). That means New Jersey was relatively more disciplined on the road than at home.

The Devils ranked #1 relative to league average in both locations, but to put it into perspective the Sens had the 6th most PPOA at home and the 19th most on the road.

Other teams that were more disciplined on the road were Atlanta, Boston, Colorado and Tampa.

Teams that were more disciplined at home included Buffalo, Columbus, Detroit, Edmonton, the Rangers, and, as I suspected, the Calgary Flames, who faced the 4th highest total of power plays against on the road but just the 18th highest number at home.

Calgary had the second-most extreme home discipline split after the Buffalo Sabres. That actually surprised me a lot for reasons that will become more obvious when I talk about Ryan Miller in the next post in this series.

overpass said...

Overpass: Having a roughly equal number of PPOA at home and on the road means that a team was actually quite disciplined on the road, given that the road team typically gets whistled for more penalties.

I should have been more clear: New Jersey appears to have pretty close to a typical home/road split. From 1998 to the present 48.3% of their PPOA came at home, compared to league average of 47.9. Holding their home numbers constant, they were shorthanded 26 times fewer than expected on the road, or about 2 per season.

Thanks for the Brodeur numbers, BTW. The dip in the early 00s is still there, but those numbers are well within the range of random variation now. Which makes sense, since I've never heard any good reason for his play to have dropped off in those years.

Olaf Kolzig is another guy who had underwhelming results relative to his reputation on the road both overall and in a Vezina winning season (.903 in 1999-00).

I always knew Belfour should have won that Vezina! Not that single season splits are necessarily significant, but when the numbers are that close already...

Reason being, I think "not taking a penalty" is a huge and very underappreciated/underrecognized part of New Jersey's "system". Players must be taught in New Jersey that penalties are bad, and that as a rule, you shouldn't take them.

Yeah, you don't hear about it much but their record in this area is remarkable. Montreal was the same way from the late 70s to the early 90s, so it may be something Jacques Lemaire brought from Montreal, along with the trap.

The Contrarian Goaltender said...

Holding their home numbers constant, they were shorthanded 26 times fewer than expected on the road, or about 2 per season.

OK. Sorry, I should have given you the benefit of the doubt on that one, it didn't seem like the kind of mistake you would make.

However, the discrepancy between your numbers and the 2004-10 numbers points out that New Jersey was not always the most disciplined team on the road from 1998-2002. Here are their rankings in fewest PPOA over those seasons:

1998: 3rd
1999: 9th
2000: 10th
2001: 6th
2002: 1st

Apparently Robbie Ftorek didn't care as much when his players took penalties, despite working as an assistant coach in New Jersey under Lemaire.

Those years correspond pretty closely with Brodeur's mid-career dip. I think you're right that it might have been just randomness, team effects or scorer bias causing it rather than Brodeur's own play.

Tom Awad said...

CG, excellent stuff. Question: would there be value to "normalizing" the home data,
say with home/road GAA, and adding it in there? I'll explain why: let's suppose some
goaltenders, for whatever reason, have a bigger home/road skew. They know their boards,
are comfortable in their home arena, throw up when traveling, whatever. We are discarding
all the value they may be contributing at home. For example, Brodeur's GAA is only 0.14
lower at home, Belfour's is 0.15, Luongo's is 0.27, Roy's is 0.29, Vokoun's is 0.35.
There seems to be a correlation between positive/negative shocks on your list and
positive/negative improvement in home performance.

Tom Awad said...

Follow-up to the last comment: if you're not comfortable using GAA because of team style, you could even look at W-L record. It exhibits different skew for different players (Kiprusoff, for example, has a 0.679 at home but 0.490 on the road).

The Contrarian Goaltender said...

I'm not comfortable with discarding the home data entirely, no, this was in a way just a kind of thought experiment ("What if the home scorers are all completely biased?"). I agree that some goalies might just play better at home, and given that sample size is always a challenge in rating goalies it would certainly be best to not have to disregard 50% of the data.

As you know there have been a bunch of different people who have looked at the issue of rink bias, and to be honest I'm just not sure that we have a good method yet to adjust for it. Total shots, GAA split, win/loss record, all of them could be just as much the team as the goalie.

But yes, I do agree that it would be best to use the entire sample, we just have to figure out the best method for properly removing team effects from it.

Bruce said...

Hey CG, long time, no write. This is good stuff. I've noticed the New Jersey shot recording bias before (way too low), I'm pretty sure we had an exchange or two about it. This seems to confirm it.

I think "not taking a penalty" is a huge and very underappreciated/underrecognized part of New Jersey's "system". Players must be taught in New Jersey that penalties are bad, and that as a rule, you shouldn't take them.

I agree with this. Now here's my thought experiment: what if that discipline to not take a penalty is extended to good to very good, even great scoring opportunities. Where other teams would be happy to take what is frequently termed a "good penalty", the Devils are happy to roll the dice that their superstar goalie will make the save. While the fewer number of penalties will result in an improved Sv% on one side of the equation, the higher number of ten-bell opportunities will have the effect of lowering his EV Sv%. Which is fine, until you get analysts who tend to discard PP (or is it PK? I can never remember) Sv% and consider only EV Sv%, then the goalie whose team has such priorities will be unfairly punished.

Impossible to prove, but worth considering IMO.

The Contrarian Goaltender said...

OK, let's continue the thought experiment. Since 2000-01, the Devils have avoided one power play against per game compared to an average team, and Brodeur has faced 27.6 shots against per 60 minutes on the road. Assume that every extra penalty avoided allowed a prime scoring chance to take place, and that by taking the penalty the other team completely took away the scoring chance while Brodeur faced a 25% chance of getting scored on. That means an average goalie would be expected to have an EV SV% of .917, while Brodeur would be expected to end up at .910.

That would be the absolute upper bound, but obviously those assumptions are highly unrealistic. If we assume instead that half of the penalties avoided were on scoring chances and that the other team's chance of scoring on those chances was 15%, then the expected EV SV% climbs to .9155.

I really doubt it had much of an impact at all. Maybe it cost Brodeur a thousandth of a point on his EV SV%, but I'd still bet that the Devils defensive play on the rest of the shots against him gained him back at least that much and most likely more.

Bruce said...

Those seem like reasonable assumptions. One or two basis points, which is not-insignificant in the realm of Sv%. After all, you're talking of "the elite" being in the realm of +.010.

Your last comment about Devils' defensive play on the rest of the shots reads more like an opinion than fact-based like the rest of this piece. I thought shot-quality was last year's theory.

The Contrarian Goaltender said...

I thought shot quality was last year's theory

Not at all. Gabe Desjardins just challenged the shot quality advocates this morning by offering money from his own pocket if they offer can offer convincing supporting evidence, but in the comments to that post he makes it clear that, quote, "we know shot quality exists in a not terribly significant way."

I'd modify that to include "in recent NHL seasons". It's fairly elementary to demonstrate that the '91-92 Sharks or the '76-77 Canadiens were shot quality outliers. If we're talking about the "top 4" goalies then our current assumptions do not necessarily hold for their entire careers.

My assessment of the Devils' shot quality is based on more than merely opinion. Even if you ignore, among other things, the attempts to quantify percentage by shot location that typically ranked New Jersey at the top of the list, the subjective assessments of the their team defence, the save percentage effects that New Jersey coaches like Lemaire and Burns had outside of New Jersey, and if you even ignore the results of Devils backup goalies, there's still one fairly good reason to suggest New Jersey's shot quality was likely at least slightly easier than average, namely playing to the score effects.

That save percentages rise on shots with a team in the lead has been clearly demonstrated in many different places. It's still not a huge effect in the aggregate, but like you say a thousandth here turns becomes a few extra wins over the course of a career.

So I'm not sure it's fair to say that my claim was entirely opinion-based, especially given that your hypothesis has no data or evidence to support it (that I am aware of). Although I do think that discipline and penalties is certainly an area that needs to be studied more closely.

Bruce said...

CG: Thanks, you make some good points. My comment about shot-quality was very definitely tongue-in-cheek, I do "believe" in it especially in small sample sizes. I am well aware of Gabe's challenge and agree with your modification of "in recent NHL seasons"; I already asked Gabe if his challenge extended to historic teams. For example it would be pretty easy to prove that my Oilers of the '80s consistently had better shot quality than all other teams, and bad expansion teams would have higher quality chances against. But in today's parity-infested NHL that whole domain is much more level. Even score effects are less of an issue in that you don't have many teams challenging for 60 wins, let alone Regulation wins which is where one would expect to find the lion's share of score effects. They do exist in the context of almost any individual game, it's just that an average team - of which there are Many - will experience such a mix of leads and deficits that they will largely wash out.

Agree on discipline and penalties, I think it's an area ripe for more research. It could well be that Lou has been way ahead of the curve in making penalty avoidance a team objective. But surely such avoidance doesn't come without some costs in terms of chances allowed, which was kind of what I was alluding to in my original remarks.

Hostpph said...

you know one of the things that I like more about your blog is the statistical part, you always include solid statistics to support your points