Monday, March 23, 2009

Belfour's Shot Prevention

When some goalies are in the net, fewer shots are taken by the other team. Although I was once skeptical of this effect, it has been shown to be pretty clear in the historical data. Martin Brodeur is a goalie who usually gets a lot of credit for his puckhandling and other skills that likely contribute to a reduced shots against total. There are, however, other goalies who are also very good at those skills.

I believe the goalie who might be the most overlooked in this aspect of goaltending is Ed Belfour. Having closely compared their shots against records, there is good evidence to suggest that Belfour's shot prevention effect may be every bit as large as Martin Brodeur's.

Belfour is a little easier to evaluate since he averaged fewer games per season and played on a number of different teams during his career. To do my comparative shots analysis, I usually look for goalies that had both substantial playing time with the study goalie as well as lots of minutes played on other teams. There are three goalies that fit these criteria that we can use to evaluate Belfour: Jeff Hackett, Roman Turek, and Marty Turco.

In this type of analysis, I prefer to take the average of yearly averages rather than simply calculate an average from the totals, because otherwise it skews the data when there are seasons with much higher or lower minutes played totals. Belfour played with Hackett from 1993-94 to 1996-97, and averaged 27.3 SA/60 per season, compared to 29.2 SA/60 for Hackett. That is a gap of about 2 shots per game. Over the rest of his career, Hackett tended to face more shots than his teammates, averaging about one extra shot per game against. That gives us an estimate for Belfour of about one shot per game saved.

Belfour played two seasons with Roman Turek in Dallas, and faced 0.5 fewer shots per game. Turek was pretty consistently about a half shot below his goalie teammates throughout the rest of his career, which again supports the one shot per game number.

In his two seasons playing with Marty Turco, Belfour was about a shot per game better than Turco, which is impressive since Turco has been half a shot per game better than his backups over the last few years (and Dallas has tended to bring in goalies who play a similar style, so Turco's effect might even greater when you consider he is being compared to guys like Johan Hedberg or Mike Smith). It could be that Turco had not yet developed his skills to the same extent early in his career, or that he learned a thing or two from Belfour that he was able to apply later on, or maybe there were some other variables that were influencing the shot numbers.

The results from all three goalies suggest that Belfour has a shots against effect of at least one shot per game. I did another piece of work where I compared how the backup goalies did while playing with a specific goalie and how they did on other teams. Ed Belfour faced 26.7 shots per game, compared to 27.9 for his backups. His backups had a weighted average of 28.4 on other teams, which suggests that although his defences were better than average at preventing shots, Belfour contributed to his team's shot prevention.

For the sake of comparison, Brodeur faced 25.4 shots per game compared to 26.1 for his backups, who had a weighted average of 28.5 shots per game on other teams. This shows that New Jersey had a significant defensive effect. Subtracting the differences between the two of them, and you get a 1.2 shot differential for Belfour and a 0.7 difference for Brodeur. It is likely that weaker competition and other small sample size factors mean that Brodeur's figure is a bit understated, and those numbers do not include the 2008-09 season where Brodeur has so far faced about 2 fewer shots per game than his teammates. Still, it is more evidence that Belfour is comparable to Brodeur in terms of non-save skills.

I am willing for now to credit both Belfour and Brodeur with preventing 1 shot per game compared to average goalies. I also think that this information is the decisive factor that separates them both from other goalies like Joseph and Vanbiesbrouck who have similar save percentage records.

It does appear that the Brodeur vs. Belfour debate is a lot closer than most people would think. If these estimates are correct, then Brodeur doesn't gain an advantage over Belfour in terms of non-save skills, and the comparison between them should be decided by puckstopping ability alone. Brodeur has a better save percentage record compared to league average, but evidence suggests he also faced easier shots, since Belfour spent some of his career on bad teams (Florida, San Jose) and other parts of his career on teams that likely allowed shots of a higher-than-average difficulty (e.g. Toronto). If you compare Belfour's Dallas numbers to Brodeur's New Jersey numbers at the same time, which were likely two fairly similar team contexts to play in, their stats are essentially identical: 2.18 GAA and .909 save % for Brodeur vs. 2.19 GAA and .910 save % for Belfour.

I think that both have had pretty similar careers - they had some great individual seasons early, had lots of playoff and team success as members of terrific two-way defensive teams, and then spent a few seasons on weaker teams yet still performed pretty well. They also had some off-years mixed in, although New Jersey's defence did a better job of camouflaging Brodeur's down seasons compared to Belfour, who actually lost his starting job twice - to Hackett in Chicago and to Turco in Dallas.

The main reason that I think there is a reasonable case for Belfour over Brodeur right now is playoff performance, where in my estimation Belfour has been pretty clearly the better goalie. If my shot quality estimates in my last post were correct, then their regular season records might actually be pretty similar as well. Having said that, however, I think I would still take Brodeur's career over Belfour's right now, and with a few years left Brodeur will likely leave little doubt before his career is over.

34 comments:

sunnymehta.com said...

I'm curious if you saw JLikens' latest post about home recording bias with regards to shots on goal. I found it really interesting. If certain arenas do indeed consistently under- or over-record shots by two or three shots a game, that would make a huge difference with regards to both overall S/60 as well as SV% numbers.

Statman said...

Good work. We know that a goalie faces more shots when killing penalties... do we know if The Backups tended to face more or less powerplays?

Perhaps a Backup's teammates are less likely to take penalties when he is in net, because they fear facing a powerplay... or, perhaps they are more likely to get penalties when they try to limit good scoring opportunities against the Backup. I'm not sure. I suppose I could ask a mainstream "legit" analyst & get some "real" insight into it gathered by watching lots of games & hanging around hockey players in the dressing room... or, we could do it properly by looking at the numbers.

[In the past, I had noticed that generally Backups tended to face fewer SA/60, & I suspect that this was because they usually play the weaker teams. I'm not sure if this lower SA/60 still holds... apparently you've found that it doesn't, at least for some goalies &/or teams. It would be nice to know the comparative strength of opposition when the Starter is in net, as opposed to when the Backup is in net.]

overpass said...

Good to see the numbers support Belfour. I thought he was better than Brodeur during the Toronto years.

What did you think of Tom Awad's recent article at Puck Prospectus ranking the top 10 goalies since 1944? He ranks them using GVT, which isn't explained but appears to be basically SV% above average with no team adjustments. His list had Belfour and Brodeur tied at 5th with Dryden and Hall.

While a rank of 5-7 all time seems about right for Brodeur (the top 100 players in history project at HFBoards ranked him 6th all-time), you don't see Belfour on many top 10 lists. I think he's definitely underrated, although he was probably only the fourth best goalie of his era.

The Contrarian Goaltender said...

Sunny: Yes, I read that post and it was indeed interesting. I've seen some of those stats before, but not as far back. That is definitely something that needs to be taken into account, and I will be looking in more detail at those results.

Overpass: It's actually save percentage above replacement, his goal with GVT is to come up with a VORP equivalent for hockey. Value over replacement gives a higher award for longevity (which is he has Roy 1st and Esposito 4th), but even despite that Brodeur is still down in 5th spot. I was intrigued with the Belfour/Brodeur tie, because that was similar to my own results. For Belfour to match Brodeur in an non-team-adjusted rating biased towards goalies with great longevity is quite impressive for Eddie the Eagle.

Bruce said...

When some goalies are in the net, fewer shots are taken by the other team. Although I was once skeptical of this effect, it has been shown to be pretty clear in the historical data.

At last, progress! :)

I believe the goalie who might be the most overlooked in this aspect of goaltending is Ed Belfour.

I've been pushing Belfour along with Brodeur pretty consistently on this aspect. I had my eyes opened in the 2000 Finals when we were visiting friends during Game 6 of the finals and the compromise was that the game was on the tube with the volume turned down. Without the usual distractions and caring hardly at all about who actually won, I just watched the flow of play and was blown away by the extremely high skill level of both goalies in this respect. The puck never seemed to stay in their zone for long, they just fielded and moved it to the right guy over and over and over again. Instructional video stuff, at both ends of the rink. The puck spent a lot of time in the neutral zone.

There are three goalies that fit these criteria that we can use to evaluate Belfour: Jeff Hackett, Roman Turek, and Marty Turco.

There's also "before" and "after" effects for a guy who moves around, such as the Luongo --> Belfour --> Vokoun triumvirate in Florida, in which all were #1 guys under Jacques Martin but Belfour's shots against totals were FAR lower than Luongo, and much lower than Vokoun.

For the sake of comparison, Brodeur faced 25.4 shots per game compared to 26.1 for his backups, who had a weighted average of 28.5 shots per game on other teams. This shows that New Jersey had a significant defensive effect.

Yeah, but Brodur's backups played primarily on bad clubs, not average, so by and large they would have swung from one extreme to the other.

I am willing for now to credit both Belfour and Brodeur with preventing 1 shot per game compared to average goalies. I also think that this information is the decisive factor that separates them both from other goalies like Joseph and Vanbiesbrouck who have similar save percentage records.

The first conclusion seems reasonable, and I agree entirely with the second.

It does appear that the Brodeur vs. Belfour debate is a lot closer than most people would think.

Both were/are elite goalies. I agree Belfour gets the least love among the big four of the Dead Puck Era, but as you know I have been consistently referring to them as the Big Four because I consider "Billion Dollar Eddie" to be in the weight class of the other more broadly-acknowledged superstar stoppers.

If you compare Belfour's Dallas numbers to Brodeur's New Jersey numbers at the same time, which were likely two fairly similar team contexts to play in, their stats are essentially identical: 2.18 GAA and .909 save % for Brodeur vs. 2.19 GAA and .910 save % for Belfour.

Bear in mind that this comparison period almost exactly brackets Brodeur's "down" period, or whatever it is you call the bottom portion of the upside-down career curve when Jersey played a more attackingbrand of hockey. They also encompass Belfour's top statisotcal seasons, and still they're pretty even.
The main reason that I think there is a reasonable case for Belfour over Brodeur right now is playoff performance, where in my estimation Belfour has been pretty clearly the better goalie.

I've got a little trouble with this statement.

Belfour 161 GP, 88-68 = .564 Win%, .920 Sv%, 2.17 GAA, 1 Cup

Brodeur 169 GP, 95-74 = .562 Win %, .919 Sv%, 1.96 GAA, 3 Cups


Their percentages are similar, except Brodeur's GAA is about 10% lower, which is entirely reflected in his lower Shots Against, of 24.3 per 60 compared to 27.0. Can't say that's all due to their individual tyles as Belfour did move around, I'm just saying there's nothing there that screams Belfour was Better. What do you see that prompts such a remark?

I think I would still take Brodeur's career over Belfour's right now, and with a few years left Brodeur will likely leave little doubt before his career is over.

Agree on both points, as well as your main point that Belfour is a lot closer than many will credit him.

Statman said...

Effect of shot prevention, assuming 1 shot prevented/60 min's & therefore shot prevention = GP:

GP SVPCT SHTPCT Goals Saved
70 0.90 0.10 7.0
70 0.91 0.09 6.3
70 0.92 0.08 5.6
70 0.93 0.07 4.9

In the playoffs, assuming a long run of 22 GP:

GP SVPCT SHTPCT Goals Saved
22 0.90 0.10 2.2
22 0.91 0.09 2.0
22 0.92 0.08 1.8
22 0.93 0.07 1.5

Ironically, the worse the goalie is at stoppping the puck - SVPCT - the more goals are saved by shot prevention.

Anyways, a savings of 4.9 to 7 goals over 70 games played in the regular season, & 1.5 to 2.2 goals over a long playoff run, isn't that impressive in the big scheme of things. Not nearly as important as stopping the puck.

Bruce said...

Statman: Shot prevention of 1 shot per 60 is the equivalent of a bump in Sv% of .003 to .004.
Sure it's only a few thousandths.

If you're talking about the difference between Kari Lehtonen and Johan Hedberg, Hedberg's shot prevention talent doesn't make up for the gulf between their shot-stopping capabilities and your point holds.

But if you're comparing top keepers a few thousandths is pretty significant. If you're making the argument that, say, Roberto Luongo and his .919 career Sv% is superior to Brodeur's .914, a little difference like a shot here or there can account for that whole "huge" difference of .005 and then some. A difference of +1 shot for one goalie vs. -1 for another -- as CG found for Luongo and Brodeur -- is the equivalent of a Sv% difference of about .007. Or by your method, the difference of 10-15 goals over the course of a season. That's the difference between having a 20-goal scorer and a 30-35 goal scorer, which is pretty significant.

Statman said...

Maybe I need a cup of coffee, but how does shot prevention interact with SV%?

Isn't - theoretically - shot prevention independent of SV%?

I see how shot prevention relates to GAA:

GP 70-- 70-- 70
MIN 4200-- 4200-- 4200
**SA/60 29-- 30-- 31
**SA 2030-- 2100-- 2170
SVPCT 0.910-- 0.910-- 0.910
SHTPCT 0.090-- 0.090-- 0.090
**GA 183-- 189-- 195.3
**GAA 2.61-- 2.70-- 2.79

The Contrarian Goaltender said...

Statman: Technically save percentage is independent of shots prevented or created, since a shot that is prevented never happens and therefore never affects either shots against or goals against. However, the net result of preventing a shot from happening and stopping that shot is exactly the same - 0 GA. Therefore, we add on an extra shot per game to someone like Brodeur, and then recalculate his adjusted save percentage to reflect that additional shot that an average goalie would have faced.

We still need to look into greater depth to see whether there are any shot quality effects associated with shot prevention, and what exactly is driving it, but I think the adjusted save percentage is the best way of taking it into account.

The Contrarian Goaltender said...

I'm just saying there's nothing there that screams Belfour was Better. What do you see that prompts such a remark?

League average during playoff career:
Belfour .909
Brodeur .913

Playoff save % above average:
Belfour +.011
Brodeur +.006

I think in many ways their playoff careers are the opposite of their regular season careers. In the playoffs, Belfour has been the consistent performer while Brodeur has been the one with more ups and downs.

Belfour had 6 straight playoff seasons at .920 or better, and arguably didn't have a single bad playoff year after 1993-94. Brodeur's save percentages and performances have been a lot more variable. If you consider team success to be important, Belfour made it past the first round in every playoff season he competed in between 1994-95 and 2000-01, while Brodeur has never gone more than three playoff years in a row without getting eliminated in the first round by some weaker opponent.

In addition to the overall edge in play, when I did my study of situational playoff performance the numbers indicated that Belfour was more instrumental in his team's results than Brodeur was. Take this, for instance:

3rd period & OT:
Belfour 1.75, .932
Brodeur 1.92, .919

Statman said...

Ok, duh.

Effect of facing 30SA/60, while preventing 1SA/60:

70
4200
30 (31)
2170
0.9129
0.087
189
2.70

GA & GAA stays the same, SV% rises from .9100 to .9129, as the goalie is credited with 100% SV pct on 70 shots (1 shot prevented x GP).

Anonymous said...

The effect on great puckhandling/rebound control has to be more than 1 shot per game. Just thinking about, especially with guys like Brodeur,Turco, Dipietro or Belfour, you have to consider multiple things, most of which I am sure you already consider, but I look at it like this: A goalie who is great with the puck can eliminate a certain number of shots by either clearing the puck, or deterring the other team from moving it. Contrarily, if this same goalie has terrible rebound control, then even though he is preventing shots one way, he is creating them in another way. Turco comes to mind with this. Then who have goalies like Hasek who were absolutely horrible with both puckhandling and rebound control. He certainly added a few shots per game because of it. Then you have somebody like Brodeur or Belfour who had a positive prevention effect in both areas, but taking it one step further, you could say by preventing shots as well as they did via puckhandling, not only did they prevent shots, but also potential rebounds from those shots, which you would have to think may account for an aggregate total of more than just one shot reduced per game. Recently there was a thing on NHL.com asking multiple goalie past and present the importance of puckhandling and rebound control, and several of them made reference to Brodeur's skills reducing between 6-10 shots a game. Now, I think those numbers may be a bit high, however 1 shot per game is definitely a bit low.

The Contrarian Goaltender said...

Recently there was a thing on NHL.com asking multiple goalie past and present the importance of puckhandling and rebound control, and several of them made reference to Brodeur's skills reducing between 6-10 shots a game. Now, I think those numbers may be a bit high, however 1 shot per game is definitely a bit low.

Do you have a link? That would be interesting to see. Those numbers (6-10 shots per game) aren't just a bit high, they are completely insane. Nobody who ever even casually glanced at the numbers would ever suggest something so ridiculous, and I can't understand how someone who played goalie could think there was that large of an impact.

I've considered all of those things, and it all comes back to the same thing: Where's the proof? Where is the effect in the numbers? If there was a 6-10 shot per game effect, how could it not be extremely obvious in the numbers?

The estimate of one shot per game is compared to average, so the difference between Brodeur and Luongo may well be around 2 shots per game.

The season-to-season gap in shot differential between goalies on the same team is usually, at most, 2-3 shots per game for a tandem, and that gap gets lower over multiple seasons. You just won't see a persistent 5 shot per game gap between goalie teammates.

Rebound control is not a major shot creator, because of the infrequency of rebound shots against per game (about 1.5 rebound shots against per game per team on average). About 5% of shots result in a rebound shot, which is not a significantly high number.

Goalie puckhandling, most of the time, does not prevent shots, it merely makes it easier for the defensive team to exit the zone. The first guy back is usually a defenceman. A good puckhandler can mean that a team doesn't have to resort so often to rimming it or chipping it off the glass, and it will allow the team to exit the zone more quickly rather than making several passes in the break out, so of course it helps but I don't see a multiple shot per game effect there.

Having said all that, hockey is a team game, and one of the reasons that these effects are not so visible is that the rest of the team adjusts to it. If you have a bad goalie in the net you probably block more shots than you would with a good goalie in the net. If a goalie is bad at giving up rebounds, the defencemen work harder to clear the crease than otherwise. I'm still looking for possible evidence that goalies can impact puck possession, shots for, and goals for. It's possible that they have some small impact on those things, which would increase the value of skills like puckhandling, but that hasn't been demonstrated yet to my knowledge.

This season has been a perfect test case with Martin Brodeur having been injured, and I think it is pretty much a validation of my results. We have one of the best at these skills (Martin Brodeur) and someone who is below average (Scott Clemmensen). Not only is there a difference in skill set but one guy has the reputation of being one of the greatest of all-time, while the other guy is an unproven minor leaguer that other coaches have been quoted as saying their teams should get the puck on them from anywhere on the ice.

Clemmensen: 29.0 SA/60
Brodeur: 27.3 SA/60

overpass said...

CG: Suppose that Brodeur's puckhandling prevents prevents the opposing team from taking 5 shots that they would have taken against Scott Clemmensen. The opposing team changes their strategy, enters the zone differently, and shoots from different locations, adding another 4 shots. Brodeur only prevents one net shot, but also changes the type and location of 4 other shots, and may create the perception that he prevented 5 shots.

Of course, if this effect exists, it only matters if it changes the shot quality faced for better or worse - and even then it only affects analyses based on shot quality, as the traditional goalie statistics will capture this effect.

Have you ever checked to see if individual goaltenders make a difference in their own shot quality faced? Not sure if it's possible with the available numbers right now, but I'll throw it out there anyway.

Bruce said...

The season-to-season gap in shot differential between goalies on the same team is usually, at most, 2-3 shots per game for a tandem, and that gap gets lower over multiple seasons.

Here's a good example from an established pairing that reinforces that the differences are Real, and can be at least as high as the 2-3 shots you are suggesting.

Season: Lehtonen / Hedberg

2006-07: 31.6 / / / 28.4 = +3.2
2007-08: 34.6 / / / 31.9 = +2.7
2008-09: 34.3 / / / 30.9 = +3.4

... showing Hedberg consistently faces almost 10% fewer shots than Lehtonen season to season. That's pretty significant, in a very decent sample size (160 GP to 84, with Hedberg playing over 20 games in each season). Lehtonen is so superior as a stopper that he is still the first choice to actually keep the puck out of the net, but I note that while his Sv% is 26% worse than Lehtonen's (.913 to .890) his GAA is "only" about 16% worse (2.91 to 3.38). At ~3 shots difference per game a shot prevention adjustment of about .010 could be derived which nicely explains the disparity between those percentages.

Forgive me for this heresy, but in cases like this -- two guys on the same team in an extended sample -- I will put more stock in their relative GAA than their Sv%. Lehtonen is definitely the better goalie, just not by the ridiculous edge that his Sv% suggests.

Statman said...

Re: Lehtonen & Hedberg...

But what about the fact that backups usually play the weaker teams, & so are likely to face less shots in any event (irregardless of puckhandling)?

Sort by team:

http://www.hockey-reference.com/leagues/NHL_2007_goalies.html
http://www.hockey-reference.com/leagues/NHL_2008_goalies.html
http://www.hockey-reference.com/leagues/NHL_2009_goalies.html

In each instance of Lehtonen/Hedberg, the starter is playing more games (duh), has the higher save pct (good move by the coach), & faces more shots/60.

Surely the level of competiton that each goalie faces has to be considered.

Is there a source that lists the winning pct (or better yet, avg shots for) of each team that each goalie faces?

Statman said...

For fun, I looked at goalies from 84-85:

http://www.hockey-reference.com/leagues/NHL_1985_goalies.html

I grouped according to team, & chose the clear "starters" as having played 50+ games for 1 team.

Of the 11 "starters", 8 of them faced more shots/60 than the goalie that played the next highest amount of games for the team. I didn't use goalies that were 3rd (or lower) in games played, due to sample size issues (goalies that see little action might have the misfortune of playing 1 GP & due to a fluke face 40 shots one night).

Adding up the "+" & "-" in shots faced (as compared to the next highest game-player), resulted in 10.01 SA/60. Divide by 11 = .91 more SA/60.

[Of course, goalies that are traded during the year might be considered to be the starters on all of their teams, but they probably wouldn't have played 50+ games for 1 team. As well, one goalie might have been considered the starter for long stretches, but then another goalie takes over as the starter. It would be nice to know the winpct of teams faced by each goalie.]

The Contrarian Goaltender said...

Overpass:

Have you ever checked to see if individual goaltenders make a difference in their own shot quality faced?

It's definitely on the to-do list. I am quite open to the possibility that goalies face different types of shots, in addition to different quantities of shots. For example, Brodeur has faced a higher shot quality against than Clemmensen and Weekes. I think there are two reasons that could explain shot differentials: prevention and deterrence. If goalies prevent shots (say, by reducing the time the other team spends in their own defensive zone through moving the puck), then that shouldn't have a major impact on shot quality. If goalies deter shots, then we would expect the shots that are deterred to be of lower quality, i.e. a shooter thinks twice about testing Brodeur from the halfboards but fires away at Clemmensen. In that case, we would expect the extra shot(s) to be of lower shot quality.

This year, there happens to be a handful of tandems around the league with a shot gap of 2-3 shots per game between starter and backup, so when I get around to it I'm planning to look into those situations and try to identify possible causes.

overpass said...

Statman - I'm not aware of a source that lists average opponent stats for goalies. I previously did a little research into the strength of opposition for Billy Smith and his backups. I pulled the game logs from the Hockey Summary Project files, used the lookup functions in Excel and the team stats from the years in question to find the average stats of opponents. You can find game logs at Yahoo Sports for more recent years.

In the case of Lehtonen vs Hedberg, when looking only at opponent Lehtonen has faced slightly better teams on average over the past 3 years. Taking the average of each seasonal difference, his average opponent's Pts% was 0.006 higher than Hedberg's, his average opponent's GF/G was 0.03 higher, his average opponent's SF/G was equal, and his average opponent's SH% was 0.1% higher.

However, Hedberg has played significantly more time on the road than at home, so it's necessary to adjust for that. After adjusting using league home/road splits, Lehtonen's average opponent has a Pts% that's lower by 0.012, 0.015 fewer GF/G, 0.3 fewer SF/60, and an equal SH% as compared to Hedberg's average opponent.

In this specific case, the backup has in fact faced more difficult opposition due to the home/road breakdown. A more detailed breakdown might include adjustments for back-to-back games also. In any case, Lehtonen gains no advantage from quality of opposition.

Statman said...

Thanks, Overpass.

Very interesting... I don't follow ATL at all, but this might be some very poor coaching/lineup-making. Why not put the strongest goalie (in terms of stopping shots) in against the stronger teams?

There could also be factors such as a particular goalie being sick or injured, etc.

overpass said...

My guess is that Atlanta plays their starter at home more often because home games are seen as games that a team "should win", and you just hope to get a point in road games. There may also be a perception that the starting goalie need a rest more often on the road.

They're not the only team that plays their backups on the road - Mikka Kiprusoff has started every home game but one in the past two years for Calgary, and Patrick Roy played significantly more games at home than on the road over his career.

Whatever the cause, home/road patterns are something that's worth looking into when comparing backups to starters, and in some cases will offset any difference in strength of opponent.

Statman said...

I've wondered how much pressure coaches feel to put the starter in at home, to put on a show for the fans aka 'win the game.'

Statman said...
This comment has been removed by the author.
Statman said...

This just in… for 2007-08, I compared the goalie on each team that played the most games (the “Starter”) to the sum of the rest of the goalies on the team (backups).

E.g. Giguere compared to his 2 backups = 0.90 SA/60 less.

Number of times Starter SA/60 was higher than 'backups': 14

Number of times SA/60 lower than 'backups': 16

On average, amongst 30 teams, the Starter faced 0.08 SA/60 more than the other goalies on the team. In % terms, 0.09% more SA/60 than the total of the other goalies on the team.

There is a very weak correlation of -0.08 between Starter GP & the shot difference, amongst all teams. (-0.09 between Starter Minutes & shot difference). I thought that perhaps when there is a clear Starter (plays most of the games) there would be a larger shot difference.

I thought there may be some other trend (other than just straight-line), but a graph of % difference SA/60 vs. Starter Minutes is all over the place. For instance, I thought that perhaps when the Starter is playing huge minutes, that the backups may just play the weakest teams (& face the least SA)…. And that if the Starter is only playing 35-45 games (e.g. for a weak team that hasn’t decided who is #1) that there might be no real correlation between SA/60 & Minutes.

Bruce said...

In the case of Lehtonen vs Hedberg, when looking only at opponent Lehtonen has faced slightly better teams on average over the past 3 years.

Overpass: Thanks for that. I was going to comment that Hedberg has played stretches of games during Lehtonen's seemingly annual injuries which should have a randomizing effect on quality of opposition. It's not like Lehtonen gets all the SouthLeast Division games or anything.

It certainly doesn't appear that the coachis picking his spots. Since February Hedberg's appearances have been equally split between strong teams -- Devils, Flyers, Penguins -- and not so strong -- Kings, Lightining, (sob!) my Oilers. It's probably to his credit that Hedberg posted a 3-2-0 record in those games, and you should be able to guess which ones he won.

In this specific case, the backup has in fact faced more difficult opposition due to the home/road breakdown.

The home/road split that results from coach's decisions could be fairly significant. In this case with Hedberg getting more than his share on the road, any adjustment correcting for that should serve to widen the shot prevention gap that already exists between him and Lehtonen. Most teams shoot more at home.

In each instance of Lehtonen/Hedberg, the starter is playing more games (duh), has the higher save pct (good move by the coach), & faces more shots/60.

My interpretation is it's a good move by the coach to play the guy with the better GAA, not Sv%. If Lehtonen's Sv% were just .005 better, with with much higher shots against he would actually have worse results than Hedberg.

One of the things I'm interested in exploring is any correlation between SA and Sv%. I anticipate there will be at least a weak one. Hopefully some time soon I'll find some time to wrap my head around it and try to develop a couple of methodologies to explore this relationship. CG may have already done something along these lines, although his interpretations always seem to differ from my own in certain respects. :) Nonetheless there's no point in reinventing the wheel if the background data has already been collected.

Statman said...

[For 07-08, using each individual goalie on each team*, there is a correlation of only 0.16 between SA/60 & SV%.

*If a goalie played for 2 teams, then it's as though there are 2 goalies. I didn't combine a multi-team goalie's stats.

That's just a really basic correlation... I'm sure you have been thinking of more complex ways to analyze it.]

Bruce said...

Basic is always a great place to start. Thanks for that, Statman.

That's +0.16 I assume from the context of your words, which matches rather well with my supposition of "at least a weak" correlation. With goalie stats, differences are subtle and it seems one always winds up mining the margins for clues. I think I'll wait until 2008-09 is complete and have a hard look at this years results through the shots-allowed prism.

The Contrarian Goaltender said...

Forgive me for this heresy, but in cases like this -- two guys on the same team in an extended sample -- I will put more stock in their relative GAA than their Sv%. Lehtonen is definitely the better goalie, just not by the ridiculous edge that his Sv% suggests.

That's not heresy at all, I agree with you that GAA is a better measure to use when comparing goalies on the same team. Not only does it account for any shot prevention effect, but it also has the benefit of being expressed in goals, which is more intuitive to understand. If I point out that Brodeur is +.009 and Luongo is +.013 in adjusted save percentage compared to backups this year, it's difficult to estimate the relative difference there. If I say instead that Brodeur is 0.25 goals per game better than his backups while Luongo is 0.40 goals per game better than his backups, that's a little easier to compare.

Re: the Atlanta example, Lehtonen and Hedberg is the highest persistent shot differential I have ever seen (in fact, it was one of the most important pieces of evidence that won me over to your side in the shot prevention debate), and I have looked at a lot of different tandems. If you can find a higher example than those two I'd be interested in knowing what it is, because I haven't found one. That's why I see the 3 shot differential between those two guys as, at least for now, the upper limit in shot differential.

One of the things I'm interested in exploring is any correlation between SA and Sv%. I anticipate there will be at least a weak one. Hopefully some time soon I'll find some time to wrap my head around it and try to develop a couple of methodologies to explore this relationship. CG may have already done something along these lines, although his interpretations always seem to differ from my own in certain respects. :) Nonetheless there's no point in reinventing the wheel if the background data has already been collected.

Everything I've ever done or seen anyone else ever do suggests that there is no general relationship between shots against and save percentage. I think it varies too much on a team-by-team basis. There are some teams that do have a positive relationship between shots and save percentage, and other teams that don't. But if you find anything interesting, please share the results.

If you look at historical NHL results or results from outside the NHL, the correlation is consistently negative, which means that good teams allow both fewer and easier shots. I think that is a basic truth of hockey - good teams are usually good at both ends of the ice. The reason this hasn't held true in the NHL in recent years is the parity across 30 teams.

Statman said...

The +0.16 correlation is based on every goalie in 2007-08 who played.... so, Matt Keetley played a total of 9 minutes & faced 2 shots & allowed no goals... his 13.33 SA/60 & 100% SV% is weighted as highly as any other goalie. Obviously that messes up the conclusion a bit.

I think it's very important to be able to be able to dig into the data & determine shot situation (penalty killing vs. even strength) &/or "shot quality", & then base correlation between SA/60 & SV% on that.

Too bad the NHL is so lame... they are sitting on decades of game sheets that could offer so much information.

Bruce said...

Re: the Atlanta example, Lehtonen and Hedberg is the highest persistent shot differential I have ever seen (in fact, it was one of the most important pieces of evidence that won me over to your side in the shot prevention debate), and I have looked at a lot of different tandems.

CG: Key word "persistent", and the fact the shots differential between the two has persisted into their third season as a pairing -- and has continued unabated after we had isolated them as a good test case -- is compelling evidence that the effect is Real.

If you can find a higher example than those two I'd be interested in knowing what it is, because I haven't found one.

I haven't spent the time you have looking at it. I wonder if you have looked at tandems from the past? We now have a quarter century of shots/Sv% data and presumably that data could be mined. Of course the position of goaltending has evolved tremendously over that time, both stopping techniques such as the Allaire butterfly block and the "soft" skills are being coached to the nth degree, so it may be that shot differentials between different styles of goalie are wider now than they've ever been. Or is it that those differences have narrowed as the game has evolved and all goalies have been forced to become at least competent in the puck retrieval game?

That's why I see the 3 shot differential between those two guys as, at least for now, the upper limit in shot differential.

It seems to be fairly reasonable. It's probably safe to assume that Atlanta doesn't somehow have both the Best and the Worst in shot prevention, but that both are fairly close to those extremes. What we do know is that team effects can largely be ruled out so surely those persistent differences largely come down to the goalies themselves.

Statman said...

Ok, it's definitive... according to Marty, without Marty the Devils would be allowing at or near the most shots/game the NHL:

http://slapshot.blogs.nytimes.com/2009/03/18/the-morning-skate-marty-and-the-media/

"For those who wish to denigrate Brodeur’s achievement and longevity because he faces fewer shots a game than other goalies, Greenberg presents this quote from Marty: “It’s nothing about style really, but about fundamentals, how I get myself in position to make the save that’s important to me. We give up 8-10 fewer shots a game here than other places because of the way I control rebounds.”"

8-10 fewer, eh? I guess he's saying his teammates are among the worst at preventing shots in the entire NHL - ouch!

So much for those "in the game" knowing more than the "nerds" who simply follow statistics. :)

The Contrarian Goaltender said...

Statman: That quote is jawdropping, thanks for the find.

Statman said...

No problem. And no, Brodeur fans, I wasn't trying to find evidence to "get" Marty... I was doing a search of K&R's Hockey Compendium... trying to find the 1950's - 70's regular season SA, GA, SV% stats listed therein.

overpass said...

Re: the Atlanta example, Lehtonen and Hedberg is the highest persistent shot differential I have ever seen (in fact, it was one of the most important pieces of evidence that won me over to your side in the shot prevention debate), and I have looked at a lot of different tandems. If you can find a higher example than those two I'd be interested in knowing what it is, because I haven't found one. That's why I see the 3 shot differential between those two guys as, at least for now, the upper limit in shot differential.

Be careful not to simply take the highest observed shot differential (Lehtonen vs Hedberg) as the upper range of shot prevention.

If you're looking for the highest shot differential, you are selecting for two things - goalie tandems with a large difference in shot prevention due to skill, and goalie tandems with a large difference in shot prevention due to chance. It's likely that the gap between Lehtonen and Hedberg is due to each of these.

Lehtonen and Hedberg have a shot differential of 2.8* over the past 3 years. The standard error on that is 0.55, so the difference is about 5 standard deviations - definitely significant. However, it's not significant at the 95% level that their shot differential skill is greater than 2, which is your estimated +/- 1 shot range.

It's likely that the highest observed shot differential will include some random variation on the high side, so I'd hesitate before concluding that 3 shots/game is the true difference between Lehtonen and Hedberg. In my opinion, your range of +/- 1 in shot prevention is good.

This is why it's important how you select data. If you just want to know Brodeur's shot differential vs his backups, there's no reason to believe that the point estimate is off, because you selected him before looking at the data. However, if you start data mining and picking the largest differentials (Lehtonen vs Hedberg), you're also selecting for luck as well as skill.

*I removed partial games from their stats, as Hedberg will face a relatively higher percentage of those, and we know teams shoot less when they are ahead - as they will be whenever a goalie is pulled and the backup comes in.