Thursday, September 11, 2008

Did Ftorek and Robinson Affect Brodeur?

There was an interesting comment posted recently by Bruce in the ongoing goalie fatigue debate that I thought deserved a closer look:

As one who doesn't put all my faith in a single column of goaltending statistics, what it indicates to me is that the Devils of 1998-2002 were a different team than the notorious "trapping" teams of the Lemaire years. The Devils of Robbie Ftorek and Larry Robinson played more offensively, took more chances, gave up more chances, and relied on their world-class goalie to limit the damage....But for those three seasons in they were a puck-possession, outscoring type of team. Such teams may or may not allow bunches of shots, but in my observation they do tend to give up a better quality of scoring opportunity when their players are tending towards the other end of the rink. One predictable result of this altered team philosophy is a lower Sv%, and Brodeur's were indeed merely "average" throughout those years.

Sounds plausible. However, all evidence indicates that it is nevertheless wrong. Brodeur's numbers did not appear to suffer from a team that took more chances. Here are his even-strength save percentage numbers from 1998-99 to 2003-04 (along with the league average):

1998-99: .915 (.916)
1999-00: .912 (.912)
2000-01: .919 (.914)
2001-02: .917 (.916)
2002-03: .921 (.918)
2003-04: .924 (.922)

That looks like the same guy doing exactly the same thing. Brodeur's performance relative to the rest of the league was very similar throughout the entire period. The only thing changing is that the league average is gradually rising. If his team was taking a lot more chances between 1998-99 and 2001-02, you would expect Brodeur to do worse relative to the league in that period, and better in 2002-03 and 2003-04 when the Devils were by all accounts a defensive powerhouse.

The main reason for the drop in Brodeur's save percentage under Ftorek and Robinson appears to be his poor play on the penalty kill, a situation that hardly seems to be affected by a team's offensive philosophy. Here are Brodeur's PK numbers (again compared to league average):

1998-99: .864 (.872)
1999-00: .908 (.866)
2000-01: .839 (.862)
2001-02: .849 (.872)
2002-03: .866 (.869)
2003-04: .878 (.867)

Note the particularly poor results in 2000-01 and 2001-02. It seems pretty unfair to blame Brodeur's team for it either - New Jersey's penalty killers were Stevens, Niedermayer, Madden and Pandolfo. In 2002-03, even with a below-average penalty kill save percentage from Brodeur, New Jersey led the league in penalty kill efficiency.

The Devils scored a lot of goals from 1998-99 to 2000-01 (their offence ranked 2nd, 2nd, and 1st in the league). But they still were excellent at preventing shots (allowing the 4th, 4th, and 2nd fewest shots in the league). It might have been a little bit more challenging to play goal for New Jersey in this period because of an increased offensive focus, but it seems unlikely that there was a large difference. Brodeur's even-strength results are pretty consistent through the period, and his save percentage variances can be explained by his performance on the penalty kill.

50 comments:

Bruce said...

Brodeur's even-strength results are pretty consistent through the period, and his save percentage variances can be explained by his performance on the penalty kill.

That is very interesting, and very, very odd. Your numbers -- where do you get them? -- certainly indicate Brodeur was his usual consistent self at evens. On the PK, his performance in the Sv% metric seems to go against the, uh, fossil record. Do you have any explanation for it? Ever the logician, I'm struggling to find any theory as to why a goalie who is otherwise on top of his game would go into a two-year slump on the PK.

Bruce said...

PS: While you're at it, can you explain why Brodeur's PK Sv% was so outstanding (+.042 better than league) in 1999-2000? That's pretty much as far above the norm as the two "bad" years put together (2 * -.023). The other three were pretty normal, and over the six years his PK Sv% was pretty much flat against the league averages you cite. That three-year fluctuation seems out of character. Is this just a fluke of small number statistics?

If it's not too much trouble, I would be interested to see the raw numbers of shots/saves in each situation (including PP, to be inclusive) for that string of seasons, if not his entire career if you have them. Thanks.

The Contrarian Goaltender said...

Your numbers -- where do you get them?

Off NHL.com.

Ever the logician, I'm struggling to find any theory as to why a goalie who is otherwise on top of his game would go into a two-year slump on the PK.

Is this just a fluke of small number statistics?


Mostly, yes. We are looking at shots against totals for the least penalized and best shot preventing team in the league, so it is not a large sample. In those two seasons Brodeur faced 279 and 251 shots while down a man. That means every extra goal would cause a save percentage drop of about .004.

In general, penalty kill save percentage is always a lot more variable than even-strength save percentage. A good example is Tomas Vokoun, here are his penalty kill save percentages since 1998-99: .839, .879, .847, .917, .873, .843, .874, .867, .895. He's a very good goalie, yet his numbers are all over the map. I am also not sure that we can assume that every good goalie is good on the PK, just like we can't assume that every good defensive player is a great penalty killer. Some guys are probably better than others on the penalty kill. Brodeur has never really been that good on the penalty kill in his career up until the last few seasons. I don't know if you read the post where I graphed it (probably not because I showed almost the exact same situational breakdown), but the improvement is substantial. Patrick Roy is another guy who had pretty average numbers on the penalty kill, at least late in his career, but he was always great 5-on-5.

Because of that variability, a lot of times you can pick out goalies with fluky seasons because they have unsustainably high save percentages on the PK. Take Chris Mason last year, for instance - .925 at even-strength, .921 on the PK. There was no way he was going to be able to duplicate that, and he didn't - his PK save % dropped to .873 and his overall save percentage dropped to .898. I am skeptical about Dan Ellis next year for similar reasons - in 2007-08 Ellis was .933 at ES, .929 on the PK. He won't be leading the league in save percentage again.

If it's not too much trouble, I would be interested to see the raw numbers of shots/saves in each situation (including PP, to be inclusive) for that string of seasons, if not his entire career if you have them. Thanks.

The data on NHL.com only go back to 1997-98, and the 1997-98 stuff appears to be incomplete. But here is the rest of it:

1998-99: 1253/1370 ES, 255/295 PK, 58/63 PP
1999-00: 1320/1447 ES, 265/292 PK, 51/57 PP
2000-01: 1300/1415 ES, 234/279 PK, 62/68 PP
2001-02: 1236/1348 ES, 213/251 PK, 50/56 PP
2002-03: 1311/1423 ES, 201/232 PK, 47/51 PP
2003-04: 1377/1491 ES, 259/295 PK, 55/59 PP
2005-06: 1494/1621 ES, 364/419 PK, 60/65 PP
2006-07: 1629/1758 ES, 311/344 PK, 71/80 PP
2007-08: 1600/1716 ES, 438/485 PK, 51/56 PP

Bruce said...

Thanks, CG. In light of the new data, I will cheerfully withdraw my theory about style-of-play effects. Unlike some people in the hockey analysis racket, I am prepared to let the facts get in the way of a good theory. It seemed logical based on the breadth of information at my disposal, but you got a few more columns that I didn't have. (And no, Anonymous, I didn't watch all 240 of those Devils games, thank goodness.)

In general, penalty kill save percentage is always a lot more variable than even-strength save percentage.

That's interesting. Your example of Vokoun is telling, and of the generally-rock-steady Brodeur even more so. I agree with your observation that not all goalies are necessarily great on the PK, but that season-over-season variation within a single goalie suggests there may be more to it. It could also be an exploited weakness on the PK unit/strategy, or even a couple of divisional rivals with deadly PP units, exacerbated by the small number statistics previously mentioned.
I've got no numbers pro or con in the case of Brodeur, I'm simply throwing these things out there as possible pieces of an interesting puzzle. These things are always more complex than the numbers that they generate.

Do you know of any place online where Sv% stats for all goalies are borken down by manpower situations?

The Contrarian Goaltender said...

Do you know of any place online where Sv% stats for all goalies are borken down by manpower situations?

NHL.com. I didn't scrape this data or anything, it is right off their stats applet. Go to Individual stats report, position: goalie, stats view: special teams, and away you go.

It seemed logical based on the breadth of information at my disposal, but you got a few more columns that I didn't have.

I must admit that your theory once seemed logical to me too, as I had a similar hypothesis about specific team effects at one point before I looked deeper at Brodeur's underlying numbers and found a lot more similarities than differences.

Anonymous said...

Interesting analysis. Intuitively, it should be much more difficult to make saves when killing a penalty, & some goalies are "penalized" (so to speak) by playing for teams that incur a lot of shorthanded situations. The "shot quality neutral sv pct" may try to incorporate this, as I recall.

"I am prepared to let the facts get in the way of a good theory."

That said, all other things being equal/relative/proportionate, which of the following goalies (same season, same league, same opponents) is "better", and why?

GP 71 70
MIN 4255 4205
W 40 27
L 19 37
T 12 6
PTS 92 60
Win% 0.65 0.43
GAA 2.21 3.10
SV% .906 .906

Or, are they so similar in skill at stopping the puck that they could be ranked as basically identical? If so, why?

Bruce said...

NHL.com. I didn't scrape this data or anything, it is right off their stats applet. Go to Individual stats report, position: goalie, stats view: special teams, and away you go.

CG: Thanks! I had no idea they had special teams stuff for goalies. Now I know.

Anon: I will make you happy and take Goalie A.

Open fire.

Anonymous said...

Bruce - why?

Anonymous said...

CG - of the 2 goalie examples I listed, what is your opinion as to who had the better season/was the better goalie that year? Why?

Or, are they basically identical?


Anyone? :)

The Contrarian Goaltender said...

CG - of the 2 goalie examples I listed, what is your opinion as to who had the better season/was the better goalie that year? Why?

Or, are they basically identical?


Well, I think I have pretty consistently said that I see very little value in games played, wins, and GAA. So based on the facts you give, and that they are tied in save percentage, it would be pretty difficult to conclusively say one is better than the other.

When I look at goalies, I look at a few things in addition to just their save percentage, including:
- how did the backups do?
- how many shots against per game?
- how good is the team?
- do we have an estimate of shot quality?
- how many shots were on the PK?
- did they win more games than they should have on that team?

I'd like to see that information before I made a judgment that I would stand behind.

However, if you put a gun to my head and made me pick one, I'd probably pick Goalie B as being better. Why? Because based on their GAA and save percentage numbers, I estimate Goalie A as facing 23.5 shots per game, and Goalie B as facing 33.0 shots per game. Also, from the Pythagorean expected points formula, I estimate that Goalie A's team is scoring about 2.7 goals per game and Goalie B's team is scoring 2.5. Assuming that we are talking about the current NHL, that means Goalie A's team is about average offensively while Goalie B's team is below average. If one of the goalies is winning a lot more games than expected based on their team's goals for/goals against ratio, that would have change the decision, but you didn't give that info.

So based on the profile of average offence/best-in-the-league shot prevention and below-average offence/worst-in-the-league shot prevention, I estimate the second team is likely to be a bad team all around and therefore more likely to give up more dangerous shots than the first one. If so, a .906 would be more impressive in that situation.

This generalization is not always true, but like I said we are missing a lot of important information. So based on those estimates in the absence of other critical information, I suppose I would take Goalie B, but I would not be very confident in that assessment.

Anonymous said...

CG: Thanks for the analysis. Correct, I used 23.5 & 33 SA/game, respectively (see your previous article where I show the #'s I used).

As for shot quality, backup success, PK shots... I intended that the 2 goalies faced similar circumstances, just that one team allowed more total shots (& a proportionately higher number of PK shots), & were equally relatively better than their backups.

So I intended that they goalies appear identical.

I'm trying to get past using "name goalies" & eliminate subjectivity, career records etc.

The Contrarian Goaltender said...

I appreciate what you are trying to do by focusing just on performance not on reputation. I agree it is difficult to evaluate goalies without letting past history affect your judgment, which is partly why I like to focus so much on the numbers.

If Goalies A & B stopped pucks at the same rate with the same shot quality against and opposing power play chances and outperformed their backups by the same relative amount, then I have no problem calling their seasons identical, whether or not one was 40-19-12 and the other was 27-37-6.

Bruce said...

Anon: Because Goalie A was more successful. Team success might be secondary to individual success (which near as I can tell is judged by Sv% and Sv% alone around here), but even if it's used as a tie-breaker one guy helped his team win more often than the other. Sorry, they don't play in a vacuum and results matter.

I have no trouble accepting that it's hard to compare, say, Gilles Meloche to Ken Dryden when their circumstances were so vastly different. The good goalie on a weak team is more likely to be the team MVP, have stolen more wins (due to more opporunity to do so), and to receive the Victoria Cross for saving his club from an embarrassing season. That's valuable, make no mistake. But given such a comparison between two strong goalies I will look across the board and choose the guy whose contributions have delivered mre team success. I know you feel differently, and we will just have to agree to disagree on this point.

If OTOH Goalie B had a Sv% of .930 and Goalie A of .900 that would give me pause as to which one actually played better. But if there's little or nothing to choose in that important category, and the others are all tilted one way, I'll take the easy way out and go with the winner.

Anonymous said...

Bruce, therein lies the problem... I purposefully created 2 identical goalies, the only difference being that one played on a weaker team & faced more SA/60 with a corresponding % of PK shots against, etc. (while his team also scored less). All other factors were designed to be the same.

Yet you still conclude Goalie A is better than B? In one sense, Goalie A had a more 'successful' season than B, because he & his team won more games, & Goalie A undoubtedly feels better about the season in general (who wants to play on a loser?), but since the goalies' abilities were IDENTICAL it was merely by chance that one had better "counting numbers" than the other... the chance that one was placed on a good team & the other on the bad team.... they are in fact exactly the same goalie. There is no reason, none at all, to choose one goalie over the other. And I think this shows why you continually point to Brodeur's seasons - even his mediocre/avg seasons - as impressive, when they aren't always so. He's had several years where any number of NHL goalies could've taken his place on the Devils & ended up with better "counting numbers", because they were just plain better at stopping the puck.

Goalies should be evaluated objectively, season by season, as though we don't know their names or their history, & then the seasonal evaluations should be accumulated for an overall career evaluation.

Anonymous said...

"...one guy helped his team win more often than the other."

Well, one team won more, but they each helped their team win (or tie, or lose) by same increment by providing the exact same skill & ability... by providing the exact same contribution to the outcome of each game they played.

Since the goalies were identical in ability to stop the puck, if you reverse their teams the teams end up with the same record as before.

Goalie A had no impact on his team winning 40+ games instead of 27+ games, and Goalie B had no impact on his team winning 27+ games vs. 40+.

The Contrarian Goaltender said...

That's not technically true, Anonymous. I agree with your general point, but a goalie can win additional games for his team with "clutch play", i.e. if the distribution of goals he allows allows his team to optimize its results.

For example, if both of the goalies you identified in your example had the same number of goals scored by their teams while they were in the game, then Goalie A's winning percentage is about .050 higher than what would be expected and is therefore more impressive than Goalie B's.

Whether or not clutch ability is a skill is a separate topic, but the distribution of goals does have an impact on team results, e.g. a goalie who lets in 0, 0, 0, and 16 goals has the same GAA and possibly save percentage as somebody who lets in 4, 4, 4, and 4, yet the first goalie will have 3 wins while the second goalie might not have any.

The Contrarian Goaltender said...

But given such a comparison between two strong goalies I will look across the board and choose the guy whose contributions have delivered mre team success. I know you feel differently, and we will just have to agree to disagree on this point.

I don't disagree with your view, Bruce, and I'm not sure Anonymous does either. The player who contributes more to winning is more valuable, because the goal of hockey is winning. Where we disagree is in our definitions of what is a contribution to team success. I think the guy who was the better goalie contributed more to winning. That is true regardless of whether or not his team actually won the game.

My view is that if a goalie stops, say, 28 out of 30 shots and his team loses 2-1, his contribution to his team winning is exactly the same as if stops 28/30 and his team wins 3-2 (assuming equal shot quality and strength of opposition). You seem to award bonus points if the sum total of the goalie's play plus the rest of the team's contribution results in the team winning the game, i.e. the goalie's play is more valuable if he is on the winning team. I see the determination of which team is the winner as a completely separate issue, and I think that is the point that Anonymous is making as well.

The evidence I have suggests that a goalie's contribution to his team's success is overwhelming by stopping the puck. If there is evidence that other factors are significant by themselves beyond their effects on save percentage (e.g. perhaps timing of saves, puckhandling, rebound control, etc.) then I will include that. I concede that lack of quantifiability is potentially a barrier to us fully valuing some of these things, but I am continuing to try to find different ways to look at the problem and hopefully we can at some point fill in any missing gaps.

Let me throw another couple of examples into the discussion. What about these two scenarios:

1) There is one minute left in a playoff game. An opponent gets a breakaway, bearing down on Goalie A. Goalie A makes the save to keep his team ahead 2-1, and his team wins the game and the series.

2) There one minute left in a playoff game. An opponent gets a breakaway, bearing down on Goalie B. Goalie B makes the save to keep the score tied and force overtime. However, his team eventually concedes a goal to lose the game and the series.

Is Goalie A's save more valuable than Goalie B's? Why or why not?

Anonymous said...

CG: in this example, their skills were to be identical so they would have equal "clutch"/consistency ability & so there would be difference in goal distribution rates or weird shutouts/16GA occurrences. In my example, whether or not they have 40 wins or 27 wins depends on how many SA the team allows & how many goals the team scores.

I agree with you about the 0, 0, 0, 16 = 4, 4, 4, 4 although the first goalie might be 0-1-3 (if his team didn't score any goals).

Anonymous said...

... there would be no difference in...

Bruce said...

Is Goalie A's save more valuable than Goalie B's? Why or why not?

Goalie A's save is more valuable because it was the last save. Ultimately Goalie B didn't stop the last shot, and his team lost.

Caveat: At the game situation of that moment, both are more or less equally huge. Ultimately, however, things will be viewed -- at least by me -- through the filter of did he win or did he lose? I stress that is not the only filter, but it's an important one. I mean, if Detroit outshoots Edmonton 57-25 and wins 3-2 in double OT (Game 1 in '06), I got a pretty good idea who I'm going to pick as the first star of the game. But when the performances are generally equal,as in both CG's and Anon's examples above, I'll take the winner.

Anonymous said...

Exactly, CG: "My view is that if a goalie stops, say, 28 out of 30 shots and his team loses 2-1, his contribution to his team winning is exactly the same as if stops 28/30 and his team wins 3-2 (assuming equal shot quality and strength of opposition)."

As for your last-minute example, I think I'd need more info. We do know that Goalie A & Goalie B made the "same save" & so with respect to that that shot (but perhaps not the entire game) they are equal... Goalie A just had the fortune to not have to go to OT.

Anonymous said...

Bruce, in my hypothetical the goalies' performances are not generally equal, they are exactly equal. The difference in counting numbers has nothing at all to do with their own abilities, but is the logical result of playing for teams of different qualities. It's impossible to conclude one is better than the other.

Bruce said...

Anon: Yeah I got that. You asked me which one I would pick and why, and I told you. I never even suggested that Goalie A probably faced fewer shots cuz he was a better puckhandler while the other guy had trouble with rebounds. :)

Now a challenge: Find two goalies in real life whose performances are "exactly equal".

Anonymous said...

Bruce, I asked you which one was better, & also I gave you the option of declaring that they were basically identical. You showed your bias.

I suppose it could be hard to find 2 real-life goalies whose performances are "exactly equal", but if we can't even come to a reasonable conclusion about hypothetical-goalie performances even when all assumptions indicate that they are "exactly equal", how on earth can one reasonably assess real goalies?

What we can do is apply a goalie's predilection for making saves (aka save pct) to circumstances different than his real circumstances (increase the SA, decrease the team GF) & come up with some accurate counting numbers.

BTW, in my example Goalie 2 would have to have a Svpct of .921 just to have a .500 record (70 pts).... still much more feeble than the mighty 40 win, 92 pt season of Goalie 1, however... even though Goalie 1 only had a .906 sv pct.

Bruce said...

Bruce, I asked you which one was better, & also I gave you the option of declaring that they were basically identical. You showed your bias.

Yup. They were identical in just one category, which happens to be the only one you seem to put any stock in. My bias is that the other columns are also a measure of the goalie's performance, and even if they are secondary yardsticks, all the tie-breakers point to Goalie A.

So sue me.

As per your last paragraph, where Goalie B is at .921, and his peripheral stats commensurately better (team at .500, GAA south of 3.00), at least I have a reason to think about who had the better season.

Anonymous said...

Bruce - So if Goalie 1 had the ability in a certain season of saving .906 of all shots, & prior to the season was traded to a much weaker team, such that his .906 translated into:

GP 70
MIN 4205
W 27
L 37
T 6
PTS 60
Win% 0.43
GAA 3.10
SV% .906

...he's now worse than he would've been with his original team?

How does that work? He's the exact same goalie with the same ability.

Anonymous said...

... I guess by your calculations, Goalie 1 (now on the weaker team) is now 33% worse (40 wins down to 27), etc.

Bruce said...

So if Goalie 1 had the ability in a certain season of saving .906 of all shots

There you go again... the goalie's entire performance can be boiled down to one stat category, with extra sympathy points for a goalie stuck on a sucky team. You have revealed your own bias.

As for hypotheticals where you know the rules and I don't, well who cares? How about a hypothetical Universe where we clone Goalie A, and set it up so Goalie A-1.0 plays on the best team in the league and Goalie A-2.0 plays on the worst. Except now we might as well clone all the other players so that all the teams are the same too.

I've answered your question twice and stand by what I said. If you want to use that as proof that I'm a biased idiot, well be my guest.

Anonymous said...

Bruce, it all starts with stopping the puck... stopping the puck leads to wins, losses, or ties, [depending mainly on how many shots he faces, shot quality, & how many goals his team scores... sorry, I guess that's too complicated so let's just say that the guy with 40 wins must be awesome... yeah...]

It's not the other way around; e.g. wins don't lead to stopping the puck. "Winning the game" doesn't mean the goalie had a good game.

A goalie can have identical performances & win, lose or tie the game. We can identify & measure this game by game, & cumulatively for a season. A player can have an identical seasonal performance & have apparently great "counting #'s", or apparently mediocre ones.

It's a mathetmatical certainty that a goalie that has a svpct of .906 can have variable seasons such as the 2 I listed. I'm curious as to why you wouldn't address my last question, but instead attacked the use hypotheticals? Certainly not very logical or scientific. I guess hypotheticals don't allow you to say, "hey it's Brodeur!! He HAS to be great!"

Your arguments are very irrational & just don't make mathematical sense. I think there are plenty of opinion-based sites to go visit, where facts & stats are too confusin' & therefore avoided...

Anonymous said...

By the way, I obviously didn't give sympathy points to the guy on the bad team, & I have no bias either way. If you understood the #'s you wouldn't have said that. They are identical goalies, who saved .906 of all shots. They played under different circumstances, leading to vastly different counting #'s. I'm interested in rationally assessing performances, not fawning over certain numbers when their performance is actually the same.

I don't think I can explain this any simpler than that.

Anonymous said...

By the way, I obviously didn't give sympathy points to the guy on the bad team, & I have no bias either way. If you understood the #'s you wouldn't have said that. They are identical goalies, who saved .906 of all shots. They played under different circumstances, leading to vastly different counting #'s. I'm interested in rationally assessing performances, not fawning over certain numbers when their performance is actually the same.

I don't think I can explain this any simpler than that.

Bruce said...

"Winning the game" doesn't mean the goalie had a good game.

Nope, it just means he accomplished the primary objective of the game. The other guy didn't. Given the choice between the two -- ESPECIALLY when all other factors are equal as per your impossible hypothetical example -- I will take the guy who was successful in helping his team accomplish its goal.

I don't think I can explain this any simpler than that.

If you want to watch a sport where one person's performance is completely divorced from all other participants, I recommend golf.

Your arguments are very irrational & just don't make mathematical sense. I think there are plenty of opinion-based sites to go visit, where facts & stats are too confusin' & therefore avoided...

Seems to me I've been citing statistics and facts all along. In fact I am the one who is willing to look at more than a single column to interpret a goalie's overall performance and impact. Meanwhile you are the one who's been whining about me being "over the top in terms of sarcasm & borderline personal insults."

Reread what you just wrote in that light. It's a two-way street, buddy. For the 50th time, let's agree to disagree ... we have a fundamental philosophical difference of opinion.

Anonymous said...

Bruce, how does a goalie accomplish the primary objective of the game? How is a guy "sucessful in helping his team accomplish its goal"? It seems that you think something magical happens, & even if he saves no more (in %) than someone else on a weaker team, he has somehow helped his team win. Strange.

You must be one of those guys who thinks a pitcher who is 22-11 with a 3.45 ERA is far better than the 15-16 guy with the 2.87 ERA (coincidentally, playing on a far worse team).

I'm going to make a wild guess & say you're an older guy, not too familiar with important hockey stats (at least not until recently), who has long marvelled over the impressive "counting numbers" that the greats of sports have accumulated... & after forming your opinions over many years, you're not about the change them, facts & numbers be damned.

By the way, what would you think of Brodeur had he played for a far worse team & therefore had maybe 2/3 the career wins, a higher GAA, less shutouts, no Cups, few or no all-star selections, etc? Everything has changed, but his actual goaltending performance (e.g. stopping the puck, mainly reflected in svpct). I suspect in that case, I'd be the one saying he is underated, & you'd say his counting numbers aren't impressive & why are we talking about him? ha

Anonymous said...

Correction to first para: It seems that you think something magical happens, & even if he saves no more (in %) than someone else on a weaker team, he has somehow done something extra, something special to help his team win.

Bruce said...

I'm going to make a wild guess & say you're an older guy,

Bingo. I've been an avid follower of the NHL since the 1962-63 playoffs, when I fell in love with the game as a seven-year-old. Played a little goal back in my youth (more soccer than hockey), and have always watched both sports from a goalie's perspective.

not too familiar with important hockey stats (at least not until recently)

Well if you are talking about "important" stats like SQNSv%, they haven't been available until recently. Otherwise you couldn't be more wrong; I have been a huge stats buff my whole life, have always stayed on top of "new" stats like +/-, Sv%, TOI, and RTSS as they have been gradually introduced to the game. A disciple of the methods of Bill James since the early 1980s, in recent years I have become a member of the Hockey Analysis Group and have followed the work of people like Alan Ryder, Iain Fyffe, Gabriel Desjardins, and other New Statistics trailblazers with interest (although this isn't my full-time job!). I do, however, tend to rely on traditional statistics as the best ("only") measure of players in the historical sense.

, who has long marvelled over the impressive "counting numbers" that the greats of sports have accumulated...

Certainly. My particular interests are hockey and baseball. While "counting numbers" are impressive, I have learned to take into account such things as era effects, team effects and so on. It is reasonable to compare Babe Ruth's 1920-23 seasons to Barry Bonds' performances in 2001-04. But I'm impressed but not overwhelmed with Hack Wilson's 190 RBI in 1930 (the year of the hitter) or Bob Gibson's 1.12 ERA in 1968 (the year of the pitcher). Great performances sure, but the numbers simply cannot be accepted at face value.

Same goes for Joe Malone's 2.2 goals-per-game in 1917-18, George Hainsworth's 22 shutouts and 0.98 GAA in 1928-29, or Wayne Gretzky's 92 goals in 1981-82. They are great performances, sure, but for proper perspective the raw numbers need be distilled through the filter of era effects. The survivors in the pantheon of greatness tend to be the ones who continue to post elite numbers atop or among the league leaders for a number of seasons (e.g. Gretzky would have been spectacular in any era, although his numbers would have been lower).

& after forming your opinions over many years, you're not about the change them, facts & numbers be damned.

Well that's your interpretation. My opinion of Marty Brodeur's greatness has actually strengthened over the course of this long discussion. I am willing to admit that he has been a net beneficiary of his era and his team; not unlike Gretzky on the free-wheeling Oilers he is in a very good situation. So his numbers need to be taken with a grain of salt. Nonetheless he taken full advantage of that situation and has been piling up those numbers year after year. I happen to think it's meaningful that the guy plays 5-30 more GP than most of his peers and the fact he maintains his percentages at elite levels adds value; you guys don't and we just disagree on that point. I also don't think that his raw Sv% numbers are truly reflective of his influence on a game (this from my soccer background), but there is no good way to quantify the subtle influences so we'll just have to disagree on that too. That doesn't make me a moron or a fanboy; we just have a different philosophy on some of these matters.

Finally I will point out that not once, never ever have I claimed that Martin Brodeur is the best goalie in history. Indeed I have unequivocally agreed with CG that Dominik Hasek was the greatest goalie during the Dead Puck Era which covers most of Brodeur's own career. I do, however, think Brodeur is the most efficient goalie I have ever seen, on one of the most efficient teams. As a result MB will wind up in the view of many as being the best statistical goalie in history, as eventually he will lead by wide margins all the "counting numbers" that you devalue so much -- GP, Wins and Shutouts -- while posting one of the very best GAA of any goalkeeper who didn't start in the late 1920s. With that statistical legacy, those Cups, gold medals, and trophies, he will be a slam dunk first ballot Hall of Famer. I sure in the hell would vote for him, let's put it that way.

Anonymous said...

So, if Brodeur had played for a worse team & therefore had maybe 2/3 the career wins, a higher GAA, less shutouts, no Cups, few or no all-star selections, etc, but his actual goaltending performance (e.g. stopping the puck, mainly reflected in svpct) remained the same, he'd be less of a goalie?

I'd say he be exactly the same... a handful of very good/excellent years, and a few mediocre yrs.

Bruce said...

Anon: My thinking is that Brodeur would have made whatever team he joined a better team. Just as he did in Jersey, a middling team that had never won more than 40 games in its history before he arrived, but which has now exceeded that milestone for 11 seasons running, with Brodeur himself having exceeded 40+ Wins on no fewer than 7 occasions. Yeah I know about Bettman points which have corrupted the statistical integrity of the league (don't get me started!); point is New Jersey immediately got far better when Brodeur joined the team, and have remained a contender ever since.

You don't think he would have made such an impact on another club? I guess that's your opinion.

Anonymous said...

I think MB would have a very similar affect on any team, whether it was the 1974-75 Capitals or the 1976-77 Habs. His ability to stop the puck wouldn't change just because his jersey has.

Whether his team won, lost or tied is a function of his ability to stop the puck, the number & quality of shots his team allowed (virtually nothing to do with him), & the number of goals his team scored (ditto).

Before & after MB joined the Devils: how did their team stats vary? (e.g. SA/60, times shorthanded) What other personnel came/went during this time?

Oh, that's so difficult to examine... let's just largely attribute the 40+ wins to MB & call him great.

Bruce said...

Before & after MB joined the Devils: how did their team stats vary? (e.g. SA/60, times shorthanded) What other personnel came/went during this time?
Oh, that's so difficult to examine... let's just largely attribute the 40+ wins to MB & call him great.


I got a better idea: let's just attribute the 40+ wins to Lou Lamoriello remaking his roster from top to bottom during the 1993 off-season, bringing in excellence in every positon except goal, where suddenly mediocrity is enough to ride the wave for the next 15 years.

Of course I'm not saying it was all Brodeur, I'm just pointing out the franchise's fortunes soared coincident with his arrival, which in my mind is NOT a coincidence.

Anonymous said...

Actually, I've already shown a hypothetical where a mediocre goalie (avg svpct) can have a 40 win season.... & it was based on 2 of Brodeur's seasons where he had just that kind of performance.

"I'm just pointing out the franchise's fortunes soared coincident with his arrival, which in my mind is NOT a coincidence." yeah, no point in trying to separate out goalie effects vs. non-goalie effects... that's for some fancydancy sportsmath site...

Bruce said...

no point in trying to separate out goalie effects vs. non-goalie effects... that's for some fancydancy sportsmath site...

By all means, have at 'er. When I tried to estimate goalie effects on shots against you basically just discounted the whole idea as if the goalie is strictly a passive bystander with no effect on the game, and ridiculed me for suggesting otherwise. Rebound control + crease management + puck retrieval + puck handling = a big fat zero in your books, it's all about stopping the puck period. Sorry, the goaltender's responsibilities are a little more complex than that these days.

how did their team stats vary? (e.g. SA/60, times shorthanded)

Here's one way the Devils stats varied: during the Ftorek-Robinson years 1998-2002 (Brodeur's "mediocre" years) the Devils had more powerplays against than for, every year. That was different from their M.O. during the years they had more defensive minded coaches.

"I'm just pointing out the franchise's fortunes soared coincident with his arrival, which in my mind is NOT a coincidence." yeah, no point in trying to separate out goalie effects vs. non-goalie effects...

Man, you are like a pit bull. You never accede to any point I ever make, you are predisposed to argue with every single thing I say. When I try to find some middle ground by saying something like "Of course I'm not saying it was all Brodeur, I'm just pointing out the franchise's fortunes soared coincident with his arrival, which in my mind is NOT a coincidence" you edit out the first part of my sentence and attack me yet again.

This conversation is going nowhere. You think all any goalie does is stop the puck; he doesn't affect the number of shots against him, he doesn't affect the quality of shots against him, he doesn't affect the flow of play, he doesn't affect the game his own team plays, he doesn't get into the heads of shooters on other teams; no, he's just an automaton with a .906 Sv% regardless of circumstances. And I say that is simplistic and wrong. We are never going to agree on this, you aren't prepared to compromise even in the slightest, so I think I'm about done with this discussion.

Anonymous said...

"Rebound control + crease management + puck retrieval + puck handling ="

equal to.... what? Can you rank each goalie in the NHL according to these categories? What effect does each category have on the game?

Awhile back here we approximated the effect of a goalie reducing the number of shots due to the factors you mention, & applied this to save pct. It wasn't a big effect. (And still, where are those rankings in each category? Are these categories objective, or more the result of 500 game announcers repeating the mantra of how great some goalie is at e.g. puckhandling... the same analysts who proclaim some goalie as the one of the best ever, yet the #'s don't prove it?)

You may have been following hockey for 45+ yrs, but we are way past just looking at GP-W-L-T-GAA-SO-SVPCT & then "eyeballing" these #'s & declaring that e.g. 60-20-35-5-3.50-2-.910 is worse than 70-40-20-10-10-2.50-.905 but better than 65-25-30-5-4-3.40-.907, because of some unknown apportionment of importance to each category, combined with hocuspocus assessments of "crease mgmt" & "puck retrieval" (let's not forget shooting, playmaking & inspiration/leadership). Very unscientific.

Bruce said...

Awhile back here we approximated the effect of a goalie reducing the number of shots due to the factors you mention, & applied this to save pct. It wasn't a big effect.

I postulated it was as much as +/- 2 shots per game, which translated to +/- .005 Sv%. Both of which I would say are a big effect. It is, as you say, impossible to measure; it's a variation of Wayne Gretzky's statement "100% of the shots you don't take, don't go in."

You may have been following hockey for 45+ yrs, but we are way past just looking at GP-W-L-T-GAA-SO-SVPCT & then "eyeballing" these #'s & declaring that e.g. 60-20-35-5-3.50-2-.910 is worse than 70-40-20-10-10-2.50-.905 but better than 65-25-30-5-4-3.40-.907, because of some unknown apportionment of importance to each category, combined with hocuspocus assessments of "crease mgmt" & "puck retrieval" (let's not forget shooting, playmaking & inspiration/leadership).

Which season is better or worse is very difficult to judge, I will readily agree. I'm actually not all that keen to say this season is better or that one, unless cornered by a pitbull running hypothetical seasons by me and getting in my face asking: "Which one's better? Huh? Huh? Which one? Huh? Pick one, pick one" and then dumping all over my conclusions. There's a lot of good goalies out there in different circumstances, and it's mighty hard to compare, say, Tomas Vokoun to Chris Osgood in 2007-08.

I am more comfortable looking at clusters of seasons or, better, entire careers and making judgements (such as they are) based on that. Some goalies are erratic, all over the map (e.g. Jose Theodore, Manny Fernandez). Others move around the league, making interpretation difficult, in part for the reasons you point out (e.g. Roberto Luongo, Nikolai Khabibulin, Dwayne Roloson, Sean Burke). Some are injury-prone (Kari Lehtonen, late-career Dominik Hasek) And a few find stable environments where they post mostly stable results over the long term (Marty Turco, J.-S. Giguere, Evgeni Nabokov).

Needless to say Martin Brodeur fits in that last category. So it's impossible to say how he would have done had he moved around a lot (as Ed Belfour did), or played on the Islanders or Panthers (as Luongo did), or what have you ... they're all hypotheticals. What is a fact is that Brodeur has been neither injury-prone nor erratic in posting across-the-board good-to-excellent numbers at an extremely high work rate for a decade and a half. He stabilized the Jersey crease pretty much from the moment he arrived and it remains stable 15 years later. Not too many goalies you can point to in the history of the league who meet that standard.

Of course if you choose to not value things like stability, durability, longevity, workload, peripheral skills (beyond simple stopping of the puck), leadership, and team success, and focus only on Sv%, well he's only among the leaders in career rate. So obviously it's all a myth.

Very unscientific.

This just in: it's a game. And for me, a hobby.

Anonymous said...

I don't think it's impossible to estimate how Brodeur or any other goalie would do under various circumstances. In fact, I say it's far easier to reasonably make this estimate for goalies than it is for skaters; apply save pct (preferably broken down into ES, PK, PP) to various team circumstances... unless you think e.g. a .906 Brodeur (NJ) season will become a .915 Brodeur (TB) season, as he increases his svpct in order to try to make his team win more often than they would've with a .906 goalie.

[+/- .005? how many shots is someone like Brodeur preventing per game compared to the avg goalie? 5, 10?? I think that's overstating it.]

The Contrarian Goaltender said...

I think those of us in the save percentage crowd need to be careful that we don't go too overboard with our reliance on the numbers, or dismiss other possibilities simply because there are certain things that we can't quantify or are difficult to estimate. I am reminded of the baseball stats guys who scoffed at defence in the 1990s because it was difficult to measure, but were later proven wrong by more advanced metrics that showed its significance. Rebound control, crease management, puck retrieval, puck handling are real skills that goalies practice at all levels of hockey. It is unlikely they have no value at all. However, I still hold that they have little marginal value because there is unlikely to be a wide spread of ability across the far-end-of-the-bell-curve sample of NHL goalies, and secondly are likely to impact a goalie's stats fairly directly, either by preventing or creating dangerous scoring chances that will raise or lower a goalie's save and goal-prevention statistics.

Do I think it is reasonable that Martin Brodeur prevents 2 shots per game compared to another goalie who is terrible at rebound control, terrible at puckhandling, terrible at crease management, etc.? Sure. But how many NHL goalies are bad at all of those things? Probably none of them, or they wouldn't be there. So how does Brodeur do against another top goalie who is good at all of those things? I'd say that margin is probably more like 0.2 shots per game than 2 shots per game. You can't give Martin Brodeur a .005 bonus because he is good at certain things without taking into account how good everyone else is at those same things. And it is that baseline is the reason I don't put much emphasis on those "soft skills".

Of course if you choose to not value things like stability, durability, longevity, workload, peripheral skills (beyond simple stopping of the puck), leadership, and team success, and focus only on Sv%, well he's only among the leaders in career rate. So obviously it's all a myth.

This is obviously a major point of disagreement between us, Bruce. I place a lot more weighting on peak value than you do. And that, I believe, is a matter of opinion rather than something that can be settled by a debate. So as long as you are consistent in applying your evaluative criteria, then you look at careers from a different angle than I do and that's fine.

By the way, if I looked at the career leaders in team-adjusted and era-adjusted save percentage, Brodeur would almost certainly not be among the leaders. Which is where the whole "myth" argument comes from. But his career is still going, and the research on goaltending is still continuing, so we'll see where he ends up when all is said and done.

Anonymous said...

I agree with CG about 'soft skills'... how many times per game (or season) do you see any NHL goalie & think, "what a terrible puck retrieval!! The goalie can hardly skate/pass/shoot... & look, it's lead to a shot/goal!!" Not too many times... I would be surprised if the absolute worst NHL goalie allows more than 5-10 goals in a year than the best NHL goalie with respect to 'soft skills' only.

Of course, not all shots against are the same, which is why I think situational factors & "shot quality" are very important adjustments & I hope they continue to be explored. But even there, players on good defensive teams (say... like NJ) tend to have their svpct's adjusted down, not up.

The 80's Oilers were my favourite team, but even I thought that Fuhr was very overrated (albeit "exciting"). I don't have anything against any particular goalie or team.

Bruce said...

I postulated it was as much as +/- 2 shots per game, which translated to +/- .005 Sv%.

[+/- .005? how many shots is someone like Brodeur preventing per game compared to the avg goalie? 5, 10?? I think that's overstating it.]

I think I stated clearly, and in the same sentence, +/- 2 shots. As in an aggressive-below-the-hashmarks goalie with good soft skills "might" affect SA by -2 compared to an average goalie, and a stick-to-his-crease and let the defencemen get the puck goalie might be +2. Then there's rebound control, and other soft, difficult-to-measure skills like a good stick to pick off goalmouth passes, caroms off the backboards, etc. I repeat this will prove very hard to measure quantifiably, but a good place to start is by closely watching this aspect of the game. I for one intend to do more of this in 2008-09, and will conduct simple experiments like counting touches of the puck by the goalies in selected games, and perhaps judging those plays as positive (a good breakout pass), neutral (simply teeing up the puck behind the net), and negative (giveaways, getting in the defenceman's way). Don't know how hard it'll be to keep track, but I'm gonna try.

I would be surprised if the absolute worst NHL goalie allows more than 5-10 goals in a year than the best NHL goalie with respect to 'soft skills' only.

For a 75 GP guy like Brodeur, 10 extra goals on 2000 shots would indeed be a Sv% swing of .005. Of course in your estimate that's the range top to bottom, whereas in mine +/- .005 is a swing of +.010. Either way it's a guesstimate, we're just trying to find an approximate order of magnitude.

Just accepting your simple number of 10 GA, at an established rate of almost 1 standings point for every 1.75 GA, a guy like Brodeur would turn a 10-goal swing into about 6 points in the standings. Which is not nothing.

The 80's Oilers were my favourite team, but even I thought that Fuhr was very overrated (albeit "exciting").

Anonymous: one thing we can agree on is our favourite team. I was a season ticket holder and attended virtually every Oiler home game throughout the 1980s. I'm not sure I agree that Fuhr was overrated, but he would be an outlier in virtually any statistical study given his unprecedented goal support and the propensity of that team to trade high-quality chances with its opponent, confident in the fact they had the better goalie and the better snipers. But that's a discussion for another day. :)

The Contrarian Goaltender said...

The question of Grant Fuhr came up on a previous day for me, and a discussion it certainly was. You Oiler fans might be interested in this lengthy debate I had with a Fuhr fan some time ago. It was the longest comment thread I had on this blog until you guys decided to break it with your epic wins vs. save percentage duel in this very comment thread.

The Contrarian Goaltender said...

I for one intend to do more of this in 2008-09, and will conduct simple experiments like counting touches of the puck by the goalies in selected games, and perhaps judging those plays as positive (a good breakout pass), neutral (simply teeing up the puck behind the net), and negative (giveaways, getting in the defenceman's way). Don't know how hard it'll be to keep track, but I'm gonna try.

It would be interesting to see what you come up with. Just to reiterate, if there is good evidence that puckhandling is an important skill and that certain goalies are real difference-makers in that area I will immediately incorporate it into my analysis. I love puck handling as a goalie, and I like watching it done well, it is just my opinion at the moment that it is not particularly valuable as a skill.

I'm definitely going to try to catch a few DiPietro games, because in my analysis of giveaways he was just off the charts. I wonder if he is really helping his team out by playing the puck, despite his natural abilities with the stick.

Anonymous said...

But Joseph averaged 2.7 saved caroms-off-backbds per game...