Courtesy of frequent commenter Statman, here is an absolutely unbelievable quote from Martin Brodeur from an article by Jay Greenberg at the New York Post:
"It's nothing about style really, but about fundamentals, how I get myself in position to make the save that's important to me," Brodeur said. "We give up 8-10 fewer shots a game here than other places because of the way I control rebounds."
I know the Post is basically a tabloid, but unless Greenberg made it up that is a fascinating quote. Brodeur is implying that the Devils are the worst team in the league at shot prevention, and are being bailed out entirely by his efforts. We must also be forced to conclude that Scott Clemmensen, Kevin Weekes, and every other backup goalie the team has had during Brodeur's career also has terrific rebound control skills, because they aren't facing 8-10 fewer shots per game (in fact, after getting peppered over the weekend, Brodeur moved to within just 0.2 shots against per game of Clemmensen). By implying that his team was allowing enough dangerous shots that an NHL-calibre goalie would give up that many rebounds, and that the other team could actually get their sticks on those rebounds even if they were available, Brodeur is throwing his defence under a bus without even realizing it. For the sake of reference, a good estimate of the average number of rebound shots faced by an NHL goalie in a single game is 2 or less.
Martin Brodeur is one of the best puckhandling goalies ever, and one of the best goalies in the league at controlling rebounds, and yet he apparently isn't even in the correct order of magnitude in terms of estimating the actual effect of those things. That is something to make one think long and hard about the relative worth of subjective evaluation compared to statistical analysis.
Subscribe to:
Post Comments (Atom)
46 comments:
Oh my.
I wonder how many journalists - sorry, sports writers - take that 8-10 figure as being accurate... I suspect a lot.
That's like Gretzky in his Oiler prime believing that he usually has 20-25 shots on goal per game (actual number 4.5ish during his big Edm yrs).
"Controlling your rebounds is something I really care about. If I control my rebounds I feel I'm able to cut down 8-10 shots a game. That's the reason why there's a difference in the (number of) shots against in New Jersey compared to other places. I believe it's the way I manage my game."
http://www.nj.com/devils/index.ssf/2009/03/martin_brodeur_developed_a_pla.html
http://msn.foxsports.com/nhl/story/9351246/Brodeur-never-one-to-chase-butterflies-
No need for statistical analysis, then. Just ask those in the game!
This quote reminds me of quotes from managers in baseball a few years back, before the statistical revolution there was widely accepted. They'd say things like "Andruw Jones (or Rey Ordonez, or Omar Vizquel) saves a run a game with his glove."
The stats guys were all over that, and of course it was ridiculous. (Of couse, the stats guys were often guilty of underrating or ignoring defense themselves when they didn't have accurate metrics to measure it.)
Anyway, I think the biggest reason that professionals overestimate these soft skills (other than hyperbole) is that they aren't really thinking about replacement level or comparing them to other players. Martin Brodeur probably thinks of 8-10 plays per game that he makes a play that might save a shot. In his mind, that equals 8-10 shots saved. He's not considering that other NHL goalies will make most of those plays, and that the few times those plays aren't made the defence will adjust.
I think Martin Brodeur is implicitly comparing himself to a very bad, non-NHL goaltender in that quote. Bobby Cox did the same when he claimed that Andruw Jones saved a run per game. Cox was probably right in the sense that Andruw Jones saved a run per game over Cecil Fielder in centre field. We know Brodeur doesn't save 8-10 shots per game over another NHL goaltender - but maybe he saves 8-10 shots per game over Lou Lamoriello in goal.
In any case, this quote should end Martin Brodeur's credibility as an objective and critical evaluator of hockey.
I've never until now seen any evidence that Brodeur has an inflated self of sense worth... it surprises me to read this. Obviously his comments were way off the mark.
Agreed with James. Brodeur has the on-ice confidence that star players have and need, in fact Jersey players and management have commented on how things like Brodeur's body language raise the confidence of his entire team. Roy had that too, in spades.
But this quote, if accurate, is both out-of-(known)-character and way off the mark. He'd be better off speaking with his body language than his mouth, if that's any indication.
I agree with that too that this is somewhat out of character for Brodeur. Usually Brodeur goes out of his way to give credit to his defence, and I think he realizes that he has benefitted from some quality players and teams in front of him. I doubt he meant it to draw attention to himself, he probably was just throwing out a off-the-cuff estimate that happened to be way off the mark.
Like Overpass said, there are lots of parallels of professional athletes and coaches massively underrating or overrating the impact of certain skills. Maybe all the media coverage is going to Brodeur's head, but it's probably more of a bad guess than evidence of lack of humility.
He's in his 15th NHL season... has faced 25,000 career NHL shots... & he is THAT far off with his estimate of rebounds prevented as compared to the avg NHL goalie? [Looking at his statements, assuming they are verbatim, I think he's comparing himself to the avg NHL goalie.]
Anyway... where are the Anonymous trolls now? haha
I'd have to agree with CG. I've read his book and although he can at times show up his teammates(the funniest in the SCF in 00 when he made the save and Stevens cleared the rebound right to Sydor at the point resulting in a wide open net, resulting in him throwing up his arms and slamming his stick) but he very openly credits his team and management with being a huge part of his success. If there is anybody who doesn't overrate him, it would be him. I'd probably believe that this came as a result on the complete magnification by the media of his anti butterfly style of play around the time of him passing Roy.
Thanks for setting the truth straight. I am sick of media guys spewing BS backed up by nothing about a goalie who has over 500 with a team that any beer leaguer could win in.
"...with a team that any beer leaguer could win in."
How dare you insult career minor-leaguer Scott Clemmensen, to say nothing of career 2nd/3rd stringer Kevin Weekes.
http://www.hockey-reference.com/leagues/NHL_2009_goalies.html
Brodeur 2.36 GAA, .920 SV%, .635 WPCT
Clemmensen 2.39 GAA, .917 SV%, .654 WPCT
Weekes 2.41 GAA, .920 SV%, .583 WPCT
Sorry, I couldn't resist.
Why weren't people so eager to make these comparisons when Brodeur had a 1.85 gaa and .930 save percentage? It's getting kind of weird how the infatuation some of you guys have with hating this guy has gotten to the point in which you sit around and wait for him to have a bad game knowing the relatively small number of games played can skew his numbers to say what you want so you can attempt to make these outlandish comparisons. Again, you should try going to a mainstream site and spewing some of this nonsense. I assure you that you will not be taken seriously. You will likely get laughed at.
Anonymous: I find it funny that, in response to a post about a mainstream media site uncritically posting an absurd and demonstrably false estimate by a professional hockey player, you apparently seem to think I will be swayed by an appeal to popularity.
I assume you know what the word "contrarian" means, and the significance of me voluntarily including that word in my chosen alias?
It's getting kind of weird how the infatuation some of you guys have with hating this guy has gotten to the point in which you sit around and wait for him to have a bad game knowing the relatively small number of games played can skew his numbers to say what you want so you can attempt to make these outlandish comparisons.
Hating the guy? Sorry for having the intellectual curiosity to be interested in what happens when one of the best goalies in the league misses a substantial period of time. Again, having Brodeur miss 50 games is a perfect test case, both of his talent and of the impact of a goalie on the rest of the team. When the stats are essentially identical, after 1500 minutes played for Brodeur and 3000 minutes by his backups, then some of us are going to be start considering that to be potential evidence of something.
And finally, I know it is "outlandish" to compare Brodeur to other goalies who play on the same team behind the same defence, but I think I'll just keep on being contrarian on that one.
"Again, having Brodeur miss 50 games is a perfect test case, both of his talent and of the impact of a goalie on the rest of the team. When the stats are essentially identical, after 1500 minutes played for Brodeur and 3000 minutes by his backups, then some of us are going to be start considering that to be potential evidence of something."
LOL yea it is evidence that his backups have played twice as much as he has this season, thus his numbers are going to fluctuate significantly on a game to game basis. It seems odd that you waited until now to bring mention to his gaa, directly following one of the worst games the team has played all year.
Assuming that because he missed 50 games that it is now the "perfect testcase" is simply retarded. He has an entire career of numbers, one season, or in this case half of one is largely irrelevant either way. Have you ever heard of the scientific method? Your "theories" and "studies" fail miserably under all ramifications of what an "accurate" test should be.
"Hating the guy? Sorry for having the intellectual curiosity to be interested"
Intellectually interested? More like borderline if not completely obsessed with Martin Brodeur. I think nobody overrates him more than you do seeing as how you have spent years now trying to belittle his accomplishments.
It seems odd that you waited until now to bring mention to his gaa, directly following one of the worst games the team has played all year.
Did you miss this post? Or maybe this one? I haven't at all waited until now to discuss New Jersey's results, it was one of the other commenters that posted Brodeur's current stats.
Have you ever heard of the scientific method?
Of course. Test a hypothesis by performing an experiment where you change one variable and hold all others constant. Compare the results to the control, and make a conclusion about the worthiness of the hypothesis.
That is, replace Martin Brodeur with Scott Clemmensen and play a bunch of games. Then put Brodeur back in on the same team, play him for a bunch of games, and compare the results. That's about as scientific as you can get for results from real-world observation.
I am not using this season to make conclusions about Brodeur's overall career value or ranking. As you say, it is just one season out of many. I do think that we can make some conclusions about his overall effect on the team, though, in terms of shot prevention and other related impacts. I think that this year's results are good evidence that those effects are not very large, and that my estimate of +/- 1 shot per game is a lot closer to the truth than, say, Brodeur's guess of 8-10.
"Have you ever heard of the scientific method?
Of course. Test a hypothesis by performing an experiment where you change one variable and hold all others constant. Compare the results to the control, and make a conclusion about the worthiness of the hypothesis.
That is, replace Martin Brodeur with Scott Clemmensen and play a bunch of games. Then put Brodeur back in on the same team, play him for a bunch of games, and compare the results. That's about as scientific as you can get for results from real-world observation."
Not really when you analylize the results. Clemmenson was receiving 3.5 goals a game of support, while Brodeur has recently been getting less than 1.5. Not too mention facing higher quality shots, more shots of recently, and less defensive support. That is hardly the same situation. Unless of course you mean to argue that the tempo and overall dominance of a team offensively has no bearing whatsoever on how easy/hard the goalies job is.
And I did not hear you deny the personal infatuation you have with Brodeur; one that has taken up years of your life, day in and day out. The more I read your nonsense the further it appears that you are grossly overrating him, not everybody else. For one, hockey in America gets next to no coverage, so it is hard to see how he is overrated there when the only people who get any attention are Crosby and Ovechkin. In Canada, hockey gets tons of coverage, yet because Brodeur plays in New Jersey, he recieves little attention over there as well. So again, where is he overrated? Because people have opinions and the most prevalent one is that he is one of the greatest goalies of all time? Well I hate to break it to you, but the title "greatest of all time" is completely opinion, and subjective at that. Therefor there will never be a definitive "greatest" only a group of candidates who get thrown into the conversation. Brodeur without hesitation is one of the guys.
Once again, a fan doesn't get the satirical nature of the blog title. Amusing, yet in a way disturbing.
The other thing that completely deflates most of what you have to say is the save percentage argument. First you say it is necessary to evaluate not using a raw save% but shot quality adjusted ones. However up until the earlier part of this decade, there was none. Yet you assume the shots Brodeur was facing were "easier" than everyone else. Assuming is hardly scientific.
The best part though, is that you again assume that save percentage is the most important thing for a goalie. That is opinion and once again subjective.
Statman said...
Once again, a fan doesn't get the satirical nature of the blog title. Amusing, yet in a way disturbing.
..........
Once again, side kick to the rescue
Another one:
"Comparing save percentage to league average"....
That is, assuming that not only was the talent level of goalies from year to year and generation to generation even, but also the talent offensively and defensively. Highly flawed
Once again, a fan doesn't get the satirical nature of the blog title. Amusing, yet in a way disturbing.
Statman said...
Once again, a fan doesn't get the satirical nature of the blog title. Amusing, yet in a way disturbing.
.......
If that is the case then why not change it? Perhaps because CG banks on his title drawing this type of attention, and knows that with such an outlandish title, no one would care about his little blog. Guarantee if he changes the name it loses all of its followers.
"Perhaps because CG banks on his title drawing this type of attention,"
Duh.
"and knows that with such an outlandish title, no one would care about his little blog. "
Huh? So the name draws attention... yet the name causes no one to care. Interesting.
"Guarantee if he changes the name it loses all of its followers."
Yes, we all HATE Brodeur & we're very jealous of him! If the name of this blog changes, I know that I certainly won't frequent it.
Yeesh.
Well, if by "followers" you mean disturbed fan-boys of Brodeur, you may be right.
Again, from my perspective this site is devoted to studying hockey goalies & trying to come up with ways to accurately measure their abilities.
On the other hand, the fan-boys who take great offense (to the name, mainly) don't seem to understand that. The fan-boys seem to think this site is a nasty junior highschool attempt at destroying the hated Martin Brodeur, who obviously is the suckiest crapiest goalie of all time & he really sucks too. (Just thought I'd put it into terms that the fan-boys could understand.)
Uh, ok.
In Canada, hockey gets tons of coverage, yet because Brodeur plays in New Jersey, he recieves little attention over there as well. So again, where is he overrated?
I live in Canada and get my hockey coverage from the Canadian media. Brodeur is rated very highly by the media here. Not sure what you mean by "recieves (sic) little attention". He doesn't play for a Canadian team, so he doesn't receive day-to-day scrutiny, but he's consistently rated as the best goalie in the game (above Luongo, who plays for a Canadian team).
He's also received a lot of coverage as he approaches and breaks the major counting numbers for goalies. Many journalists have suggested that he is the greatest of all time or is in the conversation for greatest of all time.
If your hypothesis is that "popular opinion is correct" (which seems to be the basis for most of your arguments), then by definition it's impossible for anything to be overrated. However, Brodeur is rated very highly in Canada. If he is something less than a top-3 goalie of all time (as the historical record and evidence suggests), he is overrated.
Excellent break down side kick. So what you meant ot say is that CG is more or less mooching off of Brodeur's name, and would hardly be capable of running a successful blog without it. Then I guess by your own definitions he is overrated right? I mean using somebody else to be successful?
If your hypothesis is that "popular opinion is correct" (which seems to be the basis for most of your arguments), then by definition it's impossible for anything to be overrated. However, Brodeur is rated very highly in Canada. If he is something less than a top-3 goalie of all time (as the historical record and evidence suggests), he is overrated.
.......
Waht evidence? Everything in terms of your rankings or media rankings is subjective. Being considered one of the greatest ever is different from saying "he's top 3". There is hardly any evidence that suggests Brodeur is not top 3 even if you were to argue it that way. the only case is in the save percentge arguement, however that alone is highly flawed, yet Brodeur is still 10th all time in save percentage.
"He's also received a lot of coverage as he approaches and breaks the major counting numbers for goalies. Many journalists have suggested that he is the greatest of all time or is in the conversation for greatest of all time."
No shit, really? Obviously when somebody breaks one of the more valued records in the history of his sport he is going to get attention.
I live in Canada and get my hockey coverage from the Canadian media. Brodeur is rated very highly by the media here. Not sure what you mean by "recieves (sic) little attention". He doesn't play for a Canadian team, so he doesn't receive day-to-day scrutiny, but he's consistently rated as the best goalie in the game (above Luongo, who plays for a Canadian team).
..........
Thats probably because for a while now he has been the best goalie in the game? I do not see why that is such a foreign idea to you. The best goalie in the game being regarded as such. Or are you going to split hairs and say you personally see him as the 2nd best on the game? Either way your argument regarding this aspect is silly.
Waht evidence? Everything in terms of your rankings or media rankings is subjective. Being considered one of the greatest ever is different from saying "he's top 3". There is hardly any evidence that suggests Brodeur is not top 3 even if you were to argue it that way. the only case is in the save percentge arguement, however that alone is highly flawed, yet Brodeur is still 10th all time in save percentage.
"The save percentage argument" is the correct one to use, if one wants to separate the goaltender's contribution to winning from the team's contribution. Using Wins, GAA, and shutouts to rank goalies will simply attribute major parts of the team's contribution to the goalie.
Save percentage alone isn't perfect. That's why the author of this blog has investigated ways that save percentage fail to capture the goalie's full contribution to winning. He's incorporated those into his analyses.
The point is that the process is objective. The numbers used here to analyze goalies are used because of a process of logical reasoning and investigation. They aren't based on cherry-picking numbers to make one goalie look better than another - it's been well documented here that the author has changed his opinion on several goalies through his investigations. They also aren't based on blindly accepting the decisions that the NHL made years ago on which statistics to track - decisions that were made based on what was easy more than an attempt at accurate individual evaluation.
Thats probably because for a while now he has been the best goalie in the game? I do not see why that is such a foreign idea to you. The best goalie in the game being regarded as such. Or are you going to split hairs and say you personally see him as the 2nd best on the game? Either way your argument regarding this aspect is silly.
It's not unreasonable that Martin Brodeur is regarded as the best goalie in the game. I'm not going to split hairs, I don't think they are right but I can't say with confidence that they are wrong either. In my view, the evidence isn't 100% for any goaltender as the best right now.
My point is that Brodeur is not ignored or overlooked in Canada as you suggested in this passage.
In Canada, hockey gets tons of coverage, yet because Brodeur plays in New Jersey, he recieves little attention over there as well. So again, where is he overrated?
He's also overrated by many in Canada, particularly in a historical context. See the Damian Cox column linked in a recent post here for an example.
"Clemmenson was receiving 3.5 goals a game of support"
Really? As of Feb 25, NJ had scored 182 goals in 60 games (3.0/gm). Since then, NJ has scored 47 in 17 games (2.8/gm).
Doesn't appear to be a big difference in goal support for Clemmensen vs. Brodeur, although I realize that pre-Feb 25 included some Brodeur games.
Where is your source?
Anyway, this shows how goal support is important, & how there are goalies on weak teams that would have much better basic stats (e.g. Wins) otherwise.
Not too mention facing higher quality shots, more shots of recently, and less defensive support. That is hardly the same situation.
Not exactly the same, no, but pretty darn similar.
Goal support while in the net this season:
Brodeur 2.82
Backups 2.92
From shot quality estimates, an average goalie facing Brodeur's shots would have a save percentage .003-.005 lower than an average goalie facing the shots of Brodeur's backups this season.
And as far as shots against, I thought Brodeur was supposed to be preventing those? He said so himself. But if you want to pin them on the rest of the team that's fine with me.
And I did not hear you deny the personal infatuation you have with Brodeur; one that has taken up years of your life, day in and day out. The more I read your nonsense the further it appears that you are grossly overrating him, not everybody else.
Sure, I've spent a lot of time on Brodeur. I think he represents in many ways the "Holy Grail" of goalie statistical analysis. It's very, very hard to properly evaluate the guy. We can't compare his performance on a bunch of different teams because he has played for one team, which has been nearly always very strong defensively. It's tough to estimate a team context by comparing him against teammates since Brodeur plays so many games every year. He is considered to be elite at pretty much every skill that might not be captured by save percentage, so there is also that to contend with - we need to measure and evaluate those things and factor them in.
Just because I think Brodeur is overrated, though, doesn't mean I don't find him interesting. Maybe I'm missing something important in my evaluation, and the only way to find out is by looking. I am also fascinated by how most people evaluate goalies, and by the possibility that Brodeur might become the consensus best ever despite being, in my view, nowhere even remotely close to deserving that accolade.
Also, for the record I live in Canada, and I agree with Overpass' assessment of how Brodeur is rated by the Canadian media.
How could anyone overrate a guy who, by himself, reduces 8-10 fewer shots against per game? That is extremely valuable.
Whats funny is I remember people spazzing out when Luongo's post injury numbers where brought up? Yet now there is a double standard? Go figure.
you definitely seem to be using the wrong line of thinking when defining a "shot prevented". i would say rather confidently that what brodeur is referring to is that his rebound control prevents shots in a given sequence. perhaps he misspoke, or was careless in the way he answered the question. you people spend so much time bashing brodeur and twisting information that i do not see how anybody can take any of these conclusions seriously seeing as how they are mostly based of assumptions and faulty logic.
breaking it down, what is meant by what was said is this.
a shot occurs, goalie A, makes the save and gracefully directs it out to a forward who then carries the puck to the red line, dumps it in, the other team regroups, takes the puck back down on goalie A, takes another shot, and the rebound is held. 2 shots
a shot occurs, goalie B just blocks the shot, it is directed to the corner where the attacking team picks it up and throws another shot on goal. 2 shots.
the same number of shots took place, but goalie A prevented the shot that goalie B did not.
its pretty simple, i do not know why this has not already been addressed.
the general work done on this site is pretty weak considering the absolute value placed on save percentage, and the immediate dismissal of anything that can not be quantified.
i am sure neither cg, nor side kick watched tonights devils game, but i am sure that somewhere down the line the stats from it will be used, directly or indirectly. weekes faces 12 easy shots, mostly long range wristers, and lets in one of the weakest goals you will ever see. brodeur comes in and sees nothing. when he does they are high risk chances. 2 give aways where stopped, one with a poke check on stamkos on a semi breakaway, which is not registered as a "save". the first goal is on the penalty kill in which the defense is running around aimlessly, st louis sets up stamkos on a seam to seam one timer for a goal. then oduya fans on a one timer, resulting in a malone breakaway goal. then the third goal comes on a slapper from the right circle that hits a defensemans skate and caroms directly to a tampa forward standing in front of a wide open net. but i am sure weekes is better than brodeur right? i mean thats what the numbers say right?
reading earlier comments side kick bashes mikka kipersoff for having a "low save percentage" completely blind to the fact that his team plays horrible defense and frequently gives up odd man rushes. this must mean kipersoff sucks and somebody like huet is better right? what a joke. save percentage is entirely dependant on team defense, and your shot quality neutral stat hardly accounts for shot quality, how ironic.
its pretty simple, i do not know why this has not already been addressed.
Nice try, but let's look at the quote again: "We give up 8-10 fewer shots a game here than other places because of the way I control rebounds." Marty is clearly referring to the total number of shots in a game, otherwise there would be the same number in New Jersey as in other places.
In your scenario both goalies face the same number of shots, so what exactly is being prevented? It is just slightly changing the shot distribution, and in that case, what is the value of the skill exactly?
the general work done on this site is pretty weak considering the absolute value placed on save percentage, and the immediate dismissal of anything that can not be quantified.
Because I think it is a poor excuse to simply say something can't be quantified. I've never heard a good explanation of why something can't be quantified. If a goalie does something to prevent shots, why doesn't he face fewer shots then? If he does something to help his team break out of the zone easier or assist with the offence, then the team should be facing fewer shots and/or taking more shots on the other team with that goalie in net. If you define the exact effect of a specific skill or technique, then there will be a way to quantify it. It could be lost in the noise to some degree, if it is a fairly minor effect, but run the right test on the right goalies over the right sample size and you'll find it, if it exists.
Actually, there is one explanation that makes sense to me: The rest of the team adapts to the change. For example, a goalie with poor rebound control goes into the game and they cover the crease tighter. But if they can make that adjustment with no loss in overall play, then that implies that the skill just isn't that valuable. That is essentially my position on non-save goalie skills - they aren't that valuable because the effect isn't large and the rest of the team can simply adjust their play to make up for them.
save percentage is entirely dependant on team defense
That's not true. It is certainly influenced by team defence, but it is not entirely dependent upon it. Not all the goalies on the best defensive teams are at the top of the league, and not all the goalies playing on the worst defensive teams are at the bottom.
Not to mention that if save percentage is entirely dependent on team defence, why hasn't Brodeur been at the top of the league in save percentage most seasons? The Devils have been a terrific defensive team for most of his career.
I can't help but wonder, what stats would you like to see me use instead, if you hate save percentage so much? Please suggest a better alternative. Or are you one of those "nothing is quantifiable" types?
reading earlier comments side kick bashes mikka kipersoff for having a "low save percentage" completely blind to the fact that his team plays horrible defense and frequently gives up odd man rushes. this must mean kipersoff sucks and somebody like huet is better right?
Kiprusoff has not been good this season. Shot quality estimates put Calgary allowing slightly higher than average shot quality against this year, which sounds reasonable to me, but Kiprusoff's save percentage is below average, meaning that even with the adjustment he is no better than an average goalie.
Even if we were to assume that Calgary allows the most difficult scoring chances against in the league, Kiprusoff would still only be at .913 or so, which is nowhere near the leaders. Look at the guys he is tied with at .904, Joey MacDonald and Martin Gerber, do you think Kipper has faced harder shots than either one of those guys? You can't just explain away results because somebody has a bad defence - you have to adjust the results for that bad defence and then look at them. Kiprusoff's save numbers have not been great this season by any measure.
There are also some very good Flames and Oilers bloggers that have written about how Calgary has mostly overcome Kiprusoff's mostly poor play this season, so there's some subjective evidence as well.
your shot quality neutral stat hardly accounts for shot quality, how ironic.
You obviously have no idea what shot quality is actually saying about NHL teams.
Here are my shot quality estimates for last night's Devils game:
Kevin Weekes: expected save percentage .942
Martin Brodeur: expected save percentage .899
That pretty much matches your subjective viewpoint - Weekes faced very easy shots, while Brodeur faced much tougher scoring chances.
Shot quality also says that Brodeur has faced harder shots this year than Clemmensen or Weekes, something that you agree with. Seems to me that shot quality measures pretty much agree with what you are saying, which leaves me at a loss to explain why you are so negative towards the metric.
"Here are my shot quality estimates for last night's Devils game:
Kevin Weekes: expected save percentage .942
Martin Brodeur: expected save percentage .899
That pretty much matches your subjective viewpoint - Weekes faced very easy shots, while Brodeur faced much tougher scoring chances.
Shot quality also says that Brodeur has faced harder shots this year than Clemmensen or Weekes, something that you agree with. Seems to me that shot quality measures pretty much agree with what you are saying, which leaves me at a loss to explain why you are so negative towards the metric."
Have you seen the game? An expected save percentage of .899?! Did you see anything that happened? There wasnt a goalie in the league who would have stopped 9/10 last night. It wasnt exactly a great night, but even if he stopped 9/10, that is still only a save percentage of .90, which even when adjusted is not very good. So again, these numbers hardly tell anything more than stats such as wins and goals against. If a goalie gets a lot of wins, chances are his team has faith in him. if his goals against is low, then quite obviously he is not giving up many goals. Looking at your site it seems like you rather give credit to goalies who do not win shit, over guys who get the job done.
"In your scenario both goalies face the same number of shots, so what exactly is being prevented? It is just slightly changing the shot distribution, and in that case, what is the value of the skill exactly?"
What is being prevented is obviously a higher percentage chance. If I had to place money on it, I'd confidently say that the first shot taken upon entering the zone in a non odd man rush situation, generally has less of a chance of going than the ones that follow because after that shot takes place, everyone has to react causing defenders to be at greater risk of being out of position.
Anonymous: I really find your logic to be confusing. You quite confidently said that Weekes faced much easier shots than Brodeur did two nights ago. My metric shows that to be the case - there was a huge difference between the shots faced by Weekes and the shots faced by Brodeur. You disagree with the expected save percentage predicted for Brodeur, despite it being lower than league average and much lower than Weekes', and claim that is evidence that the numbers don't tell you anything more than wins and goals against?
Seriously? In the Tampa Bay game Brodeur had a 4.39 GAA while Weekes had a 2.86 GAA. That implies Weekes had an average night, when he actually had a bad night (0.30 goals worse than expected based on shot quality against). We have two goalies who faced a very different shot distribution, and you want to use goals against to evaluate them, even though one faced much, much easier shots? Not only that, but the goalie with the 4.39 GAA and .700 save percentage "won" the game, not because he played well, but because his team scored 5 goals and outshot the opposition 39-22. You probably couldn't find a better example of the value of adjusting for shot quality than that game, and yet your conclusion is that wins and goals against are better measures. I'm completely baffled by that line of reasoning.
I saw the highlights of the Tampa game, i.e. the goals. I do actually sort of agree with you that Brodeur's expected save percentage should really be lower because of that third goal, which was an unlucky bounce and in reality was probably close to a 100% chance of being a goal. However, over an entire season those things tend to even out, there will be some chances where Brodeur will be down and out and yet the shooter will miss the net or shoot it right into him. The key thing is the relative difference between his expected save percentage and Weekes.
I do think the majority of goalies in the league would stop that Stamkos shot though, to be perfectly honest, as stacking the pads is not the high-percentage play there. Brodeur also stops around two-thirds of his shootouts, so he usually makes the save on breakaways but just happened to get beat on that particular one. I think on a good night Brodeur would have stopped 9 out of 10 of those shots.
What is being prevented is obviously a higher percentage chance. If I had to place money on it, I'd confidently say that the first shot taken upon entering the zone in a non odd man rush situation, generally has less of a chance of going than the ones that follow because after that shot takes place, everyone has to react causing defenders to be at greater risk of being out of position.
I'm not at all sure about that. It is usually very difficult to generate quality scoring chances at 5-on-5 except on the rush, because once all 5 defensive players have set up in their defensive positions they are covering the dangerous areas.
If it is true that Brodeur is preventing high quality chances, how come he has faced more dangerous shots against this year than Clemmensen and Weekes?
"If it is true that Brodeur is preventing high quality chances, how come he has faced more dangerous shots against this year than Clemmensen and Weekes?"
For somebody who seems to have sucvh in depth though out answers for everything you really do seem to do a poor job of analyzing the other side of the argument. How is it possible for him to prevent high quality chances and still face higher ones than his backup? It's simple, teams are more selective in where and when they shoot.
Your biggest flaw, and blatant error in everything you do is assume all things are equal, even after you have made you so called "adjustments". If goalie A faces a wrist shot from 16 feet out at 44 mph, that is completely different than a guy facing a 58 mph wrist shot from 15 feet. Every situation is different, which means just about every stats, but particularly save percentage is flawed
Every situation is different, which means just about every stats, but particularly save percentage is flawed
You need to take a class on probability and statistical sampling theory. You're not wrong to say that every situation is different, but you are wrong to say that means every statistic is flawed.
If we know, for example, only that Brodeur faced a 15 foot shot, we don't know who the shooter was, whether the shooter was being pressured, whether there was a screen, etc. Maybe it was Alex Ovechkin ripping one in the top corner, or maybe it was Derek Boogard fluttering one right at the goalie's chest. If we know that the league average save percentage from that distance is, say, .850, then we have a ballpark figure for comparison, so that gives more information than if we only know whether the shot went in or not, but still I concede that it doesn't fully describe that specific scoring chance.
However, if we take a look at the last 100 15 foot shots Martin Brodeur has faced, we know that they can't all have been Alex Ovechkin shooting a laser into the top corner. With that kind of sample, some of them were more or less unstoppable shots and some of them were right into the goalie. Some of them might have deflected off the defence into the net, and some of them might have been going in but were blocked by the defence. Some of the shooters were wide open, and some of them were pressured into a weak shot. As a result, the average difficulty of the scoring chance is going to converge to something very similar to the league average. This is just an example, a 100 shot sample is not really that large, but shot quality measures are assigning a probability to every shot a goalie has faced over the course of a season, and in that case the bad bounces and the good bounces are going to mostly even out.
To summarize, I'm not assuming things are equal. I am assuming that the differences of each individual shot situation will be averaged out and therefore greatly reduced over a large sample size, which means that treating scoring chances as equal results in a very good approximation of the real shot difficulty against.
"To summarize, I'm not assuming things are equal. I am assuming that the differences of each individual shot situation will be averaged out and therefore greatly reduced over a large sample size, which means that treating scoring chances as equal results in a very good approximation of the real shot difficulty against."
That is still highly erroneous given that multitude of possible scenarios, even breaking down to Alex Ovechkin shooting from 15 on a powerplay, to Ovechkin shooting from 15 feet on a 2 on 1. The situation often influences what is on the ice in more ways than one. Not in the sense that the goalie tries harder in one or the other, but in that the decision making process variously tremendously on almost every shot. Derek Boogard carrying the puck on a 2 on 1 with Gaborik to his right entails an entirely different situation than Gaborik carrying the puck with Boogard on the off wing. Maybe with Boogard carrying the puck the goalie cheats to his side assuming Gaborik will likely see the pass, whereas with Gaborik carrying the puck he probably wont cheat and as a result is open to Boogard receiving a tap in pass for an easy goal. I specifically remember hearing Brodeur discuss this earlier when he made a pretty incredible save against the Flyers because "I knew Gagne didn't like to shoot on his backhand, so I was expecting him to pass". Both situations again also greatly vary depending upon who the defenseman will be on the 2 or 1 as well. Again, do not get caught up in defending the specifics of the 2 on 1 example, because my point is that every play can vary to such an extent. Bottom line is keeping the puck out of the net, however people far too often just blindly assume that goalies are interchangeable and one guys numbers are equally comparable to another guys. You wouldn't believe how many people over at HF just ignorantly laugh at and dismiss the reality that Thomas Vokoun is a top 5 goalie, instead putting guys like Nabakov and Lundqvist ahead of him. When watching these games it is easy to take for granted a single shot or play, or when looking at just stats it is even easier to overlook what they represent. As I said, 3 years from now you could hand somebody a box score from the Tampa game, and you'd think the game was close only because of Brodeur's poor play.
Again, do not get caught up in defending the specifics of the 2 on 1 example, because my point is that every play can vary to such an extent.
I agree with you, it can vary. That doesn't mean it is "highly erroneous". You're getting lost in the details, and focusing on minutiae at the cost of the big picture. Re-read what I wrote about scoring chances averaging out.
I can try to explain it a different way, staying with Ovechkin as an example. Some games he scores on a shot off of a defender's stick, and sometimes he scores a 5 foot tap in after a teammate sets him up, and sometimes he steals the puck and beats several players, dekes the goalie, and scores a highlight reel goal. Those are different situations, and require vastly different levels of skill, but they all count the same on the scoresheet.
At the end of the season, we can look at see that Ovechkin led the league with around 60 goals. We can therefore conclude he is a terrific goalscorer. There is no need to throw out that entire number just because some of them were lucky deflections, because we know that nobody is going to score, say, 30-40 goals in a single season on lucky deflections. Obviously there must have been some terrific underlying scoring skill that explains the numbers.
Shot quality measures are based on the league average scoring levels for each type of chance, for example, a slapshot from 15 feet away, and the good ones also adjust for game situation. Ovechkin shooting from 15 feet on a power play is treated differently than Ovechkin from the same spot at even-strength. I wouldn't mind having an adjustment for whether the shooter is covered or not, or whether the shot comes on an odd-man rush or not, but then you are getting down to very small differences in probabilities at the individual shot level that, again, will almost certainly mostly disappear when looking at a sufficiently large sample size. Especially since shot quality against is likely highly correlated with odd-man rushes against.
"Whats funny is I remember people spazzing out when Luongo's post injury numbers where brought up? Yet now there is a double standard? Go figure."
The Brodeur numbers I brought up in this post were cumulative - his numbers for the entire season.
I think Anonymous continues to believe that some of us are "out to get" Brodeur. His actual performance doesn't concern me (unless he's playing for Team Canada); it's his performance vs. the general assumption of his overwhelming greatness that is my focus.
And of course, in this particular post the topic is Brodeur's bizarre claim that he alone is responsible for 8-10 fewer shots against as opposed to other goalies on other teams.
Anonymous - A model like shot quality is going to simplify things, inevitably. It's the only way we can make sense of the data.
This will introduce two sources of error - random error and systematic error. Random error can be a problem in small samples, like you say. The shot quality model won't match the actual quality of the chances in every situation. As a result, watching the game may very well give you a better sense for shot quality in every situation.
However, we expect random error to even out over a larger sample - that's just statistics and the law of large numbers.
The real problems with the shot quality model will come with systematic errors - errors that are repeated and will not average out over time. In the case of Brodeur, these could be caused by the play of Brodeur himself, by certain characteristics of the team in front of him, or by the scorers in New Jersey who report shot type and distance.
If you want to examine whether the shot quality model does a good job of estimating Brodeur's shot quality, I suggest you look at systematic error - things that are unique to Brodeur, the Devils, or the New Jersey scorers. Random error isn't an issue, except in small samples as you pointed out.
Interesting discussion, with good points made all around. I was going to mention the last 100 15-foot shots at Martin Brodeur might be better compared to the last 100 12-foot shots at Henrik Lundvist due to reporting biases, as Overpass alluded to. But perhaps the last 100 15-footers at, say, Dwayne Roloson come from an average of 4 feet closer to the side boards than the equivalent distance shots at Martin Brodeur because (hypothetically) Edmonton prefers to force the shooters wide but doesn't mind giving them shooting lanes from outside. Or perhaps the last 100 15-footers aimed at Carey Price have resulted in only 70 shots on goal cuz of his defence's tendency to block shots, but the ones that get through are a little higher difficulty cuz of tips and screens during failed block attempts. And so on. No matter how you fine-tune the shot uality adjustment there's just no way to account for all of this, and as Overpass states it's the systemic stuff related to the goalie, team, scorer and unbalanced schedule which will skew outputs no matter how large the sample size. Some of it can be reckoned and adjusted out, but it's a slippery slope. That's why I don't want to rely on any one stat, no matter how refined, and why I don't want to rely on Just stats to take the measure of the man. There's always variables, one of the largest of which is personal opinions which it will always come down to.
Post a Comment