Thursday, October 30, 2008

Estimating Brodeur's Shot Prevention

There has been an ongoing debate at this blog over how much goalies can influence the number of shots they face in the course of a hockey game. To clarify my position, I always thought it was reasonable that someone like Martin Brodeur would face fewer shots than an overmatched ECHL-level goalie with no puckhandling skills, terrible rebound control and poor positioning who just stood deep in his net quaking and looking small and leaving his teammates to take care of the defensive zone. But even in that best-possible vs. worst-possible scenario, I'd estimate a maximum difference of a few shots per game. From anywhere near the net, NHLers will likely decide to shoot no matter who is in net, and from a long distance they probably will elect to pass or stickhandle even with a weaker goalie between the pipes. Missed shots don't seem to have much of an effect, and rebound shots make up only a couple of shots against per game on average, so those things are unlikely to have a huge contribution to shots against. A top goalie might be able to intercept a pass or two that could lead to a shot, might be able to help his team exit the zone a few times with his stickhandling to avoid subsequent scoring chances against, might be able to hold onto an extra puck or two rather than allow a dangerous rebound chance, and might dissuade a few opposing shooters from shooting from moderate-level scoring positions, at least compared to our replacement level example.

When looking at NHL goalies only, however, it seems to me unlikely that anyone with very poorly developed "soft skills" would make it all the way to the best league in the world. Any differences between them would likely be substantially smaller than compared to the Brodeur/ECHL goalie example.

Can we test this hypothesis? I have repeatedly used comparisons to backup goalies on this blog, and that seems like a good tactic to address the shot prevention issue. Most of Martin Brodeur's backups were with the team only briefly and did not play many games in New Jersey, but there are a couple of backup goalies, Mike Dunham and Chris Terreri, that played a fair number of minutes with Brodeur and should provide an approximate context. I will also use results for Dunham and Terreri outside of New Jersey to estimate their own shot prevention skills for comparison.

First off, Mike Dunham. Dunham broke in with the Devils, and in two seasons in New Jersey faced 26.5 SA/60, compared to Brodeur's 24.1. Looks like Brodeur was doing well, until we realize that Dunham apparently always gives up a lot of shots. In his next six seasons, Dunham went +1.8, +1.9, +2.6, -0.9, +2.4, +1.3 compared to his backups. So he looks to be a guy who faces about 1.5 shots per game more than average. Brodeur beat him by 2.5, so that gives us an estimate of Brodeur being about a shot per game better than average.

Chris Terreri has a few years as a starter before Brodeur broke in. During those years, he was +1.3 and -0.8 playing with Sean Burke, and +0.4 and +0.1 playing with Craig Billington. Both Burke and Billington were very similar to their teammates in terms of shots against, with an average of about a half-shot per game difference between them and their teammates over their careers. Since Terreri was virtually equal with them, it implies that Terreri was about average in terms of shots against.

With Brodeur, Terreri went +1.0 in 1993-94 while sharing time, and +0.4 in 1994-95 in a backup role which may have had some effect on reducing his shots. That supports the estimate from Dunham's stats of Brodeur having a potential effect of something near a shot per game.

Interestingly, as his career went on Terreri seemed to get better at shot prevention. In 1995-96, 1996-97 and 1997-98, he played on a few bad teams and significantly outperformed his teammates in shots against (-2.8, -1.7, -1.2). When we went back to New Jersey for the 1999-2001 seasons, he was basically Brodeur's equal (24.9 SA/60 for Terreri over those 3 seasons, 24.7 SA/60 for Brodeur). Terreri may have again benefitted from weak opponents, but if he was outperforming by 1 - 1.5 shots per game in other places and then was equal with Brodeur in New Jersey, that is more evidence for a "Brodeur effect" of about a shot per game compared to the average guy.

In conclusion, there appears to be some evidence that Brodeur may prevent about a shot per game compared to average, based on comparing the results of the two backup goalies with the most games played, although the sample sizes are pretty small. Since many observers would consider Brodeur to be the best in the league at the skills that may contribute to preventing shots, I'd guess if we isolate his effect that could be considered pretty close to the maximum possible positive effect. From looking at the results of Dunham and others like him, it might be possible that the gap could be a little bigger on the other end (i.e. some goalies are substantially below average), but I think in general it is probably reasonable that goalie shots against results may vary within the range of about +/- 1 shot per game on average.

If this is the case, it raises a number of subsequent questions, such as: Are the shots being prevented/created more or less dangerous than average? Do goalies who are good at preventing shots tend to have higher save percentages? Can we isolate the specific skill that most contributes to this shot gap? Does shooter choice come into the picture at all, e.g. do shooters choose to shoot more or less often against certain goalies? Does the shot differential result primarily from even-strength play or on special teams? Is shot prevention consistent year to year? Can goalies substantially improve the number of shots they face per game? More research is certainly required in this area.

30 comments:

Bruce said...

Interesting stuff, and good questions at the end. You're getting a handle on how open-ended this whole issue is, and how difficult it will be to nail down this or that factor. Many would write it all off as details, but details are important, esp. when the differences between these guys as measured in Sv% come down to .00x.

Again I would draw the comparison to a pitcher fielding his position; it's far less important than his ability to throw strikes, but it still can be important-to-damned-important, both positively and negatively. Some guys are great at it, and some guys spend a career of spring trainings trying to become barely competent in the field and hating every minute of it.

To extend that analogy back to the crease, a guy like Ed Belfour will move the puck so sharply it appears effortless, whereas if your goalie is Curtis Joseph you're just hoping he doesn't mess up and at least gets the out at first.

Concerning the specifics of your post, the only comment I have for now is that surely Chris Terreri improved as a puckhandler during his first stay in Jersey. These are hardly static skills, it's not like some guys are born with it and some aren't and that's all she wrote. Terreri may well have learned something from watching Brodeur, and in practicing and playing (occasionally) in a team system built to rely on goaltender puckhandling. That he took said skills with him to his future clubs and was a little better than his creasemates shouldn't be a surprise.

Anonymous said...

Early in his career, maybe Brodeur felt compelled to get more involved with puckhandling to lower his shots against by up to 1.5 shots per game, in order to make up for his mediocre save pct.

Scott said...

I think it's important to recognize that we're not talking about very many goalies here and the broadest range so far is 2.8 shots. The results seem to suggest a slightly wider range of at least 3 shots per game from worst NHL goalie to best, unless of course Terreri was the best at this and Terreri's creasemate that one year was the worst. That looks to me like a significant difference.

If a team has a poor puckhandling goalie one year and then moves to a good puckhandling goalie the next it may create a significant difference. I will assume that both goalies are equal in ability at puck-stopping at a rate of 92.5% (highest reasonable percentage which should serve to minimize the effect), that he will play in sixty games (normal starter) and that the shots prevented by the goaltender are distributed randomly in terms of game state and quality. For each shot per game prevented, the goalie has prevented an extra 4.5 goals, or about one win, and that's if the effect is minimized.

The Contrarian Goaltender said...

Scott, it has not yet been established that puckhandling skill determines shot prevention. This is the popular view, but I am still not at all convinced that is the case. Shot differential probably comes from many variables. There is lots of evidence that puckhandling skill may have little to do with shot prevention. For example Arturs Irbe, one of the worst puckhandling goalies of his generation, ranks among the very best goalies in terms of facing fewer shots than his teammates. Also, I think most observers would consider Chris Osgood to be a decisively better puckhandler than Dominik Hasek, but in both of the past two seasons Hasek faced over 2 shots per game fewer than Osgood.

We need to also be careful not to put too much weighting into a single season's sample because there is a high degree of randomness. In the season with the 2.8 shot differential (1995-96), Chris Terreri was traded during the season to a San Jose Sharks team that started 1-12-4. They then won 19 games over the rest of the season. Did Terreri improve the shot prevention, or did the rest of the team improve? Probably a bit of both.

In research for another post, I looked at pairs of goalies who played together in a platoon scenario for at least 3 seasons in a row, trying to see what kind of shot differentials they were putting up. Out of 17 pairings, only 2 of them had a shot differential over 1.5 shots per game (and one of those involved Mike Dunham, mentioned above as a case of a poor shot preventing goalie). The maximum differential in the sample was 2 shots per game.

I think Dunham and Brodeur probably get pretty close to your scenario of the best vs. the worst, and the gap was 2.4 shots per game. I'm still fairly confident, for now, in my estimate of a normal range of +/- 1 shot per game on average.

Scott said...

CG,

First off, thanks for all the work that you've done. I think this subject is very interesting.

By a range of +/- 1 shot per game, do you mean +/- one shot from average?

As far as "puckhandling," that was my mistake, I really mean "shot-preventing." The point being that if one can isolate goaltenders it could be a way for teams to improve by preventing up to ten goals depending on who they've had in net the season before. If the average goalie faces 1500 shots in sixty games, but you have Goalie A who allows 1560 and Goalie B who allows 1440 that's a huge gap. If both goalies post .910 as a save percentage, if you adjust each of them to an average number of shots, Goalie A is at .907 and Goalie B is at .913. That's a pretty hefty gap for shot-preventing.

Bruce said...

Scott: In an earlier discussion I postulated that good vs. bad shot prevention -- including such "soft", immeasurable skills as rebound control; puck retrieval, handling, and movement; "housekeeping" of passes and loose pucks around the goal mouth; and the goaltender's depth in the crease -- might have an effect of as much as +/- 2 shots per game. This admittedly speculative figure based on indirect interpretation of shots data, translated into a difference of +/- .005 of Sv%, in line with your own calculations based on +/- 1 shot per game. Such differentials seems small compared to the base rate of ~.910 but are a pretty significant relative to the tiny Sv% margins between top 'tenders.

CG: One question that's percolating away unanswered in my tiny brain is, do butterfly goalies in general face more shots? As a group they seem to have better Sv%s, but they would need to if more shots faced is a cost of doing business. Can you think of a way to a) categorize them and b) analyze their collective tendencies?

Anonymous said...

brodeurs effect on the other team when in net is definitely more than you give him credit for. there is only so much statistics do in quantifying his effect. look at what barry bonds did with a mediocre giants team in 2002. he had a significant effect on other teams, and caused them to completely alter their game plan, similar to the way a lot of nhlers say brodeur alters game plans. not to mention the effect bonds had without even taking a swing. just because its harder to statistically interpret what brodeur does, doesnt mean it should be marginalized.

The Contrarian Goaltender said...

How do you know it is more than I am giving him credit for? Have you broken down the game film? Have you figured out a way to estimate his effect? Or are you simply a fan of Brodeur and aren't willing to be persuaded by statistical evidence?

By the way, your Barry Bonds analogy is a complete non-sequitur. Different sport and different level of talent. Can a historically great player have a big impact on the game? Sure. Does that mean Brodeur does? I'm still looking for the proof.

Anonymous said...

what more do you need than actrual nhl players and coaches admitting so?!?! when a coach prepares his team to play and has to alter the means by which his team plays simply because brodeur is in net, that a pretty big plus for the devils. how is it different than barry bonds? similar to brodeur the last 3 years, bonds's mere presence in the lineup took average teams to the playoffs. teams walked bonds putting men on base for other guys creating runs, and rbis. the guys hitting in front of bonds saw better pitches because of his presence. just the same as teams stopped dumping the puck in, enabling jersey to trap, thus creating more turnovers, and getting guys more scoring chances, while limiting the time spent in new jersey's end, which i'm also sure helped the +/- of guys on the ice. are you that stupid not to be able to draw parallels here or are you just ignoring logic because you are so blatantly biased on this that there is no reasoning involved in your decision making?

Anonymous said...

"when a coach prepares his team to play and has to alter the means by which his team plays simply because brodeur is in net, that a pretty big plus for the devils."

It is? Every single team tailors their approach towards their opponent's strengths & weaknesses.

The NHL isn't rec league... they actually prepare for each & every team.

Anonymous said...

"It is? Every single team tailors their approach towards their opponent's strengths & weaknesses."

no shit, what does reiterating what was already said do? obviously every team prepares for games. thats why i said that. with brodeur in net prior to the rule change, teams could not play dump and chase, which was what the majority of them played. thus they had to significantly alter the way they played, which benefitted new jerseys system. that was pretty obvious but i guess someone missed it.

Anonymous said...

"thus they had to significantly alter the way they played, which benefitted new jerseys system."

That's debatable.

The Contrarian Goaltender said...

So, the pre-lockout Devils required an elite puckhandling goalie to function? Interesting, then, that their backup goalies from 2000-01 to 2003-04 had the following stat line: 1.82 GAA, .919 save %, 22.3 shots/game.

Anonymous, I agree Brodeur has an effect that is a benefit to his team. I estimated it to be saving between 0.5 and 1 shot per game, and I have explained above the statistical reasons why I believe that to be the case. In addition, I did another study where I compared goalies to their backups, as well as how those backups did on other teams, and found Brodeur's shot differential to be 0.7 shots per game.

You, on the other hand, have simply said that I am wrong, and that Brodeur's effect is huge because a) NHLers scout opposing goaltenders, b) Barry Bonds is good at baseball, and c) you believe it is difficult to draw conclusions from goalie statistics. I simply don't find that argument to be very persuasive.

Anonymous said...

CG, one thing you may have analyzed/commented upon, but I'll bring it up:

Pre-lockout, the rules allowed for goalies to get more involved with the puck, to go to the corner & pass it up, etc. Pre-lockout, Brodeur's sv pct. was at times mediocre.

Post-lockout, the rules prevented goalies from playing the puck as much. Post-lockout, Brodeur has had a very good sv pct.

So perhaps when Brodeur spends less time wandering & showing off his amazing stickhandling & passing skills, he is better at actually saving shots directed towards the net. :)

Bruce said...

So perhaps when Brodeur spends less time wandering & showing off his amazing stickhandling & passing skills, he is better at actually saving shots directed towards the net. :)

Or now that Brodeur can't prevent quite so many shots with his amazing stickhandling and passing skills, that more shots are generated from shoot-in situations but that that they don't tend to be high percentage shots? Just guessing, but it's a fact the Devils shots allowed rate has absolutely shot up since the lockout. Brodeur's save percentage has also soared the past two years, so his GAA has remained fairly consistent. Plan B seems to be working OK.

Anonymous said...

Why would the opposition be taking more shots against Brodeur from low-% locations since the lockout?

If anything, since the new rules reduce goalie wandering/puckhandling, there should be more dump-in-chase-the-puck-cycle-it-get-a-higher-%-shot.

As for the Devils shots allowed rising since the lockout, that probably has more to do with their personnel & perhaps shot situation (EV vs. PK).

The Contrarian Goaltender said...

Don't forget the opening up of the league post-lockout. Looking at the raw shots against averages skews the perception, because the league average shots against per team went up a full 2 shots per game from 2003-04 to 2005-06.

A better way to express the numbers is as a percentage of league average. Here are the stats for both Brodeur and his backups pre- and post-lockout, shots against as a percentage of league average:

2000-01: Brodeur 90%, Backups 82%
2001-02: Brodeur 84%, Backups 85%
2002-03: Brodeur 84%, Backups 78%
2003-04: Brodeur 88%, Backups 78%
2005-06: Brodeur 98%, Backups 96%
2006-07: Brodeur 96%, Backups 97%
2007-08: Brodeur 94%, Backups 98%

The numbers rose for all goalies across the board. I think the Devils got worse as a team at shot prevention, and it had very little (if anything) to do with the goalies.

Anonymous said...

Hmmm, since 2000-01, NJ backups have faced fewer shots in 4 seasons, while Brodeur faced fewer shots in 3 seasons.

Clearly, Brodeur is not as good as his backups at reducing shots. Or perhaps the backups tend to be played against the weaker teams.

Anonymous said...

or just maybe, once again using a miniscule sample size for brodeurs backups is highly erroneous. this site used to be somewhat accurate because it used actual stats but there are way too many flawed stats put up over the past 6 months.

The Contrarian Goaltender said...

Anonymous, were you referring to my last comment or the above post when you wrote "miniscule sample size"? I definitely agree with you that the numbers for comparing pre-lockout to post-lockout are based on a very small sample size.

If you are referring to the above post, then yes it is not exactly a huge sample size but I am also drawing on additional evidence in making my +/- 1 shot claim. I did not discuss it all in detail in the post but have mentioned some of it in the comments.

Anonymous said...

i was reffering to the above post. how bout this, if i flip a coin twice, and it lands heads up both, that means i can assume it will always land heads up? for goalies, at least 55 games ( or a full season of work for the average teams netminder) is necessary to draw any conclusion

The Contrarian Goaltender said...

Chris Terreri played in 97 games as a teammate of Martin Brodeur. That is hardly a sample size of two, and is nearly double your 55 game minimum. Dunham played in 41 with Brodeur, which I agree is not a conclusive sample size but I'm not sure it is fair to call it "miniscule". Both Dunham and Terreri also played over 300 games on other teams, so I think those results can give us a pretty good estimate of their abilities with some significance.

Having said that, this post doesn't prove anything and I don't think I claimed that it did. It merely provides some evidence and an attempt to estimate an effect. I would love to make a stronger conclusion, but as you are aware the reality of the few number of games played by Brodeur's backups makes that difficult.

Anonymous said...

Missed shots don't seem to have much of an effect, and rebound shots make up only a couple of shots against per game on average, so those things are unlikely to have a huge contribution to shots against.

Probably the dumbest sentence Ive ever seen for any hockey article. Missed shots dont have effects because there not shots. Rebounds are the hardest shots for goalies to save because they are not set and square to the shooter.
So you made a post to prove nothing? And you insist on trying to determine whether Brodeur is overrated or not.

Agent Orange said...

Maybe I am missing something but doesn't preventing shots only help broduers image if the prevented shots are easy to save?

If the shots he prevents are easy it will raise his save percentage but is there really a lot of value in this? If he prevents 100 shots a season and the average goalie would stop 98 of them what benefit does that bring his team? Is 2 goals saved really significant?

If the shots are difficult then facing them would lower his save % and might cost him his job.

Seems he would be better off keeping his shot preventing to himself.

I guess what I am trying to say is that he is only as good as his save % whether he prevents shots or not.

Anonymous said...

Perhaps you should rewrite this article. Marty doesn't only prevent shots, he prevents possession, zone control, offensive moment, etc.

If you didn't watch the Devils this playoffs yet, I suggest you do. While the devils clearly implement a team game and are very effective at shot prevention, Brodeur has been unstoppable as a third defensemen moving puck out of the zone, tiring the forecheckers and sending his team up on a rush. Even when it results in a goal due to a mistake against every now and then(happend twice this playoffs) , its by far better then letting the opposition pursue the puck and gain control of the zone on every dump in.

So while stay at home butterfly goalies like Smith, Quick, and Lunquist rack up .950 save percentages this playoffs, you can use this series as a perfect example of why save percentage is not the only metric to evaluate a goalies effect on a team and winning, thus the goalie's overall value.

P.S. While you can clearly set this aside as a team factor, just ask yourself if any other goalies can do what he does back there. Only a few can. The success of the Devil's strategy relies on Brodeur's puck handling. And on the other side of the coin, save percentages are rising year after year as more and more goalies sit at home with giant pads saving rebound after rebound.

P.P.S - what has happened to your boy Bobby Lou? You should have a talk with the Vancouver Coach and GM about save percentages and averages and tell them that there is no such thing as clutch goaltending.

Anonymous said...

"Perhaps you should rewrite this article. Marty doesn't only prevent shots, he prevents possession, zone control, offensive moment, etc.

If you didn't watch the Devils this playoffs yet, I suggest you do. While the devils clearly implement a team game and are very effective at shot prevention, Brodeur has been unstoppable as a third defensemen moving puck out of the zone, tiring the forecheckers and sending his team up on a rush. Even when it results in a goal due to a mistake against every now and then(happend twice this playoffs) , its by far better then letting the opposition pursue the puck and gain control of the zone on every dump in.

So while stay at home butterfly goalies like Smith, Quick, and Lunquist rack up .950 save percentages this playoffs, you can use this series as a perfect example of why save percentage is not the only metric to evaluate a goalies effect on a team and winning, thus the goalie's overall value.

P.S. While you can clearly set this aside as a team factor, just ask yourself if any other goalies can do what he does back there. Only a few can. The success of the Devil's strategy relies on Brodeur's puck handling. And on the other side of the coin, save percentages are rising year after year as more and more goalies sit at home with giant pads saving rebound after rebound."

I completely agree with this! listen CG, maybe I haven't seen it yet on your blog but instead of trying to tarnish Brodeur (which I might add ain't working too good) you should start analyzing butterfly goalies like Quick and Rask and see how many shots they CREATE. Their save percentage is on steroids because of how many shots they see a game due to their style of play. You're worrying about Brodeur and what affect his play has on shot PREVENTION (and you prove nothing against the common belief that his unique style prevents many shots and scoring chances from occurring) but you're missing the fact that the new style of goaltender is actually creating opportunities for the opposing team and literally can allow 3-4 goals in a given game and still pull off a great SV%. Some of the gaols of their own doing. you want to try to attack the opinion that Brodeurs style prevents shots from occurring, which at the very least you must admit his play certainly doesn't add to scoring chances, but you don't (as far as i can tell) have problems with butterfly goalies who do almost everything they can to create shot and/or scoring opportunities from happening? C'mon do you really have that much of a hard-on to try and convince the masses that Brodeur is only a little better than average? Look, SV% is hardly the stat you make it out to be. It's important, but when there are variables such as the goalie himself creating shots, shot quality, and shot prevention etc, etc. It becomes as significant or in your case, as insignificant as all the other goaltending stats. I suggest you just let this go. Brodeur is a phenomenal goalie and will go down in history as the Greatest of All Time. The fact that you try to seperate him from the New Jersey Devils defense shows me how out of touch you are with his career and those devils teams he's played with over his career. Brodeur was/is an integral part of the defense. So when you talk about how great the Devils defense is/was, whether you would like to believe it or not, you are giving him just as much credit. That defense thrives on Brodeurs play and your small sample of data for his backups doesn't prove anything, which you already admitted.

alaa nile said...

شركة عزل خزانات بالرياض
شركة مكافحة حشرات بالرياض
شركة تخزين اثاث
شركة نقل اثاث الرياض
شركة كشف تسربات المياه بالرياض
شركة تنظيف بيارات بالرياض

شركة تسليك مجارى
شركة جلى بلاط بالرياض
شركة تنظيف كنب بالرياض

alaa nile said...

شركة تنظيف منازل بالرياض
شركة عزل خزانات بالرياض
شركة مكافحة حشرات بالرياض
شركة تخزين اثاث
شركة رش مبيدات بالرياض
شركة نقل اثاث الرياض
شركة تنظيف خزانات بالرياض
عزل مائي
شركة تخزين عفش بالرياض
شركة تنظيف اثاث بالرياض


شحن عفش داخل السعودية
شركات نقل وتغليف اثاث
شركة تنظيف واجهات زجاج بالرياض

راندا السيد said...

شركة نقل اثاث بجدة
شركة تنظيف بجدة
شركة مكافحة حشرات بجدة

شركة تنظيف خزانات بالرياض
شركة تنظيف مساجد بالرياض
شركة مكافحة نمل ابيض بالرياض
شركة نقل اثاث بالرياض
شركة نقل عفش بالرياض
شركة تنظيف منازل بالرياض
شركة تنظيف بيوت بالرياض
شركة تنظيف فلل بالرياض
جلى بلاط

راندا السيد said...

شركة تنظيف واجهات زجاج بالرياض
شركة رش مبيدات بالرياض
شركة عزل اسطح بالرياض
نقل عفش
تنظيف شقق بالرياض
تنظيف فلل بالرياض
شركات تنظيف بالرياض
غسيل مجالس
افضل شركة مكافحة حشرات بالرياض
رش مبيدات
رش مبيد