Friday, December 12, 2008

The Replacements

In the last couple of weeks, I've noticed a distinct lack of gleeful comments posted by Bruce giving New Jersey's goalie stats since Martin Brodeur went down. I decided to look into it myself to see just how New Jersey's replacement goalies are holding up.

Scott Clemmensen: 7-3-0, 2.19, .926
Kevin Weekes: 2-3-0, 2.86, .903
Backups Combined: 9-6-0, 2.42, .918

Martin Brodeur: 6-2-2, 2.16, .916

So far I'd say they are doing pretty well. It is a very small sample size to be sure, and it is certainly much too early to discount the possibility that Scott Clemmensen may just be playing way over his head for a couple of weeks. However, I am just as interested in the number and type of shots Brodeur's replacements are facing, rather than simply whether they are stopping them or not. I cruised over to Hockey Numbers to see what his shot-quality calculations are telling us about New Jersey's goalies. SQN% is shot-quality neutral save percentage, and SQI is shot quality index (1.00 is average, below 1.00 means a team allows easier than average shots).

Brodeur: .914 SQN%, 0.98 SQI, 25.7 SA/60
Clemmensen: .918 SQN%, 0.90 SQI, 29.5 SA/60
Weekes: .892 SQN%, 0.90 SQI, 29.6 SA/60

Clemmensen and Weekes are very similar in their underlying numbers, facing almost identical shot quality and quantity. The difference is that Clemmensen is making more saves.

What really stands out, however, is that Brodeur has faced 4 fewer shots per game than his backups goalies have. Is this finally evidence of his soft goaltending skills as a third defenceman out on the ice, or are there other factors at play? In hockey, there are pretty much always other factors at play. If we look at the shot quality numbers, Clemmsen and Weekes have faced shots that were estimated as being 10% easier than average. Brodeur's shots were only 2% easier than average. This means that while New Jersey has allowed more shots without Brodeur in net, the extra shots faced have been apparently much easier to stop. If the primary reason for the difference was Brodeur's impact on puck possession, I'm not sure that we would expect to see a difference in shot quality.

Obviously we need to track shots and shot quality over a larger number of games to see if these differences are fluke or reality (and there are a number of factors that make shot quality less than perfectly reliable, such as reporting bias, failure to consider shot angle, etc.), but there are two possible explanations that do come to mind that could explain these results (maybe New Jersey fans can weigh in if either of them seems reasonable). The numbers suggest that either opposing teams are shooting from everywhere to test Clemmensen and Weekes, or New Jersey has changed its defensive style of play to protect their goalies which has resulted in allowing a higher number of lower quality shots.

We can try to quantify the goal prevention effect from the difference in shots allowed, by estimating the expected goals against by an average goalie facing Brodeur's shot distribution and then comparing that to his backups. The league average so far is .907, so let's go with that as our baseline number. We can adjust that for the shot quality for each goalie, and then multiply that by number of shots actually faced to get an expected GAA.

Here are the results:
Brodeur: 2.33
Clemmensen: 2.48
Weekes: 2.47

Through an expected goals approach, Brodeur's 4 fewer shots per game translate into a GAA effect of -0.15 goals per game. If we want to try to express that gap in terms of save percentage, it would be the equivalent of +.005 in save percentage for a goalie with a league average save percentage facing league average shots. We don't know at this point whether Clemmensen and Weekes are better or worse than average in terms of shot prevention, so it isn't necessarily correct to attribute the entire gap to Brodeur.

I don't know the typical starter/backup split in terms of shot quality, especially in this type of situation where lightly regarded backups replace an All-Star. However, there is a very similar situation going on in Vancouver, so I'll bring that in as a point of comparison. Here is how Luongo has done compared to his replacements by all the same metrics as above (SQN% = shot-quality neutral save percentage, SQI=shot quality index, expGAA = expected GAA for a league average goalie facing the same shots):

Luongo: 2.17, .928, .930 SQN%, 30.2 SA, 1.03 SQI, 2.89 expGAA
Sanford: 2.85, .905, .897 SQN%, 30.0 SA, 0.92 SQI, 2.57 expGAA
Schneider: 2.80, .896, .878 SQN%, 26.9 SA, 0.85 SQI, 2.13 expGAA

Backups: 2.83, .902, .891 SQN%, 28.9 SA, 0.90 SQI, 2.41 expGAA

The combined shot quantity and quality for Luongo's backups is very similar to Brodeur's (same shot quality against and a difference of less than one shot against per game). Just like Brodeur, Luongo has apparently faced more difficult shots, but Luongo has also faced more of them (1.3 extra shots per game than his backups). Luongo's expected GAA is actually 0.48 goals above that of his backups because of these factors, yet his actual GAA is 0.66 lower. According to the numbers, Vancouver has been hurt a lot more by goalie injuries than New Jersey has.

What about evidence that Brodeur affects his teammates, or that his puckhandling contributes to his team's offence? I went to Time on Ice to check out the possession statistics while each goalie was in the game (using Behind the Net's numbers to estimate minutes played at 5 on 5):

Brodeur: +8.4 Shot Diff/60, +7.9 Corsi/60, +12.1 Fenwick/60
Backups: +1.8 Shot Diff/60, +4.9 Corsi/60, +5.4 Fenwick/60

Luongo: -2.6 Shot Diff/60, -3.4 Corsi/60, -4.7 Fenwick/60
Backups: -2.9 Shot Diff/60, -4.2 Corsi/60, -7.5 Fenwick/60

Both goalies have better outshooting results than their backups do. New Jersey is obviously a better outshooting team than Vancouver, but Brodeur outperforms his backups by a larger margin than Luongo. Vancouver blocks a similar ratio of shots no matter who their goalie is, while New Jersey's shot block to total shot attempts against rate is 3% higher with Clemmensen or Weekes in net than with Brodeur.

One factor at work with the possession stats could be that Luongo has been in the lead much more often than his backup goalies. That may have led to more shots against, since there is some evidence to suggest that trailing teams are likely to shoot more. Vancouver tends to reduce their offensive pressure when they are leading, focusing instead on preserving the lead. New Jersey likely usually uses a similar strategy, but New Jersey's backups have a better record than Vancouver's so this may not have affected them as much.

It's pretty early to conclude anything, but there are a few observations to be made. New Jersey is a good team, and both the shot quality numbers and Scott Clemmensen's stats suggest they are still a very good defensive team. Playing in New Jersey helps a goalie's statistics, of that there is little doubt. However, the interesting numbers are the ones that relate to Martin Brodeur's impact on his team's play beyond stopping pucks (e.g. shots against and puck possession stats), and that is certainly something to follow along with as the season goes on.

17 comments:

Anonymous said...

something looks funky about chris' shot quality model. note that at even strenth, he has only four teams with an SQA higher than 1. (1 is supposed to be average.) something like this happened last season, and i emailed him and he fixed it. you might wanna do the same.

Triumph said...

I realize there is a massive amount of selection bias here, but I haven't seen more empty nets missed or posts hit than when Scott Clemmensen began playing goal this season. I fully expect his SV% to rest between .900 and .905 when the season is over.

I also think these numbers are interesting but all the sample sizes are too small - furthermore, one could make the argument that even if Brodeur's puckhandling is preventing low-quality shots, low-quality shots can turn into high-quality shots with rebounds - that's something I would be curious about re: shot quality models.

Bruce said...

In the last couple of weeks, I've noticed a distinct lack of gleeful comments posted by Bruce giving New Jersey's goalie stats since Martin Brodeur went down.

Don't worry, I've been following on an ongoing basis and will post updates from time to time. You beat me to it this time, CG, and with some interesting documentation. I'm frankly more interested in the shot prevention data than in any particular rooting interest for Brodeur. Usually he's such a constant that there is not much to compare him to, but this year hius backups' stats will have some heft to them.

Clemmensen has played well, while Weekes has been weak. Since the latter got first shot at it, it follows there was an immediate dip after Brodeur went down that was a worst-case scenario from Jersey's point of view. Clemmensen, and the team, have righted the ship in a major way. I note our man Patrik Elias has 23 points in the last 14 games.

Interesting to follow the New Jersey and Vancouver situations in parallel as both struggle to cover the absence of their star.

Anonymous said...

You can not seriously be using Scott Clemmensons recent performance as a knock on Brodeur are you? sure Clemmenson has been great. But one thing that negates most of the comparison is that to anyone watching even a handful of devils games, is that they play a completely different style without Brodeur. Clemmenson is also receiving WAY more goal support than brodeur has over the past 3 years. Not to mention the single most important factor: the devils team now is WAY stronger than any team brodeur has had in front of him the last 3 years. It is not just a healthy Holik and Rolston that gives added 2 way depth to the lineup, but also the maturing of what used to be a young and suspect defense (Martin, Oduya, Salvador) as well as young players (Parise, Zajac, Clarkson) and the awakening of guys like Elias, Zubrus, and Gionta. Its an apples and oranges argument because with a young team like the devils, performance from year to year is not comparable, especially at one position, especially that position being goalie. Case in point, look at Boston.

The Contrarian Goaltender said...

I would say Clemmensen's performance so far is simply a validation of my view of New Jersey's team defence. Whether you consider that to be a knock on Brodeur or not depends on how you view the relationship between a team and its goalie. Clemmensen is receiving strong goal support, true, but that has nothing to do with the important goalie stats like save percentage. I also agree with your point about performance from year to year not being comparable, but I'm not comparing across years, merely equating performances within the 2008-09 season.

Having said that, the whole point of this post is not to grade Brodeur but to try to evaluate the New Jersey goaltending context and see how it differs depending who is in net. Just like Bruce, my main interest this year is what the differences are in the shot data between Brodeur and his replacements. There are a lot of different tools that can be used to evaluate how well goalies stop the puck, and we have a lot of information about Brodeur's performance in that area. Where we lack information is how he affects the game other than by stopping the puck, i.e. puckhandling, rebound control, etc.

That is what this season will hopefully tell us more than anything - what is the effect of a Martin Brodeur-type goalie on team shot prevention and goal prevention. That is a very interesting question for goalie analysis, because the magnitude of that effect will determine whether we can evaluate based predominantly on save statistics or whether we need to bring in additional evaluative criteria.

Anonymous said...

Alright that seems about right then, perhaps I jumped the gun there assuming a blog title Brodeur is a Fraud:Where we are currently sitting and watching Scott Clemmenson do our arguing for us, was bashing Brodeur.
Fair enough, although for a change when comparing the numbers it is Brodeur's sample size that is too small and only after the season when has has played lets say 20-25 games should any conclusions be drawn.
As far as the Devils themselves, it is no shock they continue to succeed, and so does who ever is in goal for them. They have a very effective system of management in which fundamental hockey and 2 way responsibility is stressed, much like Detroit has done. Whatever the knock on Brodeur is, it can not be denied however, that the Devils string of success did not start until... coincidentally?... his rookie year.
The other thing that undoubtedly separates him from most is the longevity and consistency issue. Something you have yet to do, but something I would be interested in seeing you do is look at the wave of goalies that so frequently come in and put up big numbers and then are either gone or mediocre within the span of a few years. This leads me to believe that it is not hard to do what Clemmenson, or Brodeur do for a year, or two, or three or four, but rather doing it for a sustainted period of time is what is rarely seen. Just off the top of my head you have Theodore, Kiprusoff, Jim Carey, Khabibulin, Dipietro (if he ever plays again), Giguere? and Lundqvist? if this season is an indication, etc.

Anonymous said...

Clemmenson is getting the same goal support as Brodeur gets.

And the Devils were winning before Holik and Rolston got back, so Clemmenson was doing this with the same team as last years Devils.


Brodeur is overrated, this proves it.

Its to bad most people dont see it.

Anonymous said...

The same goal support??? And if this is really the same team then I guess I missed Parise put up 90 points last year. I definitely didnt see Rolston manning the powerplay. Bryce Salvador definitely didnt play this much last year either, and I am pretty sure Patrick Elias wasnt top 10 in league scoring.
Oh well, there are idiots everywhere, but if this is truely your observation, then you shouldnt be allowed to post.

Anonymous said...

You should read my post again before you nerdgasim.

THE DEVILS WERE WINNING WITH CLEMMENSON BEFORE ROLSTON AND HOLIK GOT BACK FROM INJURY!!!!

So its the same team as last year, you know the team that would have missed the playoffs without Marty the great.

They seem to be on pace to make the playoffs this year as well, I guess Clemmenson should get a Vezina and MVP vote.

The Vezina is a joke since Brodeur started winning it.

Anonymous said...

The same team argument is crap, even the author of this blog has recognized that. This years Devils team is much stronger than last years team, mainly because they have a young core of players who have improved dramatically. Case in point is Boston. They pretty much have the exact same team they did last year minus one or two guys, yet they have a 30 point turn around?

Also, Clemmenson has not been playing as well as Brodeur was. Brodeur was 6-2-2 with 2.16 gaa and .916 save percentage. Not to mention those numbers are consistent with what Brodeur has done for 15 years. Clemmenson on the other hand is 15-9-1, with a gaa of 2.38 and .918 save percentage, much of which was the result of the early hot streak the Devils had when the won 9 of 10, since he has been shaky. And Weekes has been terrible, end of story. So again, what are you trying to prove? That the best goalie in the league is somehow less of a goalie because one of his two replacements caught fire for 6 weeks? That laughable.

The Contrarian Goaltender said...

Anonymous: You raise some good objections, but I think you are dismissing the evidence at hand a little too easily.

Scott Clemmensen has been the de facto starter for over 2 months now, and has started 25 games. That is getting close to a significant sample size, especially since there are no quality of competition effects in there like there usually would be with a starter/backup comparison.

Even in his last 10 starts, Clemmensen has a 2.50 GAA and .915 save percentage. He may have been playing shaky over that span, but those results aren't too bad at all, and that is part of the point regarding New Jersey's team defence. Clemmensen's record was 4-6 over those same games, mainly because the team scored only 23 goals (and was shut out 3 times). You can't give all the credit for wins to the team, and all the blame for losses to the goalie.

As far as Weekes, let's compare him to league average:

Weekes: 2.79, .908
Average: 2.72, .908

A goalie you describe as terrible has put up pretty much league average stats. Of course that is in a small sample of games, but still, how many other teams do think Kevin Weekes would have put up average numbers on in 9 games played?

That is, of course, my main point: The evidence from Brodeur's absence is that New Jersey is a strong defensive team. Strong defensive teams help goalie stats. Therefore, when evaluating Brodeur, we need to take into account the team he played behind. You can make an argument about how strong they were for the last 2 seasons, but I'd still say they were pretty strong. Note that your Boston example doesn't really apply to the defensive side of the game - the Bruins had a solid defence last year, and their breakout can be mostly attributed to their improved offence and goaltending.

Nobody is saying Clemmensen is as good as Brodeur, or outplaying Brodeur. That would be ridiculous, Clemmensen is a journeyman with a career .910 AHL save percentage and was at .888 in 28 NHL games heading into this season. However, I think it is a bit of a stretch to attribute everything to luck as well. I think it would be pretty much impossible at this point to continue arguing, as many have, that Brodeur is the only reason that New Jersey is still competitive.

So again, what are you trying to prove? That the best goalie in the league is somehow less of a goalie because one of his two replacements caught fire for 6 weeks? That laughable.

This season's results alone would not be enough to make that claim, but I was already claiming that Brodeur was not the best goalie in the league for a long time prior to his injury. I think the results from this season are just more evidence in favour of my point of view.

Anonymous said...

The Boston comparison almost certainly would apply. Defense wise, they have much better players than New Jersey, and have for the last few years. But if you are saying that the 30 point turn around is simply the result of more scoring and goaltending?!? you are not looking at everything equally then. I do not understand how someone such as yourself who seems to get a kick out of arguing against Brodeur, can all of a sudden give credit in Boston for "improved goaltending" when almost certainly if measured by the same stick as you do with the Devils, you will see team depth and defense emerging more as a reason for those numbers.

Also, please tell me development of young 2 way players such a Lucic, Kreicji and especially Kessel only has an effect on the offense? That would be dead wrong. Not too mention Denis Wideman's maturity, and like New Jersey, incredibly depth as a result of the maturation of young guys.

Sure Tim Thomas is a very good goalie, and likely better than anyone in the east outside Brodeur and maybe Ryan Miller, but both him and Manny Fernandez posting save percentages upwards of .930 indicates again the it is more than just excellent goaltending that is influencing the results.

I am would like to note the contradiction in the way you address Tim Thomas' performance, as opposed to the way you address Clemmenson's. Why is it that with Clemmenson you are so quick to call it "the system" yet with Thomas this year, your posts like "How Long Can the Fairytale Continue" or what ever the title was, seems to be giving Thomas credit , when in reality, you telltale backup goalie arguments seem to go against Thomas, along with the fact that Clemmenson seems if anything to be a mirror image of guess who? Tim Thomas in 05-06, a late bloomer with an awkward, unorthodox style; who simply got it together and has shown some capabilities on playing goal for a NHL team. Again I am not saying Tim Thomas doesnt deserve any credit, but the situations in Boston and New Jersey are eerily similar when comparing the turnaround for last year to this year, in terms of defensive maturation, improved depth and scoring from maturation, and a spike in goaltending results from strong 2 way play.

As far as Weekes, the argument is not there. He has not looked sharp at all with the TWO exceptions being his first game against buffalo, a 2-0 loss, and the recent performance against the Kings in which he actually played VERY well. Take out those two performance and you will have a true indication of how well you can expect him to play MOST of the time.

The Contrarian Goaltender said...

But if you are saying that the 30 point turn around is simply the result of more scoring and goaltending?!? you are not looking at everything equally then.

Sure I am. How can you not think it is the result of more scoring and goaltending? Boston has the same coach and the same style of play and has almost identical shot results to last season. Look at the breakdown:

2007-08: 28.5 SF/60, 30.0 SA/60
2008-09: 29.8 SF/60, 30.1 SA/60

So what is the difference?

2007-08: 8.8% SH%, .916 Sv%
2008-09: 12.2% SH%, .931 Sv%

This year the pucks are going into the other team's net, and they're staying out of Boston's. Obviously the Bruins have had some defensive improvement, but it certainly hasn't been Lucic and Kessel leading the charge - check out this link, and you will see that they are the worst two skaters on the Bruins in terms of shots against while they are on the ice.

If Martin Brodeur put up a .930+ save percentage, then I wouldn't credit it to team depth and defence. All the shot quality data we have seen suggests that you can't put up a .930 save percentage entirely because of the defence in front of you - you need defence and a great effort from your goalies.

Maybe Clemmensen will have a similar career path to Thomas, I don't know, but the only comparison is to suggest that Clemmensen hypothetically may have a similar late-blooming career. Comparing them today, Thomas is one of the top 10 goalies in the NHL in save percentage since the lockout and Clemmensen has a barely above average save percentage in his undistinguished AHL career. Thus the difference in treatment.

As far as Weekes, the argument is not there.

Other than his good games, he has been a bad goalie. I'm not sure your argument is much better than mine. I never said Weekes has been anything but terrible this season. He hasn't been too good from what I've seen. Yet his stats are still almost exactly league average. You do the math.

Anonymous said...

your point on weekes makes no sense because we are not talking about eliminated half of his games, i am referring to eliminated outliers so far. if a team wants to know what kind of performance they are going to get from a guy, 1. they are not going to make any judgment solely off of 9 games. 2. if they had to, they would weigh the fact that 6/9 were absolutely abysmal performances, heavier than the fact that 3/9 were decent. goaltending is before anything else, about consistency.

you say you treat thomas differently than clemmenson because he has had a high save percentage for 3 years, yet brodeur would meet the same standards, and for his career, i dont think he has ever had a below league average save percentage. so you are not knocking thomas's performance despite the fact that he has been matched statistically by his backup WITH A SUBSTANTIAL amount of playing time as well, then how can you knock brodeur for putting up good numbers BECAUSE OF THE PERFORMANCE OF HIS BACKUPS. that is the double standard i am referring to.
Also, this shot quality argument you continually raise is completely bogus as well. The guy who came up with the whole system for calculating shot quality based save percentages has stated numerous times that it is a flawed method for many different reasons, but mainly because it is subjective.

The Contrarian Goaltender said...

It's tough to call anything an outlier from a 9 game sample. If the two games you mentioned are outliers, why isn't the Ranger game (6.00 GAA, .840 save percentage) considered an outlier and similarly disregarded? Take out that game and Weekes looks better. I think in a small sample you have to include everything and take the whole thing with a grain of salt. And on the whole, his stats are basically average. Having said that, he is playing on New Jersey, so his actual performance has probably been pretty poor. I'm not trying to say Weekes has been great by any means - he lost his starting job and has been way outplayed by a minor-leaguer.

so you are not knocking thomas's performance despite the fact that he has been matched statistically by his backup WITH A SUBSTANTIAL amount of playing time as well, then how can you knock brodeur for putting up good numbers BECAUSE OF THE PERFORMANCE OF HIS BACKUPS. that is the double standard i am referring to.

I think I know the source of the confusion here. You are right, there is a double standard. Martin Brodeur is considered one of the greatest goalies in the histroy of the NHL. Tim Thomas is considered to be a decent starting goalie.

Martin Brodeur is better than Tim Thomas. That is obvious, and I have never argued otherwise. When I argue about Brodeur, I am arguing against the people who say he is the greatest of all-time. The greatest of all-time standard is different than the regular starting goalie standard, that's just the way it is.

If you want to see how I rank Brodeur relative to his current peers, then read this post.

By the way, did you actually read my article on Tim Thomas? I addressed most of the points you are bringing up. For example, there was this paragraph:

Tim Thomas is not a fluke, but his season so far appears to be. I think it is probably fair to say Thomas is a good NHL starting goalie, but he looks like he is playing way over his head so far. I don't see his numbers going anywhere but down over the rest of the season.

Re: shot quality, it is flawed because of reporting bias, because we don't have shot angle, because it doesn't take into account the skill of the shooter, and because we don't know whether the shooter is being pressured or if he has time and space to make a shot. It is not flawed because it is subjective - it is calculated by a spreadsheet that assigns a probability based on the recorded shot type and distance.

Anonymous said...

You cant talk to a Devils fan Contrarian Goaltender, its a lose lose situation.

Last year it was " the Devils have nobody, its all Marty"

This year its " the Devils are awesome, Clemmensen sucks"

If Marty didnt get hurt guess what they would be saying about this year?

How impressive is the " most Wins record" that hes going for if they can "win" without him?

Anonymous said...

UPDATE

because the author of this blog has refused to update this post without bias, i will.

roberto luongo since returning HAS NOT EVEN COME CLOSE TO PERFORMING AS WELL AS HIS BACKUPS DID, AND THUS IS NOW A FRAUD AS WELL.