Courtesy of frequent commenter Statman, here is an absolutely unbelievable quote from Martin Brodeur from an article by Jay Greenberg at the New York Post:
"It's nothing about style really, but about fundamentals, how I get myself in position to make the save that's important to me," Brodeur said. "We give up 8-10 fewer shots a game here than other places because of the way I control rebounds."
I know the Post is basically a tabloid, but unless Greenberg made it up that is a fascinating quote. Brodeur is implying that the Devils are the worst team in the league at shot prevention, and are being bailed out entirely by his efforts. We must also be forced to conclude that Scott Clemmensen, Kevin Weekes, and every other backup goalie the team has had during Brodeur's career also has terrific rebound control skills, because they aren't facing 8-10 fewer shots per game (in fact, after getting peppered over the weekend, Brodeur moved to within just 0.2 shots against per game of Clemmensen). By implying that his team was allowing enough dangerous shots that an NHL-calibre goalie would give up that many rebounds, and that the other team could actually get their sticks on those rebounds even if they were available, Brodeur is throwing his defence under a bus without even realizing it. For the sake of reference, a good estimate of the average number of rebound shots faced by an NHL goalie in a single game is 2 or less.
Martin Brodeur is one of the best puckhandling goalies ever, and one of the best goalies in the league at controlling rebounds, and yet he apparently isn't even in the correct order of magnitude in terms of estimating the actual effect of those things. That is something to make one think long and hard about the relative worth of subjective evaluation compared to statistical analysis.
Monday, March 30, 2009
Saturday, March 28, 2009
Quality Starts
Puck Prospectus has come up with a definition of a quality start for a goalie, and has a list of the leaders and trailers in the category for this season. Their definition of a quality start is a game with a save percentage above .912, or a save percentage between .883 and .912 while allowing 2 goals or less.
Defining and measuring quality starts is something I have hacked around a bit with, and I know others have as well. I think most quality start definitions include something to do with 2 goals or less, just because the winning percentage splits drop so dramatically for a team once that third goal goes in. The problem is that there is obviously a team element involved, stopping 14/16 is usually quite different than stopping 39/41, but save percentage is also team-influenced.
My ideal quality start would be awarded whenever a goalie allows fewer goals against than expected based on the shots he faced, but that would require a detailed shot quality analysis for every single game to determine.
Perhaps the most interesting bit in the article was pointing out that the median save percentage was .912. Last I checked the average was around .907, which shows how bad games can have a disproportionate impact on goalie stats. There isn't much difference between letting up 5 goals or 9 goals, in both cases your team is going to lose nearly all of the time, but obviously the latter figure is going to have a bigger impact on a goalie's save stats. Henrik Lundqvist is one example of a goalie who seems to have a couple of games a year where he gets shelled for 7 goals on 20 shots or something like that but is otherwise pretty consistent.
It is probably a good idea to look at metrics like quality starts to avoid penalizing goalies too much for a few bad games that really hurt their seasonal GAA and save percentage stats, or on the other hand to see if goalies that record a lot of shutouts are really putting up excellent performances from game-to-game.
Defining and measuring quality starts is something I have hacked around a bit with, and I know others have as well. I think most quality start definitions include something to do with 2 goals or less, just because the winning percentage splits drop so dramatically for a team once that third goal goes in. The problem is that there is obviously a team element involved, stopping 14/16 is usually quite different than stopping 39/41, but save percentage is also team-influenced.
My ideal quality start would be awarded whenever a goalie allows fewer goals against than expected based on the shots he faced, but that would require a detailed shot quality analysis for every single game to determine.
Perhaps the most interesting bit in the article was pointing out that the median save percentage was .912. Last I checked the average was around .907, which shows how bad games can have a disproportionate impact on goalie stats. There isn't much difference between letting up 5 goals or 9 goals, in both cases your team is going to lose nearly all of the time, but obviously the latter figure is going to have a bigger impact on a goalie's save stats. Henrik Lundqvist is one example of a goalie who seems to have a couple of games a year where he gets shelled for 7 goals on 20 shots or something like that but is otherwise pretty consistent.
It is probably a good idea to look at metrics like quality starts to avoid penalizing goalies too much for a few bad games that really hurt their seasonal GAA and save percentage stats, or on the other hand to see if goalies that record a lot of shutouts are really putting up excellent performances from game-to-game.
Wednesday, March 25, 2009
Development Is the Key, Not Drafting
There was an interesting passage from about a month ago at From the Rink that I have been meaning to comment on:
Whatever the predicted value, Waddell has an extra draft pick to work with. The problem with that is, well, he’s Don Waddell. No player drafted by Waddell in Rounds 2-5 has yet played 150 games in the NHL. None. Nada.
The Thrashers had 25 picks in Rounds 2-5 from 1999-04. On average, that should have yielded 4.4 NHL players based on the team’s draft position(s). Colin Stuart might still make it and so might Brett Sterling and Grant Lewis but at 23 years old they are longshots.
Does anyone really believe that Don Waddell is so clueless that he picked players who were much less talented than everyone else? Or that his scouting reports were completely different than that of every other team? Even if he just was picking random players he had never seen from Central Scouting's rankings, he would have been almost certain to have at least a couple of hits if the only factor in the equation was talent. Either Waddell is actively sabotaging his team's talent acquisition by completely misevaluating players, or his team does not do a very good job transitioning its picks from being prospects to being professionals. I guess it is possible that Waddell is incompetent, but I'll play the odds here. I highly doubt that talent evaluation is the primary cause of Atlanta's poor draft record.
My philosophy is that player development is more important than drafting for non-elite prospects, e.g. everyone taken after the first 10-15 picks in the draft. The same guy drafted by Atlanta will likely have a very different career than if he was drafted by Detroit. If he goes to Detroit, he will spend several seasons at the minor league level being painstakingly developed and taught to play a strong two-way game by top-level coaches before being eased into the NHL in a limited role as part of a team with a strong winning culture in Detroit. On the other hand if he becomes a Thrasher he might get little support from a barebones development staff, get rushed to the big leagues and thrown to the wolves in tough situations on a losing team that may or may not have strong internal leadership. When he inevitably struggles through some tough stretches, the team may downgrade him as a prospect or give up on him and ship him off somewhere else. Top prospects like Heatley and Kovalchuk are likely to succeed in Atlanta, as they most likely would in any city around the league, but for a lesser player environmental factors can have a big impact on whether they succeed or fail at the highest level of professional hockey.
Tuesday, March 24, 2009
Interesting Links, and Some Thoughts on Sports Journalism
I don't usually post links to other articles, but there are a few that I wanted to highlight as they are definitely worth checking out.
First of all, JLikens at Objective NHL demonstrates that there is one more adjustment I need to make to my save percentage rankings, an adjustment for rink reporting bias. That is certainly a topic that requires future study. I don't think that an observed shot differential makes an open-and-shut case for rink reporting bias, as I think there are some teams that are more likely to allow fewer or more shots at home based on playing style, or possibly other factors like offensive/defensive strategy, commitment to matching lines, frequency of back-to-backs played on the home vs. on the road, etc. However, it is pretty clear that certain rinks do count shots differently, and that would have affected goalie numbers.
Tom Awad at Puck Prospectus did a statistical ranking of the top 10 goalies since 1944. His numbers look pretty reasonable, although I think his value over replacement metric maybe gives too much of a bonus for longevity (Tony Esposito seems a bit out of place at #4). Martin Brodeur is ranked in a tie for 5th. Interestingly enough, given the recent discussion in this space, one of the goalies tied with Brodeur is Ed Belfour.
There was an interesting quote from John Davidson that I ran across in an article by Eric Duhatschek (hat tip: Cycle Like the Sedins):
“If you take [Brodeur's] whole career playing in New Jersey and document odd-man rushes against for that whole period, he'd have the fewest odd-man rushes against, I would think – and the fewest penalties, because they're so disciplined. Style of play, him handling the puck, which means fewer injuries to defencemen, which means they're healthy to play the full season. There's a big domino effect there.”
I don't disagree with the first part, and the second part is something that would be interesting to look into.
Any of you who hate the goalie wins stat as much as I do will probably be amused by this post at Copper 'n Blue (specifically the part about Miikka Kiprusoff). The first comment is a classic, as well.
And finally, I just had to throw in a link to this article by Damien Cox entitled "Will People Really Debate Whether Martin Brodeur is the Best Ever?". Forget Awad's numbers, or mine, or anyone else's, here comes the mainstream journalist saying that in 5 years it will be completely laughable to pick anyone other than Brodeur as the best ever.
I'll never be able to match up to the excellence of Coxbloc in skewering the ramblings of the co-writer of Brodeur's autobiography, but I'd like to address this article briefly, as there has been some discussion in the comments here lately about how someone like me can reconcile the fact that I disagree with a lot of people who spend their time around the pros and are paid to cover the sport of hockey.
I don't think journalists are stupid. I have no doubt that Cox would easily be able to understand everything I have written on this blog. I don't think he lacks basic math skills, and I don't think he even distrusts statistics. I think he just forms viewpoints without defining and testing his criteria, and he is uninterested in testing those opinions by using alternate methods.
The way I see it, the best bloggers and independent hockey analysts possess two main things that are very much lacking in the majority of mainstream sports journalism:
1. Logical reasoning skills
2. Intellectual curiosity
Let's take an example from Cox's article here. He brings up talks about New Jersey's general lack of success prior to 1993-94, and then their success afterwards. Here is his argument, split into its components:
Premise 1: Lemaire became coach in 1993-94
Premise 2: Brodeur became starting goalie in 1993-94
Conclusion: "It was equal parts of Lemaire and Brodeur, then, that made the Devils develop into the type of team they became."
They both showed up at the same time, therefore they are equally responsible? That quite clearly does not follow, and no attempt was made to quantify or even reasonably subjectively evaluate the relative contributions of each, much less the other 19 skaters on the roster. Something happened, and somebody was there while it happened - well, he must have contributed to it. Martin Brodeur's team won the Stanley Cup in 2003 while Roberto Luongo's team missed the playoffs. I guess that means Brodeur must have been the better goalie. Grant Fuhr has won a lot of championships, he must have been clutch, and so on.
Cox's whole article is a great example of a lack of clear logical argument. It is, however, a typical journalist's greatest ever article, a hodge-podge of observations, anecdotes and opinions that are somehow supposed to coalesce into a coherent demonstration of greatness. It is distinguished by its lack of an attempt to compare the subject to others, even peers who played at the exact same time Brodeur did, and its lack of defining the relevant criteria of greatness. Does Cox consider puckhandling skill to be more or less important than playoff success? How about "defining the personality of a franchise", is that more or less important than Vezina Trophies? Who knows?
Not surprisingly whenever a journalist sits down and types a bunch of unrelated items into his computer and calls it a proof, the article tends to fall a bit flat. Even more so when, as in this instance, the opinion espoused happens to be wrong in the first place.
Point 2, lack of intellectual curiosity, tends to come up when people bring numbers to the table. Fire Joe Morgan was classic at exposing mainstream journalists who would ridicule advanced statistical metrics because they disagreed with them, without even understanding what they meant.
That kind of anti-numbers bias comes up in hockey as well, but there is more to it than that. To me, journalists seem to be satisfied with simple explanations, and seem to have little interest to investigate further to see whether something is true. I have read the attempts of a number of different journalists to account for the effect of New Jersey's excellent defensive play on Brodeur's success, and I've never seen any of them talk about shot quality. They'll usually pull up some carefully selected irrelevant metric to show that it doesn't matter (e.g. "Brodeur ranked in the top 10 in shots against seven times in the last 10 seasons"), or they'll gloss it over by talking about at best tangentially related things like New Jersey's offensive performance or the number of Devils coaching changes, or they'll demonstrate that New Jersey did not have excellent defensive personnel or excellent shot prevention in 2007-08 and skillfully extrapolate those results backward over the prior 14 years of Brodeur's career.
The reason I focus on Brodeur so borderline obsessively is that I find his career to be very interesting. He has had spent almost his entire career within an extreme team context, and he has a large gap between the subjective evaluation of hockey observers and the objective evaluation of the statistical record. He played behind a great defence - how much does that help a goalie? He played a lot of games - how hard is that to do, and how valuable is it to a team? He is great at all the goalie non-save skills - how big of an impact does that have on the rest of his team? His team won a lot of games and Cups - was that because of great/clutch goaltending, or because of superior team play?
Answer all of those questions for Brodeur and you can pretty much do it for anyone, and therein lies the challenge. But getting a complete answer to any individual one of them is not easy, much less properly evaluating the whole, and that is why you can forgive me if I don't have a ton of respect for journalists who haven't even attempted any of that legwork trying to tell me the case is already closed.
First of all, JLikens at Objective NHL demonstrates that there is one more adjustment I need to make to my save percentage rankings, an adjustment for rink reporting bias. That is certainly a topic that requires future study. I don't think that an observed shot differential makes an open-and-shut case for rink reporting bias, as I think there are some teams that are more likely to allow fewer or more shots at home based on playing style, or possibly other factors like offensive/defensive strategy, commitment to matching lines, frequency of back-to-backs played on the home vs. on the road, etc. However, it is pretty clear that certain rinks do count shots differently, and that would have affected goalie numbers.
Tom Awad at Puck Prospectus did a statistical ranking of the top 10 goalies since 1944. His numbers look pretty reasonable, although I think his value over replacement metric maybe gives too much of a bonus for longevity (Tony Esposito seems a bit out of place at #4). Martin Brodeur is ranked in a tie for 5th. Interestingly enough, given the recent discussion in this space, one of the goalies tied with Brodeur is Ed Belfour.
There was an interesting quote from John Davidson that I ran across in an article by Eric Duhatschek (hat tip: Cycle Like the Sedins):
“If you take [Brodeur's] whole career playing in New Jersey and document odd-man rushes against for that whole period, he'd have the fewest odd-man rushes against, I would think – and the fewest penalties, because they're so disciplined. Style of play, him handling the puck, which means fewer injuries to defencemen, which means they're healthy to play the full season. There's a big domino effect there.”
I don't disagree with the first part, and the second part is something that would be interesting to look into.
Any of you who hate the goalie wins stat as much as I do will probably be amused by this post at Copper 'n Blue (specifically the part about Miikka Kiprusoff). The first comment is a classic, as well.
And finally, I just had to throw in a link to this article by Damien Cox entitled "Will People Really Debate Whether Martin Brodeur is the Best Ever?". Forget Awad's numbers, or mine, or anyone else's, here comes the mainstream journalist saying that in 5 years it will be completely laughable to pick anyone other than Brodeur as the best ever.
I'll never be able to match up to the excellence of Coxbloc in skewering the ramblings of the co-writer of Brodeur's autobiography, but I'd like to address this article briefly, as there has been some discussion in the comments here lately about how someone like me can reconcile the fact that I disagree with a lot of people who spend their time around the pros and are paid to cover the sport of hockey.
I don't think journalists are stupid. I have no doubt that Cox would easily be able to understand everything I have written on this blog. I don't think he lacks basic math skills, and I don't think he even distrusts statistics. I think he just forms viewpoints without defining and testing his criteria, and he is uninterested in testing those opinions by using alternate methods.
The way I see it, the best bloggers and independent hockey analysts possess two main things that are very much lacking in the majority of mainstream sports journalism:
1. Logical reasoning skills
2. Intellectual curiosity
Let's take an example from Cox's article here. He brings up talks about New Jersey's general lack of success prior to 1993-94, and then their success afterwards. Here is his argument, split into its components:
Premise 1: Lemaire became coach in 1993-94
Premise 2: Brodeur became starting goalie in 1993-94
Conclusion: "It was equal parts of Lemaire and Brodeur, then, that made the Devils develop into the type of team they became."
They both showed up at the same time, therefore they are equally responsible? That quite clearly does not follow, and no attempt was made to quantify or even reasonably subjectively evaluate the relative contributions of each, much less the other 19 skaters on the roster. Something happened, and somebody was there while it happened - well, he must have contributed to it. Martin Brodeur's team won the Stanley Cup in 2003 while Roberto Luongo's team missed the playoffs. I guess that means Brodeur must have been the better goalie. Grant Fuhr has won a lot of championships, he must have been clutch, and so on.
Cox's whole article is a great example of a lack of clear logical argument. It is, however, a typical journalist's greatest ever article, a hodge-podge of observations, anecdotes and opinions that are somehow supposed to coalesce into a coherent demonstration of greatness. It is distinguished by its lack of an attempt to compare the subject to others, even peers who played at the exact same time Brodeur did, and its lack of defining the relevant criteria of greatness. Does Cox consider puckhandling skill to be more or less important than playoff success? How about "defining the personality of a franchise", is that more or less important than Vezina Trophies? Who knows?
Not surprisingly whenever a journalist sits down and types a bunch of unrelated items into his computer and calls it a proof, the article tends to fall a bit flat. Even more so when, as in this instance, the opinion espoused happens to be wrong in the first place.
Point 2, lack of intellectual curiosity, tends to come up when people bring numbers to the table. Fire Joe Morgan was classic at exposing mainstream journalists who would ridicule advanced statistical metrics because they disagreed with them, without even understanding what they meant.
That kind of anti-numbers bias comes up in hockey as well, but there is more to it than that. To me, journalists seem to be satisfied with simple explanations, and seem to have little interest to investigate further to see whether something is true. I have read the attempts of a number of different journalists to account for the effect of New Jersey's excellent defensive play on Brodeur's success, and I've never seen any of them talk about shot quality. They'll usually pull up some carefully selected irrelevant metric to show that it doesn't matter (e.g. "Brodeur ranked in the top 10 in shots against seven times in the last 10 seasons"), or they'll gloss it over by talking about at best tangentially related things like New Jersey's offensive performance or the number of Devils coaching changes, or they'll demonstrate that New Jersey did not have excellent defensive personnel or excellent shot prevention in 2007-08 and skillfully extrapolate those results backward over the prior 14 years of Brodeur's career.
The reason I focus on Brodeur so borderline obsessively is that I find his career to be very interesting. He has had spent almost his entire career within an extreme team context, and he has a large gap between the subjective evaluation of hockey observers and the objective evaluation of the statistical record. He played behind a great defence - how much does that help a goalie? He played a lot of games - how hard is that to do, and how valuable is it to a team? He is great at all the goalie non-save skills - how big of an impact does that have on the rest of his team? His team won a lot of games and Cups - was that because of great/clutch goaltending, or because of superior team play?
Answer all of those questions for Brodeur and you can pretty much do it for anyone, and therein lies the challenge. But getting a complete answer to any individual one of them is not easy, much less properly evaluating the whole, and that is why you can forgive me if I don't have a ton of respect for journalists who haven't even attempted any of that legwork trying to tell me the case is already closed.
Monday, March 23, 2009
Belfour's Shot Prevention
When some goalies are in the net, fewer shots are taken by the other team. Although I was once skeptical of this effect, it has been shown to be pretty clear in the historical data. Martin Brodeur is a goalie who usually gets a lot of credit for his puckhandling and other skills that likely contribute to a reduced shots against total. There are, however, other goalies who are also very good at those skills.
I believe the goalie who might be the most overlooked in this aspect of goaltending is Ed Belfour. Having closely compared their shots against records, there is good evidence to suggest that Belfour's shot prevention effect may be every bit as large as Martin Brodeur's.
Belfour is a little easier to evaluate since he averaged fewer games per season and played on a number of different teams during his career. To do my comparative shots analysis, I usually look for goalies that had both substantial playing time with the study goalie as well as lots of minutes played on other teams. There are three goalies that fit these criteria that we can use to evaluate Belfour: Jeff Hackett, Roman Turek, and Marty Turco.
In this type of analysis, I prefer to take the average of yearly averages rather than simply calculate an average from the totals, because otherwise it skews the data when there are seasons with much higher or lower minutes played totals. Belfour played with Hackett from 1993-94 to 1996-97, and averaged 27.3 SA/60 per season, compared to 29.2 SA/60 for Hackett. That is a gap of about 2 shots per game. Over the rest of his career, Hackett tended to face more shots than his teammates, averaging about one extra shot per game against. That gives us an estimate for Belfour of about one shot per game saved.
Belfour played two seasons with Roman Turek in Dallas, and faced 0.5 fewer shots per game. Turek was pretty consistently about a half shot below his goalie teammates throughout the rest of his career, which again supports the one shot per game number.
In his two seasons playing with Marty Turco, Belfour was about a shot per game better than Turco, which is impressive since Turco has been half a shot per game better than his backups over the last few years (and Dallas has tended to bring in goalies who play a similar style, so Turco's effect might even greater when you consider he is being compared to guys like Johan Hedberg or Mike Smith). It could be that Turco had not yet developed his skills to the same extent early in his career, or that he learned a thing or two from Belfour that he was able to apply later on, or maybe there were some other variables that were influencing the shot numbers.
The results from all three goalies suggest that Belfour has a shots against effect of at least one shot per game. I did another piece of work where I compared how the backup goalies did while playing with a specific goalie and how they did on other teams. Ed Belfour faced 26.7 shots per game, compared to 27.9 for his backups. His backups had a weighted average of 28.4 on other teams, which suggests that although his defences were better than average at preventing shots, Belfour contributed to his team's shot prevention.
For the sake of comparison, Brodeur faced 25.4 shots per game compared to 26.1 for his backups, who had a weighted average of 28.5 shots per game on other teams. This shows that New Jersey had a significant defensive effect. Subtracting the differences between the two of them, and you get a 1.2 shot differential for Belfour and a 0.7 difference for Brodeur. It is likely that weaker competition and other small sample size factors mean that Brodeur's figure is a bit understated, and those numbers do not include the 2008-09 season where Brodeur has so far faced about 2 fewer shots per game than his teammates. Still, it is more evidence that Belfour is comparable to Brodeur in terms of non-save skills.
I am willing for now to credit both Belfour and Brodeur with preventing 1 shot per game compared to average goalies. I also think that this information is the decisive factor that separates them both from other goalies like Joseph and Vanbiesbrouck who have similar save percentage records.
It does appear that the Brodeur vs. Belfour debate is a lot closer than most people would think. If these estimates are correct, then Brodeur doesn't gain an advantage over Belfour in terms of non-save skills, and the comparison between them should be decided by puckstopping ability alone. Brodeur has a better save percentage record compared to league average, but evidence suggests he also faced easier shots, since Belfour spent some of his career on bad teams (Florida, San Jose) and other parts of his career on teams that likely allowed shots of a higher-than-average difficulty (e.g. Toronto). If you compare Belfour's Dallas numbers to Brodeur's New Jersey numbers at the same time, which were likely two fairly similar team contexts to play in, their stats are essentially identical: 2.18 GAA and .909 save % for Brodeur vs. 2.19 GAA and .910 save % for Belfour.
I think that both have had pretty similar careers - they had some great individual seasons early, had lots of playoff and team success as members of terrific two-way defensive teams, and then spent a few seasons on weaker teams yet still performed pretty well. They also had some off-years mixed in, although New Jersey's defence did a better job of camouflaging Brodeur's down seasons compared to Belfour, who actually lost his starting job twice - to Hackett in Chicago and to Turco in Dallas.
The main reason that I think there is a reasonable case for Belfour over Brodeur right now is playoff performance, where in my estimation Belfour has been pretty clearly the better goalie. If my shot quality estimates in my last post were correct, then their regular season records might actually be pretty similar as well. Having said that, however, I think I would still take Brodeur's career over Belfour's right now, and with a few years left Brodeur will likely leave little doubt before his career is over.
I believe the goalie who might be the most overlooked in this aspect of goaltending is Ed Belfour. Having closely compared their shots against records, there is good evidence to suggest that Belfour's shot prevention effect may be every bit as large as Martin Brodeur's.
Belfour is a little easier to evaluate since he averaged fewer games per season and played on a number of different teams during his career. To do my comparative shots analysis, I usually look for goalies that had both substantial playing time with the study goalie as well as lots of minutes played on other teams. There are three goalies that fit these criteria that we can use to evaluate Belfour: Jeff Hackett, Roman Turek, and Marty Turco.
In this type of analysis, I prefer to take the average of yearly averages rather than simply calculate an average from the totals, because otherwise it skews the data when there are seasons with much higher or lower minutes played totals. Belfour played with Hackett from 1993-94 to 1996-97, and averaged 27.3 SA/60 per season, compared to 29.2 SA/60 for Hackett. That is a gap of about 2 shots per game. Over the rest of his career, Hackett tended to face more shots than his teammates, averaging about one extra shot per game against. That gives us an estimate for Belfour of about one shot per game saved.
Belfour played two seasons with Roman Turek in Dallas, and faced 0.5 fewer shots per game. Turek was pretty consistently about a half shot below his goalie teammates throughout the rest of his career, which again supports the one shot per game number.
In his two seasons playing with Marty Turco, Belfour was about a shot per game better than Turco, which is impressive since Turco has been half a shot per game better than his backups over the last few years (and Dallas has tended to bring in goalies who play a similar style, so Turco's effect might even greater when you consider he is being compared to guys like Johan Hedberg or Mike Smith). It could be that Turco had not yet developed his skills to the same extent early in his career, or that he learned a thing or two from Belfour that he was able to apply later on, or maybe there were some other variables that were influencing the shot numbers.
The results from all three goalies suggest that Belfour has a shots against effect of at least one shot per game. I did another piece of work where I compared how the backup goalies did while playing with a specific goalie and how they did on other teams. Ed Belfour faced 26.7 shots per game, compared to 27.9 for his backups. His backups had a weighted average of 28.4 on other teams, which suggests that although his defences were better than average at preventing shots, Belfour contributed to his team's shot prevention.
For the sake of comparison, Brodeur faced 25.4 shots per game compared to 26.1 for his backups, who had a weighted average of 28.5 shots per game on other teams. This shows that New Jersey had a significant defensive effect. Subtracting the differences between the two of them, and you get a 1.2 shot differential for Belfour and a 0.7 difference for Brodeur. It is likely that weaker competition and other small sample size factors mean that Brodeur's figure is a bit understated, and those numbers do not include the 2008-09 season where Brodeur has so far faced about 2 fewer shots per game than his teammates. Still, it is more evidence that Belfour is comparable to Brodeur in terms of non-save skills.
I am willing for now to credit both Belfour and Brodeur with preventing 1 shot per game compared to average goalies. I also think that this information is the decisive factor that separates them both from other goalies like Joseph and Vanbiesbrouck who have similar save percentage records.
It does appear that the Brodeur vs. Belfour debate is a lot closer than most people would think. If these estimates are correct, then Brodeur doesn't gain an advantage over Belfour in terms of non-save skills, and the comparison between them should be decided by puckstopping ability alone. Brodeur has a better save percentage record compared to league average, but evidence suggests he also faced easier shots, since Belfour spent some of his career on bad teams (Florida, San Jose) and other parts of his career on teams that likely allowed shots of a higher-than-average difficulty (e.g. Toronto). If you compare Belfour's Dallas numbers to Brodeur's New Jersey numbers at the same time, which were likely two fairly similar team contexts to play in, their stats are essentially identical: 2.18 GAA and .909 save % for Brodeur vs. 2.19 GAA and .910 save % for Belfour.
I think that both have had pretty similar careers - they had some great individual seasons early, had lots of playoff and team success as members of terrific two-way defensive teams, and then spent a few seasons on weaker teams yet still performed pretty well. They also had some off-years mixed in, although New Jersey's defence did a better job of camouflaging Brodeur's down seasons compared to Belfour, who actually lost his starting job twice - to Hackett in Chicago and to Turco in Dallas.
The main reason that I think there is a reasonable case for Belfour over Brodeur right now is playoff performance, where in my estimation Belfour has been pretty clearly the better goalie. If my shot quality estimates in my last post were correct, then their regular season records might actually be pretty similar as well. Having said that, however, I think I would still take Brodeur's career over Belfour's right now, and with a few years left Brodeur will likely leave little doubt before his career is over.
Monday, March 16, 2009
Brodeur, Roy and Hasek: The Debate Rages On
With Martin Brodeur having tied Patrick Roy's wins record, there is a lot of debate going on about who was better and who is the greatest goalie of all-time. Everyone else is doing it, so I might as well join in. I've done similar things before, but I've got a couple of new wrinkles this time.
The typical debate is set up as a choice between having 4 Cups and 3 Conn Smythes vs. having the career wins and shutouts record. I think all of those things are strongly team dependent and are correspondingly overhyped, so I'm not even going to bring them up. Having done a lot of research lately into shots against by goaltender, I have come up with some estimates of shot prevention effects, and would like to use those numbers to review the save percentage vs. league average results I posted some time ago.
I prefer to rank goalies by taking their save percentages and adjusting for all the important factors. Therefore, I have included a shot prevention adjustment (by giving goalies credit for shots they likely prevented and subtracting from the goalie's shot total for shots they likely had a hand in creating; if you haven't read any of my work on the topic check out this post estimating the effect for Brodeur) and a team defence adjustment based on the quality of scoring chances against, which gives an adjusted save percentage that can be compared against league average to account for differences in league scoring levels. I also ran the numbers for a few others (Belfour, Joseph, and Luongo) who should also be in the conversation about the best goalies in recent years.
These adjustments were mainly derived by comparing performance by backup goalies while on the same team to their performance on different teams, and using that that to assess the overall team context. There are many variables still in play in that type of comparison, such as quality of backups, quality of teams, small seasonal sample sizes, and the usual year-to-year random fluctuations, so I mostly took what the numbers said but did make a few subjective adjustments. They included adjusting downward the shot quality numbers for Roy and Brodeur, who both had results that were far overstated because their backups played a lot of games on bad teams in other cities, and increasing Brodeur's shot prevention effect and lowering Luongo's based on a more detailed comparative analysis of their performance relative to their teammates.
Here are my estimates:
Patrick Roy: 2.5% easier shot quality, 1 shot created/gm
Martin Brodeur: 5% easier shot quality, 1 shot prevented/gm
Dominik Hasek: 2.5% harder shot quality, 0.5 shots created/gm
Roberto Luongo: 0% harder shot quality, 1 shot created/gm
Ed Belfour: 0% harder shot quality, 1 shot prevented/gm
Curtis Joseph: 2.5% harder shot quality, 0.75 shots created/gm
That leads us to the following results:
These numbers allow us to compare the adjusted save pecentages to league average and calculate a number of goals above average for each goalie that combines quantity and quality:
After letting some of the air out of Roy's numbers, Dominik Hasek reigns supreme at the top of the rankings list. The surprising result is how close to the rest of the pack Patrick Roy ends up, although these numbers are from the regular season only.
Ed Belfour's ranking is also a point of interest. I think nearly everyone would rank him 4th out of this group but well behind the top 3. That's probably underrating Belfour. I have long thought that there was much more of a gap between Hasek and Brodeur than between Brodeur and Belfour, and these numbers certainly support that.
I ran the numbers for the playoffs as well, but without the same ability to compare backup results those numbers are entirely based on guesswork. I'm sure the shot quality factors were different, since the opposition is tougher and the number of games played on each team in the regular season is different than the number of games for each team in the playoffs. Belfour likely faced easier shots in the playoffs because of all the deep playoff runs he had in Dallas, and Roy's difficulty probably rose because of strength of competition and more games played in Colorado. This is an imprecise measure to be sure, but I'd guesstimate the playoff goals above average numbers as something like the following:
1. Patrick Roy, 74.6
2. Ed Belfour, 51.8
3. Dominik Hasek, 31.3
4. Martin Brodeur, 24.3
5. Curtis Joseph, 22.3
6. Roberto Luongo, 8.1
Obviously the inputs determine the results in a study like this, so there can and should be considerable debate about the adjustments I've made. I am not claiming to have ironclad proof on any of them and I am open to the possibility of any or all of them being wrong. The most discussion-worthy adjustments, and possibly the most difficult ones to estimate, are the strength of team defensive adjustments for Patrick Roy and Martin Brodeur. I think it is pretty clear both goalies benefitted from their teams, but the question is how much.
I think Roy's context is probably a little tougher to evaluate. Playing for Montreal in the 1980s was probably relatively similar to playing in New Jersey in the 1990s, with great team defence and fewer than average power plays by the other team. Then Roy was traded to Colorado, who had a different style of play. The shot quality data we do have available from '03 and '04 indicates that New Jersey was much better than Colorado at preventing opposing scoring chances, which is primarily why I gave Roy a different overall shot quality rating than Brodeur.
Overall, I think it is pretty clear that Dominik Hasek is the best goalie of the last 20 years (and I would argue the best goalie ever). As far as Brodeur vs. Roy goes I'd still take Roy. I think Roy's peak is well ahead of Brodeur's (illustrated graphically here), and Roy has also been better in the playoffs, but it does seem like the more work I do on St. Patrick the more question marks are raised about how much of his success was from team factors.
If Brodeur keeps up his current level of play for several more seasons (the previous three seasons account for over one-third of Brodeur's career goals over average total), then he will be at least moving into the conversation with Roy. If we are able to more precisely drill down to the relative team effects and adjust for them maybe we'll find that the team factors between the two aren't that different after all, and in that case the debate could get very interesting by the time Brodeur's career is done.
The typical debate is set up as a choice between having 4 Cups and 3 Conn Smythes vs. having the career wins and shutouts record. I think all of those things are strongly team dependent and are correspondingly overhyped, so I'm not even going to bring them up. Having done a lot of research lately into shots against by goaltender, I have come up with some estimates of shot prevention effects, and would like to use those numbers to review the save percentage vs. league average results I posted some time ago.
I prefer to rank goalies by taking their save percentages and adjusting for all the important factors. Therefore, I have included a shot prevention adjustment (by giving goalies credit for shots they likely prevented and subtracting from the goalie's shot total for shots they likely had a hand in creating; if you haven't read any of my work on the topic check out this post estimating the effect for Brodeur) and a team defence adjustment based on the quality of scoring chances against, which gives an adjusted save percentage that can be compared against league average to account for differences in league scoring levels. I also ran the numbers for a few others (Belfour, Joseph, and Luongo) who should also be in the conversation about the best goalies in recent years.
These adjustments were mainly derived by comparing performance by backup goalies while on the same team to their performance on different teams, and using that that to assess the overall team context. There are many variables still in play in that type of comparison, such as quality of backups, quality of teams, small seasonal sample sizes, and the usual year-to-year random fluctuations, so I mostly took what the numbers said but did make a few subjective adjustments. They included adjusting downward the shot quality numbers for Roy and Brodeur, who both had results that were far overstated because their backups played a lot of games on bad teams in other cities, and increasing Brodeur's shot prevention effect and lowering Luongo's based on a more detailed comparative analysis of their performance relative to their teammates.
Here are my estimates:
Patrick Roy: 2.5% easier shot quality, 1 shot created/gm
Martin Brodeur: 5% easier shot quality, 1 shot prevented/gm
Dominik Hasek: 2.5% harder shot quality, 0.5 shots created/gm
Roberto Luongo: 0% harder shot quality, 1 shot created/gm
Ed Belfour: 0% harder shot quality, 1 shot prevented/gm
Curtis Joseph: 2.5% harder shot quality, 0.75 shots created/gm
That leads us to the following results:
| Goalie | Save % | SA/60 | GAA | SA Prev | Adj SA | SA AdjS% | SQ AdjS% |
| Hasek | .922 | 28.32 | 2.20 | 0.5 | 27.82 | .921 | .923 |
| Luongo | .919 | 31.82 | 2.58 | 1.0 | 30.82 | .916 | .916 |
| Brodeur | .914 | 25.46 | 2.20 | -1.0 | 26.46 | .917 | .913 |
| Roy | .910 | 28.24 | 2.54 | 1.0 | 27.24 | .907 | .904 |
| Joseph | .907 | 29.75 | 2.79 | 0.75 | 29.00 | .904 | .906 |
| Belfour | .906 | 26.66 | 2.50 | -1.0 | 27.66 | .910 | .910 |
These numbers allow us to compare the adjusted save pecentages to league average and calculate a number of goals above average for each goalie that combines quantity and quality:
| Rank | Goalie | AdjSV% | LgAvg | % Diff | SA | GOA |
| 1. | Hasek | .923 | .903 | +21% | 20,220 | 402.3 |
| 2. | Roy | .904 | .895 | +9% | 28,353 | 267.2 |
| 3. | Belfour | .910 | .900 | +10% | 24,751 | 238.0 |
| 4. | Brodeur | .913 | .905 | +8% | 24,720 | 190.3 |
| 5. | Joseph | .906 | .900 | +6% | 26,676 | 165.3 |
| 6. | Luongo | .916 | .907 | +10% | 16,051 | 149.1 |
After letting some of the air out of Roy's numbers, Dominik Hasek reigns supreme at the top of the rankings list. The surprising result is how close to the rest of the pack Patrick Roy ends up, although these numbers are from the regular season only.
Ed Belfour's ranking is also a point of interest. I think nearly everyone would rank him 4th out of this group but well behind the top 3. That's probably underrating Belfour. I have long thought that there was much more of a gap between Hasek and Brodeur than between Brodeur and Belfour, and these numbers certainly support that.
I ran the numbers for the playoffs as well, but without the same ability to compare backup results those numbers are entirely based on guesswork. I'm sure the shot quality factors were different, since the opposition is tougher and the number of games played on each team in the regular season is different than the number of games for each team in the playoffs. Belfour likely faced easier shots in the playoffs because of all the deep playoff runs he had in Dallas, and Roy's difficulty probably rose because of strength of competition and more games played in Colorado. This is an imprecise measure to be sure, but I'd guesstimate the playoff goals above average numbers as something like the following:
1. Patrick Roy, 74.6
2. Ed Belfour, 51.8
3. Dominik Hasek, 31.3
4. Martin Brodeur, 24.3
5. Curtis Joseph, 22.3
6. Roberto Luongo, 8.1
Obviously the inputs determine the results in a study like this, so there can and should be considerable debate about the adjustments I've made. I am not claiming to have ironclad proof on any of them and I am open to the possibility of any or all of them being wrong. The most discussion-worthy adjustments, and possibly the most difficult ones to estimate, are the strength of team defensive adjustments for Patrick Roy and Martin Brodeur. I think it is pretty clear both goalies benefitted from their teams, but the question is how much.
I think Roy's context is probably a little tougher to evaluate. Playing for Montreal in the 1980s was probably relatively similar to playing in New Jersey in the 1990s, with great team defence and fewer than average power plays by the other team. Then Roy was traded to Colorado, who had a different style of play. The shot quality data we do have available from '03 and '04 indicates that New Jersey was much better than Colorado at preventing opposing scoring chances, which is primarily why I gave Roy a different overall shot quality rating than Brodeur.
Overall, I think it is pretty clear that Dominik Hasek is the best goalie of the last 20 years (and I would argue the best goalie ever). As far as Brodeur vs. Roy goes I'd still take Roy. I think Roy's peak is well ahead of Brodeur's (illustrated graphically here), and Roy has also been better in the playoffs, but it does seem like the more work I do on St. Patrick the more question marks are raised about how much of his success was from team factors.
If Brodeur keeps up his current level of play for several more seasons (the previous three seasons account for over one-third of Brodeur's career goals over average total), then he will be at least moving into the conversation with Roy. If we are able to more precisely drill down to the relative team effects and adjust for them maybe we'll find that the team factors between the two aren't that different after all, and in that case the debate could get very interesting by the time Brodeur's career is done.
Friday, March 13, 2009
Special Teams and GAA
What would you rather do, play 4 minutes at even-strength, or get a 2 minute power play followed by a 2 minute penalty kill? If you had a normal team that didn't have an unusually dominant power play or penalty kill, then it likely wouldn't matter much in terms of winning or losing the game since you would not gain an advantage over the opposition in either scenario. If you are the goalie, though, you'd much rather play 5 on 5, because it means a lower chance of getting scored on.
Based on the numbers this season at Behind the Net, an average team that played all 60 minutes at even-strength would score about 2.4 goals per game. An average team that played 30 minutes on the power play and 30 minutes on the penalty kill would score about 3.7 goals per game, or about 50% more often. The scoring rate on the power play is roughly triple the even strength rate. The overall shots are likely to be fairly similar in both scenarios, the difference is that power play shots are much more likely to go in. Save percentages on the penalty kill usually hover around .870 or so, while the average save percentage at even-strength is up around .920.
Some teams appear to be more low-scoring than they actually are because they don't spend much time on the power play. The New Jersey Devils have been an example of this type of team, and another is the Minnesota Wild. The opposite is true for often-penalized teams like the Anaheim Ducks and the Philadelphia Flyers. A team that wants to improve its goals against without bringing in extra defensive talent can simply coach its players to take fewer penalties.
Teams that take few penalties tend to also draw fewer penalties, but a team that was terrific at drawing penalties while being also disciplined enough to avoid taking penalties would have a big advantage. That description fits this year's Carolina Hurricanes, who lead the league in power play opportunities for while also having faced the 5th fewest power play opportunities against. The Hurricanes special team units are pretty mediocre (Carolina has the 21st ranked power play and the 18th ranked penalty kill), so they haven't really taken advantage of this as much as they could have, but the team is still +4 in goal differential on special teams compared to -5 playing 5 on 5. If the Hurricanes had been exactly average in taking and drawing penalties, they would be expected to score 48 power play goals and allow 55, so the net result of their team discipline is +11 goals.
Combine team discipline with a weak division and some good play/luck in close games (the 'Canes are 19-7-6 this year in games decided by one goal), and you have a team that is in the playoff hunt. This just illustrates the potential impact and importance of penalties in today's NHL.
Subscribe to:
Posts (Atom)
