Pat Burns: A Goalie's Best Friend

What is the impact of a coach in hockey? That is one of the most difficult analytical questions in hockey, as indeed it is in all sports. Certainly there are many coaches who have reputations either built or ruined because of the talent of their teams. For the most part, I think most observers would agree that talent is the most important factor in determining which teams are successful. However, when it comes to evaluating goalie performance we are not only concerned with overall results (i.e. wins and losses), but also how well the goalie stops the puck and how many goals he allows. This a coach can certainly impact through establishing a team's style of play.

Jacques Lemaire is well-known as a defensive guru, the statistics of his goalies have been consistently excellent in both New Jersey and Minnesota (see this post for some relevant numbers). However, it is difficult to evaluate Lemaire because except for a short stint in Montreal he has worked for only two teams, New Jersey and Minnesota, and as the only coach the Wild has ever employed it is very difficult to estimate how well anyone else would have done in that position. We can guess based on how goalies did on other teams compared to how they did on a Lemaire-coached team, but we don't really know how good the other teams are and the sample sizes are mostly small. Ideally, we would prefer to look at a coach who coached on multiple teams, and evaluate how the team did before he arrived, during his tenure, and after he left. This would allow us to evaluate the overall shot prevention effect and shot quality/goaltending effect of his defensive system.

Pat Burns fits the criteria of a defensive coach with multiple stops, so I decided to focus on him. My method was to look at the two seasons before he was hired, and the two seasons after he was fired, and compare those results to how he did as coach. Since Burns was fired very early in the 2000-01 season, I decided to make the judgment call of counting that campaign as the first season after Burns, rather than his final season.

I looked at how Burns did in Montreal, Toronto, Boston and New Jersey. In the years prior to Burns showing up, his teams allowed 3.05 goals per game, 27.6 shots per game, and posted an .889 save percentage. In the years after Burns left, his teams allowed 2.78 goals per game, 28.7 shots per game, and his goalies stopped shots at a .903 rate. With Burns as coach, his teams alowed 2.57 goals per game, faced 27.5 shots per game and had their goalies post a .907 save percentage. If we look at shutouts, something that often influences Vezina voting, the goalies had .047 shutouts per 60 minutes before Burns, .074 shutouts per 60 with Burns, and .060 shutouts per 60 after Burns, so they were about 25-30% more likely to record a shutout with Burns behind the bench.

Just from the raw numbers, it looks pretty clear that Pat Burns has a positive effect on team defence. The problem here is obviously that most of the years prior to Burns' arrival came in the late 1980s or early 1990s, when scoring levels were higher, whereas his "after" years came when you would expect reduced scoring and improved goalie performance. We therefore need to normalize the numbers to league average. I took the average goals per game, shots against per game, and save percentages for Burns' career (3.08, 29.0, .897) and used them as adjustment factors to remove league effects for each individual season.

Prior: .517, 2.95, .893, 27.5 SA/60
Burns: .567, 2.57, .907, 27.6 SA/60
After: .542, 2.96, .899, 28.9 SA/60

With adjustments, the numbers are clear: Pat Burns helps his goaltenders out to a substantial degree, something in the neighbourhood of a .010 boost in save percentage and a .40 reduction in GAA.

The argument could be made that Burns had better goalies in net. Felix Potvin and Byron Dafoe both became starters in Burns' first year as coach for their respective teams, and they were better goalies than their predecessors. However, both were still around after Burns had left, and we see how the numbers fell back down to earth. I think the small discrepancy between the before and after results can be partly explained by goalie quality, but for the most part the same goalies were employed both during Burns' tenure and in the seasons after he left, so talent does not account for the performance improvement under Burns.

To try to avoid the goalie talent issue altogether, we can look at just the Montreal and New Jersey results, since in both cases Patrick Roy and Martin Brodeur were in net before, during, and after Pat Burns was coach (again the numbers are adjusted to league averages):

Prior: .608, 2.51, 26.0, .905
Burns: .610, 2.37, 26.2, .911
After: .611, 2.75, 28.2, .906

The results are similar, although less striking (which is partly because Montreal was already a strong team and had less room to improve than some of the other teams Burns worked for). Of course player talent is another consideration - for example, the New Jersey Devils would have been expected to suffer some dropoff in defensive play with the loss of Stevens and Niedermayer. However the persistence of the defensive improvement across four different teams makes it pretty likely that Burns' defensive philosophy was responsible for improved play.

Team defence is made up of shot prevention, shot quality against, and goaltending. Pat Burns' defensive effect seems to be primarily based on reducing opposing shot quality. His teams gave up almost exactly the same relative number of shots per game as they did previously, yet the goalies saw their numbers skyrocket under Burns' watch. Pat Burns is known for wearing out his welcome in a city after a few seasons, but I doubt he ever was that unpopular with the goalies. This is more evidence that goalies can receive a significant advantage by playing under the right coach in the right defensive system.