Close Games, Part 2

After getting some feedback on my post looking into New Jersey's record in close games, I decided to do a bit more research. It turns out that the reason for their success was mainly because they did well in overtime and in the shootout. Here are the top 5 teams in regulation one-goal victories and regulation one-goal winning percentages since the lockout:

Regulation one-goal wins:
1. Calgary, 61
2. San Jose, 54
3. New Jersey, 53
4. Detroit, 52
5. Carolina, 51

Regulation one-goal winning percentage:
1. Carolina, .585
2. Detroit, .578
3. Calgary, .573
4. New Jersey, .572
5. San Jose, .567

By these numbers New Jersey is still good at winning close games, but they are not head and shoulders above the rest of the league. Here are the numbers for overtime and shootout wins:

Wins in overtime and shootout combined:
1. New Jersey, 56
2. Atlanta, 50
3. Dallas, 49
4. N.Y. Rangers, 48
5. Buffalo, 45

Winning percentage in games that go into OT:
1. New Jersey, .659
2. Dallas, .613
3. Atlanta, .602
4. Buffalo, .570
5. Colorado, .569

New Jersey was by far the best team in the league in games tied after 60 minutes. In this light, the team's recent playoff performances perhaps don't seem as disappointing. Their regular season records were largely influenced by their ability to perform well in 4 on 4 overtime and in shootouts. Unfortunately for them, the Devils weren't able to take advantage of those situations in the playoffs.

Another variable brought up by someone in the comments was empty net goals. This was indeed a factor that helped boost the Devils' number of one goal wins, since New Jersey has been one of the worst teams in the league at scoring empty net goals since the lockout. New Jersey scored 19 times with the other goalie pulled, which was tied for the second-lowest total in the league behind only the weak Toronto Maple Leafs, a team that faced many fewer empty net chances than the Devils. Assuming they never scored two empty netters in any one game, New Jersey scored an empty net goal in 14% of their regulation wins, the second worst percentage in the league behind only San Jose's 13%.

The Devils allowed 25 empty netters against, or an ENG against in 23% of their regulation losses, which ranked them slightly worse than the league average of 22%.

I am not sure how many of the empty netters came when leading/trailing by one goal and how many came when there was a two goal margin on the scoreboard. I decided to assume that empty net goals scored came with a one and two goal lead came in the same proportion as the team's number of one and two goal wins, e.g. a team with the same number of one goal and two goal wins would score half of their empty netters in each situation. It is likely a few teams would by chance have a very different ratio, but that probably puts most teams in the ballpark. Combined with the OT/shootout numbers, that allows us to estimate a team's regulation-only one-goal game record with empty-netters removed.

I'll refer to any game that goes to overtime or is decided in regulation by a one goal margin (empty netters excluded) as a close game. Here are the close game records for all teams since the lockout (not including 2009-10), along with their close game points percentage (games tied after regulation count as 1 point), the team's winning percentage in games decided by 2 goals or more, the total points earned in overtime and shootouts, and the percentage of close games that went to OT.

Rank	Team	Close W-L-OT	Close W%	2G+ W%	OT/SO	% OT
1.	Detroit	71-37-66	.598	.740	94	38%
2.	Calgary	82-50-58	.584	.536	79	31%
3.	Carolina	70-43-64	.576	.497	99	36%
4.	San Jose	65-42-69	.565	.658	101	39%
5.	New Jersey	65-42-85	.560	.522	141	44%
6.	Nashville	64-46-71	.550	.537	109	39%
7.	Philadelphia	59-50-76	.524	.441	107	41%
8.	Montreal	51-43-74	.524	.519	112	44%
9.	Anaheim	61-53-80	.521	.597	119	41%
10.	Vancouver	58-54-78	.511	.540	121	41%
11.	Buffalo	46-44-79	.506	.591	124	47%
12.	Florida	53-54-80	.497	.461	113	43%
13.	Toronto	46-47-79	.497	.442	115	46%
14.	Minnesota	57-59-75	.495	.533	115	39%
15.	Dallas	50-52-80	.495	.582	129	44%
16.	Phoenix	56-58-54	.494	.358	84	32%
17.	Boston	49-54-79	.486	.507	114	43%
18.	Columbus	49-54-72	.486	.359	111	41%
19.	Ottawa	44-49-58	.483	.644	79	38%
20.	Edmonton	50-57-78	.481	.413	121	42%
21.	Colorado	56-66-65	.473	.496	102	35%
22.	Pittsburgh	40-50-83	.471	.516	124	48%
23.	Washington	44-56-77	.466	.470	112	44%
24.	Rangers	37-51-92	.461	.581	140	51%
25.	St. Louis	44-59-82	.459	.378	113	44%
26.	Tampa Bay	46-62-74	.456	.414	110	41%
27.	Atlanta	35-52-83	.450	.430	133	49%
28.	Islanders	42-60-74	.449	.375	112	42%
29.	Los Angeles	46-65-68	.447	.389	99	38%
30.	Chicago	39-60-79	.441	.447	116	44%

A few teams have interesting profiles here. Carolina and Calgary are teams that do much better in close games, and have tended to win the close ones in regulation. Over the last two seasons, both teams have seen both their shot ratio and percentages improve in the third period, so perhaps it could be argued that these teams have shown some clutch ability. In contrast, Phoenix and Ottawa also don't make it overtime that often, but they tend to lose the close ones and would be better off in the standings if they could hold on a bit longer to earn a few more loser points. The Coyotes and Senators both saw their third period percentages tumble over the last two years. I'm not sure whether that is a sign of poor performance late in games or simply bad luck that led to losses.

In this table New Jersey doesn't look much different from other teams, other than their league-leading total of 141 overtime and shootout points. Their rivals the New York Rangers were only one point behind. The Rangers were not as good at picking up the extra point, but they took a lot of loser points since they played more overtime games than any other team. That suggests the Rangers have made aiming for shootouts part of their team strategy. However, it looks to me like that strategy might have been suboptimal for them. Either that or the Rangers did a poor job of carrying it out, because a team that went to OT less often but won more games in regulation would have ended up with more points at the end of the day.

The Rangers took 74 points from one goal wins, 92 points from making it to overtime, and 48 points for winning by OT or shootout for a total of 214 points from close games. Given the same number of close games a typical team would have gone to overtime only 76 times, but would have won half the remaining games for a total of 52 regulation wins. That means they would only need to win 35 out of their 76 overtime games (46%) to earn more points than the Rangers did.

The Rangers had a pretty good record in games decided by 2 goals or more. They also had the third best winning percentage in the league when trailing after 2 periods, yet had a slightly below average winning percentage when leading after 2 periods. All that tends to reinforce the theory that the team would have been better off going for more wins in regulation rather than sitting back on a lead or trying to take a tie game into overtime, because it looks like too often they saw that strategy backfire.

Close Games

It is rare that professional teams can outperform their peers in a particular area for a long time. If a team has a strategic advantage, the rest of the league will study them and adjust their coaching strategy to compensate. If a specific talent or skillset is undervalued, a clever GM might be able to gain a short-term advantage, but again if the other teams are following along the market should correct the valuation.

As a result, when I run across a team that is a substantial outlier in any area then I take notice, and when that outlying result comes in an area that I mostly attribute to luck then it is especially interesting.

Here are the top 5 teams in games won by a one-goal margin since the lockout (not including this season so far):

1. New Jersey, 109
2. Dallas, 89
3. Vancouver, 87
3. Nashville, 87
5. San Jose, 86
5. Carolina, 86
5. Anaheim, 86

And here are the top 5 teams in winning percentage in games decided by one goal:

1. New Jersey, .740
2. Carolina, .709
3. Nashville, .690
4. Detroit, .673
5. San Jose. 670

The Devils are at the top of both of those lists, and it's not even close. That begs the question, just what are they doing differently than everyone else in the league?

The simple answer, and the one that 90% of journalists would probably respond with, is that the Devils have Martin Brodeur in net. There is, however, one fairly significant piece of evidence that suggests there is more to the story than that, namely New Jersey's record in one-goal games in 2008-09. Despite losing Brodeur to injury for almost 4 months, the Devils posted the best close game record they have ever had, going a remarkable 25-5-4 in one-goal games. Compared to an average team, New Jersey picked up an extra 12 points by winning the nailbiters, which made the difference between them winning their division and finishing as the #7 seed.

I looked at New Jersey's record since the lockout when leading, trailing, and tied after 2 periods. If they were a particularly clutch team, we would expect them to have a lot of wins in games that were tied heading into the third. The Devils did do well in that situation with a .644 winning percentage, good for 6th best in the league since the lockout. However, the Devils actually played a relatively low number of games that were tied after two, which was somewhat surprising for a low-scoring team. The team won a total of 36 games where they were tied after 2 periods, which was right about the average number (the Devils ranked 16th in the league).

New Jersey was also pretty good at coming from behind. Their winning percentage of .250 ranked 6th in the league, well above the average of .200. The Devils won 26 games that they trailed after 2 periods, which was tied for 4th in the league. It is likely that many of those wins would have been one-goal wins, although the average team won 20 so this would only account for part of their close game success.

By far most of New Jersey's wins came in games they were already leading after two periods. The Devils went 130-6-7 when ahead after two, for a .934 winning percentage that was the league's best. Only Detroit, San Jose and Ottawa converted a higher number of second intermission advantages into victories. However, none of those teams had anywhere close to as many one goal wins as New Jersey.

This indicates that New Jersey's terrific one-goal game record is mostly from their ability to get ahead and hold onto the lead. The Devils outplay the other team early, get a lead, and then try to close out the game by protecting that margin rather than trying to extend it, a strategy that if successful leads to a lot of one-goal victories.

I have scoring data broken down by period for the last two seasons from the Hockey Summary Project to support this thesis. New Jersey's offence dropped in the third, with the Devils ranking 25th in the NHL in third period goals (empty-netters removed). Their shots taken also dropped, from an average of 10.3 shots per period in the first two (6th in the league) to an average of 9.4 shots per period in the third (15th). This was from a greater focus on defensive play, as the team also was able to cut down on the number of shots against (from 9.5 per period to 8.8 per period in the third).

Despite fewer shots against, the team's GAA actually went up in the third period, from 2.27 in the first two to 2.34 in the third. This was because the team save percentage dropped from .920 to .911. This suggests that shot prevention, rather than clutch goaltending, was the main reason the team was so effective at preserving leads. Keep in mind that New Jersey had a relatively high success rate in mounting comebacks, which shows that they had the ability to score if they wanted to. Instead, the Devils traded offence for defence, and the reason they won so many games by a single goal was because they scored fewer late insurance goals than other strong teams.

New Jersey was actually outscored 128-124 in the third period over the last two seasons. The team's positive goal differential came entirely from its success in the first two periods. I looked at a few of the other top teams in winning close games as well as some of the worst teams, and their records usually could not be explained by their third period save percentages or shot ratios. In fact, few teams were particularly clutch in terms of their percentages. In general the teams that outshot and/or out-"percentaged" their opponents in the first two periods had good records in close games, while teams that got outshot usually did not.

Announcers and writers often focus on the late "big save" that supposedly "won the game". However, most of the time that is giving too much credit to the goalie. The reason that save looks important is that the team had already built a lead in the game. The goalie does have to make the saves to keep the team ahead, of course, and if they are facing sustained pressure sometimes they need to be excellent to keep the other team off the scoreboard, but since the average shot has a 91% chance of being stopped the odds are very much in favour of the leading team. Because of this goalies have a very high success rate in holding off the other team late in the game, in the same way that baseball closers usually manage to "save" the game when they enter in the ninth inning with their team already in front.

New Jersey has won a lot of close games by outshooting and outscoring the opposition early, and then locking down the game to reduce scoring chances in the third period. Their goaltending should get credit for its strong overall performance, but it does not appear to warrant any additional recognition for "making the big saves".

It looks like the Devils have kept their "competitive advantage" going this year, with all three wins this season coming by a shootout or by a one goal margin. They will likely need to continue to excel at winning the close ones to stay competitive in a tough Atlantic Division.

Situation-Adjusted Save Percentage

My conclusion from a recent look at special teams performance was that a goalie's play while his team is killing a penalty is important for evaluation, but that his performance when his team was on the power play was not. The latter conclusion was based on the lack of correlation between EV SV% and PP SV%, which suggests that the results are either highly influenced by luck or the rest of the team.

Gabe Desjardins over at Puck Prospectus agrees with me, and has a nice article up with a more advanced look at evaluating goalies based on only their EV and PK play. His method looks like a solid one for comprehensive goalie evaluation, but I thought of developing a much simpler formula that can be used as a quick-and-dirty way to take special teams into account.

Over the last decade, 76.3% of the shots have come at EV, 3.7% on the PP, and 20.0% on the PK. That number is skewed up slightly by the 2005-06 season, as in both of the last two seasons 19.7% of the shots taken were by a team on the power play. If we ignore the PP shots, a good approximation of the average EV/PK split is 80/20. By assigning an 80% weighting to the goalie's EV SV% and a 20% weighting to his PK SV% we can quickly adjust for special teams factors.

This adjustment doesn't actually make much of a difference for most goalies, but it does impact goalies who either faced a disproportionate number of shots on the penalty kill or for whatever reason allowed a unusual number of shorthanded goals. Here are the top 20 in situation-adjusted save percentage since the lockout (min. 100 GP):

1. Niklas Backstrom, .923
2. Tomas Vokoun, .922
3. Roberto Luongo, .920
4. Tim Thomas, .919
5. Cristobal Huet, .918
5. Henrik Lundqvist, .918
5. Dominik Hasek, .918
8. Martin Brodeur, .917
8. J.S. Giguere, .917
10. Miikka Kiprusoff, .914
10. Chris Mason, .914
12. Manny Fernandez, .913
13. Martin Biron, .912
13. Ryan Miller, .912
13. Kari Lehtonen, .912
16. Ilja Bryzgalov, .910
16. Marc-Andre Fleury, .910
16. Ray Emery, .910
19. Rick Dipietro, 909
19. Dwayne Roloson, .909
19. Manny Legace, .909

I think the best use of this adjustment would be as a quick check when comparing goalies. Let's say you were voting on the 2009 Calder Trophy and you wanted to compare the performances of Steve Mason and Pekka Rinne. If you look at the raw save percentages it was pretty close, with Mason at .916 and Rinne at .917. What that doesn't show, however, is that Rinne faced an unusually low number of shots against on the penalty kill. If we multiply their EV SV% by 80% and their PK SV% by 20%, Mason edges ahead .917 to .914.

The formula also adjusts for goalies who were lucky or unlucky with shorthanded scoring chances against. Henrik Lundqvist and Cam Ward both had .916 save percentages last season. However, Lundqvist allowed 11 shorthanded goals against compared to Ward's 5, as the Rangers allowed a lot of shots and presumably a lot of scoring chances against on the power play. If we look at EV and PK play only, Lundqvist jumps to .919 while Ward falls to .915.

To simply adjust for special teams factors remember the "80/20 rule", and you'll be able to pick out the goalies who have the burden or good fortune of facing heavy or light work on the penalty kill.

The Win Threshold

I am firmly against comparing goalies based on wins. This is not because wins aren't important or desirable. Wins are what every player and team wants more than anything else. What makes wins a poor stat is that every team situation is different. If every goalie played the same schedule with identical teammates in front of them, then we could just give the Vezina to the guy with the most wins at the end of the season. In real life, goalies do not compete on a level playing field.

The two most important team factors that affect a goalie's ability to win are his goal support and the number of shots he has to face. More goal support means that more of his mistakes are covered up, and facing fewer shots against means fewer opportunities to allow goals.

Last year the Detroit Red Wings had the league's best offence, scoring 3.52 goals per game. They also allowed the second fewest shots against per game with 27.7. On the other end of the scale, the New York Islanders finished second last in both goals for (2.42) and shots against (33.5) per game. Quite obviously an Islander goalie would need to be much better than a Red Wing goalie for their teams to have the same chance at winning, because he would have to make up for his team scoring one less goal per game and he would have to do it while facing an extra half-dozen shots against.

We can calculate what I'll call the "win threshold" for the goalies on each team by taking (shots against - goals for) / shots against. This gives us the save percentage that would result in the team ending up with an equal number of goals for and goals against over the course of the season. If the goalie's save percentage is above that number, the team is likely to win more than the lose, while anything below the threshold means that the team should end up sub-.500 (or sub-.550 in the shootout era).

In 2008-09, Detroit's win threshold was .873, which was the lowest in the league. The Islanders' win threshold was .928, which was not only the highest mark in the league but also the highest of any team since the lockout.

Expressed a different way, Detroit is likely to win if their opponents have a shooting percentage of 12.6% or worse. The New York Islanders are likely to win only if their opponents have a shooting percentage of 7.1% or worse. The shooting percentage against Detroit needs to be almost 80% higher than the percentage against the Islanders for the teams to have the same likelihood of winning the game.

Naturally, comparing win totals on goalies playing on the Islanders to goalies playing on the Red Wings is completely senseless. It would be like comparing two students in terms of how many course credits they attained, where the first student passes their courses if they achieve a mark of 50% or better while the second student only passes if they score 90% or higher. With that advantage, the first student is much more likely to pass his courses and achieve a higher overall number of passes. Even if the second student is exceptional and the first student is mediocre, it is likely that the first student will have a similar or better score because of their inherent advantage.

I ran the formula for every team since the 1997-98 season, including an adjustment for average league goals and shots per game.

Top 10 since 1997-98:
1. 2001 Devils, .860
2. 2004 Senators, .866
3. 2000 Blues, .867
4. 2003 Senators, .868
5. 2008 Red Wings, .869
6. 1998 Blues, .872
6. 1999 Blues, .872
6. 1998 Stars, .872
6. 2006 Red Wings, .872
10. 2001 Avalanche, .874
10. 2003 Blues, .874
10. 2009 Red Wings, .874

Bottom 10 since 1997-98:
1. 1998 Lightning, .937
2. 2002 Thrashers, .935
2. 2000 Thrashers, .935
4. 2003 Panthers, .934
5. 2002 Blue Jackets, .933
6. 2002 Panthers, .932
7. 1999 Lightning, .931
7. 2000 Islanders,. 931
7. 2004 Panthers, .931
10. 2001 Wild, .930
10. 2004 Blue Jackets, .930

If you ever wondered how Roman Turek managed to get 42 wins in a season, or how Patrick Lalime won 39, here's your answer. On the other hand, note that four of Roberto Luongo's teams show up in the bottom 10. Why didn't the Florida Panthers make the playoffs? Because the team was terrible. It had nothing to do with the goaltending.

Note that these are team totals that need to be achieved, which make it even more difficult for goalies on the worst teams than it appears at first glance. If they have a backup who plays around 20-25 games at .900, then the starting goalie would need to be at .940 or better for the team to have a goal differential of zero. Even then, the team is unlikely to make the playoffs without scoring more goals than they allow.

The average win/loss record of the teams in the top 10 list was 48-22-12. The average win/loss record of the teams in the bottom 10 was 22-46-14. What is interesting, however, is that the goaltending performance was quite similar:

Save % of top 10 teams: .905
Save % of bottom 10 teams: .904

The teams on the top list didn't win because of great goaltending or because their goalie gave them clutch saves. They won because their teams scored a lot of goals and didn't allow many shots against. Similarly, the teams on the bottom list lost because they struggled to score and allowed too many shots against, not because their goaltenders were poor. This is further proof that win totals are a team stat, and should not be used to evaluate individual goalies.

The Underrated Allan Bester

I discussed recently how goalies are often evaluated based on a few memories. For some goalies it means they are forever remembered as winners. For others, it means that they are defined by one bad goal against. The latter group is often very underrated. For example, all our memories of ten years of league average save percentages and a previously stellar international career vanished in a flash as soon as that puck bounced in off of Tommy Salo's head.

Another example of this phenomenon is Allan Bester. Bester is remembered as a joke among many Toronto Maple Leafs fans. Take this quote off a Leaf blog, for example:

"Raycroft is the worst starting goalie I’ve seen play for the Leafs since Allan Bester. Remember him? He attempted suicide one night. No, really. He jumped in front of a bus! But it went between his legs." (Leaf Club)

This is of course a reference to the the overtime goal scored by Sergio Momesso in the 1990 playoffs at Maple Leaf Gardens. Momesso's shot beat Bester five-hole, a memory that Leaf fans will forever associate with the tiny (5'7, 155) netminder.

However, the numbers paint a very different picture of Allan Bester. According to the statistics, Bester outperformed his teammates in every league that he ever played in, and was easily the best goalie the Leafs had in the 1980s. Despite this, the Momesso goal was essentially the end of Bester's NHL career.

Bester broke into the league early as a 19 year old right out of junior, and had all of his NHL success in his early twenties. With fellow youngster Ken Wregget, Bester made up a promising goalie tandem for the Leafs in the mid-1980s. The 1988-89 season was probably the high point of Bester's career, as he finished in the top 10 in save percentage on a team that missed the playoffs. Bester was even named to Team Canada's world championship squad, although he was unable to participate because of injury.

In 1989-90 Bester had an off-year, and then he struggled in the playoffs. Everything started to unravel after that. He was displaced in the Leaf net in 1990-91 by rookie Peter Ing and then got traded to the Detroit Red Wings for a draft pick.

The Wings buried Bester in the AHL, but he played well in the minors. In 1992, the Adirondack Red Wings won the Calder Cup, and Bester was named playoff MVP. The next year he outplayed Chris Osgood, who was seen as Detroit's goalie of the future. Without an opportunity to move up in the Red Wings organization, Bester looked to catch on with one of the expansion teams. He was passed over in the expansion draft, but managed to catch on with the Anaheim Mighty Ducks as a free agent.

Unfortunately for Allan Bester, Anaheim had the best goaltending of any of the expansion teams, with the solid tandem of Guy Hebert and Ron Tugnutt. Bester was competing with Mikhail Shtalenkov for the #3 role in the organization, and despite similar IHL numbers the team decided to give the NHL opportunities to Shtalenkov. Bester continued to play in the IHL until he got one last cup of coffee at the NHL level as an injury replacement for the Dallas Stars in 1996. He acquitted himself well in a 10 game stint, but returned to the IHL where he remained until he retired in 1997-98 at the age of 33.

Having established the biography, let's look at the numbers. I compared Bester's numbers to his teammates for every season of his professional career, except when he only played a few games which would not be a representative sample.

Allan Bester solidly outplayed his teammates at the NHL level:

Bester: 4.00 GAA, .883 save %, .432 win %, 2.5 SO/70 GP
Others: 4.30 GAA, .869 save %, .370 win %, 0.5 SO/70 GP

Bester has a clear edge in every stat, including a large edge in shutouts. For goalies playing on bad teams, shutouts are often a fairly good indicator of dominance, since they aren't able to post the easy shutouts that goalies on winning teams often get. On the mid-'80s Leafs, the only way anybody was going to get a shutout was through an outstanding performance, and Bester had a very respectable 2.5 shutouts per 70 games played (I prefer to express shutout rates per 70 games because the per-game rates get pretty small). This was five times the rate of his teammates. The raw total was 7 shutouts in the equivalent of 178 full games, compared to his teammates' total of just 2 in 309.

Question to Maple Leaf fans of that era: Does one bad playoff goal make up for a winning percentage difference of .062 compared to Toronto's other goalies, the equivalent of a 5 extra wins over a full season?

In the AHL, Bester again outplayed his teammates:

Bester: .487 win %, 3.74 GAA, 1.4 SO/70 GP
Others: .456 win %, 4.05 GAA, 1.5 SO/70 GP

Bester's playing partners included Wregget, Tim Bernhardt, Rick St. Croix, Damian Rhodes, Scott King and Chris Osgood. Bester spent his prime age 27 and 28 years in the minors as a Red Wing, outplaying his teammates and being named the MVP of a Calder Cup winning team, yet only getting to play 31 minutes in the NHL.

One contributing factor to this was likely the overall improvement in league goaltending during the early- to mid-1990s. A new wave of goalies was entering the league, bringing with them the modern butterfly style and displacing many veterans who weren't able to keep up. Bester certainly wasn't the only goalie of his age that got caught up in a numbers game, but having said that he was still putting up numbers that were as good as or better than several young goalies who were headed for the NHL. It is possible that his declining performance in 1989-90 and 1990-91 showed that he no longer had the ability to compete at the NHL level, but his minor league performance suggests that it is also possible that he had an off-year and didn't get a chance to play his way back into form.

In the IHL, Bester was in his thirties but was still usually the best goalie on the team. Some of his partners were scrubs, but others were legit goalies (e.g. Shtalenkov, Essensa).

Bester: .630 win %, 3.22 GAA, 1.9 SO/70 GP
Others: .563 win %, 3.43 GAA, 1.6 SO/70 GP

Just as winning goalies shouldn't be judged based on one shining playoff run, losing goalies should not be judged based on one bad goal. Allan Bester was likely at least a league average goalie, and for a short time probably even better than that.

I'll leave you with one final stat: From 1983-84 to 1989-90, Allan Bester finished 13th in the NHL in save percentage among goalies with at least 150 GP, playing on one of the worst teams in the league. I'll post a section of the standings so we can see the goalies just above and below him:

11. Tom Barrasso, .883
12. Don Beaupre, .883
13. Allan Bester, .883
14. Mike Liut, .883
15. Billy Smith, .882
16. Grant Fuhr, .881
17. Kirk McLean, .881
18. Mike Vernon, .881
19. Pete Peeters, .881

In the 1980s those 9 goalies combined for 5 First Team All-Stars, 4 Second Team All-Stars, and 9 Stanley Cups. Every single one of them had long NHL careers. And yet nobody gave Bester another NHL shot.

I think the scouts got it wrong on this one. Bester may have been small and he may have let a few through the wickets, but the evidence suggests that he was a legitimate pro goalie and he deserved another shot at an NHL job after the age of 25. He should be remembered for what he was, the best Toronto Maple Leafs goalie of the 1980s.

Shot Quality Refined

I have to throw up a link to the latest work by Ken Krzywicki on shot quality. He tackles the issue of arena reporting bias, and has updated his model to take that into account. I think there are some very good reasons to question the importance and/or the accuracy of shot quality measures, but I still think it is possible that some of the problems are a result of the data or the models rather than the theory that different teams allow different types of scoring chances. That shot quality differences are observed in the game of hockey is beyond doubt, the debate is about whether those differences are significant at the NHL level.

To me, the revised model makes more sense, because we see a smaller spread of values at the team level. The best team has an expected save percentage of .915, while the worst is at .904. Compare that to the .900 - .918 which would be predicted based on the raw numbers. Half of the teams are within .002 of league average, which matches our expectation that most teams do not have significant shot quality effects.

I would like to see the shots separated out by situation into EV and PK. I think that would allow us to better be able to test the results. However, the model must be projecting some even strength variations, since the correlation between expected save percentage and power plays against in 2008-09 was -0.03.

Some of the team results are a bit surprising. In particular, Boston is rated as having allowing the 4th most difficult shots against, which seems to strongly contradict the numbers of Thomas and Fernandez, the Bruins' low number of penalties taken, and the coach and team style of play.

Alan Ryder also has a method for evaluating shot quality in his annual NHL review. He says he has tried to make some adjustments for rink bias as well, yet some his numbers differ from Krzywicki's. One of them is in his measurement for Boston whom Ryder grades as about average, which I suspect might be a more accurate assessment.

By comparing Krzywicki's numbers with Ryder's, and also keeping in mind each team's actual save percentage as an additional check, I think there are about ten teams that we can target as possible shot quality outliers in 2008-09, with New Jersey, Phoenix, Buffalo, Minnesota and Columbus the candidates for easier than average shot quality and Toronto, Dallas, Carolina, Montreal and Calgary possibly allowing harder than average shots against.

Of those, a few could be the rest of measurement error, score effects, randomness, etc., but I think is likely that there are a few teams that must be ahead or behind the rest of the league. To me, Minnesota, New Jersey, Toronto and Dallas are the most likely candidates for 2008-09. For those four teams, the shot quality results match both the subjective perception and the actual results in the crease.

If this is correct, that would have implications for our evaluation of some goalies. That doesn't mean that goalies like Turco or Toskala can be let off the hook, however. The expected save percentage of an outlier team is still likely somewhere in the range of .003-.005 above or below average, which means that outstanding or terrible save percentages cannot be explained by the rest of the team and must fall mostly on the shoulders of the goalie that posted them.

Winning Means the Rest Doesn't Count

Goalies are rated far too much based on their team success. One reason for this is that we all have limited memories, which causes certain moments to be more easily recalled while the rest is forgotten. For some goalies that's a good thing, because we can clearly recall them doing things like stopping Pavel Bure on a penalty shot in the Finals or making a save with 2 seconds left in game 7 to help their team win the Cup. For others we remember only the weak moments that they would rather forget. As a result, goalies who have not experienced team success often see a few of their worst performances get scrutinized to the near-exclusion of all else, while similar performances by other goalies get ignored because they have a ring.

Patrick Roy was a terrific playoff goalie, but he had his fair share of weak performances. If you play 17 playoff seasons you are naturally going to have a few that are forgettable, of course, but some of Roy's flops were actually quite spectacular (losing his job in 1987 and 1988, his problems against the Bourque/Neely Bruins, a few first-round upsets against much weaker opponents, and of course the famous game 7 of the 2002 Western Conference Finals). Yet his fans will often talk about 4 Cups and 3 Conn Smythes, as if that's all that happened. Again, Roy's playoff record is strong by any measure, but it obviously looks even stronger when you only focus on the best of the best.

For other goalies more of a focus is put on their negatives than their positives. Glenn Hall is generally considered a playoff underachiever and is blamed by many for the Hawks' relative lack of playoff success in the 1960s, despite a strong effort in helping his team win the 1961 Cup and even though Chicago was outshot in nearly every playoff series that decade. Another one would be Curtis Joseph, who often led his team past better teams with excellent performances in the early rounds, but usually gets blamed because of his teams' failure to perform well deeper in the playoffs.

If we adjust for era and opponents and compare the playoff numbers for Roy and Joseph, Roy is something like a .916 playoff goalie while Joseph is at .909. Adjust for power plays against and the gap might be even smaller. That's a significant difference over a decent sample size, certainly enough to conclude that Roy was better, but the actual gap is just one goal every 143 shots. You have to be pretty good at subjective evaluation to notice a difference of one extra goal saved every five games.

If we rely on our memories, however, then the differences are usually exaggerated. If I think of Roy and Joseph I might recall Roy's performances in the Stanley Cup Finals in 1993 and 2001, while what comes to mind for Joseph is how he played in the Conference Finals in 1999 and 2002. Based on that my memory is going to tell me that Roy was a lot better, because I'm effectively comparing a .934 goalie against an .890 goalie. That's the equivalent of Tim Thomas vs. Vesa Toskala in 2008-09. That is a gap that you can easily see with your eyes, and the result is substantially overrating the difference between playoff Roy and playoff Joseph.

There are a few goalies in the league today who benefit from the "only memorable playoff successes count" method of goalie evaluation. Marc-Andre Fleury is an obvious one, with his big save on Lidstrom still fresh in everyone's memories. Yet perhaps the goalie who benefits the most is Cam Ward. Many fans overlook his mostly mediocre regular season play (essentially everything up until December of 2008) in favour of the 6 playoff series of above average play that have built Ward something of a reputation as a money goalie. Some are claiming that Ward is one of the top 5 goalies in the NHL and deserves a chance to become Canada's starter in 2010, neither of which is supported by his history. Almost nobody took Ward to task for his performance in the Pittsburgh series despite getting shelled in four straight games. That's the power of reputation: If you're considered "clutch", then you can get blown out and the fans blame the rest of the team. If you're considered unclutch, then the fans blame you.

Ward just signed a contract extension for $37.8 million over 6 years. That's more than he deserves based on his past record, but Ward has shown some real improvement over the last few seasons (his EV SV% numbers have gone .899-.917-.926). Ward probably got rushed to the NHL before he was ready, and when he caught lightning in a bottle in the 2006 playoffs his team unwisely decided to throw him even further over his head by handing him the starting job a couple years before he deserved it. His performance over the last two seasons is likely more indicative of what he can be expected to do until the end of his this contract. Still, we don't have a lot to go on to be able to assess Ward's true skill level. Is he still improving? Was last season a good year or a typical year? All that remains to be seen.

In the new NHL, teams are paying players for their projected performance instead of their past performance, which is what I'm sure Carolina was thinking when they inked Ward to his big deal. If his 2008-09 numbers represent his true skill, then Ward is a top-10 goalie in the league. I'm not a big fan of paying out $5-6M+ on goalies, unless it is to an elite goalie with an established track record, because there always seems to be Craig Anderson-types available who have the potential to give you 90% of the performance for 20% of the cost and you don't want to get stuck paying big money to a goalie giving you average performance. However, if Ward is legitimately a .925 puckstopper at EV than it shouldn't be too bad of a contract for Carolina. If it turns out he's more of a .917 guy then that's a big overpayment.

I think it is pretty likely that Ward has become and will continue to be at the very least a solid NHL goalie, but I'm not at all convinced that he is some incredible pressure performer. If you look at his playoff performances throughout his career (junior, AHL, NHL), the only time that his playoff results look way out of line compared to his regular season numbers is his surprise Cup run in 2006. He did do pretty well in the playoffs in junior, but in the 2005 AHL playoffs he had one good series and one not-so-good one (.895 save percentage in a second-round loss to a lower-seeded team). Ward's international record is also fairly spotty, albeit in a very small sample, with a .901 combined save percentage in world championship games against the other top hockey nations (USA, Russia, Sweden, Finland).

To me Ward's performance in 2006, given what he did directly before and after it, is less a sign of his clutch ability than evidence of what kind of a charmed run the Hurricanes were fortunate enough to go on for the entire 2005-06 season. Referring back to the recent topic of percentages, I'm not sure a team has ever seen their shooting percentage jump more from one season to the next then those Hurricanes did when they improved from 7.2% in 2003-04 to 11.2% in 2005-06. No wonder the team went from dead last in goals and out of the playoffs to the 3rd best offence and the 4th best record in the league.

Winning once or twice does not mean that the rest shouldn't count. If you want to evaluate goalies on their playoff or international play, then first of all be aware of the sample size issues. Secondly, evaluate them based on their individual success, not their team success. And finally, be sure to take their entire performance into account, not just the very best or the very worst.