Fun With Small Sample Sizes

I couldn't help but find this amusing, even though it covers just two handfuls of games:

Chris Osgood, career against the Detroit Red Wings:

1 win, 7 losses, 2 ties, 3.35 GAA, .879 save percentage

The Florida Outlier

In honour of Ed Belfour's Hall of Fame induction, I thought to take a closer look at one of the weirdest single-season statistical anomalies that I can think of since the lockout.

The Florida Panthers have routinely been one of the worst teams in allowing shots against. Over the past seven seasons the team has had consistently good goaltending and consistently bad shot prevention. There was, however, one very large exception. See if you can pick it out:

2003-04: .924 save %, 34.0 SA/60
2005-06: .913 save %, 34.3 SA/60
2006-07: .899 save %, 28.9 SA/60
2007-08: .922 save %, 33.2 SA/60
2008-09: .925 save %, 34.3 SA/60
2009-10: .920 save %, 33.6 SA/60

2010-11: .918 save %, 31.5 SA/60

That raises the intriguing question: Did the lower shot totals in 2006-07 cause the goalies to have a correspondingly lower save percentage? Or was it the lower save percentage in the first place that caused the other team to take fewer shots against? Or were there some other factors at play?

The Panthers' starting goalie that year was Belfour, in the final season of his illustration career. Throughout his career, Eddie the Eagle typically faced fewer shots against than the other goalies that he has played with. Therefore, it seems quite likely that at least some portion of the effect is likely because of him. However, given the numbers through the rest of his career that suggest he prevents about one shot against per game compared to an average goalie, I don't think there is any chance at all that a 41-year old Belfour was able to prevent 4-5 shots per game entirely through his own efforts.

Here are the home/road splits for each goalie for 2006-07 along with the two seasons before and after:

Luongo, '04: 2.55, .926, 34.6 at home, 2.28, .935, 35.3 on road

Luongo, '06: 2.67, .920, 33.3 at home, 3.28, .909, 36.1 on road

Belfour, '07: 2.75, .901, 27.8 at home, 2.80, .903, 28.8 on road

Vokoun, '08: 2.55, .921, 32.2 at home, 2.85, .916, 33.9 on road

Vokoun, '09: 2.21, .933, 32.9 at home, 2.92, .915, 34.3 on road

From those splits it doesn't look like Florida changed their official scorer, as shots went down significantly in all venues.

Looking at the numbers for the backup goalies shows that there was a lot more to it than any individual discrepancies between Belfour and Luongo or Vokoun:

Steve Shields, 2003-04:

Starts: 3-6-1, 3.67, .874, 29.1 SA/60

Non-starts: 0-0-0, 2.37, .904, 24.6 SA/60

Jamie McLennan, 2005-06:
Starts: 2-3-2, 3.71, .895, 35.3 SA/60

Non-starts: 0-1-0, 1.43, .941, 24.4 SA/60

Alex Auld, 2006-07:
Starts: 7-13-5, 3.28, .889, 29.6 SA/60

Non-starts: 0-0-0, 4.55, .857, 31.9 SA/60

Craig Anderson, 2006-07:

Starts: 1-1-1, 2.54, .924, 33.3 SA/60

Non-starts: 0-0-0, 0.00, 1.000, 23.6 SA/60

Craig Anderson, 2007-08:
Starts: 7-5-1, 2.23, .936, 35.0 SA/60

Non-starts: 1-1-0, 2.33, .924, 30.8 SA/60

Craig Anderson, 2008-09:
Starts: 15-7-3, 2.75, .924, 36.0 SA/60

Non-starts: 0-0-2, 2.27, .931, 32.8 SA/60

The special teams numbers don't give much of a hint either, given that Florida had a below-average rate of power play opportunities and a higher-than-average rate of power plays against. Those two factors would typically result in more shots against, rather than fewer. The only major difference in the special teams numbers for '06-07 was the number of shots against per PK:

2003-04: 374 PPOA, 571 SA, 1.53 SA/PP

2005-06: 514 PPOA, 744 SA, 1.45 SA/PP

2006-07: 443 PPOA, 530 SA, 1.20 SA/PP

2007-08: 374 PPOA, 532 SA, 1.42 SA/PP

2008-09: 311 PPOA, 513 SA, 1.65 SA/PP

(PPOA=Power play opportunities against)

Jacques Martin was the coach from 2005-06 to 2007-08, which makes it even more surprising that the numbers would change so suddenly.

Team shots for:

2003-04: 2273

2005-06: 2724

2006-07: 2730

2007-08: 2549

2008-09: 2412

Given that '05-06 was skewed by a high rate of power plays, this suggests that the Panthers were a much better territorial team in '06-07.

Florida had similar personnel in '06-07 compared to '05-06. Jay Bouwmeester was the #1 defenceman and Olli Jokinen was the #1 forward. Nathan Horton (21) and Stephen Weiss (23) both probably took large steps forward in 2006-07, and may have had some impact on turning results around, although the two of them remained in Panther uniforms until 2009-10 while Florida's results dropped back to their usual levels. The defence replaced Sean Hill, Lukas Krajicek and Joel Kwiatkowski with Ruslan Salei, Bryan Allen and Steve Montador, which is probably an upgrade. Still, it remains curious that there was such a huge difference in results. If a personnel change was responsible for the sudden shot drop, why did it reappear again the following season?

Breaking down shots against by period from the Hockey Summary Project, the biggest drop in shots against from '05-06 to '06-07 came in the first period:

First Period: -196 (-20%)

Second Period: -104 (-11%)

Third Period: -113 (-13%)

Overtime: 0 (0%)

I looked at the leading and trailing numbers, and 2006-07 does not appear to be an outlier in that regard. The Panthers may have played a more low-event game early on, suppressing shots against in the first period. However, there was clearly a significant shot prevention effect throughout the entire 60 minutes.

The last thing I thought to look at was blocked shots:

2003-04: 463 home, 549 away, 1012 total

2005-06: 403 home, 454 away, 857 total

2006-07: 501 home, 520 away, 1021 total

2007-08: 410 home, 570 away, 980 total

2008-09: 485 home, 620 away, 1105 total

2009-10: 584 home, 819 away, 1403 total

2010-11: 507 home, 618 away, 1125 total

This may explain some of the effect, perhaps reflecting a strategic shift by the Panthers' defence to adjust for the team's weaker goaltending. There is still a large amount left unexplained, however, as combined blocked shots and recorded shots against still show a drop of about 300 during 2006-07 compared to the seasons before and after.

For what it's worth, given that shot quality measures have tended to be unreliable and not predictive year-to-year (see Gabe Desjardins' summary rant on the subject), Alan Ryder estimated the Panthers' shot quality at 0.957 in 2005-06, 1.014 in 2006-07, and 1.008 in 2007-08 in his annual NHL reviews. Numbers below one indicate easier than average shots against while numbers above one indicate more difficult than average shots. That would support the hypothesis that the Panthers were able to block or prevent more long-range shots in '06-07 compared to '05-06. However, the shot quality estimate was similar in '07-08 compared to '06-07, despite the large increase in shots against.

I still don't entirely know what to make of the Florida's team defensive performance in 2006-07, it looks like a confluence of factors was responsible for the one-year dip in the team's shots allowed. It makes sense that the Panthers would have played more conservatively that year because of the goaltending change, leading to more blocked shots. The metrics suggest that team improved in terms of puck possession and spent more time at the other end of the rink, which helped cut down shots against. Switching goaltenders from Roberto Luongo to Ed Belfour may also have had a shots against impact of 1-2 shots per game. Other contributing factors could have been improved penalty killing as well as facing fewer opposing power plays against as players adjusted to the new post-lockout rules.

Yet while that may account for the changes from '05-06 to '06-07, it still doesn't explain why the Panthers dropped back to their usual awful shot prevention level during '07-08 and following years. It looks like coach Martin and his skaters put everything together for one season, but then for whatever reason were unable to recreate that success. Unfortunately for them, the one season where they managed to put together a pretty strong team effort (.898 win threshold, 13th best in the league) was also the one season where the team's goaltending was poor, and the result was yet another non-playoff appearance for the Florida Panthers.

Why Mike Richter Might Be Overrated

Mike Richter is a goalie that is remembered pretty fondly by many. That shouldn't be too surprising, as he played a long career on a big market team, he won a Cup, and he represented the U.S.A. admirably in a number of international tournaments. However, looking at his career numbers and especially his Vezina and All-Star voting record makes it pretty clear that Richter was not a member of the goaltending elite. Pretty much the only thing that was elite was his paycheque; in his entire career nobody ever gave Richter a first place vote for the postseason All-Star teams.

I recently realized, however, that Richter may not even have been as good as his numbers suggest. The reason is that there is some evidence that Richter may have benefited from a generous home scorer during the prime of his career in New York. Madison Square Garden has long been known as a rink that produces abnormal statistics for things like shot distance. During the mid-1990s, they might have been recording some screwy numbers when it comes to total shots as well.

For seven consecutive seasons from 1991-92 to 1997-98, Mike Richter's backup goalies faced a higher rate of shots against at home than on the road. Richter faced a higher rate himself in five out of the seven seasons and narrowly missed the two other times, finishing 0.8 lower in '91-92 and 0.5 lower in '97-98. Having a higher rate of shots against at home in any season is relatively rare, given that teams typically play better at home. When it happens seven years in a row, it is a major outlier.

Richter ('92-'98):

Home: 99-60-24, 2.71, .911, 30.5 SA/60

Road: 71-66-13, 3.01, .896, 29.1 SA/60

NYR backup goalies ('92 to '98):

Home: 45-25-18, 2.81, .910, 31.2 SA/60

Road: 47-60-14, 3.01, .894, 28.4 SA/60

The home/road GAA splits are quite normal. The backups had a very skewed record at home vs. on the road, which implies that the Rangers played a lot better in front of them at home. Given that, one would not expect shots against to go up by nearly 3 per game. The only effect that could somewhat account for that would be score effects. The Rangers playing to the score might explain why the backups had more of a differential between home and road than Richter did, but Richter himself had a more typical home/road split yet still had a higher shots against rate at home. Thus it seems that all Rangers goalies were getting extra credit for saves at MSG.

It is at least possible that the Rangers played a very high event game at home, although if that were true it would be expected that the goalie's home save percentages would have dropped or stayed the same rather than rose substantially compared to their numbers on the road.

Richter's numbers at home and on the road were almost identical to those of his backups. For a several of those seasons that was nothing to be ashamed of, as quality veteran John Vanbiesbrouck was Richter's playing partner, but for the rest of it the Rangers had a fairly undistinguished collection of backups, led by Glenn Healy. Healy's numbers cratered once he left the Rangers to play on the Leafs, which probably had a lot to do with age, but may have also had something to do with artificially inflated home numbers.

League average over the period was roughly .898. Richter's overall save percentage checked in at .905, suggesting that he was a pretty valuable goalie, worth nearly two wins above average to his team per season. However, the numbers show that nearly all of his excess value was coming based on the performance he recorded on home ice.

If we assume that in reality Richter faced the same rate of shots against at home and on the road and that the difference was due to generous scorekeeping, his home save percentage would drop to .907 and his overall save percentage for the period would fall to .902. If Richter actually faced one fewer shot against per 60 minutes at home, his numbers would fall even further to .904 at home and .900 overall, a result that would leave him about 20 goals above an average goalie. That's still pretty good, but it would have a pretty dramatic impact on Richter's career numbers. It would cost him about half of his career value in terms of goals above average, causing him to plunge well out of the "decent starters" range on this list.

On the road, Mike Richter was almost exactly an average goalie, based on his save percentage numbers. At home, his numbers were up among the best in the league. Given that the numbers of his backups followed the same pattern, it seems unreasonable to conclude that this was due to anything related to Richter himself. Furthermore, it seems unlikely that the Rangers would have been successfully suppressing shot quality against at least 10% better than an average team while playing at home yet for some reason choosing not to do the same thing on the road. Perhaps there were some team effects, but on the whole it seems like the best explanation is probably that there was some degree of shot padding at Madison Square Garden in the mid-1990s which boosted Mike Richter's statistics.

The Razor-Thin Edge of a Winning Reputation

If you ask hockey fans who is the best goalie in the league, you'll probably usually hear a handful of names listed before Cam Ward enters the conversation. But ask instead which goalie they would want if they were playing in game 7 of the Cup Finals, and don't be surprised if you'll hear Ward mentioned in the first 2 or 3 names.

Why is that? Simply because Cam Ward has a reputation as a winner. He won a Stanley Cup, he has a Conn Smythe, he's only lost one playoff series in his career. And if you get into an argument with a Hurricanes fan about their goaltender, it usually doesn't take long at all for them to break out Cam Ward's career record in game sevens: 4 wins, 0 losses.

That is an impressive statistic, to be sure, but like any win/loss record it is dependent on Ward's teammates and opponents. The reality is that Cam Ward was very, very close indeed to being 0-2 in his career in game sevens.

In his first career game 7, Ward allowed Jochen Hecht to bank in a shot from behind the net with 5 seconds left in the second period to give Buffalo a 2-1 lead. In the entire 2006 playoffs, teams leading after two periods were 60-9, but Ward's Carolina teammates scored three times in the third to bail out their goaltender and advance to the Cup Final, where the 'Canes again won in seven games.

In 2009, Carolina went to game seven in the first round against New Jersey, and again found themselves trailing in the third period. This time the outlook was even bleaker, with New Jersey leading 3-2 at home with less than 90 seconds remaining before Jussi Jokinen and Eric Staal combined to stun the Devils and snatch victory from the jaws of defeat. Excluding that game, teams were 59-2 in the 2009 playoffs when leading after two periods, which shows just how miraculous the 'Canes comeback was.

If not for two terrific comebacks, Cam Ward would be 0-2 in game sevens and nobody would have the impression that he was a particularly clutch goalie. That is not to say that Ward had no impact at all in those comebacks; clearly another goal or two against would have made much more difficult for his team to pull off the improbable. Nevertheless, it is an entirely reasonable counterfactual to suggest that with a bit less help Ward could easily have had not just no Cup and no Smythe, but not even a single game seven victory to his credit.

The game seven argument is also fairly meaningless because it represents such a small sample of Ward's career. Win/loss record aside, it is correct to say that Ward has performed well in game sevens (1.85 GAA and .932 save percentage), but we don't have to go very far to find examples of important games where Ward didn't manage to get the job done. Twice he has played the final game of the season in "win and you're in" situations, and both times he gave up four goals in a Hurricanes loss that eliminated them from playoff contention. Ward also gave up five goals in a world championship final loss on Canadian ice in Quebec City in 2008. These performances show that his performances in must-win games are far more variable than the "4-0 in game sevens" narrative implies.

Cam Ward is a very good goalie, and his recent performance has been trending solidly upward (.919 save percentage over the past three seasons). Is he unusually clutch in pressure situations? I think we still have to simply wait and see.