I compared shots per game averages for starters to backup goalies during the 2007-08 season, and was a bit surprised to find that over half the league had a substantial difference in shots faced averages between the #1 and #2 goalies. Sixteen out of the 30 teams had a difference of 1.5 shots per game or better between the starter and the backup, with a few more teams just missing the cutoff at 1.3 or 1.4. So having a sizeable shot differential was not rare but commonplace around the league last season.
I decided to look at just the teams with a substantial shot differential, using 1.5 as a cutoff, trying to isolate any variables that could explain the differences. I broke it down first by starter vs. backup (again using the goalie with most games played as the starter):
Starter: 29.3 SA/60, 26.8 SA/60 5 on 5, 43.2 SA/60 4 on 5
Backup: 29.3 SA/60, 26.1 SA/60 5 on 5, 41.1 SA/60 4 on 5
I also looked at two different measures of shot quality, both from Behind the Net and Hockey Numbers. Behind the Net gives the expected save percentage for each goalie based on the shots they faced. Hockey Numbers gives only the shot-quality neutral save percentage, but we can calculate the expected save percentage from that number. Behind the Net had starters and backups almost equal at .908 and .907 respectively. Hockey Numbers had a bit more of a gap, with starters facing slightly easier shots (.907 expected save percentage compared to .903 for the backups).
Other than the penalty kill discrepancy noted earlier, there isn't much difference there.
I then broke it down into the goalies with high shots against averages compared to those with lower shots against averages.
High shots: 30.7 SA/60, 27.9 SA/60 5 on 5, 42.6 SA/60 4 on 5
Low shots: 28.0 SA/60, 25.1 SA/60 5 on 5, 41.1 SA/60 4 on 5
There is a significant even-strength gap that is driving the results. Let's look at the shot-quality numbers for this group:
High shots: .908 (BtN), .908 (HN)
Low shots: .908 (BtN), .904 (HN)
The goalies who were facing more shots per game were not facing more difficult shots. Again, this does not imply that the extra shots they were facing were rebound shots or other prime scoring chances.
However, those extra chances did have an effect on goals against. The GAAs for the two groups were almost identical, although it was actually the goalies facing more shots that had a slightly lower GAA (2.78 to 2.80). They did much better at stopping the puck by both shot-quality measurements (.30 better in Behind the Net's GAA above average, and .006 better in SQNSV%), but the extra shots they faced resulted in more goals against.
There were two other things I looked at: shots for, and missed shots. Surprisingly to me, the backups had more shots for than the starters, and the goalies facing fewer shots against had more shots for than the goalies facing more shots. Starters had just 25.9 shots for/60 at even-strength compared to 26.7 for their backups, and the teammates of the high shot goalies took 26.1 shots/60 compared to 26.4 for the low shot goalies.
The starters and the backups were very close in missed shots, both at even-strength (10.9 and 11.1) and at 4 on 5 (18.8 and 18.7 respectively). In the high vs. low shot comparison, missed shots were very close at even-strength (10.8 to 10.9), but there was a bit of a gap at 4 on 5 (18.2 to 18.8).
I looked at the numbers for 2006-07 for high shot vs. low shot goalies, and they confirmed the basic principles: the goalies facing more shots tended to have better save percentages (.918 to .914 at even-strength, unadjusted), but the GAAs were almost identical on average (2.89 and 2.88).
So nothing really jumps out as a cause for the shot discrepancies. That these differences were found league-wide makes me think that a lot of it might be random noise, and it is unclear how much of the gap is repeatable from season to season. That is the key question: did the guy who is outperforming his teammates in terms of SA/60 also do it last year, and is he likely to do it next year? I think it is still very difficult to tell which goalie is allowing fewer shots than the other. Try this little quiz: I put together a list of 10 goalie tandems who had a difference of 1.5 or greater in their shots against averages in 2007-08. Guess which of the two goalies allowed fewer shots per game. If you score better than 50% you are outperforming me (I'll post the answers in the comments):
Cam Ward or John Grahame?
Ryan Miller or Jocelyn Thibault?
Tim Thomas or Alex Auld?
Chris Osgood or Dominik Hasek?
Niklas Backstrom or Josh Harding?
Carey Price or Cristobal Huet?
Chris Mason or Dan Ellis?
Martin Gerber or Ray Emery?
Marc-Andre Fleury or Ty Conklin?
Olaf Kolzig or Brent Johnson?