Monday, November 17, 2008

Shots Attempts Against

With the issue of shots against featuring prominently in recent discussion here and elsewhere in the blogosphere, I thought to take a slightly different look at the issue. Generally attempts to define the shots against/save percentage relationship look at total numbers by season (like this one, for example). I was interested in how teams performed over multiple seasons, and how similar shot preventing teams did in terms of save percentage.

Based on the results from the even-strength save percentage while tied numbers, I was also interested in exploring the results of blocked shots and missed shots against. The frequency of blocked or missed shots is quite high. Over the last 8 seasons, there were 555,531 shots, 215,508 blocked shots, and 208,889 missed shots, meaning that 43% of shot attempts never make it on net. It would therefore seem that a team's commitment to shot blocking could have a big impact on the number of shots actually faced.

The "fewer shots = lower save percentage" argument is usually based on one of the following two premises: 1. Goalies who face infrequent shots lose focus and are less physically prepared for each shot that comes than those who are facing more frequent shots, or 2. Goalies who face infrequent shots are facing higher quality chances. The first one could possibly be true, but based on my personal experience I do not think it is a major factor. It is, however, a difficult one to test (it would require analysis of the play-by-play records to do it properly), so this post is directed at premise 2.

Before I continue, just a caveat: with RTSS data there is always the possibility of systematic errors. It is, in fact, quite likely that teams differ in their reporting criteria, based on what we have discovered from past shot and shot distance reporting results. I will present the data as is, but if there is reason to believe that the numbers aren't quite correct please point it out.

I took the last 8 seasons (1999-00 to 2007-08), and collected the total minutes, shots against, saves, blocked shots and missed shots for each team. I then divided the teams up into 5 groups, based on where they rank in shot attempts against (Att), defined as saves plus goals allowed plus blocked shots plus missed shots. I have also included shots against (SA) and save percentage numbers for each team.

Low Shot Group:
Anaheim: 46.4 Att, 28.2 SA, .913 Sv%
Detroit: 46.5 Att, 25.9 SA, .910 Sv%
Chicago: 46.7 Att, 28.2 SA, .899 Sv%
Dallas: 46.9 Att, 25.0 SA, .910 Sv%
San Jose: 47.1 Att, 27.0 SA, .910 Sv%
New Jersey: 47.5 Att, 25.5 SA, .912 Sv%

If a low shots against total was the only handicap preventing Martin Brodeur from posting elite save percentage numbers then we would expect the Devils' goaltending to outperform the rest of this group. The fact that Anaheim, Detroit, Dallas and San Jose have virtually identical save percentages while facing even fewer shot attempts than New Jersey seems to be strong evidence against that viewpoint.

Anaheim and Chicago do not block many shots, at least according to NHL scorers, so even with average shot totals they both move up into the top group here. Chicago is definitely the outlier, finishing far behind the other teams. Could the Chicago scorers possibly be underreporting blocks and missed shots? Or do the Hawks simply suffer from bad goaltending and/or high shot quality against?

Moderately Low Shot Group:
Boston: 48.4 Att, 28.5 SA, .903 Sv%
Nashville: 48.7 Att, 29.3 SA, .912 Sv%
Ottawa: 48.8 Att, 26.8 SA, .908 Sv%
Vancouver: 48.8 Att, 27.2 SA, .906 Sv%
St. Louis: 48.9 Att, 25.4 SA, .901 Sv%
Calgary: 48.9 Att, 27.5 SA, .907 Sv%

St. Louis ranks near the very best in shots against, but the reason seems to be that they block so many shots. Given the goaltending they have had that is maybe not too surprising, but lumping in St. Louis with the top possession teams during the study period appears to be incorrect. Ottawa is similar to the Blues. On the other hand, Nashville apparently lets a lot more shots through, and they had the best save percentage in this group.

Average Shot Group:
Colorado: 49.3 Att, 27.4 SA, .912 Sv%
Tampa Bay: 49.5 Att, 28.3 SA, .898 Sv%
Philadelphia: 49.8 Att, 27.1 SA, .908 Sv%
Toronto: 49.9 Att, 27.9 SA, .903 Sv%
Buffalo: 50.0 Att, 27.9 SA, .909 Sv%
Minnesota: 50.0 Att, 28.5 SA, .916 Sv%
Phoenix: 50.0 Att, 29.4 SA, .905 Sv%

The average group had the widest range of save percentage results, from Minnesota (best in the league) to Tampa Bay (worst in the league). Even though Minnesota blocked a normal number of shots and allowed an average number of shots on goal, their goalies had very high save percentages, which suggests that Jacques Lemaire knows how to make life easier for his goalies. Both Colorado and Buffalo had good results even after Roy and Hasek left town, which suggests that they were good defensive teams.

Moderately High Shot Group:
Columbus: 50.3 Att, 30.3 SA, .905 Sv%
Carolina: 50.4 Att, 27.5 SA, .901 Sv%
Los Angeles: 50.8 Att, 27.7 SA, .900 Sv%
Washington: 50.9 Att, 29.9 SA, .905 Sv%
Edmonton: 50.9 Att, 27.2 SA, .904 Sv%

The save percentages are very similar for all 5 teams and none of these teams had elite goalies in the period (other than maybe a couple of Kolzig seasons), so the results likely generally reflect team shot quality against. Edmonton had the second highest total of blocked and missed shots in the league while Columbus had one of the lowest, creating a difference of over 3 shots on goal per game, but the two teams had almost identical save percentages.

High Shot Group:
NY Islanders: 52.2 Att, 29.6 SA, .904 Sv%
Pittsburgh: 52.2 Att, 30.4 SA, .900 Sv%
Florida: 52.9 Att, 31.9 SA, .913 Sv%
NY Rangers: 52.9 Att, 29.1 SA, .904 Sv%
Montreal: 53.3 Att, 30.1 SA, .914 Sv%
Atlanta: 53.3 Att, 31.6 SA, .898 Sv%

For teams that give up a lot of chances, the norm appears to be mediocre save percentages rather than higher ones. Florida and Montreal are the only above-average teams and they rank far ahead of everyone else.

Here are the overall averages for each group:

Low Shots: 46.9 Att, 26.6 SA, .909 Sv%
Mod. Low: 48.8 Att, 27.5 SA, .906 Sv%
Average: 49.8 Att, 28.1 SA, .907 Sv%
Mod. High: 50.7 Att, 28.5 SA, .903 Sv%
High Shots: 52.8 Att, 30.5 SA, .906 Sv%

There is no real evidence of a pattern in terms of goaltending success. There are a few teams that have outlier results based on their groupings, either underperforming (Chicago, St. Louis, Tampa Bay) or overperforming (Nashville, Florida, Montreal, Minnesota) their expected save percentages. If we remove the best and the worst team in each group to deal with potential outliers, the save percentages by group go .911 - .906 - .907 - .903 - .905. This evidence certainly doesn't show that playing on a top defensive club hurts one's save percentage; if anything it suggests the opposite. However, save percentage is not a perfect proxy for shot quality - the goalies themselves obviously have an impact. If top teams tend to have better goaltending, for instance, then we would expect this result.

These numbers are polluted by a few variables, like suspect RTSS data and special teams play. The topic of possession, outshooting, and scoring percentages is still being investigated. However, I remain unconvinced that playing on a strong defensive team makes it tougher to put up high save percentages. I did not really evaluate possession effects here, merely shot prevention, so it could be that there is an effect for outshooting teams. In any event, even if a general relationship is established, this post is evidence that there is still substantial variability from team to team even within similar shots against groupings. This variability means that we cannot necessarily go from an established general result to the specific case for individual goaltenders or individual teams without additional supporting evidence.

These findings also show significant differences in shot-blocking tactics, which support the use of Fenwick or Corsi numbers rather than raw shot totals to evaluate team puck possession. Finally, there is some evidence of a relationship between blocked shots and goalie save efficiency (-0.30 correlation between blocked shots per game and save percentage). It remains to be determined whether this may be because shot blocking makes it harder to stop pucks, or because teams with bad goaltending simply try to make more saves themselves.


JLikens said...

Highly interesting.

As you said, there doesn't seem to much a relationship between the two variables.

The negative correlation between blocked shots and save percentage is interesting, though.

Is the correlation based on an analysis of the entire sample, or the average of individual seasons (or neither)?

The Contrarian Goaltender said...

The correlation is based on the entire sample.

Kent W. said...

Im very interested in the negative correlation between shot blocking and save percentage. Both because it seems counter-intuitive to me and because the Flames coaching staff has been actively discouraging shot blocking since the Stanley Cup run (even though Kipper's effectiveness has been falling).

It would be nice to verify whether they're on the right track or not with that strategy.

overpass said...

The negative correlation between shot blocking and SV% is interesting. It makes intuitive sense also, because some coaches and goalies clearly prefer that their players block fewer shots (including the Flames, as alluded to by Kent W.)

I don't know if this tactic is a net loss or a net gain, but they would be crazy to do so if blocking shots had no effect on SV%. If that was the case, teams should be going all-out to block shots. NHL teams have been known to pursue sub-optimal strategies, but surely not to this degree.

It's easy to think of ways that blocked shots might affect SV%. If blockable shots tend to be lower quality shots, teams that choose to block them will face better quality shots on average than teams that choose not to. I think this is likely the main cause of the correlation. Also, teams that sell out to block shots may leave themselves open to the other team creating higher quality opportunities, by screening the goalie or by taking themselves out of the play.

It would be interesting to see more work on this, particularly the effect it has on individual goaltenders SV%. Of course, that's difficult because of the RTSS issues CG mentioned in his post, but if a home-road breakdown of those stats is available it should be possible to remove the scorer bias. said...

Guys, correct me if I'm missing something here (or if this is just extremely obvious), but the above SV% numbers are for shots that hit the net, not the total "save percentage" on shots that were ATTEMPTED.

So in other words, even if blocking shots worsens save percentage on shots that get to the net, shot blocking might still be a huge net gain because it prevents a ton of shots from even hitting the net.

E.g., let's say Team A shoots the puck 100 times.

In example 1, Team B decides to block none of them. The goalie posts a save percentage of .920 and 8 total goals are allowed.

In example 2, Team B decides to try and block all of them. 40 are blocked and don't hit the net. Of the 60 that hit the net, the goalie posts a .900 save percentage. Note that 6 total goals are allowed - 2 less than the first example, despite the goaltender posting a significantly lower SV%.

JLikens said...

I'm wondering to what effect the relationship between shots blocked and save percentage is independent of shot quality (as measured by Alan Ryder.

Take Anaheim for example.

Last season, the Ducks ranked 29th out of 30 teams in terms of shot quality against (i.e. they were the 2nd worst team in this regard). And yet, they had the 2nd best save percentage.

What's interesting is that the Ducks blocked the 2nd fewest shots. Perhaps this allowed them to have such a high team save percentage despite ostensibly allowing such dangerous shots on average.

overpass said...

jlikens - That's an interesting question. I'd like to see the relationship between SQNSV% and shots blocked.

If blocking shots simply stops lower percentage shots from getting through and raises the average shot quality of the shots that do make it to the net, that should show up in Ryder's shot quality model. The "shot-blocking effect on SV%" would be fully accounted for in the model.

If, on the other hand, shot-blocking causes a lower SV% because missed shot-blocks screen the goalie and distract him, the effect should be independent of SV%.

So in other words, even if blocking shots worsens save percentage on shots that get to the net, shot blocking might still be a huge net gain because it prevents a ton of shots from even hitting the net.

sunnymehta - I think you're right about that. Players who block shots aren't concerned about maximizing their goalie's SV%, they're trying to minimize GAA.

Bruce said...

If, on the other hand, shot-blocking causes a lower SV% because missed shot-blocks screen the goalie and distract him, the effect should be independent of SV%.

Bingo. The goalie gets no credit for the shots blocked by his team. But the shots that get through the traffic are going to be harder for him to see and stop. Also, as Don Cherry is wont to point out about a thousand times a season, sometimes attempted shot blocks wind up being deflections. No problem if they're deflected wide, but again no credit to the goalie. If they're on net, though, it's generally a tough stop.

Whereas if a team is coached to allow the goalie to see the puck and make the first save while clearing rebounds and traffic from around the goal mouth, that would correspond well with results like Anaheim's. There's more to shot quality than distance.

Sunny Mehta said...

Regarding SQNSV%, I actually emailed Chris from Hockey Numbers about this exact thing last season.

So far as I know, the NHL does not record (in their play-by-play data) when a goal is scored on a deflection by the opposing team.

For example, if Lidstrom shoots the puck at the blue line and it deflects off an opposing team player in front of the net, the NHL records "GOAL: Lidstrom, Slapshot, 60 feet."

Whereas, if Lidstrom shoots the same shot and it gets deflected the exact same way, but off Zetterberg, the NHL records "GOAL: Zetterberg, Tip-in, 5 feet."

I don't know how significant it is in the overall SQ model, but it sure seems like a pretty big error to me. Maybe someone can email Ryder.

The Contrarian Goaltender said...

I did a bit more research on this, and here are a couple of comments:

Blocked shots are less correlated with shot quality against than they are with save percentage. For the shot quality numbers I have from 2005-2008, the correlation with blocked shots was 0.14.

Another thing I thought to check was who exactly was blocking all the shots. If only point shots were being blocked, for instance, then we would expect forwards to block a large portion of the shots. A better way to check this would be to figure out who had their shot blocked, because defencemen could be blocking point shots closer to the net, but I don't have that data. It turns out that defencemen account for 66% of the blocked shots this season. Considering that defencemen have only taken 26% of total shots, this implies that most of the blocks are taken closer to the net. If so, teams that are blocking a lot of shots seem to be helping their shot quality against.

In my experience, it is usually tougher to make a stop when a defencemen makes a failed shot block attempt than a forward, because they are usually closer to the net so you have less time to react and are often partially or completely screened. I would guess most own-goal deflections come off of defencemen.

One thing we could investigate is how much of shot blocking is based on the system and how much is based on discretion. For example, is there a team that doesn't block many shots with the starter in the net yet tries to block everything when the backup plays? If there are discrepancies, that would suggest that part of the relationship would be explained by goalie quality. If worse goalies are less likely to stop the puck, and teams are more likely to block shots for worse goalies, then that would certainly account for some of the effect.

Steven said...

I think there are too many variables between each team for you to make a comparison like this. I think a much more accurate comparison would be to compare the Same goalie's SV% between facing more shots and facing less shots. For example. What is a Goalie's SV% in a season where he faced 30+ shots vs when he only faced 20 or lower?

I think that would be a more accurate analysis of corelation between shots taken and SV%. However even then, it still would vary between goaltenders, as some styles would favor one or the other.

As for shot blocked, last year over 1/3 of the goals on Brodeur were from deflections from his own team. I don't think you can really factor in blocked shots correctly because they simply don't have that data of what happened after the block was attempted.