Courtesy of Wikimedia
Welcome back to our semi-regular segment where I will touch on a few trending topics in hockey statistics in a less mathematical and more discussion-based format.
This week we will explore the debate on player defensive impact on shot quality and save percentage.
So let’s begin.
The topic of control over save percentage is extremely important analytically. There are two ways a team reduces goals against: reduce shots against and reduce the probability a shot goes into the goal. It’s one of only two parts that impact goals.
The impact of goaltending talent versus player talent (and therefore value) on winning games solely rests on how much control players and teams have on shot quality.
While the understanding in shot quantity may not be fully developed, the basics are there. We can already identify with respectable confidence which teams and players are successful in shot repression. We can estimate how events like zone entries and exits success impacts shot volume. We know many of the outside factors that impact shot repression, like player usage, and are starting to account for discrepancies.
The same is not easily said with save percentage. We know not all shots are created equal in probability of goal. We know that save percentage models goaltender true-talent after a large enough sample. After that it gets murky.
Most research in shot quality –albeit with admittedly limited data– has yet to return much in support for importance in shot quality control.
Former Hockey-Graphs blogger Emmanuel Perry has created a sample significance tableau which helps illustrate this point:
The above graph shows the out-of-sample determination of correlation for the statistic relative Corsi percentage for defensemen. Since out attempting is a byproduct of performing well and players attempt to achieve success in these areas, we expect the correlation to become stronger as the sample approaches the halfway point in the season.
Essentially, if something is desired and players can significantly control their success in it, then there should be some sort of relationship between those who were successful in the past and those who are successful in the future. The larger the samples, the stronger relationship.
But, what if we look at a relative save percentage for the same players?
There is no discernible pattern and the R^2 values are extremely small, with the max being just barely over 0.006 at the 25 game mark, compared to the 0.4 peak we see in relative Corsi.
We’ve shown before that even when you increase the sample to two seasons on either side as opposed to half a season, the results don’t change very much.
Why is that? Should defensemen not care about reducing shot quality?
Obviously defensemen should still care and try to reduce the quality of shots. Coaches should not approach team systems without care for their defensemen’s drive for reducing shot quality.
It’s not that defensemen have no impact on save percentage; it’s that there is no observable difference in successfully improving save percentage in the NHL.
These are two completely different elements.
The distribution of impact is quite small. In today’s NHL a combination of systems, coaching, and skill has created an environment where the distribution in relative save percentage is extremely small.
As an example, you have two swimmers, one much better than the other. If all things were equal, the better swimmer is relatively much faster. However, if you place each in separate streams with a strong current, the differences between current strength may be far larger in impact than the difference in swimmer skill.
Now let’s expand this example to many swimmers in many different streams where the distribution in swimming talent is a matter of a few meters per minute while the distribution in stream current strength is a matter of a few meters per second. All of a sudden which stream a swimmer is placed in, something outside of the person’s control, becomes a greater factor than their actual skill.
However, if a good portion of the swimmers just gave up and not swim, or even swim in the opposite direction, this would all change.
Defensemen (and forwards) are similar. It’s not that defensemen have no impact over save percentage, but as it stands the observed distribution between them is much smaller than other factors. If a large percentage of NHL defensemen all of a sudden stopped caring, we would likely see a shift in these results.
During a panel discussion at the MIT Sloan Sports and Analytics Conference, Tyler Dellow reminded us that the results we see with players exists in a particular environment and that we must take that into account when discussing impact.
Hockey Graphs 