Image from Wikimedia commons

Currently save percentage is the single best statistic for evaluating goaltenders… which is unfortunate as save percentage is extremely rudimentary and a suboptimal statistic.

There are two important factors for a statistic to be useful: that it impacts wins and the individual can either control or push the needle. Save percentage has both.

Robert Vollman showed previously how a goaltender’s save percentage impacts a team’s chance to win that game:

We also know that save percentage is the goaltender statistic they have the most control of (versus win% or GAA).

A team wins by out scoring their opponent, which can be accomplished by both increasing goals for and decreasing goals against. Goaltenders have a near negligible impact on goal scoring, with their primary role being to block shots.

Goals against are a combination of shots (or attempts) against and the percentage of shots (or attempts) that enter the net. Goaltenders have some control over shots against with rebound control and playing the puck, but all indications are the impact is small.

So then there’s only two possible outcomes: either goaltenders have a very small impact on winning from the variables above or goaltenders impact the game by blocking more shots per than their peers.

There still exists two major issues with save percentage…

Shot Quality

One unanswered question is how much impact does a team have on a goaltender’s save percentage.

The prevailing current theory is that a goaltender pushes the needle far more than their team over the long run. This theory has arisen from multiple different studies on player and team effects on save percentage. There is also save percentages relationship with things like back-to-backs and aging curves.

However, a team having less impact than their goaltender is not the same as no impact.

There have been many attempts to account for shot quality. In 2011 Michael Schuckers published Defense Independent Goaltender Ratings. One of the most recent is War-on-Ice’s shot location Adjusted Save Percentage.

The idea is adjusting a goaltender’s numbers with looking at their performance relative to their peers for different shot locations. Gabriel Desjardins previously showed that the major driver to a player’s shooting percentage is shot location.

The above regression curve above compares the regular (or unadjusted) save percentage to adjusted save percentage for all goaltenders with at least 40 games played since the 2011-12 season (with all the fun numbers added in the tables for those interested in such things).

There is not much information added in the long term by adjusting for shot location. Regular save percentage accounts for just under 90% of the variation we see in the adjusted numbers. How much of that remaining 10% is shot quality factors being weeded out remains to be seen.

This does not mean that numbers like adjusted save percentage do not have possible use.

Sample Size

By nature, goals are an extremely rare and highly susceptible to variance. Due to these factors, save percentage requires large sample sizes to become remotely indicative of talent.

Gabriel Desjardins created the above chart using a coin-toss model to illustrate how huge number of shots are required in order for save percentage to become less “luck” driven (see here for article).

The necessity for large sample sizes creates a very inefficient statistic from a managerial perspective. A team does not have the time to give their up-and-coming, young back up 3000+ shots against before determining whether or not he should supplant the older veteran starter.

While adjusted save percentage may have diminishing returns in the long run, it is possible that such adjustments may allow for “earlier detection”. The next step would be to go back back in history and see whether adjusted or regular save percentage predicts future (unadjusted) save percentage better.

Something like Eric Tulsky‘s work with even strength versus regular save percentage:

For more reading material, Tulsky has indexed multiple goaltending articles here.