The usefulness of on-ice save percentage (and derivative metrics such as Sv% Rel and Sv% RelTM) has been the source of many, many heated debates in the analytics blogosphere. While many analysts point to the lack of year-over-year repeatability that these metrics tend to show (past performance doesn’t predict future performance very well) as evidence of their limitations, others (primarily David Johnson of HockeyAnalysis.com) have argued that there are structural factors that haven’t been accounted for in past analyses that artificially deflate the year-to-year correlations that we see.
David’s point is a fair one – a lot can change about how a player is used between two samples, it’s not unreasonable to think that those changes could impact the results a player records. But we don’t just have to speculate about the impact those factors have – we can test the impact, by building a model that includes measures of how these factors have changed and seeing how it changes our predictions.
Hockey Graphs 