Evaluating goalies is hard. Goalie performance varies more than anything else in hockey and today’s terrible goalie can randomly turn into an elite goalie next season….and then turn back into a terrible goalie. The best measure we have for evaluating goalies is Save Percentage and so we often tend to use a player’s career SV% as a way of forecasting what to expect from a goalie in the future.
However, it would make more sense not just to take a goalie’s career average SV% when forecasting future performance, but rather to take a weighted average in which we place greater importance on more recent data. Eric Tulsky recently did this at his must-read blog, Outnumbered, and looked at what weight he should give each recent year’s data to forecast the next three years of a goalie’s performance:
So in my base case, I’m using years 1-4 to try to predict years 5-7. The best predictions came from weighting things like this:
- Each shot faced in year 3 counts 60 percent as much as shots in year 4
- Each shot faced in year 2 counts 50 percent as much as shots in year 4
- Each shot faced in year 1 counts 30 percent as much as shots in year 4
This is particularly similar to the baseball forecasting system invented by Tom Tango, known as the Marcel Forecasting System. Marcel, named after the monkey, is one of the most basic projection systems possible – it simply weights each of the last three years with weights of 5/4/3, adds a very basic regression to the mean, then adds a very basic aging projection. Marcel is very basic on purpose – it’s still pretty damn accurate, and if a more complicated forecasting system can’t beat Marcel in baseball, it’s useless. Surprisingly, most forecasting systems don’t improve upon Marcel by very much.
Hockey Graphs 