Evaluating goalies is hard.  Goalie performance varies more than anything else in hockey and today’s terrible goalie can randomly turn into an elite goalie next season….and then turn back into a terrible goalie.  The best measure we have for evaluating goalies is Save Percentage and so we often tend to use a player’s career SV% as a way of forecasting what to expect from a goalie in the future.

However, it would make more sense not just to take a goalie’s career average SV% when forecasting future performance, but rather to take a weighted average in which we place greater importance on more recent data.  Eric Tulsky recently did this at his must-read blog, Outnumbered, and looked at what weight he should give each recent year’s data to forecast the next three years of a goalie’s performance:

So in my base case, I’m using years 1-4 to try to predict years 5-7. The best predictions came from weighting things like this:

• Each shot faced in year 3 counts 60 percent as much as shots in year 4
• Each shot faced in year 2 counts 50 percent as much as shots in year 4
• Each shot faced in year 1 counts 30 percent as much as shots in year 4

This is particularly similar to the baseball forecasting system invented by Tom Tango, known as the Marcel Forecasting System.  Marcel, named after the monkey, is one of the most basic projection systems possible – it simply weights each of the last three years with weights of 5/4/3, adds a very basic regression to the mean, then adds a very basic aging projection.  Marcel is very basic on purpose – it’s still pretty damn accurate, and if a more complicated forecasting system can’t beat Marcel in baseball, it’s useless.  Surprisingly, most forecasting systems don’t improve upon Marcel by very much.

Eric’s Hockey Weights come out to weights of 5/3/2.5/1.5, which is pretty similar to the 5/4/3 of baseball’s Marcels.  So let’s use these weights to create Hockey Marcels.

Of course, as Eric noted, we can’t simply use these weights as is (or well, doing so will work, but won’t be as accurate as you’d like).  We still need to regress each player toward the average, especially in the cases of players with smaller than optimal sample sizes – after all, we’re a lot more confident in the weighted average of Lundqvist of a .9228 on 3327 shots than we are in Cory Schneider’s .9295 on 1869.7 shot sample.   Tango did this by adding league average at bats until he had a certain # of at bats for each player, and we can do the same thing here.  In this case, I added shots saved at the average rate until each player’s sample was 4000 shots strong.  This is the weakest part of this method by the way, since my selection of 4000 was kind of arbitrary – 4000 is a general minimum for when we feel somewhat confident in a goalie’s stats, although it’s usually the # used for even strength shots and here we’re using all situations.    However, it leads to all goalies facing at least SOME regression adjustment, which is what we would want.

The end result is in the chart down below.  However, we shouldn’t forget the last part of baseball Marcels, the aging adjustment.  Unfortunately, Hockey aging curves, especially for goaltending, aren’t quite as well founded as for baseball, and I couldn’t find one that I could use to create a very simple formula to adjust the data.  So the below data does not include an aging adjustment.  However, it should be fairly simple to mentally adjust the data downwards for players on the wrong side of 30, where goalies clearly start to decline.

NOTE:  The Following Data is through 1/31/14.

And without any further ado, the data:

You’ll note only three players show expected SV%s, after regression toward the mean, above .920, the standard for the very elite, over the next three years (Rask, Schneider, Lundqvist).  Of these three, we’re clearly the most confident in Lundqvist due to his 3300 shots, which is why his projection barely changes after regression, whereas Rask and Schneider’s projections drop a ton.

Similarly, only 5 goalies show up under .910, which is kind of terrible for starting goalies facing this # of shots.  Admittedly, for two of these goalies we have very tiny samples (Poulin and Lindback both give us samples barely over 1000 shots), but those samples have been so poor, that even the huge regression hasn’t put them above .910.  By contrast, Ondrej Pavelec somehow gives us our 2nd biggest sample of the above-goalies – so the regression barely helps him.  Of course, when we consider aging, Brodeur probably should project to have a worse performance in the future than Pavelec, but that’s not saying much.

One final note: These #s act as if the league itself isn’t going to change, but of course, SV% has been rising for years.  If it does, one would expect most of these guys to perform “better” than the results above, although their rankings should still be the same.  The point isn’t really to project absolute SV% for the next year as much as it is to project the quality of goalies over the next three years.  I think this seems like a pretty solid way of doing so.