Forecasting Future Goalie Performance with Four Year Hockey Marcels:

Evaluating goalies is hard.  Goalie performance varies more than anything else in hockey and today’s terrible goalie can randomly turn into an elite goalie next season….and then turn back into a terrible goalie.  The best measure we have for evaluating goalies is Save Percentage and so we often tend to use a player’s career SV% as a way of forecasting what to expect from a goalie in the future.

However, it would make more sense not just to take a goalie’s career average SV% when forecasting future performance, but rather to take a weighted average in which we place greater importance on more recent data.  Eric Tulsky recently did this at his must-read blog, Outnumbered, and looked at what weight he should give each recent year’s data to forecast the next three years of a goalie’s performance:

So in my base case, I’m using years 1-4 to try to predict years 5-7. The best predictions came from weighting things like this:

  • Each shot faced in year 3 counts 60 percent as much as shots in year 4
  • Each shot faced in year 2 counts 50 percent as much as shots in year 4
  • Each shot faced in year 1 counts 30 percent as much as shots in year 4

This is particularly similar to the baseball forecasting system invented by Tom Tango, known as the Marcel Forecasting System.  Marcel, named after the monkey, is one of the most basic projection systems possible – it simply weights each of the last three years with weights of 5/4/3, adds a very basic regression to the mean, then adds a very basic aging projection.  Marcel is very basic on purpose – it’s still pretty damn accurate, and if a more complicated forecasting system can’t beat Marcel in baseball, it’s useless.  Surprisingly, most forecasting systems don’t improve upon Marcel by very much.

Eric’s Hockey Weights come out to weights of 5/3/2.5/1.5, which is pretty similar to the 5/4/3 of baseball’s Marcels.  So let’s use these weights to create Hockey Marcels.

Of course, as Eric noted, we can’t simply use these weights as is (or well, doing so will work, but won’t be as accurate as you’d like).  We still need to regress each player toward the average, especially in the cases of players with smaller than optimal sample sizes – after all, we’re a lot more confident in the weighted average of Lundqvist of a .9228 on 3327 shots than we are in Cory Schneider’s .9295 on 1869.7 shot sample.   Tango did this by adding league average at bats until he had a certain # of at bats for each player, and we can do the same thing here.  In this case, I added shots saved at the average rate until each player’s sample was 4000 shots strong.  This is the weakest part of this method by the way, since my selection of 4000 was kind of arbitrary – 4000 is a general minimum for when we feel somewhat confident in a goalie’s stats, although it’s usually the # used for even strength shots and here we’re using all situations.    However, it leads to all goalies facing at least SOME regression adjustment, which is what we would want.

The end result is in the chart down below.  However, we shouldn’t forget the last part of baseball Marcels, the aging adjustment.  Unfortunately, Hockey aging curves, especially for goaltending, aren’t quite as well founded as for baseball, and I couldn’t find one that I could use to create a very simple formula to adjust the data.  So the below data does not include an aging adjustment.  However, it should be fairly simple to mentally adjust the data downwards for players on the wrong side of 30, where goalies clearly start to decline.

NOTE:  The Following Data is through 1/31/14.

And without any further ado, the data:

Player Age Projected 3 Year SV% After Regression 3 Year Weighted Sample Projected 3 Year SV% w/out Regression
Tuukka Rask 26 0.9223 2295.3 0.9280
Cory Schneider 27 0.9216 1869.7 0.9295
Henrik Lundqvist 31 0.9214 3327.0 0.9228
Ben Bishop 27 0.9198 1711.5 0.9266
Ryan Miller 33 0.9189 3517.2 0.9195
Tim Thomas 39 0.9184 2373.8 0.9210
Pekka Rinne 31 0.9184 2537.6 0.9206
Roberto Luongo 34 0.9182 2718.0 0.9198
Jonathan Bernier 25 0.9179 1837.7 0.9216
Ben Scrivens 27 0.9176 1122.7 0.9251
Jonathan Quick 28 0.9171 2643.2 0.9183
Robin Lehner 22 0.9171 1039.8 0.9238
Carey Price 26 0.9168 3496.9 0.9172
Kari Lehtonen 30 0.9167 3413.4 0.9170
Anton Khudobin 27 0.9166 732.1 0.9254
Jimmy Howard 29 0.9166 2857.4 0.9174
Braden Holtby 24 0.9165 1885.9 0.9186
Semyon Varlamov 25 0.9164 2909.9 0.9171
Antti Niemi 30 0.9164 3390.8 0.9167
Marc-Andre Fleury 29 0.9157 3092.2 0.9160
Sergei Bobrovsky 25 0.9156 2404.0 0.9162
Mike Smith 31 0.9153 3142.6 0.9154
Jaroslav Halak 28 0.9152 2100.5 0.9156
Brian Elliott 28 0.9150 1750.9 0.9154
Craig Anderson 32 0.9149 2951.5 0.9150
James Reimer 25 0.9144 2145.2 0.9142
Jonas Hiller 31 0.9141 2852.4 0.9139
Jhonas Enroth 25 0.9138 1181.3 0.9118
Jean-Sebastien Giguere 36 0.9136 1295.5 0.9115
Al Montoya 28 0.9134 1130.3 0.9102
Corey Crawford 29 0.9132 2670.4 0.9125
Cam Ward 29 0.9131 2605.2 0.9123
Peter Budaj 31 0.9130 1093.0 0.9085
Justin Peters 27 0.9130 1149.0 0.9087
Michal Neuvirth 25 0.9129 1409.1 0.9097
Niklas Backstrom 35 0.9125 2299.5 0.9109
Evgeni Nabokov 38 0.9123 1989.4 0.9099
Ray Emery 31 0.9118 1260.6 0.9055
Dan Ellis 33 0.9112 1069.7 0.9018
Scott Clemmensen 36 0.9108 1218.0 0.9019
Ilya Bryzgalov 33 0.9106 2578.1 0.9084
Jonas Gustavsson 29 0.9103 1429.5 0.9023
Devan Dubnyk 27 0.9101 2645.1 0.9078
Steve Mason 25 0.9099 2620.4 0.9074
Anders Lindback 25 0.9099 1164.0 0.8982
Kevin Poulin 23 0.9096 1052.8 0.8952
Martin Brodeur 41 0.9081 2214.3 0.9029
Ondrej Pavelec 26 0.9062 3507.1 0.9050

You’ll note only three players show expected SV%s, after regression toward the mean, above .920, the standard for the very elite, over the next three years (Rask, Schneider, Lundqvist).  Of these three, we’re clearly the most confident in Lundqvist due to his 3300 shots, which is why his projection barely changes after regression, whereas Rask and Schneider’s projections drop a ton.

Similarly, only 5 goalies show up under .910, which is kind of terrible for starting goalies facing this # of shots.  Admittedly, for two of these goalies we have very tiny samples (Poulin and Lindback both give us samples barely over 1000 shots), but those samples have been so poor, that even the huge regression hasn’t put them above .910.  By contrast, Ondrej Pavelec somehow gives us our 2nd biggest sample of the above-goalies – so the regression barely helps him.  Of course, when we consider aging, Brodeur probably should project to have a worse performance in the future than Pavelec, but that’s not saying much.

One final note: These #s act as if the league itself isn’t going to change, but of course, SV% has been rising for years.  If it does, one would expect most of these guys to perform “better” than the results above, although their rankings should still be the same.  The point isn’t really to project absolute SV% for the next year as much as it is to project the quality of goalies over the next three years.  I think this seems like a pretty solid way of doing so.

3 thoughts on “Forecasting Future Goalie Performance with Four Year Hockey Marcels:

  1. Someday, it would be cool to do something like this while demonstrating the error on either side of the projection (or even just expressing it as .906-.910 for someone like Pavelec).

    By the way, I despise the Word Press tables.

  2. First time visiting your site, like it. I’ve read some interesting stuff on how coaches can influence save percentage. Can that be considered? Albeit in an admittedly small sample Pavelec looks way better with Maurice coaching. From watching the games it makes sense, the Jets were often guilty of incredible defensive breakdowns that left the goalie out to dry, which can’t help but kill save %. Thoughts?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s