Projecting Future Goalie Performance: Updating and Improving Hockey Marcels

In February, I introduced my hockey version of Baseball’s Marcel forecasting system – a system that uses the last few years of a player’s career, weights the more recent seasons more heavily, and uses that to project future performance.  In particular, I was using the system to take an attempt at projecting goalies – who are of course the most unpredictable of hockey players.

Now that the season is over, it’s time to take another stab at goalie projections using Marcels. However, we can do better than we did last time: last time, our projections used Eric’s weights plus a mostly arbitrary regression mechanism. This time, we can use a more realistic (and non-arbitrary) regression as well as attempt to account for aging as well. In short, we SHOULD be building a better projection system for goalies, the most unpredictable of players.

METHODOLOGY (Skip if you just want the results, but it’s important)

The earlier post was building upon Eric Tulsky’s work which found that the following weights gave the best results at projecting future performance by goalies over the next three years:

So in my base case, I’m using years 1-4 to try to predict years 5-7. The best predictions came from weighting things like this:

  • Each shot faced in year 3 counts 60 percent as much as shots in year 4
  • Each shot faced in year 2 counts 50 percent as much as shots in year 4
  • Each shot faced in year 1 counts 30 percent as much as shots in year 4

Now as we noted in the last post, simply using these weights isn’t enough to make a projection of a goalie – goalie performance is simply too affected by random events (well this is true for non-goalies as well, but especially so for goalies).  As such, we need to regress the goalies’ #s toward the mean (in this case a SV% of .9142) in order to compensate for this.  Naturally we regress less for goalies with higher samples and more for those with much smaller samples.

In the previous post we did this simply by adding shots at the league average rate till we hit 4000 shots, which was basically arbitrary, and had the issue of leaving some goalies with basically no regression at all (which isn’t likely even for goalies with near 4K shot samples).  Instead, we should regress by adding the same # of shots to every goalie, and determine that # by using the correlation between goalie performance from one year to the next (and using the average goalie sample size to find out how many shots said regression would take).

In short, we should add 1525 to every goalie’s weighted sample, all saved at the league average.  The end result of this is that the 49 goalies in our sample were regressed on average 40% to the mean, with Ryan Miller’s #s being 26.97% regressed to the mean, while Viktor Fasth’s #s facing a regression of 67.4%.  In other words, we’re basically regressing Fasth two thirds to the Mean here due to his small sample size, while Miller is only being regressed a little more than one quarter.

But while adding regression is necessary to make a projection, a projection system needs a third component: a way to adjust for aging.  We didn’t do this in our last attempt at Marcels because we didn’t have an easily accessible goalie aging curve available.  As it happens, we just looked at goalie aging last month.  So now we can add this in (if you’re curious, I’m using the goalie aging line discussed in that post rather than the curve.)  And this gives us our completed Marcels.

So without further ado:

THE PROJECTIONS (If you skipped the methodology, stop here):

Note that the below are projecting performance over the next three years. You’ll apply less of an aging adjustment (and slightly different weights) for one year projections.

Player Age Raw Marcel Regressed Marcel (no Aging) Complete Marcel (Includes Aging) % Regressed
Tuukka Rask 26 0.9284 0.9234 .9207 35.27%
Cory Schneider 27 0.9260 0.9212 .9180 39.92%
Sergei Bobrovsky 25 0.9206 0.9185 .9165 32.33%
Anton Khudobin 27 0.9259 0.9197 .9164 52.91%
Semyon Varlamov 25 0.9200 0.9183 .9162 29.60%
Carey Price 26 0.9204 0.9187 .9160 27.40%
Ben Bishop 27 0.9223 0.9191 .9158 39.35%
Jonathan Bernier 25 0.9203 0.9179 .9158 39.27%
Henrik Lundqvist 31 0.9238 0.9211 .9155 27.65%
Robin Lehner 22 0.9173 0.9157 .9154 49.99%
Braden Holtby 24 0.9172 0.9160 .9145 39.41%
Josh Harding 29 0.9216 0.9177 .9133 51.57%
Ben Scrivens 27 0.9186 0.9165 .9133 47.15%
Jaroslav Halak 28 0.9185 0.9169 .9131 36.81%
Jonathan Quick 28 0.9175 0.9164 .9125 32.71%
Kari Lehtonen 30 0.9184 0.9172 .9122 27.60%
Brian Elliott 28 0.9172 0.9159 .9120 43.91%
Thomas Greiss 28 0.9177 0.9156 .9118 58.69%
Jhonas Enroth 25 0.9133 0.9137 .9116 50.01%
James Reimer 25 0.9134 0.9137 .9116 37.72%
Mike Smith 31 0.9179 0.9168 .9111 29.53%
Michal Neuvirth 25 0.9122 0.9132 .9111 48.64%
Marc-Andre Fleury 29 0.9154 0.9150 .9106 29.14%
Antti Niemi 30 0.9162 0.9156 .9105 28.15%
Pekka Rinne 31 0.9168 0.9159 .9103 34.05%
Roberto Luongo 34 0.9193 0.9177 .9102 32.04%
Corey Crawford 29 0.9148 0.9146 .9101 32.20%
Jimmy Howard 29 0.9146 0.9145 .9100 30.61%
Ryan Miller 33 0.9169 0.9161 .9093 26.97%
Steve Mason 25 0.9100 0.9113 .9092 32.09%
Craig Anderson 32 0.9153 0.9150 .9087 30.30%
Al Montoya 28 0.9108 0.9125 .9087 51.62%
Justin Peters 27 0.9087 0.9118 .9085 57.03%
Viktor Fasth 31 0.9128 0.9137 .9081 67.43%
Cam Ward 29 0.9113 0.9123 .9079 34.51%
Jonas Hiller 31 0.9126 0.9131 .9075 32.20%
Devan Dubnyk 27 0.9078 0.9101 .9069 36.57%
Tim Thomas 39 0.9176 0.9164 .9060 35.57%
Anders Lindback 25 0.9004 0.9077 .9057 53.60%
Kevin Poulin 23 0.8952 0.9064 .9055 59.16%
Ray Emery 31 0.9070 0.9106 .9050 50.65%
Ondrej Pavelec 26 0.9047 0.9073 .9047 27.99%
Ilya Bryzgalov 33 0.9096 0.9112 .9043 34.12%
Jean-Sebastien Giguere 36 0.9117 0.9129 .9043 49.63%
Niklas Backstrom 35 0.9108 0.9121 .9041 39.78%
Jonas Gustavsson 29 0.9021 0.9080 .9035 49.14%
Evgeni Nabokov 38 0.9086 0.9107 .9009 39.31%
Dan Ellis 33 0.8938 0.9048 .8980 54.35%
Martin Brodeur 41 0.9033 0.9074 .8958 37.95%

A quick explanation about how to read this table.  The first SV% column is the Raw Marcel projection before any aging or regression.   As such, there are some silly small sample results here  (Anton Khudobin, 3rd best NHL goaltender, .9259 SV%.)  The second SV% column are the Marcels after the regression talked about above.  Basically, those are the #s we came up with in our last post, albeit now we have a more solid regression method.

The third SV% column, the one in bold, is the most important one however – that’s the completed Marcel projections when we include in the effects of aging.  The effects are dramatic – only two goalies 30 or over remain in our top 20 goalies over the next 3 years – and those two are ages 30 and 31 (not exactly that old).  Moreover, only 11 of these goalies are expected to be better than average over the next three years.

The final column by the way details the amount of regression done on the #s of each player – the higher regression %s come obviously from smaller sample sizes, and vice versa.  The lower this number the more confident we are in the result – We’re much more confident in our projection Bobrovsky as a .9165 goalie going forward the next 3 years than Anton Khudobin being a .9164.

SOME THOUGHTS ON THE PROJECTIONS:

First of all, Tuukka Rask is by far the league’s best goalie and should continue to be so.  He leads in raw, regressed, and complete marcel projections, and the #2 goalie going forward in the complete Marcels isn’t even close  (almost .003 away!)   There’s also a gap between Cory Schneider as the #2 goalie in the league and Bobrovsky as the #3 and then the pack becomes much more muddled.

Second, due in large part to the aging adjustment, finding goalies in free agency to act as a multi-year starter is basically a losing play.  No UFA goalie is projected to be above average over the next 3 years, and only three: Jaroslav Halak, Brian Elliott and Thomas Greiss project as being in the top half of these forty nine goalies going forward*   Greiss of course is partly the result of being regressed almost 60% while there are reasonable concerns about Elliott.

*If you’re curious about the discrepancy between the above statements, we will have over the next three years a bunch of newer younger goalies who will take up the top half of the next version of this list – and a few of those guys say entered the league this year (Andersen) but don’t have enough sample for this projection right now.

Elliott actually showcases a limitation of Hockey Marcels – the four years of data it has on Elliott include Elliott’s only two good seasons in his career (including his incredible 2011-2012) -so it’s including two good seasons and two poor ones.  Of course, Elliott’s 3 prior seasons to these last 4 were all lousy, which suggests that our projections are overrating him a little – we still need to be a bit Bayesian about how we treat these projections of course.  That said, the point of weighting more recent data so much more is to adjust for the fact that players do change and sometimes improve over seasons, and it’s possible that Elliott actually has become a decent goalie.  If a team is thinking about handing a multi year deal to Jonas Hiller or Ryan Miller, it might stop to think if Elliott might be a better (not to mention less costly investment).

Really, the biggest pointer of this is that the best way to obtain goalies is through development and early signings – the UFA market sucks for long term goalie stability due to aging (not to mention the best goalies being locked up through their better years).   For teams without any such goalies in their systems, the road is rough indeed.

About these ads

2 thoughts on “Projecting Future Goalie Performance: Updating and Improving Hockey Marcels

  1. My brother recommended I might like this blog. He was totally right.
    This post truly made my day. You cann’t imagine simply how much time I had spent for this info!

    Thanks!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s