In February, I introduced my hockey version of Baseball’s Marcel forecasting system – a system that uses the last few years of a player’s career, weights the more recent seasons more heavily, and uses that to project future performance.  In particular, I was using the system to take an attempt at projecting goalies – who are of course the most unpredictable of hockey players.

Now that the season is over, it’s time to take another stab at goalie projections using Marcels. However, we can do better than we did last time: last time, our projections used Eric’s weights plus a mostly arbitrary regression mechanism. This time, we can use a more realistic (and non-arbitrary) regression as well as attempt to account for aging as well. In short, we SHOULD be building a better projection system for goalies, the most unpredictable of players.

METHODOLOGY (Skip if you just want the results, but it’s important)

The earlier post was building upon Eric Tulsky’s work which found that the following weights gave the best results at projecting future performance by goalies over the next three years:

So in my base case, I’m using years 1-4 to try to predict years 5-7. The best predictions came from weighting things like this:

• Each shot faced in year 3 counts 60 percent as much as shots in year 4
• Each shot faced in year 2 counts 50 percent as much as shots in year 4
• Each shot faced in year 1 counts 30 percent as much as shots in year 4

Now as we noted in the last post, simply using these weights isn’t enough to make a projection of a goalie – goalie performance is simply too affected by random events (well this is true for non-goalies as well, but especially so for goalies).  As such, we need to regress the goalies’ #s toward the mean (in this case a SV% of .9142) in order to compensate for this.  Naturally we regress less for goalies with higher samples and more for those with much smaller samples.

In the previous post we did this simply by adding shots at the league average rate till we hit 4000 shots, which was basically arbitrary, and had the issue of leaving some goalies with basically no regression at all (which isn’t likely even for goalies with near 4K shot samples).  Instead, we should regress by adding the same # of shots to every goalie, and determine that # by using the correlation between goalie performance from one year to the next (and using the average goalie sample size to find out how many shots said regression would take).

In short, we should add 1525 to every goalie’s weighted sample, all saved at the league average.  The end result of this is that the 49 goalies in our sample were regressed on average 40% to the mean, with Ryan Miller’s #s being 26.97% regressed to the mean, while Viktor Fasth’s #s facing a regression of 67.4%.  In other words, we’re basically regressing Fasth two thirds to the Mean here due to his small sample size, while Miller is only being regressed a little more than one quarter.

But while adding regression is necessary to make a projection, a projection system needs a third component: a way to adjust for aging.  We didn’t do this in our last attempt at Marcels because we didn’t have an easily accessible goalie aging curve available.  As it happens, we just looked at goalie aging last month.  So now we can add this in (if you’re curious, I’m using the goalie aging line discussed in that post rather than the curve.)  And this gives us our completed Marcels.

So without further ado:

THE PROJECTIONS (If you skipped the methodology, stop here):

Note that the below are projecting performance over the next three years. You’ll apply less of an aging adjustment (and slightly different weights) for one year projections.

A quick explanation about how to read this table.  The first SV% column is the Raw Marcel projection before any aging or regression.   As such, there are some silly small sample results here  (Anton Khudobin, 3rd best NHL goaltender, .9259 SV%.)  The second SV% column are the Marcels after the regression talked about above.  Basically, those are the #s we came up with in our last post, albeit now we have a more solid regression method.

The third SV% column, the one in bold, is the most important one however – that’s the completed Marcel projections when we include in the effects of aging.  The effects are dramatic – only two goalies 30 or over remain in our top 20 goalies over the next 3 years – and those two are ages 30 and 31 (not exactly that old).  Moreover, only 11 of these goalies are expected to be better than average over the next three years.

The final column by the way details the amount of regression done on the #s of each player – the higher regression %s come obviously from smaller sample sizes, and vice versa.  The lower this number the more confident we are in the result – We’re much more confident in our projection Bobrovsky as a .9165 goalie going forward the next 3 years than Anton Khudobin being a .9164.

SOME THOUGHTS ON THE PROJECTIONS:

First of all, Tuukka Rask is by far the league’s best goalie and should continue to be so.  He leads in raw, regressed, and complete marcel projections, and the #2 goalie going forward in the complete Marcels isn’t even close  (almost .003 away!)   There’s also a gap between Cory Schneider as the #2 goalie in the league and Bobrovsky as the #3 and then the pack becomes much more muddled.

Second, due in large part to the aging adjustment, finding goalies in free agency to act as a multi-year starter is basically a losing play.  No UFA goalie is projected to be above average over the next 3 years, and only three: Jaroslav Halak, Brian Elliott and Thomas Greiss project as being in the top half of these forty nine goalies going forward*   Greiss of course is partly the result of being regressed almost 60% while there are reasonable concerns about Elliott.

*If you’re curious about the discrepancy between the above statements, we will have over the next three years a bunch of newer younger goalies who will take up the top half of the next version of this list – and a few of those guys say entered the league this year (Andersen) but don’t have enough sample for this projection right now.

Elliott actually showcases a limitation of Hockey Marcels – the four years of data it has on Elliott include Elliott’s only two good seasons in his career (including his incredible 2011-2012) -so it’s including two good seasons and two poor ones.  Of course, Elliott’s 3 prior seasons to these last 4 were all lousy, which suggests that our projections are overrating him a little – we still need to be a bit Bayesian about how we treat these projections of course.  That said, the point of weighting more recent data so much more is to adjust for the fact that players do change and sometimes improve over seasons, and it’s possible that Elliott actually has become a decent goalie.  If a team is thinking about handing a multi year deal to Jonas Hiller or Ryan Miller, it might stop to think if Elliott might be a better (not to mention less costly investment).

Really, the biggest pointer of this is that the best way to obtain goalies is through development and early signings – the UFA market sucks for long term goalie stability due to aging (not to mention the best goalies being locked up through their better years).   For teams without any such goalies in their systems, the road is rough indeed.