In my last post on neutral zone tracking, I went over repeatability of metrics using the data to measure performance in all three zones (Neutral, Offensive, Defensive).  As I noted, the results didn’t quite measure up to Eric T’s initial hypothesis:

Eric’s initial thesis as to the importance of neutral zone play and the lack of repeatability in offensive and defensive performance certainly does fit the Flyers.  But it doesn’t fit anywhere near as well other teams that have been tracked.  This suggests to me there’s a heavy coaching influence going on here (although the Flyers did shift coaches this year).  Neutral Zone Play is certainly incredibly important.  But teams have seemed to succeed via offensive and defensive zone success WITHOUT neutral zone success, which suggests it may not be AS important as Eric’s initial work suggested.

On twitter, Eric pointed out that his initial posts also dealt with Minnesota Wild data and that it mentioned that the Wild did show some defensive zone repeatability.  Eric however, was skeptical of these results:

The two data sets conflict to some degree on whether players possess a defensive zone skill for suppressing shots against. For the Flyers, the defensive zone metrics all showed up as irreproducible. In contrast, for the Wild, the defensive zone metrics show fairly strong reproducibility.

So what does this mean? Undoubtedly, it means that the picture for defensive zone performance remains muddled. However, there is a reason to mistrust the apparent reproducibility of the Wild defensive zone data.

[snip]

While I generally put quite a bit of faith in statistics, this seems a little bit fishy. If we take the defensive zone metrics at face value, then the conclusion would be that the Wild consistently gave the most ice time to the defensemen who played the worst defense.

[snip]

I am skeptical that the Wild would have used their worst defensive zone performers as their top pairing. So while we definitely need more data to draw firm conclusions [] the combination of seeing irreproducibility for the Flyers and seeing unintuitive results for the Wild leaves me suspecting that defensive zone performance is not reproducible at the season level.

Eric argued this same point on twitter – do the guys with the best Offensive and Defensive Zone #s make sense?  For the Isles, it certainly seems so to an extent – In 2014, he best DZone #s belonged to de Haan, then Hamonic, then Lubo, then Hickey, your #1-#4 Dmen respectively.  In 2013, again you saw your top 4: Hickey, then Visnovsky, then AMac, then Hamonic, as your top 4 D Men in DZone performance.  Pretty logical here.  Offensive zone performance however was a lot murkier and wasn’t as logical.
However, we can do one better in our analysis: since I have two years of Isles Data, we can see if there is repeatability in OZ and DZ performance (and neutral zone performance) from the lockout season to 2014.  The results are below*:

*Note that OZone and DZone Performance are not being measured in the form of DZ100 and OZ100, but the effect is the same.  The Metric is the same, just presented slightly differently – here 50% = average results in Offensive and Defensive Zones while above that is better and below it is worse.  

As we’d expect, Neutral Zone Performance is the most reliable of the three, with a decent sized correlation.  But the R^2 for OZone and DZone performances aren’t THAT small, and are in fact relevant.  The OZone #s may not make the most sense for who the players are (that’s Matt Carkner in the top right), but they do appear at least a bit repeatable.

Conclusion:

On twitter, Eric suggested that certain teams may be able to consistently get above average offensive or defensive zone performance, like the Sharks, even if individuals might not be consistantly above average relative to their teams.  And I believe that’s definitely true.  But again here once again we see some repeatability in offensive and defensive zone performance on an individual level.  So there’s something else going on here.