Hockey Talk: Why the layman eye-test and results sometimes differ for defenders?

Patrick Wiercioch - Ottawa Senators.jpg
Patrick Wiercioch – Ottawa Senators” by Lisa GanskyFlickr: IMG_6499. Licensed under CC BY-SA 2.0 via Commons.

Hockey Talk is a (not quite) weekly series where you will get to view the dialogue amongst a few of the Hockey-Graphs’ contributors on a particular subject, with some fun tangents.

This week we look at what makes up an effective defensemen and why the eye test sometimes differs from the results:

Jack Han (@ml_han): What are some things we should care more (or less) about when evaluating NHL defensemen? Some case studies Carolyn and I have been discussing: Kris Russell, Jakub Kindl, Patrick Wiercioch, Jordie Benn; Sheldon Souray. And I’d add Slava Voynov somewhere in there too.

Carolyn Wilke (@classlicity): Ahh, see I thought we were gonna go with “fancy stats darlings that fail the eye test” which is pithier.

Micah Blake McCurdy (@ineffectivemath): This business about Wiercioch grates me the wrong way somehow. He doesn’t have great “underlying numbers”, he has great shot results. They’re outputs, not “under” anything. ^c^v

Jack: I appreciate Wiercioch as a player. But ultimately hockey is a game of goal differential, not shot differential. Not saying it’s the case for Wiercioch in particular, but at a certain point, an ability to drive shot differential can be washed out by enough bird-brained plays in the D-zone. Coaches emphasize that, perhaps too much, but there is a break-even point. Hence the “Wiercioch Line”.

Same idea with Jakub Kindl, who not many people in Detroit love on that team despite his pretty decent shot differentials over the years.

Micah: Wiercioch goal differential has always been strong too. His results have always been great.

Conor Tompkins (@null_hhockey): Coaches might be getting caught up on things they can coach. Pivots, skating, wall battles. They value those skills because it reinforces their job security. Those things are only important so far as they lead to the results we care about.

Micah: They can coach passing and positioning, too. That’s how guys like Wiercioch succeed.

Jack: At that level, some things you can teach, and some things you can’t. Positioning and decision-making might be teachable. We can pretty much see that the things that drive Corsi percentage are much less so.

So yes PW has value for pretty much every team out there. But there are other players with similar tendencies that maybe you wouldn’t want.

Conor: I think the main issue is that we have a very limited ability to gauge defensemen, so choosing some traits seems arbitrary.

Jack: Of interest: Kris Russell’s HERO chart BEFORE he joined Calgary:


And now Russel in Calgary:


Conor: We’re looking at all these inputs (passing, positioning, wall battles, shot) but don’t have clarity on how to weight them.

DTM About Heart (@dtmaboutheart): I don’t think Russell was ever that impressive. Career GAR of -0.33. I also put little to no stock in a defencemen’s production numbers save for a few elite players (ex: Karlsson, Subban, Byfuglien).

Jack: So if a guy like Russell, but at age 24-25 is on the market, what’s one to do? Great shot numbers, not enough of a sample to tell with goal numbers, maybe has room to grow if you give him more ice time.

Prime example of this being Jordie Benn.

DTMAboutHeart: Short term, low AAV.

Carolyn: Benn’s one of those guys who does a great job driving possession via zone exits – he likes to carry out and pass through the neutral zone to a forward, but the longer he plays, the less crisp he gets mentally.

Jack: What if you’ve got no choice, but to give term and money or pass. Would you assume Dougie Hamilton projects better because he’s been “vetted” by traditional scouting and was a high pick?

Nick Mercadante (@nmercad): We are a bit all over the place here. This is like 10 topics in one. Defense is more complex and positioning/system driven than offense.

Some guys have high aptitude for gap control entering zone, less so for gap and area control in zone. So they may not show well in a zone style system. Or vice versa. Don’t gap control on the rush well and get burnt, leading to visible big mistakes. But maybe their in-zone coverage is more solid.

There is no one build that is successful. You have to know your system and know your players’ strengths and weaknesses. Seek not to expose them as much as possible.

DTMAboutHeart: Dougie is just a better player so you can’t really compare them.

Carolyn: Defense is SLIGHTLY more system driven, but player aptitude makes a huge difference. Coaches who stay with the same team for years can see large swings in shots against during their tenure.

Nick: Many coaches implement their system their way. Expect one size fits all.

GMs and coaches have to know why something works for what they are driving (easier), or inject complexity and train players to recognize what to do when certain personnel are on ice (harder).

I tire of analysis that doesn’t consider environmental (team strategy, system) impact on performance. Especially when the player has only primarily been in one environment. Especially when it’s defense, and often much more environmentally driven than offense.

Most offenses are – enter zone this way, forecheck with these numbers, beyond that be creative. Utilize your skills. That’s how offense is practiced in large part at all levels.

Defense is 5 guys working more as a unit to pick up coverage. Control areas of ice. Gaps. Mobilize upon a turnover to breakout and transition.

So if we don’t analyze a defence within their given setting, and scout to pinpoint particular skills better/worse suited for it (or for another system), we probably aren’t analyzing with enough context. And I think this is where that “underlying numbers” trope comes from. Coaches want more to inform their decision. Doesn’t mean Micah is wrong to criticize it. He is right in saying they aren’t underlying. They are the results. Just that coaches want more analysis in context. For better or worse.

Carolyn: Well, I think that’s part of what we’re talking about here, especially with respect to Russell & Benn.

The numbers they put up – the results – change depending on usage. Which is why I was being particularly flippant about fancy stats vs eye test in this topic.

And Wiercioch too. There’s a tendency for fanalysts to say “this guy has amazing stats he should be 1st pair D” and not take into account what a guy does poorly that keeps him on his pairing.

I think the main frustration there is that guys like Ceci and Phllips were getting bigger minutes over a guy that looks bad but gets results. The Wiercioch line is that line between “fancy stats shows more than eye test” and “eye test shows more than fancy stats” regardless of system.

Nick: Good point above about fanalyst lamp post.

If Hamilton doesn’t pull it together in Calgary there’s a good argument that he doesn’t work in Hartley’s system.

It doesn’t make him a “system” player (a guy that only succeeds because of a given system), he just may not be being used correctly as compared to as he was in Boston.

Nick: I think a big issue is that a lot of NHL coaches aren’t very good at analyzing talent and making the best use of them. They rise up the ranks because they are seen as experts in their profession. It would make sense that they like things their way and don’t want to change them to suit talent at disposal.

Or they rise up the ranks based on past results. So they continue doing those same things. They are not adaptable. This mindset is pervasive in coaching circles.

Garret Hohl (@garrethohl): I just enjoy every coach saying Enstrom is great, and Enstrom getting great numbers, and Jets fans just absolutely hating on Enstrom.

Ben Wendorf (@benjaminwendorf):  I love Toby. Forever.

Nick: Many coaches also suffer from warm blanket loyalty. Girardi is an example.

He had that one year where he seemingly carried the defense because everyone was injured. Played way above his norm. Still not particularly good. But insane minutes and better than any other time in his career. Hank and everyone else lauded it. It made his career.

Jack: So what should coaches look for when evaluating talent then. One thing to say they’re bad. Another thing to figure out how to get better.

Ben: Re: Wiercioch opinions…why isn’t there a yarn about him that says “It isn’t pretty, but it gets the job done.”

I mean, that was what carried Douglas Murray through his terrible career. Why does it work for some, and not others?



Conor: I don’t think you can make Girardi better. You can drastically reduce his minutes (third pair, and he’s actually good on the PK) to reduce his negative impact on the team.

Nick: Right. But it won’t happen. Because warm blanket loyalty.

“He did this amazing thing for us once so I’ll never fire him.” Happens in all professions.

Conor: If I’m a coach, I look for the ability to do basic offense-driving activities while under pressure: breakout passes. ability to skate out of danger. A guy that doesn’t just make the safe dump off the glass when a forechecker enters his zipcode.

Of course its tough to tell whether that’s ingrained in a player or if that’s the work of previous coaching.

Sather may not have fired him, but you have a new GM now. Turnover will probably be high next year.

Zone exits
Zone entries
Denial of exits
Denial of zone entries

Jack may have said it right when the largest factor in defenders’ impacting the game is their effect on zone transitions.

Carolyn: Conditioning.

The fact that defensemen play more minutes than forwards is very important. Guys like Keith are just as good at minute 22/23 as they are at minute 10, and that’s conditioning.

Nick: Net front coverage is very important. Ability to disrupt passing lanes for premium opportunities. Anticipating coverage. Boards work on loose pucks.

Sharks fans were very frustrated with Brent Burns’ defensive work last season (and in his not-partnered-with-Paul-Martin games this year), and I couldn’t help wondering if it was just too great a change to make. His offensive skill made for tremendous Corsi numbers, but in his own end he struggled with positioning and making good decisions.

Outlet and stretch pass accuracy and decisiveness. Gap control and taking away middle of ice. Forcing/angling offense wide. Disrupting the blue line.

Burns is an interesting case: what happens when a talented defenseman plays forward for 2 seasons, and then gets pushed into a top 4 role at the back?

Jack: Andrei Markov played C until the year he came to North America (or the year before). So it’d be interesting to see if that transition could create some good defencemen.

Jonathan Ericsson another example.

Carolyn: John Klingberg was a forward until about 15.

Jack: One thing Id see is that a kid who plays forward never learns bad habits like putting the puck off the glass and out and generally gets more touches with the puck in different situations.

Sean Tierney (@SeanTierneyTSS): At the risk of going all Stars here, I find Goligoski interesting.

Carolyn: Goligoski does a lot of the little things right but often gets called out for having “deadly” turnovers.

Sean: Right – Goligoski gets virtually no publicity relative to what he’s able to do. Instead, the turnovers story seems to follow him.

In that sense, he’s the West’s answer to Jake Gardiner. Turnovers overblown in a way that masks their true utility.

Gardiner -does- make his mistakes. It seems to be a product of having the puck on his stick so often and being unafraid to be creative. I don’t know Goligoski as well as Gardiner but wonder if his contributions are overshadowed in the same way.

Jack: Goligoski’s so underrated that hockeydb doesn’t even know he shoots left.

Carolyn: I think, and this is true of forwards too, turnovers can be over blown. The more you have the puck, the more chances you have to turn it over.

I believe Datsyuk has one of the highest “give aways” per 60 in the league. He also has one of the highest “takeaways” per 60.

Sean: Mark Stone was listed similarly last year. I guess the question boils down to “fun” high-event play vs. “dull” low-event styles.

Jack: …or holding the puck vs dumping it.

Almost every dump is a giveaway anyway.

Carolyn: Fancy Stats like Goligoski and the average fan’s “eye test” doesn’t. Falls UNDER the Wiercioch line. Although, Wiercioch falls under the Wiercioch line.

Who is the player best represented by eye test meets fancy stats then? Who did we get EXACTLY right?

Jack: Depends who you ask.

Sean: Karlsson and Subban have advanced stats that essentially confirm what we see, if that’s what you mean.

Jack: McDonagh maybe. Everyone loves him but he teeters between underrated and overrated… or Duncan Keith.

Someone from a big market with a big reputation, who’s not particularly underrated.

Carolyn: The Subban-Karlsson Median?

Nick: McDonagh will never get fairly assessed until he can be free of his shackles.

From earlier: I think a lot of forwards would make good D. Many young players end up on defense for lack of foot speed or lack of dangles… or if they are just big.

Literally. That is a ton of the position choice assessment early on.

Carolyn: Ugh, too accurate.

Jack: One of the coaches at McGill I work with used to coach elite minor hockey in Montreal. One year, his #1 D was Alex Biega and Jason Demers was on the 3 pairing. They were maybe 15-16yr old.

Nick: Some forwards would make great defenders. If they can skate w their head up and make accurate passes that is half the battle.

I would look for some playmakers that are good at assessing spacing with and without the puck. And accurate decisive passing. Those guys have skills transferable to D.

Bonino always used to want to play defense, because it has a bunch of his best attributes. He knows how to control space. Anticipate plays and jump lanes. Gap cover puck carriers to usher them into areas that aren’t dangerous. Make quick and accurate heads up passes up ice.

He used all these as a forward. It is part of why he is so trusted on penalty kill.

Sean: In reverse, would Franson make an even better forward because you eliminate the backwards skating dilemma.

Jack: Who would he make a forward pass to, then? I don’t really see it.

Conor: The net… kidding- I don’t think he’d make a good forward. Engelland might be a better 4th line forward than defenseman.

Nick: I don’t know enough about Franson to say either way. But yea some of these guys aren’t dynamic backwards skaters. It kills Girardi. And Marc Staal a bit.

Staal would be a good forward like his brothers. He just isn’t because somebody had to play defense when they were kids.

That would actually be an interesting article.

Garret: Back on the conversation of “turnovers”, I also find people struggle with losing puck vs losing puck relative to ice time or possession. You instinctively notice aggregation of events, but miss rates.

Enstrom and Byfuglien lead the Jets in defensive zone possession losses, but are the Jets best relative to TOI and to puck touches.

Carolyn: I’m getting my friends up to speed on fancystats. That’s the first thing I told them: throw the aggregate RTS out the window.

Jack: …or giving up possession in ways other than a classic tape-to-tape turnover.

Some defensemen could have a much better Corsi% if they develop the habit of circling back and looking for a fresh linemate to make a pass to, instead of dumping the puck in at the redline when they/their teammates are on a change.

What they’re doing when they dump it in is technically a turnover but just far enough from their net that no one really notices. Then their teammates spend half a shift skating their bags off trying to get the puck back.

“Petbugs” (@petbugs13): (too Carolyn’s earlier comment on TOI) This is something I’ve always wanted to look into but don’t have the time or R skills

During Gillis’ time here it when he and AV were all into sleep doctors and all kinds of stuff to squeeze extra performance out of players, they apparently had a hard TOI limit for Kevin Bieksa. They had concluded that his performance dropped significantly after the 25 min mark.

Would be interesting to evaluate player performance against TOI on a wholesale basis to see if there is something there.

Carolyn: I’d imagine that it’s true for certain players (and teams) and less for others. I know I’d read a lot about the Blackhawks, conditioning and nutrition, and how they were basically ahead of the game with a lot of stuff and they definitely thought it gave them a competitive advantage.

Petbugs: Way late in this topic but I will add one thing that I don’t think I saw mentioned.

One factor that I do think can play a big part in the shot metrics is the product of coaching system and player decision making in the offensive zone, specifically to do with pinching.

If you’re playing in an aggressive pinching system and have the smarts to know when to take that risk and when not to, it can have a huge impact on your shot attempt differential.

And perhaps mask poor decision making in the defensive end.

So you wind up with good “underlying” stats but look awful through the neutral zone and/or in your own end.

Conor: I think I’ve moved away from straight CA60 or any relative derivations as a measure of being good defensively (as opposed to never being in your own zone). Scoring chances against per 60 as a % of Corsi Against per 60 makes more sense.

Jack: That’s an interesting thing to know. Suspect it would make the guys liable to get hemmed in their end look a bit better than they are.

You’d want to look at both on a scatter plot. Who is never in their zone vs who gives up good chances when they are in their zone.

Jack: Hence itd be an interesting thing to know

Arik Parnass (@arikparnass):  Yeah studying how TOI relates to performance is something ive wanted to do for a while, but not sure how u could control for all the necessary factors.

Jack: Maybe you’d get somewhere by taking a large enough sample of relative performance in 1st, 2nd, 3rd periods

Arik: Maybe take results in mins ex. 20+ for a player relative to their mins 1-20. Feel like i thought of that before and it’s missing something though.

Jack: It’s something that would be easier to tease out with body monitors than PBP data.

If physical output goes way down past certain games relative to teammates.  That’s on average. So most players will be above or below that curve.

Carolyn: I mean, that’s something that was alluded to at RIT. I think it was something like 20% of speed? 15% of speed is lost by the 3rd period?

Jack: Teams already know certain things from the beep test and other physical evaluations

How much those things matter is a question our coaches certainly keep it in mind when tracking progress.

Arik: Yeah need to know how it affects each player, and how that affects results.

Carolyn: Actually, Hilary Knight and the US women’s Olympic team have done interviews about nutrition too. I seem to recall Vitamin D being a big thing? but I read that like over a year ago.

Jack: The most interesting thing I’ve read regarding nutrition and defensemen specifically was Al MacInnis and Barret Jackman around 2001. In ESPN magazine, IIRC.

Talked about Macinnis taking Jackman under his wing and specifically dietary changes Jackman made as a rookie – meat and veggies as opposed to high-carbs.

First I’ve ever heard about athletes in hockey doing low-carb diets.

Macinnis was saying how when he was young, a postgame meal was beer and a plate of chicken wings.

Nicholas Emptage (@npemptage): There is of course the famous anecdote about Mario Lemieux’s “training” diet.

Lemieux was known for not caring about conditioning early in his career. One year when he was still young, a reporter in training camp asked him what changes he made to get in shape for the season. His reply: “I leave the ketchup off my French fries”.

One thought on “Hockey Talk: Why the layman eye-test and results sometimes differ for defenders?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s