Hockey Talk is a (not quite) weekly series where you will get to view the dialogue among a few Hockey-Graphs contributors on a particular subject, with some fun tangents.
This week we started from a Twitter conversation suggesting that expected goals calculations (xG) might underweight “shot quality”. A topic that HG contributors are hardly short of opinions on.
dtmaboutheart: Follow @DTMAboutHeart
If there is no good way to measure shot quality then how can anyone test if xG is underweighting it?
mattcane: Follow @Cane_Matt
xG is literally the most direct way to measure shot quality that exists, given the current data. Unless there is a way to measure the information about where the puck was moving prior to the shot, you’re not going to squeeze anything more out of the current data.
nmercad: Follow @NMercad
People talk about shot quality but they don’t really understand it. What it is. What it means. Where and when it is appropriate or worthwhile to consider in analysis.
Chasing shot quality is chasing the dragon.
No coach is going to listen to anyone who says shot quality doesn’t matter. They also aren’t going to listen to someone who says it matters but can’t show how.
If they can’t show how, a coach, a hockey person, will use their own empirical evidence. Your best bet is still to show a coach how in a large sample its impact gets largely washed out.
So don’t focus too much on it in the grand scheme of things. Focus on other things that will help produce more sustained offense. Then let your players create out of that. Use intuitive solutions for quality. Like lateral passes and quick touches prior to scoring plays. Mess with goalie angle lines. Use traffic against your opponent. Etc.
bwendorf: Follow @BenjaminWendorf
Honestly, even with richer data, the improvements to existing shot quality models are going to be small increments. There are no leaps to be had, because the finer you slice the data the more troubling your samples are, and you can’t really escape how heavily shooting regresses.
Which is more a plea to quit with the idea that richer data will transform what we already know. The value in richer data will be in getting more answers related to the “how,” and making analytics more prescriptive.
rjessop: Follow @Thats_Offside
I think if we keep trying to answer whether shot quality exists, we’re gonna keep missing the point because we haven’t framed the question right.
We know shot quality exists. Shootout shooting percentage is higher than PP shooting percentage is higher than 5v5 shooting percentage. It is completely, abundantly obvious that shot quality exists.
The question to ask then is: “why does shot quality have so little relative impact on long-term results?”
My working hypothesis would be the impact of contemporary coaching basically creates an environment where generating shot quality is hard. Everyone sections off the middle of the ice, keeps wingers low, takes away the slot, boxes guys out, has sticks in lanes, etc. So if everyone’s doing this, it’s going to be harder and harder to get to the areas where “quality shots” are generated, so they’re going to be scarce. Also, if everyone is similarly successful playing their system, relative differences are going to be tiny.
If this is right (and it may not be but I have no way to test it either way), we would also be completely wrong to tell a coach that he shouldn’t worry about shot quality. Because it matters a TON.
The entire ecosystem in which we analyze is based on a leveled playing field in terms of shot quality due to contemporary systems. Ignore that, and the sort of shot quality equilibrium that’s been established falls apart, relative differences grow, and you see bigger effects.
The message then isn’t: “ignore quality, focus on quantity.” It’s: “we need to find ways to boost the number of pucks directed the right way within the framework of what we have here.”
Of course, this is all theory and conjecture. Makes sense to me though.
bwendorf: I think coaching systems are attempted workarounds to the general facts of play, instincts, and reflexes in hockey. Which is to say, it is always going to be harder to get shots from closer in, and easier to get them from further out.
jackhan: Follow @ml_han
That’s about right, Rhys. The NHL is hockey being played by generally comparable players, within generally comparable systems.
At the fringes you’ll find the Vaneks and Pat Kanes and the Bobby Ryans. But you can see why just by watching them. In women’s hockey you’ll find more players who really drive on-ice shooting %, but in most cases they drive volume too. Microstats gives you a pretty good ide of why that is.
Guys like Desharnais, Tanguay and (to a lesser extent) Hudler shoot for higher % because they’re more selective. In fact it might be because they’re less confident in their shooting skills. Either they pass off or they try to get closer. Whereas Pacioretty, Kessel and Ovi will be a bit more ambitious and take the shot even if it’s not a gimme.
petbugs: Follow @petbugs13
And which set of players scores more goals? That right there should say everything about chasing shot quality.
Part of the problem I find in the shot quality discussion is the tendency to conflate shot context with shooting skill. It appears that people think they’re talking about shooting skill when in fact it’s usually the context that’s actually producing the observations they’re trying to explain.
The contextual qualities of the shot, like distance, angle, puck movement, screening, whether it’s a rebound, etc. are much more a product of collective offensive system and player creativity than of individual shooter skill. Shooters impact aim and velocity.
That split between the contextual factors, which again are a product of the offensive and defensive systems and the execution of all 10 skaters on the ice, and the shooting skill is what keeps getting left out of the discussion. So trying to find or show the repeatability of the skill component of shooting percentage is a Sysiphean task.
We already know that the impact of shot quality (context + skill) is miniscule in comparison to other factors, but to then try and split it out even further is going to be incredibly complex. Especially when it will be nigh on impossible to control for the myriad of factors that go into the contextual component.
And to be honest, I still have this sneaking suspicion that defensive system and execution has the largest impact on the contextual component. If true, this would mean that what we think of as shot quality is actually a function of the defense and not the shooter. So no wonder it’s difficult to show the repeatability.
jackhan: Still a concept worth teaching. Like faceoffs are worth teaching, even they don’t really drive results. You still want to be adequately prepared.
Simple postgame talking point we had during a video session: outside the dots off the rush, a forward shoots about 2%. Cut inside the dots and you can expect to shoot closer to 8-10%. Much more efficient to take 1 shot instead of 4, so cut in if you can.
Also, in my experience, shooting % for dmen is mostly related to how well they short the zone, not how hard they shoot.
petbugs: And that is exactly the kind of actionable insight the analytics community should be identifying. Chasing the holy grail of shooting skill is a fool’s errand.
bwendorf: I’ll leave it open for someone to say otherwise, but I don’t think anyone here would say shot quality doesn’t exist. I only make a point to say that because I still feel like there are people that believe we say that.
petbugs: Agreed. I think that was something that was said years ago, early on in this argument. But it was quickly amended to “it exists, but the impact is small so you can ignore it for the most part”.
It was an off-the-cuff, imperfect reaction to the suggestion that shot quality was what really matters. But it quickly evolved to the realization that yes, it does exist; it’s just rather inconsequential and difficult, if not impossible, to isolate. So why spend much effort trying to do so?