Photo by Arnold C, via Wikimedia Commons

I think it’s fair to say that people familiar with hockey scouting and stats analysis know that there is a bit of a rift between the two (not unlike what exists in baseball). The former, as in baseball, has a long history as the standard in hockey analysis, being at-or-near the forefront of drafting, trading, and free agency decisions for teams. The latter is expanding its reach exponentially into league offices, and has many a pro-stats person questioning the abilities of scouts to analyze players (and vice versa). There are at least preliminary attempts to reach out, on the part of Corey Pronman at Hockey Prospectus (and ESPN), but scouting and stats analysis both have a lexicon, methods, and best practices, and devotees of one probably don’t have much time to develop proficiency in the other.

Yet, therein lies a problem and a solution. There is a common thread between these two groups, the desire to usefully analyze hockey players. They each have their own approach, but neither necessarily contain such complicated concepts that they cannot be read by a conscientious analyst. But most importantly, they have something to offer one another that could improve both areas of analysis.

Scouting, Cross-Checking Results, & the Value of Possession

Pronman was early to the game on the latter of these; he recognized the value of stats analysis, found a useful ability to build into his analysis, and he’s earned a strong following with his work (and taken it to ESPN; hey maybe they’ll like hockey someday). I do have to say, being on the other side of the stats-scouting situation, he’s probably the first person I know who crossed that line and did something quite useful.* And to this day, he engages stats debates and otherwise is one of the most prolific scouts I know who regularly examines players using scouting and stats lexicon.

But that’s not the only contribution stats can make. In recent years, there have been a large number of analyses that looked back on draft results and worked towards finding the values of draft picks (trade value as well). As Pronman has pointed out, though, a hindsight or results-based analysis of draft picks doesn’t peel enough layers off the onion; each scouting group (plus coaches, GMs) has an intention going in, and a selection is sparked by particular reports and (sometimes) data. Stats could help shape pre-draft evaluations into a tool that can be cross-checked, so scouting teams can be assessed more accurately (rather than just by rudimentary results analysis). Do you anticipate this person to be a strong possession player, or a player that’s strong offensively but needs to play sheltered minutes? Do you anticipate this player playing all-minutes, or will they more likely be shorthanded specialists? Do you project this player as a top 4 defenseman? Do you anticipate they are league-ready, or will take X years to reach their peak? To a degree, these questions are asked already, but I have as of yet to see it matched up with earnest Corsi or Fenwick-based analysis. And for the record, I think Pronman provides a promising starting point to getting there.

Cross-checking can also help scouts determine just what observed skills are translating to success at the NHL level (from what it sounds like, this was something that influenced Pronman). Does Bob the Scout have a clear picture of how X player’s “compete level” would contribute to future success? Over time, has that compete level demonstrated itself in any way? Is possession observable? If it is, certainly scouts should join Pronman in making it part of their assessments.

Stats & the Importance of Intuition

“Watch the games, genius.” … so I was told by THN’s Ken Campbell not so long ago. And it’s true, that watching NHL games is important. He was off the mark though, as I not only watch games but have also watched them again and again, tracking their stats and quantifying actions. I’ve also played the game since I was 3 years old, refereed it for seven years, drove the Zamboni for five years, and even coached for a couple. I come from a hockey family. My father and a friend started the youth league in our area, coached in it for fifteen years, and continues to referee. My sister played since she was five, and played boy’s high school hockey. My brother played competitively for eight years, and we still play pickup outdoors in the winter. I have over 2,000 hockey cards that I’ve read religiously and sorted by a dozen different variables, was editor-in-chief (as well as contributing author) of a Winnipeg Jets inaugural preview magazine, and I have been writing for hockey stats blogs for four years. In other words, I have lived the game from every imaginable angle, for as long as I can remember. I have intuition about the game, about what works and makes sense; it’s this same intuition that drew me to the possession-focused analysis I saw at Gabe’s Behind the Net blog long ago.

Intuition is a vital component of stats analysis. It is crucial to have background knowledge of what you are analyzing, because when you begin to observe trends and decided what data is relevant, your intuition will guide your assessment of what’s important. More importantly, your intuition will press you to take a critical eye to your work: Does this match what we’ve seen from X player? (many stats analysts will speak to whether a positive Corsi player “passes the eye test”) Does it seem beneficial to carry the puck in the zone rather than dump it in? Intuition opens areas of inquiry, theory; it brings us to test factors, and come to conclusions that have changed the game as we know it. So I have to laugh when I hear that “watch the games” comment, because there are few stats analysts in hockey that don’t watch games, primarily because they know the importance of intuition and “the eye test” for stats work.

But not everyone can do all the things that I’ve done, and I’d be the first to say you don’t need to in order to develop good intuition for the game. As some recent journalism has revealed, simply having a label like “hockey analyst” for 30 years and watching games doesn’t mean you have a strong intuition for hockey analysis. The intuition comes from active watching and observation, and corroborating what matters statistically to what seems to be having impact on the game. I can guarantee you that nobody watches the game (and particularly, players) much closer than scouts, and furthermore scouts provide the largest body of literature anyone has on each player in the NHL (and those looking to break into its ranks). That information is the epitome of “the eye test,” and can greatly inform the intuition of a more holistic form of hockey analysis.

Conclusion

In sum, while in some cases stats analysts and scouts have found some common ground, too often it seems that these two sides are wary of one another. My hope is we can realize the value of both; I can think of no greater example than eliteprospects.com, where nearly every player in the modern era is presented with stats back to Mini-Mites and a scouting report blurb. Take Raphael Diaz’s page, for example. It’s just a blurb, surely not the entire book on Diaz, but doesn’t it tell you so much more about just who he is on the ice? Get eliteprospects.com some possession-focused data, and I think we would have the leading edge of hockey analysis.

Hey, at least it’ll help snuff out those immature “watch the game” comments.

Postscript

Corey Pronman made a good point on Twitter not long after I published this, that a good demonstration of possession-focused scouting is that, generally speaking, NHL coaches seem to play better possession players more. Gabe Desjardins has said as much, and Tom Awad used even-strength ice-time as the basis of his great “Good Players” series. I think Darryl Sutter sums it up perfectly here (starting at 1:50).

*Of course, there are cases where coaches have done this in the name of scouting, so to speak. Harry Sinden, Jim Corsi come to mind. And certainly, stats bloggers in the past (Alan Ryder, Ken Krzywicki, Vic Ferrari) believed in the scouting value of their NHL-focused analysis, and Gabe Desjardins put together a very rough set of minor/European/collegiate league equivalencies. But none of these were completely earnest efforts to bridge the divide.