Look, nobody knows what analytics actually is anyway, so why are we still talking about it? At its most basic, analytics is simply a tool. Much like a hammer is a tool.

Maybe too much like a hammer. As the old saying goes, when all you have is a hammer, everything looks like a nail. The same may be true for analytics. At least in some contexts. Yes, analytics is simply a way to draw meaning out of data, but just because you finally figured out how to apply gradient boosting to your ridge regression model doesn’t mean you should.

Once you think of analytics as a tool, a means to an end, then it’s much easier to see that it’s not just a tool, but an entire toolbox. And when you reach into that toolbox, the tool you take out should depend on what you want to accomplish.

This is where things get interesting. Hockey stats are typically thought of as being either descriptive or predictive, or sometimes both, depending on how they are used. But when you break it down, there are really three contextual uses for data and statistics, whether observed or derived, depending on whether you want to describe what happened, diagnose why it happened, or forecast what might happen:

The data and the analytical technique(s) you choose should fit the use. Data and statistical methods necessary to provide a game recap are typically less sophisticated than analytical techniques that will help you diagnose why things are going the way they are. And those techniques, in turn, will be less sophisticated than the complex regression models and machine learning algorithms involved in predicting future results based on the underlying descriptive data, or perhaps prescribing the changes in behaviours or strategies necessary to produce the intended results.

But while this is where things start getting interesting, it is also where those of us who are new to the field and don’t have the experience and training with advanced statistical analysis will start to fall to the wayside. (And I say this as someone with an engineering degree that included more math and statistics than I care to remember.) The toolbox contains some tools that take time to learn and master. Tools that will look like arcane contraptions that we can’t make heads of tails of. But in the hands of a skilled craftsperson, they are tools that can produce incredible results.

And by all means, if that hard core data analysis and modeling is something you are attuned to, dig in. Learn about it, play with it, apprentice in it, master it. These are translatable skills that are highly sought after, and can lead to lucrative careers. Just not with a professional hockey team.

That said, any NHL franchise could go out and buy themselves an analytics department today. But adding a few people with the skills to crunch numbers won’t in and of itself add value to the critical decisions that are made in the organization. And that’s where the real market inefficiency is.

At least it is in the NHL. That’s not so much the case in Major League Baseball. If you read part two of the McKinsey interview with Jeff Luhnow, the GM of the Houston Astros, you might remember this:

So we were able to start with a fresh piece of paper and say, “OK, given what we think is going to happen in the industry for the next five years, how would we set up a department?” That’s where we started, “OK, are we going to call it ‘analytics’ or are we going to call it something else?”

We decided to name it “decision sciences.” Because really what it was about for us is how we are going to capture the information and develop models that are going to help the decision makers, whether it’s the general manager, the farm director who runs the minor-league system, or the scouting director who makes the draft decisions on draft day. How are we going to provide them with the information that they need and that will allow them to do a better job?

This excerpt really stood out for me and cuts to the core of what a data-driven culture should be about: making better decisions. It’s not about the tools you use, it’s about the decisions you make. And the way you make better decisions is:

1. Have a clear objective in mind;
2. Identify all available options;
3. Evaluate them against each other; and
4. Decide.

Easy, right? I mean, the first two steps are pretty self-explanatory. So much so, that we usually just skim past them and focus on the third one, i.e. reach for the toolbox. But those first two steps are probably the most important. As Albert Einstein once said,

“If I had an hour to solve a problem and my life depended on the solution, I would spend the first 55 minutes determining the proper question to ask… for once I know the proper question, I could solve the problem in less than five minutes.”

As applied to decision-making, this means that most of your effort should be going into defining the problem you are trying to solve, what you want to achieve, and what all of your options truly are. Once you’ve done that, then you can determine the best analytical tool for the options you want to evaluate.

Absent that decision context, stats, no matter how fancy, are just numbers. It’s fine to have lots of data and statistics, but that data has to be in service of the overarching needs and goals of the organization. And the way to do that is to ensure the data and analytics capabilities are deployed in support of making better decisions. Because ultimately, it is the decisions made every day at all levels within an organization that determine whether it meets its long term goals.

What does this mean for you, the amateur hockey data analyst?

It means start with a decision that you might expect a GM or coach to have to make:

• should they trade for player A?
• what formation should they use on the power play?
• who should they target in free agency?

Consider the objectives are they trying to achieve. What metrics would help you evaluate whether those objectives are met? Now consider the options available to them and collect the data needed to assess them against those metrics.

When you can design the analytical process in support of making a better decision, you are that much closer to being in a position to provide relevant input into the process.

This is the first installment in a series on applying best-practices in data-driven decision-making to the world of professional hockey. You can find the next installment here.

Follow me on Twitter: Follow @petbugs13