Introducing Box Plus-Minus

EDMONTON, AB - OCTOBER 23: Connor McDavid #97 of the Edmonton Oilers skates during a game against the Washington Capitals on October 23, 2015 at Rexall Place in Edmonton, Alberta, Canada. (Photo by Andy Devlin/NHLI via Getty Images)

(Photo by Andy Devlin/NHLI via Getty Images)

This is Part 3 of a 5 part series detailing the my WAR model, Part 1 of the series can be found here and Part 2 of the series can be found here.

Introduction

Box Plus-Minus (BPM) is a box score-based metric for evaluating a hockey player’s quality and contribution to the team. It is very different than an Expected Plus-Minus type model, which is a play-by-play regression metric. BPM relies on a player’s box score information to estimate a player’s performance relative to replacement level. Box Plus-Minus type metrics have long populated basketball circles, there is a great summation of some of the original creations here, with many newer versions popping up including Dredge, DRE and Player Tracking Plus Minus. A version has even already been brought to hockey in the form of Game Score. Here I will attempt to create my own version of Box Plus-Minus for the NHL.

Methodology

We are going to set up our model similarly to most box scored-based metrics. The setup of our model will look like this:

  • Response Variable
    • Long Term Offensive GPM (for offense) and Defensive XPM (for defense)
      • Long Term here being 9 years (2007-2016)
  • Explanatory Variables
    • The relevant box score metrics:
    • Offensive BPM
      • Individual Goals per 60
      • First Assists per 60
      • Second Assists per 60
      • Individual Expected Goals per 60
      • Individual Fenwick (unblocked shots) For per 60
      • Individual Expected Fenwick Shooting Percentage per 60
      • Quality of Teammates
      • Giveaways
      • Takeaways
    • Defensive BPM
      • Giveaways
      • Takeaways
      • Blocked Shots
      • Hits For
      • Hits Against

The purpose of using long term statistics here is to achieve maximum stability with our numbers and properly determine the weights for each of our box score metrics. A difference between our model here and typical basketball models is that we will use Goals Plus-Minus (determined using the same methodology discussed here, except the target variable in that equation will be actual goals instead of Expected Goals) for estimating our offensive weights and Expected Plus-Minus for estimating our defensive weights. The reason for this is due to the overwhelming evidence that NHL players have a minimal to zero effect on their on-ice save percentage. Therefore, using Defensive XPM instead of Defensive GPM will give us a more accurate target variable. 

Another large difference between this BPM model and others is that we will determine our ratings and weights using machine learning techniques instead of a traditional linear regression. The choice to use machine learning leaves us with a trade off as it will most likely increase the predictive accuracy of our model, but at the cost of making the impact of our inputs harder to interpret compared to a linear regression. There were four machine learning techniques used to create each BPM rating; Random Forest, Generalized Boosted Regression, Generalized Additive Model and Neural Network. The model was run separately for even-strength and special teams, offense and defense as well as forwards and defensemen. Each of the four machine learning algorithms were run separately and then combined to produce the lowest error. While, we can’t see the direct value of the weights we can extract the “importance” of each of our metrics for our Generalized Boosted Regression and our Neural Network. The plot below shows the importance of each metric based on those two models.

screen-shot-2016-10-25-at-4-08-51-pm

A common hockey adage is that you can’t hit if you have the puck. Therefore, in order to try and maximize the value in our box score metrics they were possession adjusted using the sigmoid method found here. Variables that were sigmoid adjusted are noted with the prefix “sig”.

An interesting observation for all offensive categories is that assists were more important than goals for both forwards and defensemen, especially since this is the opposite of how the values are weighted in Game Score. Looking at the forward offensive variables, the relationship between actual goals scored and expected goals scored seems to flip depending on whether we are looking at even-strength or power-play. This will get touched on again later, but Defensive Box Plus-Minus ends up being more of a fun exercise than a useful tool, mainly due to the lack of reliable and useful data to evaluate defensive play found in a box-score.

Box Plus-Minus 15/16 Leaderboard

Data will be released for public consumption in the final section of the write up. BPM is presented as a per 60 rate stat. Below are the top 10 offensive and defensive performances from the 2015-2016 season for both forwards and defensemen according to BPM:

screen-shot-2016-10-25-at-6-35-11-pm

screen-shot-2016-10-25-at-6-35-25-pm

screen-shot-2016-10-25-at-6-35-35-pm

screen-shot-2016-10-25-at-6-35-48-pm

Conclusion 

A huge improvement to this model could come in the future when more data (player tracking) becomes available. It could also be valuable to have this metric reconstructed either with different variables or using a different method. A linear regression would allow a greater insight into the exact influence of each metric included in the regression. I plan on releasing the long term GPM and XPM data required to create such a model and encourage the public to try and improve on the methodology I have laid out here.

Please let me know if you have any thoughts, questions, concerns or suggestions. You can comment below, reach me via email me here: DTMAboutHeart@gmail.com or via Twitter @DTMAboutHeart.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s