*This post is co-authored by DTMAboutHeart and Ryan Stimson*
Recently, the topic of Quality of Competition has been at the forefront of Hockey Twitter. This post hopes to articulate some of the nuance associated with Quality of Competition, as well as Quality of Teammate, metrics and how impactful they are. To do that, we will revisit methods outlined here by Eric Tulsky, namely splitting the competition and teammate quality by position and measuring the impact of each split. Ryan recently wrote about this at the NCAA level, but it has not been looked at with much rigor at the NHL level.
Both Quality of Competition and Quality of Teammates matter. They also don’t matter. It depends on the position and metric you’re looking at. All TOI data is 5v5 and from Corsica. Ryan had the game files of who was on the ice during each 5v5 shot from Micah Blake McCurdy, so that data was used as well. Also, thanks to Muneeb for feedback during this process. Thanks to all!
The Problem
How an analyst accounts for the quality of a player’s given competition and teammates is the crux of this issue. In the past, we’ve looked at the distribution of QoC and QoT primarily by CF% and TOI of the players. A distribution is often helpful, but shouldn’t be the endpoint for your analysis. Take the following for example.
These are the distributions of the QoT and QoC, based on ice time of the opposing forwards and defense as well as the forward and defense teammates. You’ll see that once you break it down like this, the distributions are slightly different based on the position you’re evaluating. So, we need to account for this when we analyze the impacts these metrics have on performance.
The Methodology
What we did was take the 5v5 TOI% a player had given the available 5v5 TOI he could have played. We then used this to create a TOI/60 metric for each player. Then, whenever a player was on the ice for a shot, that shot was weighted by the TOI/60 of each player on the ice (opposing players fall into the TOI QoC metric, and teammates fall into the TOI QoT metric). However, a point of departure from more common QoC/QoT metrics is that rather than two, we create four. For each player we then have an approximate value of the Forward and Defenseman QoC and QoT that a player faces or steps onto the ice with.
We then ran a multi-linear regression with all four metrics for both forwards and defensemen. We added in the team’s CF/60 to control for team effects when looking at offensive production, and replaced it with the team’s CA/60 to account for team defense. Only players with more than 200 5v5 minutes from the 2015 – 2016 season were included.
The Results
Below is a table showing the adjusted R^2, coefficients, and p-values for each of the regressions.
So, how you would read this is that a forward’s Fwd QoC, Fwd QoT, Def QoC, and Def QoT, while controlling for team effects, will explain 69% of his shot rates for. Upon looking closer at the regression, we see the bulk of the significance is due to the defensemen he plays against (Def TOI QoC) and the forwards he plays with (Fwd TOI QoT). They appear equal in importance. The defenders he plays with and the forwards he plays against do not appear to have much significance.
When looking at a forward’s shot suppression, the forwards he plays with and plays against appear to be the significant factors.
For defensemen, there was only one metric of significance for both offense and defense. Def TOI QoT for CF60 and Fwd TOI QoC for CA60.
When we remove the variables whose p-values were greater than 0.05 and rerun the regression, we have the following.
So, now we know that these are the variables we want to use and we can refine what our coefficients should be. The adjusted R^2 values do not change as these variables were driving the correlations. We see some small changes from the original regression, notably that when explaining a forward’s CF60, the QoC of the defensemen they face appears to matter slightly more than who the QoT of the forwards they played with.
So What?
Being able to properly analyze a player within the context of the nine other players on the ice starts with knowing how to value each group (opposing forwards, opposing defensemen, forward teammates, defense teammates). Splitting QoC and QoT by position is a necessary first step in doing this. We can also take the residuals of our regression and understand how much a player under/overperformed what the model predicted given the QoC and QoT metrics.
Having a better understanding of the factors that impact a player’s performance, a coach may be able to better optimize his lineup, something DTM recently presented on at #RITHAC.
Simply put, we believe this is an important and necessary step if you’re trying to account to the quality of competition and teammates governing the player’s environment. Both matter, one simply matters more than the other based on the position (F/D) and phase (offense/defense) you’re evaluating. Much of this is logical as well: for forwards trying to create offense, the defensemen they are matched up against will have an impact.
Future Work
Since this is a shot-based metric, there is no effort made to control for individual scoring talent and expected goals. Working in a goal-based component would, in theory, enhance the quality of competition and teammates.
Since these are in-sample correlations looking at explaining what goes into a player’s shot rates, seeing how well these factors predict shot rates can be something we explore as well. For now, we believe this is an important piece to begin untangling how much impact positional competition and teammates have on a player’s performance.
You can look at the 2015 – 2016 5v5 Positional QoC, QoT, and Residual Performances via the below links.
Hockey Graphs 
Nice Work, two things:
1. Are you controlling for the player in your regression (doesn’t look like you are)? Because the player himself obviously influences in the outcome and would therefore have an effect on the dependant variable. So you would nee to control for him. The general pattern should stay the same but the coefficients would change.
2. I always felt the best way to do this was at the game level and not the season level (this obviously involves a lot more work). I can’t explain it well, but I think doing it at the season level wouldn’t properly capture the effect. Because at the game level you break it down better and see how individual players do against varying levels of QOC and QOT. I guess I’m just not convinced that doing it at the season level would properly control for the player’s talent.
Interesting work, thanks. Nice to see that you controlled for team effects, but rather than running regressions, multilevel models (nesting players within teams) would provide more accurate results. This approach might yield different results, since there appears to be a pretty good sized association between team CF (or CA) and the DVs.