Offside challenges are, to say the least, a controversial topic. While many have advocated for the benefit of getting the call right even at the cost of a delay in the game, it’s almost indisputable that the introduction of the offside challenge has slowed down the flow of the game. Over the past two years, coaches have challenged any play that was remotely close with the hopes of getting lucky on the video review, to the dismay of basically anyone other than replay technicians.

Those spurious challenges are one reason why the NHL modified the rules around coach’s challenges yesterday. Starting next season, instead of a failed challenge simply resulting in the loss of a team’s timeout, clubs will now face a 2 minute penalty for losing an offside challenge. Upon hearing of this change many fans were apoplectic, complaining that this rule change could bury teams who were already reeling from giving up a goal against, and would severely limit the willingness of coaches to challenge even legitimate missed offside calls.

Fan reaction notwithstanding, however, the question coaches should be asking is whether they should be changing their approach in response to the new rules. The threat of killing off a penalty for a failed challenge may seem like a big deal, but it’s important to note that teams only score on roughly 20% of their power play opportunities. Fans will surely remember when a failed challenge leads to a power play goal against, but there will certainly be occasions when the potential gain from overturning your opponent’s goal outweighs the risk.

The question for any coach then is how sure you have to be of your video coach’s recommendation in order to call for a video review. If you think there’s only a 5% chance of success it’s unlikely that the 1 in 20 odds of taking a goal away outweigh the cost of the power play you’re going to give up 95% of the time. But that decision won’t always be so easy: what if you’re 33% sure you’ll be successful, but you’re already down 1 with just 10 minutes left to play? Is it worth the risk of going down two goals with half a period left for a 1 in 3 shot of being tied?

To answer this question we can look to the always excellent insight of Micah Blake McCurdy of Hockeyviz.com. Last year, Micah wrote about the concept of leverage, which he defined as the cost (benefit) in expected standings points for a team allowing (scoring) a goal given the current score state and time left in the game.

Offensive leverage is the increase in expected points from scoring a goal, while defensive leverage is the cost in expected points from conceding a goal. Leverage allows us to estimate how big the cost of a failed challenge will be by looking at how a team’s expected points will change based on the success or failure of their challenge.

Because a team that’s deciding whether or not to challenge has already conceded a goal, a coach who decides to challenge faces two possible outcomes:

• The challenge is successful, and the goal is erased. This is effectively equivalent to the challenging team scoring a goal (since if they choose not to challenge, the score stays the same)
• The challenge is unsuccessful, and their team goes on the penalty kill. This is effectively equivalent to the challenging team conceding 0.2 goals (assuming the league average power play rate is 20%).

We can use this information to model how a team’s expected points will change based on the probability of success of a challenge.[i]

Change in Expected Points = P(Success) * Offensive Leverage + P(Fail) * 0.2 * Defensive Leverage

As long as a team’s Change in Expected Points is positive, a coach should feel confident that challenging a goal is the right choice. We can then calculate the Break Even Certainty, which is the confidence that a coach needs to have in order to ensure that the Change in Expected Points is positive.

Break Even Certainty = (0.2 * Defensive Pressure)/(0.2 * Defensive Pressure + Offensive Pressure)

With this formula and Micah’s leverage data, we can plot the Break Even Certainty for a team playing at home given the current score (after the goal that’s being challenged) and time remaining. So, when should teams challenge under the NHL’s new rule?

For a home team that’s tied before the challenge (i.e. they were up 1 and gave up a goal that they think may have been offside, the green line in the plot above), the break-even rate is pretty consistent around the 18% mark until roughly the 35^th minute of the game, at which point the break-even rate starts to rise steadily throughout the rest of the game. This makes a lot of sense – when you get closer to the end of the game, the cost of giving up a goal on the penalty kill rises, as you have less time to score a tying goal afterwards (and you lose the OT point). As such, you need to be more sure you’re going to get it right before you call for a replay review.

The other curves appear mostly where we’d expect[ii] – a team that’s leading needs to have a much higher certainty than a team that’s trailing, which isn’t exactly surprising. What’s interesting to note is how low a team’s certainty needs to be in order to make challenging worthwhile. A team that’s trailing at basically any point of the game needs less than a 15% chance of success to see a positive expected value,. And even a team that’s ahead by 2 goals can still challenge with less than 50% certainty for most of the entire game.

All of which is to say that while these rule changes may get rid of a few longshot challenges, it will still be worthwhile for a team to challenge in many instances where their odds are far below even. While killing off a 2 minute penalty may seem like a substantial deterrent, the benefits of potentially saving a goal are simply too large to ignore in many instances. As long as the offside challenge remains an option, finding ways to discourage its abuse will be difficult for the league.

The flipside of this, however, is that teams should continue to challenge even if they’re less than certain on their likelihood of success. The value of a timeout is significantly reduced now that teams can’t call one after an icing, and as we’ve seen above, the required success rates are generally so low that coaches can afford to take a flyer every now and then with the hopes of saving a goal.

[i] We’ll ignore the lost 5-on-5 time for simplicity here. Practically speaking, this would be a further net negative for a trailing team, but it should be small enough that it doesn’t significantly impact our conclusions.
[ii] There are a few weird areas where small samples and/or smoothing may make things look odd, but in general the modelled results are pretty intuitive.