Last night Dave Hakstol and the Flyers were the first team to get burned by the NHL’s new offside challenge rule. With a one-goal lead over Nashville and just 2:41 left in the 3rd period, Philadelphia was dinged for not one but two minor penalties at the same time. And on the ensuing 5-on-3 power play, Scott Hartnell banged in a loose puck to tie the game up.
Philly, however, decided there was something not quite right about Hartnell’s goal. They thought that Filip Forsberg may have snuck into the offensive zone just slightly ahead of the puck on the zone entry that preceded the tying marker. The Flyers decided to challenge, hoping that video review would negate the Preds’ goal and put them back on top with just under two minutes to play.
When news first came out of the league’s proposal to change the rules, there was a lot of skepticism that it would act as much of a deterrent to frivolous challenges. While no coach wants to see their team go on the penalty kill after conceding a goal, the odds were still stacked pretty heavily in favour of challenging even in low probability scenarios. In a normal even-strength situation, your probability of success doesn’t need to be all that high in order to make a challenge worthwhile, in fact you’re safe challenging a lot of the time with less than a 25% certainty of success.
But the Flyers situation was anything but normal – they were already guaranteed to be down at least one man after the goal regardless of whether they challenged or not. Even if they were successful, they’d still need to kill off another 64 seconds at 6-on-3 and then survive for an additional 41 seconds at 6-on-5.
While the play was definitely tight (Forsberg even turned back immediately, indicating he may have even been worried he was ahead of the puck), it was far from an easy call, and the risks to the Flyers were definitely high. It’s impossible to know how frequently plays like this one end up getting overturned in the long run, but we can figure out how certain the Flyers should have been that they’d be right in order to make the challenge worthwhile.
To do this, we have to consider the 3 possible scenarios for Philly:
- They don’t challenge, and face 36 seconds of 4-on-5, and then, if they survive, plays the remaining 41 at 5-on-5.[1]
- They challenge and win. They’d then face 68 seconds of 3-on-6, followed by up to 41 seconds of 5-on-6.
- They challenge and lose. This is the nightmare scenario that Hakstol et al faced last night, where the game wound up tied anyways, and they had to kill 36 seconds at 3-on-5 followed by another 41 seconds at 4-on-5.
To run the math on each of these scenarios we need to know how often each team is likely to score in each strength-state, ignoring the possibility of a non-empty-net shorthanded goal.
| Strength State | Goals Per 60 |
| 6-on-3 | 23.6 |
| 6-on-4 | 10.7 |
| 6-on-5 | 6.9 |
| 5-on-3 | 20.0 |
| 5-on-4 | 6.2 |
| 3-on-6 | 2.6 |
| 4-on-6 | 9.3 |
| 5-on-6 | 15.4 |
With these numbers we can calculate Philadelphia’s expected points in each of the three scenarios above, and then figure out how often Hartnell’s goal would need to be overturned to make the challenge worthwhile.
No Challenge
The simplest scenario to look at is if Dave Hakstol leaves the bench, fumes for a minute or two, and doesn’t call for the linesmen to review the play. In this case we have 2 possible outcomes to consider:
- Nashville scores in their remaining 36 seconds of 5-on-4 time, and the Flyers take home 0 points.
- Philly kills off the rest of the penalty, and the game likely goes to overtime, which is worth about 1.5 points for each team.
What’s the probability of the Flyers getting through those 36 seconds? If teams score 6.2 goals per 60 at 5-on-4 that translates to a 6.2% chance of the Predators scoring during their 36 seconds of 5-on-4 time.
Philly’s expected points are then:
xPoints (No Challenge): 6.2% * 0 Points + (100% – 6.2%) * 1.5 Points = 1.41 Points
Challenge and Lose
The next simplest scenario to look at is the one that actually happened. The refs reviewed the play, determined there was no conclusive evidence that Nashville was offside, and the Preds took advantage of their extra-man advantage to score the game winning goal. Once again there are 2 basic outcomes of this scenario:
- Nashville scores on either the 36 seconds of 5-on-3 or the 41 additional seconds of 5-on-4, and the Flyers go home empty handed.
- Philly keeps the Preds off the board, and take home 1.5 points on average from an OT or shootout game.[2]
This time our expected points calculation is a bit more complicated. At 5-on-3 teams score 20.0 goals per 60, which means there’s a 20% chance Philly will concede on the 5-on-3, while at 5-on-4 there’s a 7.1% chance a team will score in 41 seconds. We have to reduce that 7.1% chance though to account for the fact that a second goal won’t have the same impact if Nashville scores on the 5-on-3.
Therefore the probability that they’ll score at least one goal is:
Probability >1 Nashville Goal = 20% + (100% – 20%) * 7.1% = 25.7%
And their expected points when we lose the challenge are:
xPoints (Lose Challenge): 25.7% * 0 points + (100% – 25.7%) * 1.5 Points = 1.12 Points
We can see right away that this is a decent drop from the No Challenge scenario, as the odds are reasonably high that with over 30 seconds on a 2-man advantage you’ll score. The question then is how much of a boost the Flyers get from erasing a goal.
Challenge and Win
Lastly, we need to consider the case where Dave Hakstol is proven right and looks like a hero (at least temporarily). This is the most complicated scenario since we have to take into account that the clock would reset back to 1:45 if the play was indeed offside, giving Nashville an additional 28 seconds of 2-man advantage to play with.
We also need to assume the Preds would keep their goalie pulled (at least until they tie the game) which means we have to consider the possibility of a shorthanded goal by Philly.
To simplify our calculations, we’ll use Nashville’s net goal differential in each empty-net scenario to model our problem. There are 3 cases we need to consider for this scenario:
- Nashville scores 2 power play goals, one at 6-on-3 and one at 5-on-4. The Flyers earn 0 points in this case.
- Nashville scores 1 power play goal, either at 6-on-3 or at 6-on-4. Philly takes home 1.5 points on average in this case.
- Nashville scores 0 goals, Philadelphia takes home a full 2 points, and Dave Hakstol is hailed as a maverick cowboy genius or something.
What’s the probability of each scenario?
The net goal differential per 60 of a team at 6-on-3 is 21.0, which over 64 seconds makes a 37.3% chance of scoring. We have to multiply that number by the probability they’ll also score at 5-on-4 after that. Assuming Nashville gets 32 seconds at 5-on-4 (half of the 64 seconds from the first penalty), 32 seconds is worth about 0.055 goals at 6.2 goals per 60. That makes the total probability of Philly giving up two in the last 36 seconds of their penalty kill equal to:
Probability of 2 Nashville Goals = 37.3% * 5.5% = 2.1%
In the 1-goal scenario, Nashville has 64 seconds to score at 6-on-3, or, if they fail to do that, 41 seconds to score at 6-on-5. The probability of one goal in 64 seconds of 6-on-3 is again 37.3%, while the probability that they score at 6-on-5 is actually negative at -6.1%, as teams who have pulled their goalie actually get outscored in the long run.[3]
Probability of 1 Nashville Goal = 37.3% + (100% – 37.3%) * (-6.1%) = 31.3%
This makes the total expected points for the Flyers if they win the challenge equal to:
xPoints (Win Challenge): 2.1% * 0 points + (31.3%) * 1.5 Points + (100% – 2.1% – 31.3%) * 2 = 1.80 Points
Here’s where you see the real benefit to challenging – if they wind up being right, they’ve earned nearly half a point more than if they just let the play go on. So while there’s risk to the decision, there’s also a decent reward if they’re proven correct.
Break Even Certainty
Knowing Philadelphia’s expected points in each possible scenario, we can calculate the Flyers’ Break Even Certainty for the challenge. The Break Even Certainty is simply the probability of overturning the call on the ice which makes the expected points in the challenge scenarios equal to the expected points if they don’t challenge.
[Break Even Certainty] * xPoints (Win Challenge) + (1 – [Break Even Certainty]) * xPoints (Lose Challenge) = xPoints (No Challenge)
Re-arranging, we get:
[Break Even Certainty] = (xPoints (No Challenge) – xPoints (Lose Challenge)) / (xPoints (Win Challenge) – xPoints (Lose Challenge))
which in our case works out to 42.4%. That is to say, as long as Hakstol and his staff thought they had better than a 42.4% chance of being right, challenging the call was the correct move.
While that level of required certainty doesn’t make it an easy decision, it’s also not a super high probability. The play was definitely close, and you can’t blame Hakstol for taking the chance on potentially icing the game at that point. Our calculations may even overstate the true risk to the Flyers: the numbers we’ve used assume average goal scoring rates, but the Predators would be starting in the neutral zone if the call did get overturned, and would need to re-enter the Flyers end before they get any offensive opportunities.
Though Dave Hakstol is certainly going to live with that failure in the back of his mind for a long time, the reality is that he may have made the right call and just got a bit unlucky. Even in a worst-case scenario like this one, the odds still lean towards challenging, as the benefits of erasing one of your opponent’s goals will almost always outweigh the risk of going on (or staying on) the penalty kill.
[1] We’ll ignore the case where they give up a goal at 5-on-4 and then score after that for simplicity’s sake.
[2] Again, we’ll ignore the fact that Nashville gets about 43 seconds of 4-on-3 time in OT to make the math easier.
[3] This is a conservative estimate, as it includes a lot of delayed penalty time from the middle of the game where it’s (nearly) impossible for the defending team to score.
Hockey Graphs 