Recently, one of my VanHAC slides was used to justify the Bruins’ decision to take Trent Frederic over Alex DeBrincat (and many others) in the first round of the 2016 NHL Draft. Needless to say, I haven’t slept soundly since then.

Unfortunately, crafting a solid rebuttal isn’t as easy as saying “DeBrincat has a higher ceiling.” To that end, I present a framework for evaluating these types of draft decisions. There are two basic questions to consider:

1. What are the outcomes for each player?
2. How can we appropriately value these outcomes?

Variance: What Does It Look Like?

First, we must engage with the idea that prospect outcomes are inherently variable. This sounds easy enough, but many draft analyses, including my own previous work, have treated players’ NHL careers as fixed and predetermined.

A great example of the variability of draft outcomes is this case of two draft-eligible prospects in 2006. Here are some of their relevant stats up until draft day:

And here are some relevant stats after the draft:

Player X is Claude Giroux, current captain of the Philadelphia Flyers, and Player Y is François Bouchard, current third highest-scoring forward for Bordeaux in the French league. The latter was drafted by the Capitals, spent some time in the AHL, and then made his way over to Europe. (This makes some of HFBoard’s takes especially fun in hindsight.) In short, their careers ended up diverging dramatically, and it’d be helpful for NHL teams to know why.

Now that we know their eventual outcomes, we could try to emphasize any differences we find. Their pre-draft ratings were roughly equivalent, but Giroux had a bit more skill and Bouchard had a bit more size, and maybe that’s why Giroux was always meant to be a good NHLer. Or maybe the Flyers’ development efforts were superior, but of course the Capitals developed some excellent players from the same draft. Maybe Giroux had more opportunities to crack the Flyers’ lineup, but good players are (hopefully) able to break through eventually. Maybe Giroux “wanted it more,” but I would think that Bouchard’s extensive European career implies a similar commitment to the sport.

We can also just say that Claude Giroux has achieved one of his best possible outcomes, and François Bouchard has lived one of his worst possible outcomes. Retroactively, we might label Giroux a draft steal and Bouchard a draft bust, but we will never know what would happen if we could run another “trial” of reality.

Estimating Outcomes

In general, predicting all possible draft outcomes is difficult to do in a principled manner. Few GMs go into the draft saying “This prospect has a 73% chance of success,” though some are showing an increasing willingness to do so. Even fewer GMs say “This prospect’s career NHL games played are best modeled by a zero-inflated negative binomial distribution.” But all of them have some idea of whether they’re looking at a relatively sure bet or a more boom-or-bust prospect.

This language was used at length to address the Bruins’ decision of drafting Trent Frederic rather than someone like Alex DeBrincat. The Bruins’ own estimation of Frederic was as follows:

[Frederic] is not going to be a top-two-line guy, we know that.¹

Meanwhile, 5’7″ DeBrincat had this to say about his potential:

You have guys like [Johnny] Gaudreau and Tyler Johnson that are doing really well in the NHL, and it shows guys like me that they can play at the NHL level.²

This provides us with enough information to do some back-of-the-envelope calculations and comparisons. Let’s say Frederic, based on his size and apparent #grit, is a sure bet to become a third liner in the NHL. (Please don’t tweet at me saying that this is false. I know it’s false. Feel free to adjust any estimates as you see fit.)

Furthermore, let’s say DeBrincat has some probability of becoming the next Johnny Gaudreau. However, if he doesn’t do that, he’ll miss out on the NHL completely.

If Frederic is offered an entry-level contract (ELC) soon, he’ll play his first three NHL seasons with a maximum cap hit of $925,000. DeBrincat has already signed his ELC with the Blackhawks at an $809,167 cap hit. We’ll go ahead and use the $925,000 figure for both players because it would’ve been our best estimate at the time of the draft.

In general, it’s fair to assume either $925,000 or $894,167 for an eventual ELC cap hit on an NHL roster. If a team has a good prospect, they often offer him a maximum-valued ELC, and then an entry-level slide (see here for more details) may decrease his cap hit slightly.

The actual value they end up creating for their respective teams will be closer to the cap hit of their second contract. For example, Sidney Crosby’s first three NHL seasons cost the Penguins $850,000 of cap space each, but he was producing at a rate better estimated by his current $8,700,000 cap hit. This valuation provides us with a good enough proxy to inform draft decision-making.

For Frederic, we need to find the monetary value of a pretty good third liner. We can estimate this to be around a $2,000,000 cap hit in today’s NHL. If we look at active players on non-entry-level contracts (as of 3/25/2017), the cap hit of the 211^th player (30 teams * 7 more expensive forwards on each roster = 210 more expensive forwards) is $2,000,000. Fittingly, the 211^th player is Tom Wilson, who was a similarly controversial draft decision by the Washington Capitals in 2012. He is also proof that cap hit is not directly equivalent to on-ice value, but I digress.

For DeBrincat, we can cheat a bit and use Johnny Gaudreau’s second contract cap hit of $6,750,000 to represent the best case scenario. If he doesn’t pan out, that value drops to $0 because he wouldn’t be offered a second contract.

Now, we can compute the expected value of each player. Frederic’s expected value is always $2,000,000 because we’re assuming that he’s basically a lock to play in the NHL in a decent bottom-six capacity. DeBrincat’s expected value ranges from $0 to $6,750,000 based on your best guess as to the probability he makes it to the NHL.

This analysis implies that if you think DeBrincat has a > 30% chance of success, you should go ahead and take him over Frederic.

Valuing Outcomes

But now we should think a bit more like economists. So, let’s start with the simplest question: why is the draft important at all? There are two main reasons.

1. Drafting allows you to acquire players that would be exceedingly hard/expensive to find through trades or free agency.
2. Entry-level contracts allow you to pay young players next to nothing for up to three years while giving them significant on-ice responsibility.
   [This is not cool or fair, by the way. There is no good reason to take away up to three years of significant earnings from the young superstars who make this sport worth watching.]

The second point requires further study. When a team considers drafting a player, they should consider the opportunity cost of not drafting him. For example, you might want to draft a projected fourth liner who is worth exactly $925,000, and then offer him a max ELC with a cap hit of $925,000. But if you don’t draft him, the cost to find someone like him in free agency would be around $925,000, and there are many players available every year who fit the description. So you don’t lose anything but you don’t really gain anything, either.

At the other extreme, if the Penguins didn’t draft Sidney Crosby (because of concerns about his finger-slashing habits, let’s say), the cost to acquire a player of his caliber would be astronomically high. Players like him rarely exist in free agency, but even if they did, the Penguins would have had to take on a cap hit of at least $7,850,000 ($8,700,000 – $850,000) more than his ELC cap hit.

In general, when considering a prospect, we should want to know how much value he will end up creating above and beyond the dollar value of his ELC. If the difference is negligible, teams could acquire such players in other channels.

Based on our earlier calculations, we can easily estimate the yearly excess value created by Frederic and DeBrincat. For Frederic, this is always $2,000,000 – $925,000 = $1,075,000 since we assume he has a 100% chance of making it to the NHL and fulfilling each year of a max ELC. So, not drafting Frederic and then searching for a player with a similar skill set would cost about $1,075,000 more than what you’d pay for him. For DeBrincat, this excess value is $6,750,000 – $925,000 = $5,825,000 if he pans out, and $0 if he doesn’t. If DeBrincat never makes it to the NHL, his cap hit won’t be relevant to his team’s roster decisions.

Now, the probability threshold at which we choose DeBrincat over Frederic is 11 percentage points lower (roughly 18.5% instead 29.6%). Based on these numbers, there is still some rationale for drafting Frederic if you think DeBrincat’s probability of success is particularly low. But if you think he has even a 1 in 5 chance of becoming the next Gaudreau, he’s worth the risk.

In every part of this assessment, you can modify the specific assumptions as you see fit, but the general result should hold.  When we consider the alternatives to drafting each player, we value risky players with higher ceilings more, because they are harder and more expensive to replace if they pan out.

Concluding Thoughts

This was a very quick look at a complicated problem, and every assumption I’ve made should really have its own well-validated model. But the goal of this analysis is to show that the baseline for a good draft decision is not “does this prospect have some positive expected value?” Rather, the question to ask is “how much value can this prospect add beyond what I can find in other channels?”

Of course, we can’t run multiple trials of reality, and so boom-or-bust prospects will often look like bad decisions in hindsight. But if teams remain consistent in their decision-making, keep and acquire as many picks as possible, and aren’t incredibly unlucky, seeking variance should eventually pay big dividends.

In summary: every team needs third and fourth liners, but that doesn’t necessarily mean you should draft them. Instead, consider taking a chance on a player you wouldn’t be able to find anywhere else. For my sake.

https://twitter.com/nnstats/status/844957897746530304

Special thanks to CapFriendly and Spotrac for providing all contract information.