SALO ability estimates are now stated on a WAR/82 scale with a new method titled MARKOV, which stands for “MARKOV Approximation for Reasonable Konstruction of Overall Value”.

The methodology, and the question it answers, are a little different from most or all existing WAR methods. The output is an estimate not of how many wins each player did contribute to their real team, but how many they would contribute (through their own effect on shot rates alone) to a standardized team over a stylized season. That is, MARKOV estimates value over replacement (VORP), not wins above replacement (WAR), although the scale of the result is still WAR/82.

Here’s how it works.

Procedure

MARKOV operates by constructing a Markov chain to represent the hockey games a player appears in, based on SALO and some key assumptions.

A Markov chain is defined by a space of distinct states and a matrix of transition probabilities for moving from any state to any state (including the same one) in the short term. The probability of being in any state at an arbitrary time in the future can be looked up using the initial state and the transition matrix. If the space of states includes, say, the lead held by a given team at any time, the chain yields that team’s expected winning percentage.

SALO itself predicts the short-term evolution of each team’s count of shots on goal, given the players on ice and some context factors. MARKOV takes over from there with the following steps:

  1. Imagine each individual player in a standardized context to turn SALO ability estimates into (short-term) shot probabilities.
  2. Make some simplifying assumptions to turn shot probabilities into transition probabilities between relevant game states.
  3. Use matrix multiplication to turn transition probabilities into winning percentages.

The difference between a player’s estimated winning percentage and that of a synthetic replacement player (also based on SALO) is taken to be their value over replacement (per game). Repeating the procedure for each Monte Carlo draw from the SALO model allows the error estimates from SALO to propagate into MARKOV, giving error bars for value over replacement.

Discussion

The winning percentage found in the last step is, in a sense, exact – it is merely a mathematical consequence of the preceding ones. The assumptions used in the previous steps are pretty strong (at least so far), but they’re also easy to make entirely explicit. Thus, MARKOV mostly or entirely avoids depending on any hidden, or even merely obscure, assumptions to generate estimates. I think that’s a pretty neat place to start from.

Thinking about the game in terms of a MARKOV chain makes it natural to avoid double-counting the impact of any estimated effects. The short-term probability of transitioning from, for instance, one goal ahead to two is given in terms of the on-ice player abilities and the appropriate score effect for the current period. Thus, players’ second-order effect on outcomes – through the score effects that themselves change as players change the score – is included automatically in the estimate of their value.

Assumptions

The following assumptions are used to define the state space and generate the transition matrix:

  • The player gets exactly average minutes for their position.
  • All other players are always exactly average.
  • Both goalies save shots on goal at a flat 0.915 rate.
  • All play is 5v5.
  • A team that takes a seven-goal lead always wins.

Except for the last, each of these assumptions describes some aspect of the game not directly modeled by SALO. (The final one simply gives the transition matrix a finite size. No team in NHL history has ever blown a seven-goal lead.)

These assumptions are neutral – they ensure that individual estimated value is a function only of individual estimated ability – but they aren’t weak. Indeed, they are the simplest (and therefore strongest) neutral assumptions that could be made regarding the phenomena they describe.

Over time, more realistic assumptions will indeed replace those stated above. The main driver will be improvements to the scope and realism of SALO itself. Even these initial assumptions, however, are a sensible enough starting point that I find it worthwhile to report the estimates they yield.

Two more assumptions are used in reading out wins per 82 games:

  • Any game tied after regulation never happened.
  • A team has average luck with either a real player or a replacement.

These are only for convenience. A little more coding effort would replace each with a more realistic assumption, but I don’t think doing so in either case would improve the estimates enough to be worth that effort right away.

Lastly, replacement level is defined as the ability level of the proverbial “next man up” – an otherwise typical NHL player who gets zero games – as estimated by SALO.

Method

State space

The MARKOV state space consists of the home team’s lead and the period.

Transition matrix

Wins per 82

Replacement level