What do I mean when I say I want to use a Markov chain in place of linear steps in deriving WAR from modeled player effects on event rates?

Previously, I wrote about about why that would be a good idea. In this longer post, I’ll cover how to do it.

- First, I explain the basics of Markov math and of setting up the Markov chain implied by a model like SALO.
- Second, I lay out and discuss all the assumptions used to get there from the starting point of SALO.
- Third, I detail the math by which those assumptions are implemented.

A **Markov chain** is a mathematical picture of a real-world system whose state evolves randomly in discrete time.

The main component of a Markov chain is a square **transition matrix** \(M\), with one row and column for each state the system may be in.

The value \(M_{jj'}\) in row \(j\) and column \(j'\) of the matrix equals the probability that the system will be in state \(j'\) a moment from now **conditional** on being in state \(j\) now. (For this reason, the values in any row of the matrix add up to 1.)

As long as the **conditional** probabilities do not change over time, the probability of observing state \(j'\) *two* moments from now if the current state is \(j\) equals \((M \times M)_{jj'}\), or \(M^2_{jj'}\), and so on. (Making sure that’s true mostly comes down to conceptualizing the space of possible states in sufficient detail.) The fact of the conditional probabilities remaining constant over time is called the *Markov property*.

One real-world system that changes state randomly over time is a hockey game. Let’s think about how we’d depict it with a Markov chain.

Before starting, we’d have to settle on the meaning of a “moment”, since a game actually evolves in continuous rather than discrete time. In MARKOV, I take a moment to be half a second:

- NHL play-by-play data has 0.5s temporal resolution; SALO models outcomes explicitly at this resolution.
- 0.5s is short enough to mostly rule out multiple events, making event rate and probability identical.

If we have momentary probabilities for play-by-play events conditional on game state, perhaps from a model previously fitted to some NHL data, we can hope to construct momentary conditional probabilities of state change as well.

From a matrix \(M\) of probabilities for (among other things) any particular score a moment from now conditional on (among other things) the current score, we can compute probabilities of any score after regulation thanks to the *Markov property*.

Probabilities for any possible final score, naturally, imply a win percentage for each team.

Then, if the momentary event probabilities are also conditional on the actual players on ice, we can imagine plugging in different skaters one at a time to assess how many wins each would contribute, all else held equal.

This is how I construct MARKOV, using SALO to define that state space and transition probabilities on it.^{1}

SALO models the probability that each team will take a shot on goal or a penalty within half a second conditional on:

- the
*period*; - the home team’s
*lead*(i.e., the*score*); - how many players each team has on ice (their
*strengths*); and *who*those players are.

These four aspects will be taken as the whole notion of the game state. (Of the four, the period will be handled specially.)

Conditional event probabilities given by SALO *largely* suffice to imply probabilities of any *state* a moment later:

- It’s always known what period it will be in half a second.
- The short-term probability of a change in the score mostly varies with probability of a shot (which SALO models).
- The short-term probability of a change in team strength mostly varies with the probability of a penalty or goal.
- The fourth aspect, who is on ice, is the thing to be fixed so as to use this method for assessing player value.

Markov transition probabilities will be derived from SALO using these four principles.

The gaps between the *event* probabilities SALO outputs and the *state* probabilities required must be filled with explicit *assumptions*.

Each assumption I use concerns how one of the four aspects of game state defined above evolves over time. Below, I lay them all out, discussing briefly what problems they solve and how they might be improved on in future work.

It will be handy to classify the assumptions used into two different kinds:

*Fortifying*assumptions substitute for substantive model content not present in SALO.*Finiteness*assumptions cut off the space of states SALO could permit, for tractability.

SALO’s limited scope makes for non-trivial assumptions to begin with.^{2} In some cases, gentler assumptions than those used would work in principle, but I choose harsh ones anyway to save work.

All the same, this method has two really worthwhile properties:

- It may make assumptions more
*transparent*than those inherent (but possibly implicit) in other techniques. - It allows isolating many or all assumptions to specific game circumstances rather than making them all global.

So what do I assume?

The big idea is to depict a hockey game that one particular skater happens to dress for, with all else held equal.

That leaves a lot to pin down in the Markov chain’s treatment of who is on ice. Who are the other players on both teams? What portion of game time is the player off ice for? Who is at home?

SALO has no model of shifts or roster composition, so these questions are all answered with assumptions:

*Fortifying*: the player assessed sees exactly average ice time: 15 minutes per game for forwards, 20 otherwise.*Fortifying*: all teammates and opponents are exactly average.*Finiteness*: the player’s probability of being on ice doesn’t change with game state (including team strength).*Finiteness*: the player assessed is at home exactly half the time and away otherwise.

With these assumptions, the matter of who *all* is on ice reduces to whether *one skater* is on ice. As covered later, independence of ice time from state means that separate on- and off-ice states don’t even show up in the matrix – I need only *average over* the two cases.

In reality, ice time varies with ability and with game state: better players get more minutes; power-play skaters are disproportionately forwards; and so on. Some of these issues could be ameliorated through more realistic assumptions, while others would have to wait for a better model.

The period doesn’t change randomly. Variation in period is handled by simply constructing a separate matrix for each period, as detailed later. This doesn’t require tacking any assumptions on to SALO. Nonetheless, one is used:

*Finiteness*: any game tied after regulation is replayed in full.

This comes entirely from me being lazy. Overtime could be handled correctly (although not the shootout) with only a little more effort. I’m not doing it anyway. Most games have a winner in regulation.

Depicting how the score changes is straightforward. The assumptions I use are as follows:

*Fortifying*: SoG save percentage is a constant 0.915 in all game states.*Finiteness*: a team that takes a seven-goal lead at any time always wins.*Finiteness*: only one shot can be taken at any moment.

Under these assumptions, there are fifteen possible values of the lead held by the player’s team. If I stopped here, the matrix would be \(15 \times 15\).

Obviously, the fortifying assumption could be improved by plugging in separate save percentages for each strength state. This is a target for future versions of MARKOV. Further improvement would come from modeling skaters’ own on-ice Sh% effects in SALO.

The first finiteness assumption seems rather innocuous by contrast. No NHL team has ever blown a seven-goal lead.

The last assumption is already used in SALO: at any moment, either the home team shoots, the away team shoots, or no one shoots, and either team only shoots once. Over a half-second span, this is pretty close to true.

Strength is harder to depict than score, mostly because strength changes via three different processes: two random (penalties and power-play goals) and one deterministic (expiration). I stitch these processes together as follows:

*Fortifying*: penalty expiration happens randomly at a flat rate of once every two minutes.*Finiteness*: a shorthanded team always gets a player back when a penalty ends.*Finiteness*: extra attackers never appear.*Finiteness*: only one penalty can be taken at any moment.*Finiteness*: only one penalty can expire at any moment.

Under these assumptions, there are nine possible values of (joint) team strength, from 3 to 5 players each. Together with the fifteen possible lead values, this means the transition matrix \(M\) is \(135 \times 135\).

From the first assumption I get a *tractable* matrix. The true rate at which (minor) penalties expire *is* once every two minutes, but (to put it mildly) the short-term probability of expiration varies greatly with how long ago a penalty was taken. Depicting that variation, however, would require separate game states for every moment remaining on each penalty.^{3} With 240 half-second moments per penalty and up to two ongoing penalties per team, this would generate a transition matrix much too large to compute with.^{4} Likewise, allowing for different kinds of penalties with different ending conditions would expand the matrix unhelpfully; I keep the most common kind.

The next two tie up other loose ends. First, I assume players don’t pile up in the box – a team can’t be down two skaters with another penalty to start when a previous one expires. Second, I assume situations with a sixth skater don’t arise. Once again, both of these things happen in reality, but, once again, not *much*.

The final two, as with the last assumption about how the score changes, come from the functional form of SALO. SALO only imagines one penalty being called at a time, so simultaneous ones are recoded as having happened a moment apart. This means the (conditional) rate at which penalties are called is not distorted by the model setup.

That’s a bunch. On the other hand, depicting evolution of team strength based on SALO doesn’t require assuming a conversion factor between events and state changes the way depicting evolution of the score did. So hooray?

It’s finally time to turn SALO into MARKOV.

As stated above, take the term “moment” to mean a half-second of hockey gameplay.

Take the set \(\check{U} = \{-1, 1\}\) to represent the *teams* in a hockey game.

These numeric values will be used with different meanings in different settings:

- SALO deals with
*home*and*visiting*teams. Let \(\check{U}_s = \{v, h\}\) map to \(\check{U}\) for this purpose. - MARKOV instead deals with
*for*and*against*. Let \(\check{U}_m = \{a, f\}\) map to \(\check{U}\) for that purpose.

In either setting, take \(U = \{-1, 0, 1\}\) to mean the set of teams augmented with \(0\) for *neither* team.

The numeric values of each of these team labels will be used in equations ahead.

An *event* can be represented mathematically simply as the *team* \(u \in U\) it happens to at any moment.

SALO and MARKOV concern different types of events:

- SALO event types \(y_s \in \{s, p\}\) are
*shots*on goal and*penalties* - MARKOV event types \(y_m \in \{g, p, x\}\) are
*goals*,*penalties*, and*expirations*.

In each case, one event of each type is defined as occurring each moment, albeit possibly (indeed usually) to the null team \(0 \in U\).

Observe that this implements assumptions 8, 12, and 13 stated above.

Finally, denote by the tuple \(\bar{y}_m = (g, p, x) \in {U_m \times U_m \times U_m}\) the *joint MARKOV event* at any moment.

Besides player effects, SALO takes three aspects of game context as relevant to expected event rates:

- the
*period*\(t \in \{1, 2, 3\}\); - the
*lead*\(\ell_h \in \mathbb{Z}\) for the home team; and - the
*strength*\(n_u \in \{3, 4, 5, 6\}\) of each team \(u \in \check{U}_s\).

MARKOV restates these in for/against terms.

- Denote by \(j_s = (\ell_h, n_v, n_h) \in \mathbb{Z} \times \{3, 4, 5, 6\} \times \{3, 4, 5, 6\}\) a
*SALO state*.^{5} - Denote by \(j_m = (\ell_f, n_a, n_f) \in \{-7, ..., 7\} \times \{3, 4, 5\} \times \{3, 4, 5\}\) a
*MARKOV state*.

Note that MARKOV’s state space \(J_m\) is a *proper subset* of SALO’s space \(J_s\), setting up assumptions 7, 10, and 11.

Note also that the period \(t\) is left out of the notion of a state. It will be used in a unique way.

SALO is a model for the momentary probability of any SALO event of any type at any moment:

\[P(y = u) = \Lambda(\alpha_y + u (X^{\top}\beta_y + Z^{\top}\gamma_y)), \forall u \in \check{U}, \forall y \in \{s, p\}\]

Probabilities must add to \(1\), so \(P(y = 0) = 1 - P(y = v) - P(y = h)\) by default.

What do the moving parts of that equation represent?

- \(\alpha_y\) is an estimated baseline.
- \(X^{\top}\beta_y\) adds the impact of game context:
- \(X\) is a vector of context indicators encoding \(t\), \(j_s\), and \(u\); and
- \(\beta_y\) is a vector of estimated context effects.

- \(Z^{\top}\gamma_y\) adds the impact of individual players on-ice:
- \(Z\) is a vector of on-ice indicators equal to \(-1\) for visitors and \(1\) for hosts; and
- \(\gamma_y\) is a vector of estimated player effects.

- \(\Lambda(\cdot)\) is the
*logistic function*\(\text{exp}(\cdot) / (1 + \text{exp}(\cdot))\).

Note that \(u\) multiplies the two impact terms: home and visiting teams see equal and opposite impacts of each feature. This is what it means for SALO to be an *ordinal* model.^{6}

Call \(p_{yu}(t, j_s, Z^{\top}\gamma_y) = P(y = u | \alpha_y, \beta_y, \gamma_y; X, Z)\) a *SALO event probability*. (Forget about writing out \(\alpha\) and \(\beta\) henceforth.)

Oh, right, the point is still to quantify what a particular skater contributes to a team.

Suppose SALO team \(u_i\) dresses a skater \(i\) with estimated effects \(\gamma_i = (\gamma_{si}, \gamma_{pi})\).

Suppose, further, that all other skaters on teams \(u_i\) and \(\bar{u_i} = -u_i\) are exactly average (\(\gamma = 0\)).

Then there are only two possible values of \(Z^{\top}\gamma\):

- When \(i\) is off ice, \((Z^{\top}\gamma)_{\text{off}} = 0\).
- When \(i\) is on ice, \((Z^{\top}\gamma)_{\text{on}} = u_i\gamma_i\).

Let \(f(i) = 1\) if \(i\) is a forward and \(0\) otherwise. Let \(g(i) = \frac{1}{3 + f(i)}\).

Suppose that \(i\) gets exactly average ice time by position: fifteen minutes per game if \(f(i) = 1\), twenty otherwise.

Then each SALO team sees *average SALO event probabilities*

\[\bar{p}_{yu}(t, j_s, u_i, \gamma_y) = g(i) p_{yu}(t, j_s, u_i\gamma_{yi}) + (1 - g(i)) p_{yu}(t, j_s, 0)\]

over the two possible on-ice scenarios.

Observe that this implements assumptions 1, 2, and 3.

Likewise, suppose half of games \(i\) appears in are home games and the other half are road games.

Then each *MARKOV* team \(w \in \check{U}_m\) sees average SALO event probabilities

\[\bar{p}_{yw}(t, j_m, \gamma_i) = (p_{yh}(t, j_m, u_i, w\gamma_i) + p_{yv}(t, j_m, u_i, -w\gamma_i)) / 2\]

over the two venue possibilities.

This implements the fourth of the assumptions.

From here forward, the subscript \(m\) will be dropped; all objects can be construed as the MARKOV sort.

Construct *MARKOV event probabilities* from SALO analogues as follows:

- \(q_{gw}(t, j, \gamma_i) = \bar{p}_{sw}(t, j, \gamma_i) \times 0.085\)
- \(q_{pw}(t, j, \gamma_i) = \bar{p}_{pw}(t, j, \gamma_i)\)
- \(q_{xw}(t, j, \gamma_i) = 1 / 240\)

for all \(w \in \check{U}\).

Observe that this implements *fortifying* assumptions 6 and 9 on goal and expiration probabilities respectively.

Denote by \(j'(j, \bar{y})\) the function returning the state *following* a joint event \(\bar{y}\) in state \(j\).

Recall that a joint MARKOV event \(\bar{y} = (g, p, x) \in U \times U \times U\).

By the MARKOV state definition, \(j'(j, \bar{y}) = (\ell_f'(\ell_f, \bar{y}), n_f'(n_f, \bar{y}), n_a'(n_a, \bar{y}))\).

Then let the following rules define \(j'(j, \bar{y})\):

- \(\ell_f'(\ell_f, \bar{y}) = \ell_f + g\) if \(|\ell_f| < 7\) and \(\ell_f\) otherwise.
- \(n_w'(n_w, \bar{y}) = \min(\max(n_w + I(g = -w) + I(x = w) - I(p = w), 3), 5)\).

Observe that these rules implement *finiteness* assumptions 7, 10, and 11.

Given the above, the probability of a state \(k \in J\) following a state \(j\) is given by

\[m_{jk}(t, \gamma_i) = \sum_{\bar{y} \in \bar{Y}} \bar{p}_{\bar{y}}(t, j, \gamma_i)I(j'(j, \bar{y})=k)\]

which is the sum, over all joint events \(\bar{y}\) that lead from \(j\) to \(k\), of the conditional probability of \(\bar{y}\).

There are \(15 \times 3 \times 3 = 135\) possible MARKOV states. Number them (arbitrarily) from \(1\) to \(N = 135\).

For each period \(t\), construct the *transition matrix*

\[M_t(\gamma_i) = \left[ \begin{matrix} m_{11} & \cdots & m_{1N} \\ \vdots & \ddots & \vdots \\ m_{N1} & \cdots & m_{NN} \end{matrix} \right](t, \gamma_i)\]

from the momentary MARKOV state probabilities defined above.

By the *Markov property*, the matrix \(M_t(\gamma_i)^{2400}\) gives *period-end* MARKOV state probabilities conditional on the state at the beginning of period \(t\).

Since each period immediately follows the next, the matrix

\[M^*(\gamma_i) = M_1(\gamma_i)^{2400} \times M_2(\gamma_i)^{2400} \times M_3(\gamma_i)^{2400}\]

gives *game-end* state probabilities, conditional on the state at the start of period 1.

Denote by \(\hat{j} = (0, 5, 5)\) the known initial state (tied at full and even strength).

Then \(M_{\hat{j}}^*(\gamma_i) = [m^*_{\hat{j}1}, ..., m^*_{\hat{j}N}](\gamma_i)\) gives actual game-end state probabilities.

Denote by \(J_w\) the subset of \(J\) such that \(w\ell_f > 0\). Then

\[W(\gamma_i) = \sum_{j \in J_f} m^*_{\hat{j}j}(\gamma_i) \Big/ \sum_{j \in J_f \cup J_a} m^*_{\hat{j}j}(\gamma_i)\]

is the expected winning percentage for \(i\), discounting tied games to implement the fifth (and last remaining) of the assumptions.

The value \(W(\gamma_i) - W(\gamma_r)\) for a suitably defined replacement player \(r\) is MARKOV’s WAR per game value.

In this post I don’t cover how I get error bars or how replacement level is defined; these features are add-ons, however, cleanly separable from the main portion of the method written up here.↩

In general, the richer and more complete your underlying model, the fewer and more limited the assumptions you’d have to add to fill out a transition matrix. The bigger suite of regressions underlying Corsica’s WAR numbers, for instance, would require a reduced set of assumptions.↩

One for the final moment when the probability of expiration is 1, one for the previous moment when the probability that the next is the final moment is one, and so on.↩

12.5 billion rows and columns, accounting for the score. 100

*exabytes*of double-precision numbers. Um.↩\(\mathbb{Z}\) is the set of all integers.↩

This is a convenient way to write the model for explanation; the actual coding is different, though mathematically equivalent.↩