Lecture 1: Zero-sum perfect information games, Reachability games

Teachers: Wieslaw Zielonka

Game Theory techniques in CS: Introduction

Finite two-player zero-sum perfect information games

Simplest games: finite ones, finite trees, outcomes on the leaves

$u_i$ utility function (payoff function) for player $i$

Start from the leaves, and backward induction: compute what player 1 earns provided both players act rationally:

  • player 1 tries to maximises his/her outcome

  • player 2 tries to minimise player 1’s outcome.

Couple (profile) of strategies:
\[(σ^1, σ^2)\]

$Σ_1, Σ_2$: sets of strategies of player 1 & 2.

Payoff:
\[u: Σ_1 × Σ_2 ⟶ ℝ\]
Game value:

$v ≝ u(σ^1, σ^2)$

\[∀τ_1 ∈ Σ_1, τ_2 ∈ Σ_2, \; u(τ^1, σ^2) ≤ v ≤ u(σ^1, τ^2)\]

For zero-sum games, one talks about optimal strategies.

Theorem (Zermelo): Finite perfect information zero-sum games have a game value.

Exs:

  • chess is a finite game.

  • combinatorial game: the Hex game is a finite zero-sum game with only two outcomes: win or lose (the first player to have a path from a side of his/her color to the opposing side wins: there’s always a winner).

    • the first player (white) to play has a winning strategy

Extensive perfect information games

You may have “chance nodes” in the tree: the child chosen from such nodes is drawn from a fixed probability distribution.

⟶ Value to maximise, for each player: the expectancy of his/her gain

For each stategy profile $(σ^1, \ldots, σ^n)$, one compute the value $\sum\limits_{ i } p_i o_i$, where $p_i$ is the probability of the outcome $o_i$.

Nash equilibria

Strategy profile $(σ^1, \ldots, σ^n)$ is a Nash equilibrium:

iff for all $τ^i ∈ Σ_i$, \(u_i(σ^1, \ldots, σ^n) ≥ u_i(σ^1, σ^{i-1}, τ^i, σ^{i+1}, σ^n)\)

NB: it’s in no player’s interest to change his/her strategy.

Each subgame perfect equilibrium is a Nash equilibrium. But the converse is not true.

Games

Game:
  • there are 2 players (Left and Right)
  • there are several positions with a fixed starting position
  • for each position: a set of available moves specifying the next position
  • Left and Right move alternatively
  • both players have complete information
  • no chance/probabilistic move
  • the loser is the one who can’t move anymore at the end

Games as directed acyclic graph (DAG): vertices = positions. Edges in either player’s color, or in a neutral color (if the move can be take by both players).

There is no infinite path (even though the tree may be infinite).

Winning positions:

all the positions such that the player who plays in this position (the current player) has a winning strategy

Losing positions:

all the positions such that the player who plays in this position (the current player) has no winning strategy

All the vertices with no outgoing edge are losing (the player can’t play ⟶ he/she loses).

The game of Nim

Typical combinatorial game.

Operation $\oplus$:

$n \oplus m$ = bitwise XOR between $\underbrace{(n)2}{\rlap{\text{binary representation}}}$ and $(m)_2$

Winning strategy: when $\bigoplus_i n_i ≠ 0$ (where the $n_i$’s are the number of coins on the $i$-th stack).

Poker Nim: each player has a personal stack of coins which is finite, and new move: each player can add a certain number of their personal coins to any stack on the board.

⟶ can be reduced to the Nim game: when the opponent is in a losing position and adds personal coins, all the winning player has to do is to remove the coins the opponent just added.

Reachability games

Simplest zero-sum games

Let $G = (V, E)$ be a directed graph. Two players: A(lice) and B(ob): $V ≝ V_A ∪ V_B$.

Suppose that for each vertex, there is at least one outgoing edge.

Reachability game:

we have a set of vertices $R ⊆ V$ and the question is: « Does Alice have a strategy such that someone ends up in $R$? »

Notation:

\[vE ≝ \lbrace w \mid (v, w) ∈ E \rbrace\]

What does it mean to win in one step?

\[V ⊇ 1\text{-Step}_A(R) ≝ \lbrace v \mid v ∈ V_A, ∃ (v, w) ∈ E; \; w ∈ R\rbrace \\ ∪ \lbrace v \mid v ∈ V_B, \underbrace{∀ (v, w) ∈ E, \; w ∈ R}_{∀w ∈ vE, \; w ∈ R}\rbrace ∪ R\]

Similary, one defines $n\text{-Step}_A(R)$:

  • $0\text{-Step}_A(R) ≝ R$
  • $1\text{-Step}_A(R)$ as above
  • $(n+1)\text{-Step}_A(R) ≝ n\text{-Step}_A(R) ∪ 1\text{-Step}_A\Big(n\text{-Step}_A(R)\Big)$

Clearly, $(n\text{-Step}A(R)){n ∈ ℕ}$ is non-decreasing. There exists $i$ such that

\[i\text{-Step}_A(R) = (i+1)\text{-Step}_A(R) ≝ Reach_A(R)\]

  • Let $V_A \ni v ∈ Reach_A(R)$. Let $i$ be the smallest index such that $v ∈ i\text{-Step}_A(R)$. If $i > 0$, Alice can play a move $(v, w)$ which makes the index decrease stricly. Otherwise, we’re done.

  • If $v ∈ V \backslash Reach_A(R)$, all Alice’s moves remain in $V \backslash Reach_A(R)$, and similarly, Bob has a strategy to remain in this set.

We’ve calculated the (smallest) fixed point of

\[ψ ≝ V ⊇ X ⟼ 1\text{-Step}_A(X) ∪ R\]
\[\underbrace{Reach_A(R)}_{\text{reachability set}} = \bigcup_{i=0}^{∞} i\text{-Step}_A(R)\]

each vertex has a finite number of outgoing edges.

NB: if there were an infinite number of outgoing edges for Bob, it may be the case that each one of them are in a $i\text{-Step}_A(R)$ (ex: the first one is in $1\text{-Step}_A(R)$, the second one in $2\text{-Step}_A(R)$, etc…) but there exists no finite $j$ such that $j\text{-Step}_A(R)$ contains them all (as there’s an infinite number of them). Same reasoning for the complement.

The first such games were considered: in the 50’s.

Topological games: consider an infinite binary tree (going on the left (resp. right): labelled $0$ (resp. $1$)). $V = (0+1)^\ast$

  • Height zero (root): Player 1’s move
  • Height one (root): Player 2’s move
  • Height two (root): Player 1’s move
  • and so on…

$W ⊆ \lbrace 0, 1 \rbrace^ω$. The first player wins if $p ∈ W$.

Distance:

\[d(p, q) ≝ \begin{cases} 0 \text{ if } p=q \\ 2^{-n} \text{ else, where } n ≝ \min \lbrace i \mid p_i ≠ q_i \rbrace \end{cases}\]
Open sets for this metric:

the sets of the form $X \lbrace 0, 1 \rbrace^ω$, where $X ⊆ \lbrace 0, 1 \rbrace^\ast$

For open sets $W ≝ X \lbrace 0, 1 \rbrace^ω$: there’s a winning strategy: it’s the reachability set of $X$.

And with the axiom of choice, we can show that there are sets that have no winning strategy.

Leave a comment