Question

Consider a Markov Chain with state space \(S = \{0, 1\}\) and the transition probability matrix \(P = \begin{pmatrix} 1 & 0 \\ 0.5 & 0.5 \end{pmatrix}\). What is the value of \(P^{(2)}_{10}\)?

• A. 1
• B. 0.5
• C. 0.75
• D. 0

Hint:

Use the Chapman-Kolmogorov relation \(P^{(2)}_{10}=\sum_k P_{1k}P_{k0}\), or simply square the given 2 by 2 matrix.

Answer 1

The Correct Option is C

Step 1: Compute $P^2$ directly by squaring the matrix: $P^2 = \begin{pmatrix} 1 & 0 \\ 0.5 & 0.5 \end{pmatrix}\begin{pmatrix} 1 & 0 \\ 0.5 & 0.5 \end{pmatrix}$.

Step 2: Row 0 of the product: $(1\cdot1+0\cdot0.5,\ 1\cdot0+0\cdot0.5) = (1,0)$. Row 1 of the product: $(0.5\cdot1+0.5\cdot0.5,\ 0.5\cdot0+0.5\cdot0.5) = (0.75,\ 0.25)$. So $P^2 = \begin{pmatrix} 1 & 0 \\ 0.75 & 0.25 \end{pmatrix}$, and its $(1,0)$ entry is $0.75$.

Step 3: Cross check by enumerating sample paths from state 1 to state 0 in exactly two steps: path $1\to0\to0$ has probability $0.5\times1=0.5$, and path $1\to1\to0$ has probability $0.5\times0.5=0.25$. These are the only two ways to land on state 0, and being mutually exclusive their probabilities add: $0.5+0.25=0.75$, matching the matrix computation.

\[\boxed{0.75}\]