Question

Consider a Markov Chain with transition probability matrix \(P = \begin{pmatrix}0 & 1 & 0\\0.5 & 0 & 0.5\\0 & 1 & 0\end{pmatrix}\) with state space \(S=\{0,1,2\}\). Then:

• A. All the states are aperiodic (period 1)
• B. All the states are periodic with period 2
• C. All the states are transient
• D. All the states are null

Hint:

Trace the smallest number of steps a state needs to return to itself in this chain, and check whether every diagonal entry of \(P\) (and of \(P^2\)) is zero or positive.

Answer 1

The Correct Option is B

Step 1: Compute $P^2 = P\times P$ using $P = \begin{pmatrix}0 & 1 & 0\\0.5 & 0 & 0.5\\0 & 1 & 0\end{pmatrix}$.
Step 2: Row 0 of $P^2$ = (row 0 of $P$) $\times P$ = $1\cdot(\text{row }1\text{ of }P) = (0.5,\ 0,\ 0.5)$. Row 2 of $P$ equals row 0 of $P$, so row 2 of $P^2$ is also $(0.5,\ 0,\ 0.5)$.
Step 3: Row 1 of $P^2$ = $0.5\cdot(\text{row }0\text{ of }P) + 0.5\cdot(\text{row }2\text{ of }P) = 0.5(0,1,0)+0.5(0,1,0) = (0,\ 1,\ 0)$.
Step 4: So $P^2 = \begin{pmatrix}0.5 & 0 & 0.5\\0 & 1 & 0\\0.5 & 0 & 0.5\end{pmatrix}$. Every diagonal entry of $P^2$ is strictly positive (return possible in 2 steps), while $P$ itself has an all-zero diagonal (no return in 1 step, and by the same alternating structure no return in any odd number of steps).
Step 5: A state that returns to itself only at even time steps, with gcd of return times equal to 2, has period 2. All three states are periodic with period 2, and being a finite irreducible chain it is positive recurrent (not transient, not null).
\[\boxed{\text{Period} = 2 \text{ for all states}}\]