Question

Consider a discrete time Markov chain with state space \(S=\{1,2,3\}\) and the transition probability matrix

• A. The Markov chain is irreducible
• B. \(1\) is a recurrent state
• C. \(2\) is a transient state
• D. \(3\) is a recurrent state

Hint:

In a finite Markov chain, every state belonging to a closed communicating class is recurrent.

Answer 1

The Correct Option is D

Step 1: Check who reaches state $1$.
From state $1$ the chain can leave to $2$ or $3$, but $p_{21}=p_{31}=0$, so nothing returns to $1$.

Step 2: Note it is not irreducible.
Since $1$ does not communicate with $2,3$, the chain is reducible, so (A) is out.

Step 3: Find the closed set.
States $2$ and $3$ only move among themselves ($p_{23},p_{22},p_{32}$), so $\{2,3\}$ is closed.

Step 4: Classify.
In a finite chain a closed class is recurrent, so $2$ and $3$ are recurrent and $1$ is transient. That makes (B) and (C) false, while (D) 'state $3$ is recurrent' is true.

Step 5: Conclude.
\[ \boxed{(D)} \]