Let \(\{X_n:n\ge0\}\) be a homogeneous Markov chain with state space \(S=\{1,2,\ldots,7\}\) and transition probability matrix \[ P= \begin{pmatrix} \frac{1}{3} & 0 & \frac{2}{3} & 0 & 0 & 0 & 0 \\ 0 & \frac{1}{3} & 0 & \frac{1}{3} & 0 & \frac{1}{3} & 0 \\ \frac{1}{2} & 0 & \frac{1}{2} & 0 & 0 & 0 & 0 \\ 0 & \frac{1}{2} & 0 & \frac{1}{4} & 0 & \frac{1}{4} & 0 \\ \frac{1}{2} & 0 & 0 & 0 & \frac{1}{4} & \frac{1}{4} & 0 \\ 0 & \frac{2}{3} & 0 & \frac{1}{6} & 0 & \frac{1}{6} & 0 \\ \frac{1}{3} & \frac{1}{3} & 0 & 0 & 0 & 0 & \frac{1}{3} \end{pmatrix}. \] Then which of the following statements is correct?
Show Hint
States \(\{1,3\}\) and \(\{2,4,6\}\) each form a closed set the chain can never leave. State 5 can jump into either closed set and never return, so it is transient.
Step 1: Draw the picture in words. Look at where probability mass can flow out of each state, just the 'can go to' arrows: state $1$ only talks to state $3$ and itself; state $3$ only talks to state $1$ and itself. So $\{1,3\}$ is a self contained loop, nothing escapes it.
Step 2: Find the second loop. State $2$ only talks to $2,4,6$; state $4$ only talks to $2,4,6$; state $6$ only talks to $2,4,6$. So $\{2,4,6\}$ is a second self contained loop. Together these two loops use up six of the seven states, and neither loop sends anything to the other.
Step 3: Recall what closed implies for a finite chain. On a finite state space, a closed set of states you can never leave once entered is always recurrent. So every state in $\{1,3\}$ and every state in $\{2,4,6\}$ is recurrent. That already rules out state $2$, state $3$ and state $4$ from being transient.
Step 4: Look at what is left over. States $5$ and $7$ are the two states not inside either closed loop. Row $5$ sends $1/2$ to state $1$, $1/4$ to itself, and $1/4$ to state $6$. Both $1$ and $6$ belong to closed loops. So over time, with probability $1$, the chain started at $5$ gets pulled into one of the two closed loops and stays trapped there.
Step 5: Confirm state 5 never comes back. Since neither closed loop ever sends anything back out, once the chain leaves $5$ for state $1$ or state $6$, it can never revisit $5$ again. A state the chain eventually leaves forever is transient.
Step 6: Match against the answer choices. (A) says $2$ is transient, but $2$ sits in the closed loop $\{2,4,6\}$, so this is wrong. (B) says $3$ is transient, but $3$ sits in the closed loop $\{1,3\}$, so this is wrong too. (C) says $4$ is transient, same problem. (D) says $5$ is transient, and that is exactly what Step 5 showed.
Step 7: Conclude. \[ \boxed{\text{5 is a transient state}} \]
