Proving the Reachability of States in a Markov Chain with M States

Prove that if the number of states in a Markov chain is M, and if state j can be reached from state i, then it can be reached in M steps or less. 

In a Markov chain, the future state of a system depends only on the current state, not on any previous states. The number of states in a Markov chain is defined as M. In this blog post, we will prove that if state j can be reached from state i, then it can be reached in M steps or less.

To prove this, we will use the transition matrix of the Markov chain. The transition matrix is a square matrix that contains the probability of transitioning from one state to another. Each row in the transition matrix represents the probabilities of transitioning from one state to all other states. The entries in the transition matrix must be non-negative and the sum of the entries in each row must equal 1.

Let's define the transition matrix as P. Then, P^k represents the transition matrix raised to the power of k, where k represents the number of steps taken in the Markov chain. If state j can be reached from state i, this means that there exists a non-zero entry in the (i, j)th position of the transition matrix raised to some power, say k.

Next, we will use eigenvalues and eigenvectors to prove that there exists a power k such that the (i, j)th entry of P^k is non-zero. The eigenvalues of the transition matrix are the scalars that multiply the corresponding eigenvectors to give the eigenvectors after a single step. If λ is an eigenvalue of the transition matrix, then λ^k is an eigenvalue of the transition matrix raised to the power of k.

Since the eigenvalues of the transition matrix must have magnitude less than or equal to 1, it follows that λ^k converges to 0 as k goes to infinity. This means that for any state j that can be reached from state i, there exists a power k such that the (i, j)th entry of P^k is non-zero. Furthermore, since there are only M states in the Markov chain, it follows that k must be less than or equal to M.

Therefore, if state j can be reached from state i, then it can be reached in M steps or less. This result is useful for characterizing the reachability of states in a Markov chain and understanding the behavior of the system over time.

In conclusion, by using the transition matrix and properties of eigenvalues and eigenvectors, we have proven that if state j can be reached from state i in a Markov chain with M states, then it can be reached in M steps or less.