Class structure

It is possible to break a Markov Chain into smaller pieces, each of them easy to understand. The key idea is communicating classes.

Def We say that $i$ leads to $j$ and write $i\to j$ if exists some $n\ge 0$ such that

$$P_i(X_n=j)=P_{ij}^{(n)}>0$$

Def We say $i$ communicates to $j$ and write $i\leftrightarrow j$ if both $i\to j$ and $j\to i$.

Remark It is clear that $\leftrightarrow$ is an equivalence relation on the state space $I$, so that we can partitions $I$ into communicating classes.

Def We say that $C$ is a closed class if

$$i\in C,i\to j⟹j\in C$$

a closed class is one from which there is no escape.

Def A state $i$ is called absorbing if $\{i\}$ is a closed class.

Def A non closed class is called open.

Def (Irreducibility) A Markov Chain is said to be irreducible if $I$ is a single class. Just check that $\forall i,j$ $i\to j$.