Computation for Markov Chains

Abstract

A finite, homogeneous, irreducible Markov chain $\mC$ with transition probability matrix possesses a unique stationary distribution vector. The questions one can pose in the area of computation of Markov chains include the following:

How does one compute the stationary distributions?

How accurate is the resulting answer?

In this thesis, we try to provide answers to these questions.

The thesis is divided in two parts. The first part deals with the perturbation theory of finite, homogeneous, irreducible Markov Chains, which is related to the first question above. The purpose of this part is to analyze the sensitivity of the stationarydistribution vector to perturbations in the transition probabilitymatrix. The second part gives answers to the question of computing the stationarydistributions of nearly uncoupled Markov chains (NUMC).