Autonomous Solution Methods for Large-Scale Markov Chains

Abstract

One of the roadblocks to greater application of Markov chains is that non-numerically sophisticated users possess the detailed domain knowledge needed to construct a large Markov chain but may have a difficult time deciding which numerical solution method might be best suited to their applications. A realistic Markov chain model can easily contain hundreds of thousands of states, yet users may severely restrict their models to keep them small enough to fit within the constraints of certain software packages or solution methods. Even after selecting a solution method, implementation details imposed by compact storage schemes and the nature of the solution method itself may pose additional barriers. By making judgments about the Markov chain, an experienced researcher or practitioner can sometimes propose a solution technique in a short amount of time. This research examines methods to obtain a proposed solution technique without the services of an expert and with little or no intervention from the novice user. We take advantage of information readily available in the Markov chain to aid in the selection and execution of a solution method. We demonstrate a computer tool with a graphical user interface (GUI) and embedded expert system to make large-scale Markov chain analysis more accessible. The computer tool receives a user's Markov chain, examines the chain, determines its primary characteristics, and then gives the user useful information and recommendations about how to analyze the model. This can be done without the user being an expert in the various solution techniques and their respective areas of applicability.