Electronic Transactions on Numerical Analysis
its book of mathematics on numerical analysis.
30 Classification of States
In a Markov chain, each state can be placed in one of the three classifications.1 Since
each state falls into one and only one category, these categories partition the states. The secret
of categorizing the states is to find the communicating classes. The states of a Markov chain can
be partitioned into these communicating classes. Two states communicate if and only if it is
possible to go from each to the other. That is, states A and B communicate if and only if it is
possible to go from A to B and from B to A.
31 Geometric Series
Geometric series are a basic artifact of algebra that everyone should know.1 I am
teaching them here because they come up remarkably often with Markov chains. The finite
geometric series formula is at the heart of many of the fundamental formulas of financial
mathematics. All students of the mathematical sciences should be intimately familiar with this
topic and have all the formulas memorized.
32 Averages
You should remember the arithmetic average. Given n data points, their arithmetic
average is their sum divided by n. Now suppose that we have the average of n numbers, An. We
are given a new data point x and we would like to compute the new average of all n + 1
numbers, An + 1. Many people simply add up all n + 1 numbers and then divide by n + 1.
However,
33 The Gambler's Ruin
There are many variations of the gambler's ruin problem, but a principal one goes like
this. We have i dollars out of a total of n. We flip a coin which has probability p of landing
heads, and probability q = 1 ! p of landing tails. If the coin lands heads we gain another dollar,
otherwise we lose a dollar.
34 Ergodic Chains
Until this point in the book, results have been derived or proven, or at least the nature of
the proof has been indicated. However, proofs of the techniques of Sections 34, 35 and 36 as
well as the core concept of Chapter 37 are beyond this book and have been omitted. Do not let
this deter you.
35 Mixed Chains
In this chapter we learn how to analyze Markov chains that consists of transient and
absorbing states. Later we will see that this analysis extends easily to chains with (nonabsorbing)
ergodic states.
36 Poisson Processes
We are going to look at a random process that typifies what most of us think of as
random. This process has the added virtues that it is easy to work with and it is used a great deal
in mathematical modeling. In fact, it may be used more than any other process of its type. In
particular what I am talking about is the Poisson1 process which is described by both the Poisson
distribution and the exponential distribution.
37 A Little Game Theory
Game theory is one of the most interesting topics of discrete mathematics. The principal
theorem of game theory is sublime and wonderful. We will merely assume this theorem and use
it to achieve some of our early insights. To appreciate the theorem it is not necessary to know
the proof. Do not let any math pedant tell you otherwise.1 By a game mathematics refers to a
conflict between individuals (or entities) with conflicting goals.
Introduction to Methods of Applied Mathematics
For the past few years I have been working on an open source textbook. It contains material on calculus, functions of a complex variable, ordinary differential equations, partial differential equations and the calculus of variations. The text is still under development, but I believe that the current version will be useful for students and instructors.
29 Markov Chains - Free eBook 29 Markov Chains - Download ebook 29 Markov Chains free
|