28 Bayes' Rule

EBook Description: Bayes' rule is a probability benchmark. Once you understand it, it indicates that you have moved on to a new level. Suppose that we have partitioned events into mutually exclusive and exhaustive cases E1, E2, ..., En. That is, exactly one of E1, E2, through, En will occur. Ei might be the weather (such as temperature will be between 10 and 30 degrees). Suppose also, that for any Ei we know the probability of X (which might be the event that the Raiders win). That is,

its book of mathematics on numerical analysis.

29 Markov Chains

Markov chains are one of the most fun tools of probability; they give a lot of power for very little effort. We will restrict ourselves to finite Markov chains. This is not much of a restriction. We get much of the most interesting cases and avoid much of the theory.1 This chapter defines Markov chains and gives you one of the most basic tools for dealing with them: matrices.

30 Classification of States

In a Markov chain, each state can be placed in one of the three classifications.1 Since each state falls into one and only one category, these categories partition the states. The secret of categorizing the states is to find the communicating classes. The states of a Markov chain can be partitioned into these communicating classes. Two states communicate if and only if it is possible to go from each to the other. That is, states A and B communicate if and only if it is possible to go from A to B and from B to A.

31 Geometric Series

Geometric series are a basic artifact of algebra that everyone should know.1 I am teaching them here because they come up remarkably often with Markov chains. The finite geometric series formula is at the heart of many of the fundamental formulas of financial mathematics. All students of the mathematical sciences should be intimately familiar with this topic and have all the formulas memorized.

32 Averages

You should remember the arithmetic average. Given n data points, their arithmetic average is their sum divided by n. Now suppose that we have the average of n numbers, An. We are given a new data point x and we would like to compute the new average of all n + 1 numbers, An + 1. Many people simply add up all n + 1 numbers and then divide by n + 1. However,

33 The Gambler's Ruin

There are many variations of the gambler's ruin problem, but a principal one goes like this. We have i dollars out of a total of n. We flip a coin which has probability p of landing heads, and probability q = 1 ! p of landing tails. If the coin lands heads we gain another dollar, otherwise we lose a dollar.

34 Ergodic Chains

Until this point in the book, results have been derived or proven, or at least the nature of the proof has been indicated. However, proofs of the techniques of Sections 34, 35 and 36 as well as the core concept of Chapter 37 are beyond this book and have been omitted. Do not let this deter you.

35 Mixed Chains

In this chapter we learn how to analyze Markov chains that consists of transient and absorbing states. Later we will see that this analysis extends easily to chains with (nonabsorbing) ergodic states.

36 Poisson Processes

We are going to look at a random process that typifies what most of us think of as random. This process has the added virtues that it is easy to work with and it is used a great deal in mathematical modeling. In fact, it may be used more than any other process of its type. In particular what I am talking about is the Poisson1 process which is described by both the Poisson distribution and the exponential distribution.

37 A Little Game Theory

Game theory is one of the most interesting topics of discrete mathematics. The principal theorem of game theory is sublime and wonderful. We will merely assume this theorem and use it to achieve some of our early insights. To appreciate the theorem it is not necessary to know the proof. Do not let any math pedant tell you otherwise.1 By a game mathematics refers to a conflict between individuals (or entities) with conflicting goals.

• Free IAS eBooks
• Free GRE eBooks
• Free MBA eBooks
• Free GATE eBooks