Today's lecture was a high point in our course: the proof by coupling that for an ergodic Markov chain $P(X_n=i)\to\pi_i$ as $n\to\infty$.
A couple points raised by questions that students asked me about at the end:
I mentioned Vincent Doblin (1915-40) [also known as Wolfgang Doeblin] to whom is due the coupling proof of Theorem 9.8. There is a good article about his life in a 2001 article in The Telegraph: Revealed: the maths genius of the Maginot line. Some of Doblin's work was only discovered in summer 2000, having been sealed in an envelope for 60 years.
I quoted J. Michael Steele on coupling:
Coupling is one of the most powerful of the "genuinely probabilistic" techniques. Here by "genuinely probabilistic'' we mean something that works directly with random variables rather than with their analytical co-travelers (like distributions, densities, or characteristic functions.
I like the label "analytic co-travelers". The coupling proof should convince you that Probability is not merely a branch of Analysis. The above quote is taken from Mike's blog for his graduate course on Advanced Probability at Wharton (see here). Mike has a fascinating and very entertaining web site, that is full of "semi-random rants" and "favorite quotes" that are both funny and wise (such as his "advice to graduate students"). I highly recommend browsing his web site. It was by reading the blogs for his courses that I had the idea of trying something similar myself. So if you have been enjoying this course blog - that is partly due to him. Here is a picture of Mike and me with Thomas Bruss at the German Open Conference in Probability and Statistics 2010, in Leipzig.
The playing cards for the magic trick that I did in the lecture were "dealt" with the help of the playing card shuffler at random.org.
The following is a little sidebar for those of you who like algebra and number theory.
In the proof of Lemma 9.5 we used the fact that if the greatest common divisor of $n_1,\dotsc, n_k$ is $1$ then for all sufficiently large $n$ there exist some non-negative integers $a_1,\dotsc, a_k$ such that
\begin{align}
n &= a_1 n_1 + \cdots + a_k n_k.\tag{*}
\end{align}
Proof. Think about the smallest positive integer $d$ that can be expressed as $d = b_1 n_1 + \cdots + b_k n_k$ for some integers $b_1,\dotsc, b_k$ (which may be negative as well as non-negative). Notice that the remainder of $n_1$ divided by $d$ is also of this form, since it is $r =n_1 − m (b_1 n_1 + \cdots + b_k n_k$) for $m=\lfloor n_1 /d\rfloor$.
If $d$ does not divide $n_1$ then $r< d$, and $d$ fails to be the smallest integer that can be expressed in the form $d = b_1 n_1 + \cdots + b_k n_k$. So $d$ divides $n_1$. The same must be true for every other $n_j$, and so $d=\text{gcd}(n_1,\dotsc,n_k)=1$.
So now we know that it is possible to write $1 = b_1 n_1 + \cdots + b_k n_k$, and so we can also write $j = j (b_1 n_1 + \cdots + b_k n_k$), for all $j=1,\dotsc, n_1$. Finally, we can leverage this fact to conclude that for some large $N$ we can write all of $N, N+1, N+2,\dotsc, N+n_1$ in the required form (*), and hence we can also express in form (*) all integers $N + m n_1 + j$, where $m$ and $j$ are non-negative integers, i.e. we can do this for all integers $n\geq N$.
A couple points raised by questions that students asked me about at the end:
- If a finite state space Markov chain is aperiodic and irreducible then it is regular. I think I forgot to include the words "finite state space" when lecturing, but they are in this notes. This not true for an infinite state space chain. Consider the Makov chain on the non-negative integers with $p_{i,i+1}=p_{i,0}=1/2$. Then $p_{100,100}^{(n)}>0$ only if $n\geq 101$. So there is no $n$ for which $P^n>0$, i.e. $p_{ij}^{(n)}>0$ for all $i,j$.
- In the proof of Theorem 9.8 the assumption of positive recurrence guarantees that there exists an invariant distribution $\pi$. From this we can see that $\pi\times \pi$ is the invariant distribution of the chain with state $W_n=(X_n,Y_n)$, and it existence implies that this chain is recurrent, and hence that $P(T<\infty)=1$. If we only knew that $X_n$ were recurrent (and so possibly null recurrent), we would only be able to claim the existence of an invariant measure $\lambda$, and $\lambda\times\lambda$ would be an invariant measure for $W_n$. But existence of an invariant measure does not imply recurrence (see Example 9.3).
I mentioned Vincent Doblin (1915-40) [also known as Wolfgang Doeblin] to whom is due the coupling proof of Theorem 9.8. There is a good article about his life in a 2001 article in The Telegraph: Revealed: the maths genius of the Maginot line. Some of Doblin's work was only discovered in summer 2000, having been sealed in an envelope for 60 years.
I quoted J. Michael Steele on coupling:
Coupling is one of the most powerful of the "genuinely probabilistic" techniques. Here by "genuinely probabilistic'' we mean something that works directly with random variables rather than with their analytical co-travelers (like distributions, densities, or characteristic functions.
I like the label "analytic co-travelers". The coupling proof should convince you that Probability is not merely a branch of Analysis. The above quote is taken from Mike's blog for his graduate course on Advanced Probability at Wharton (see here). Mike has a fascinating and very entertaining web site, that is full of "semi-random rants" and "favorite quotes" that are both funny and wise (such as his "advice to graduate students"). I highly recommend browsing his web site. It was by reading the blogs for his courses that I had the idea of trying something similar myself. So if you have been enjoying this course blog - that is partly due to him. Here is a picture of Mike and me with Thomas Bruss at the German Open Conference in Probability and Statistics 2010, in Leipzig.
The playing cards for the magic trick that I did in the lecture were "dealt" with the help of the playing card shuffler at random.org.
The following is a little sidebar for those of you who like algebra and number theory.
In the proof of Lemma 9.5 we used the fact that if the greatest common divisor of $n_1,\dotsc, n_k$ is $1$ then for all sufficiently large $n$ there exist some non-negative integers $a_1,\dotsc, a_k$ such that
\begin{align}
n &= a_1 n_1 + \cdots + a_k n_k.\tag{*}
\end{align}
Proof. Think about the smallest positive integer $d$ that can be expressed as $d = b_1 n_1 + \cdots + b_k n_k$ for some integers $b_1,\dotsc, b_k$ (which may be negative as well as non-negative). Notice that the remainder of $n_1$ divided by $d$ is also of this form, since it is $r =n_1 − m (b_1 n_1 + \cdots + b_k n_k$) for $m=\lfloor n_1 /d\rfloor$.
If $d$ does not divide $n_1$ then $r< d$, and $d$ fails to be the smallest integer that can be expressed in the form $d = b_1 n_1 + \cdots + b_k n_k$. So $d$ divides $n_1$. The same must be true for every other $n_j$, and so $d=\text{gcd}(n_1,\dotsc,n_k)=1$.
So now we know that it is possible to write $1 = b_1 n_1 + \cdots + b_k n_k$, and so we can also write $j = j (b_1 n_1 + \cdots + b_k n_k$), for all $j=1,\dotsc, n_1$. Finally, we can leverage this fact to conclude that for some large $N$ we can write all of $N, N+1, N+2,\dotsc, N+n_1$ in the required form (*), and hence we can also express in form (*) all integers $N + m n_1 + j$, where $m$ and $j$ are non-negative integers, i.e. we can do this for all integers $n\geq N$.
