1. We make the definition that state i is recurrent if $P_i(V_i = \infty)=1$. It is defined to be transient otherwise, i.e if $P_i(V_i = \infty) < 1$. Later (in Theorem 5.3) we show that if $i$ is transient then actually $P_i(V_i = \infty)=0$ (but this is a consequence, not part of our starting definition of transience).
2. In the proof of Theorem 5.4 we use the fact that $p_{ii}^{(n+m+r)} \geq p_{ij}^{(n)} p_{jj}^{(r)} p_{ji}^{(m)}$. Please don't think that we are using any summation notation! (We never use summation convention in this course.) This inequality is a simply product of three terms on the right hand side and is a simple consequence of the fact that one way to go $i\to i$ in $n+m+r$ steps is to first take $n$ steps to go $i\to j$, then $r$ steps to go $j\to j$, and finally $m$ steps to go $j\to i$. There is a $\geq$ because there are other ways to go $i\to i$ in $n+m+r$ steps.
3. The proof of Theorem 5.3 can be presented in one line:
\begin{align*}
\sum_{n=0}^\infty p_{ii}^{(n)}&=
\sum_{n=0}^\infty E_i\left(1_{\{X_n=i\}}\right)
=E_i\left(\sum_{n=0}^\infty 1_{\{X_n=i\}}\right)=E_i(V_i)\\[12pt]
&=\sum_{r=0}^\infty P_i(V_i> r)
=\sum_{r=0}^\infty f_i^r=
\left\{\begin{array}{cc}
\infty, & f_i=1\\[6pt]
\frac{1}{1-f_i}, & f_i<1 \end{array}\right. \end{align*} Remember that if $A$ is an event then $P(A)=E(1_{\{A\}})$, where $1_{\{A\}}$ is the indicator random variable that $=1$ or $=0$ as $A$ does or does not occur.
\begin{align*}
\sum_{n=0}^\infty p_{ii}^{(n)}&=
\sum_{n=0}^\infty E_i\left(1_{\{X_n=i\}}\right)
=E_i\left(\sum_{n=0}^\infty 1_{\{X_n=i\}}\right)=E_i(V_i)\\[12pt]
&=\sum_{r=0}^\infty P_i(V_i> r)
=\sum_{r=0}^\infty f_i^r=
\left\{\begin{array}{cc}
\infty, & f_i=1\\[6pt]
\frac{1}{1-f_i}, & f_i<1 \end{array}\right. \end{align*} Remember that if $A$ is an event then $P(A)=E(1_{\{A\}})$, where $1_{\{A\}}$ is the indicator random variable that $=1$ or $=0$ as $A$ does or does not occur.
4. In Theorem 5.5 we gave an important way to check if a state is recurrent or transient, in terms of the summability of the $p_{ii}^{(n)}$. This criterion will be used in Lecture 6. There are other ways to check for transience. One other way is explained in Theorem 5.9. This is to solve the RHE for the minimal solution to
\[
y_j = \sum_k p_{jk} y_k,\quad j \neq i,\quad \text{and } y_i=1.
\]So $y_j =P_j(\text{return to }i)$. Now check the value of $\sum_k p_{ik} y_k$. If it is $< 1$ then $i$ is transient. This is essentially the content of Theorem 5.9, which I have put in my published notes (in blue type) but am not going to discuss in lectures. However, you may find it helpful to read the Theorem. It's proof is simple.
\[
y_j = \sum_k p_{jk} y_k,\quad j \neq i,\quad \text{and } y_i=1.
\]So $y_j =P_j(\text{return to }i)$. Now check the value of $\sum_k p_{ik} y_k$. If it is $< 1$ then $i$ is transient. This is essentially the content of Theorem 5.9, which I have put in my published notes (in blue type) but am not going to discuss in lectures. However, you may find it helpful to read the Theorem. It's proof is simple.
