This post is devoted to a quick proof of a version of the law of large numbers for non-negative random variables in $\mathrm{L}^2$. It does not require independence or same distribution. It is a variant of the famous one for independent random variables bounded in $\mathrm{L}^4$.

The statement. If $X_1,X_2,\ldots$ are non-negative $\mathrm{L}^2$ random variables on $(\Omega,\mathcal{A},\mathbb{P})$ such that $\lim_{n\to\infty}\frac{1}{n}\mathbb{E}(S_n)=\ell\in\mathbb{R}$, and $\mathrm{Var}(S_n)=\mathcal{O}(n)$ where $S_n:=X_1+\cdots+X_n$, then $$\lim_{n\to\infty}\frac{S_n}{n}=\ell\text{ almost surely}.$$

A proof. We have $$\mathbb{E}\Bigr(\sum_n\Bigr(\frac{S_{n^2}-\mathbb{E}(S_{n^2})}{n^2}\Bigr)^2\Bigr)=\sum_n\frac{\mathrm{Var}(S_{n^2})}{n^4}=\mathcal{O}\Bigr(\sum_n\frac{1}{n^2}\Bigr)<\infty.$$ It follows that $\sum_n\Bigr(\frac{S_{n^2}-\mathbb{E}(S_{n^2})}{n^2}\Bigr)^2<\infty$ almost surely, which implies that $\frac{S_{n^2}-\mathbb{E}(S_{n^2})}{n^2}\to0$ almost surely. This gives that $\frac{1}{n^2}S_{n^2}\to\ell$ almost surely. It remains to use the sandwich$$\frac{S_{n^2}}{n^2}\frac{n^2}{(n+1)^2}\leq\frac{S_k}{k}\leq\frac{S_{(n+1)^2}}{(n+1)^2}\frac{(n+1)^2}{n^2}$$ valid if $k$ is such that $n^2\leq k\leq (n+1)^2$ (here we use the non-negative nature of the $X$’s).

Note and further reading. I have learnt this proof recently from my friend Arnaud Guyader who found it in a paper (Theorem 5) by Bernard Delyon and François Portier. It is probably a classic however. It is possible to drop the assumption of being non-negative by using $X=X_+-X_-$, but this would require to modify the remaining assumptions, increasing the complexity, see for instance Theorem 6.2 in Chapter 3 of Erhan Çinlar book (and the blog post comments below). The proof above has the advantage of being quick and beautiful.

Last Updated on 2019-04-25

1. yihong 2019-01-21

Yes this type of subsequence argument can be found in e.g. Theorem 6.2 of E. Cinlar “Probability and Stochastics”

https://www.springer.com/us/book/9780387878584

Therein it was pointed out that for iid sequence one can apply the argument to X_i^+ and X_i^- separately so positivity can be assumed without loss of generality.

2. Djalil Chafaï 2019-01-21

Thanks for the $X_{\pm}$, I have just updated the post accordingly. However the proof in this book is less elegant.

3. amic 2019-01-21

Then you need something more if you do not suppose nonnegativity, right ? iid ?
Because the assumptions on $S_{n,+}$ and $Var(S_{n,+}$ may not be satisfied…

4. Djalil Chafaï 2019-01-22

Yes, and in fact I prefer the initial version. So I reverted the post to its initial version!

5. yihong 2019-01-22

I did not see any essential difference. The “Borel-Cantelli” part is the same. Lemma 1.7 in the book is the sandwich bound of the ratios.

6. Djalil Chafaï 2019-01-22

The difference is essentially in aesthetics and kind of generality, I would say. Nevertheless, I have just added precise references to better suit readers sharing your way of thinking.

7. Rodrigo Pena 2019-01-24

To complement yihong’s answer, it’s Theorem 6.2 of *Chapter III* of Çinlar’s book (the theorems numbers are reset at eat chapter).

8. Jerry 2019-04-22

I don’t understand the Sandwich take X_1 X_2 .. to be deterministic

2, 2, 2, 2, -3, -4, 0, 0, 1

and case n=2, S_4/9=8/9 > S_6/6=1/6.

Maybe you mean X_1 X_2 .. positive, but then the theorem is less interesting.

9. Djalil Chafaï 2019-04-25

Yes, thanks, I’ve fixed the post. Still it remains remarkably beautiful.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Syntax · Style · .