Press "Enter" to skip to content

Law of large numbers

This post is devoted to a quick proof of a version of the law of large numbers for non-negative random variables in $\mathrm{L}^2$. It does not require independence or same distribution. It is a variant of the famous one for independent random variables bounded in $\mathrm{L}^4$.

The statement. If $X_1,X_2,\ldots$ are non-negative $\mathrm{L}^2$ random variables on $(\Omega,\mathcal{A},\mathbb{P})$ such that $\lim_{n\to\infty}\frac{1}{n}\mathbb{E}(S_n)=\ell\in\mathbb{R}$, and $\mathrm{Var}(S_n)=\mathcal{O}(n)$ where $S_n:=X_1+\cdots+X_n$, then $$\lim_{n\to\infty}\frac{S_n}{n}=\ell\text{ almost surely}.$$

A proof. We have $$\mathbb{E}\Bigr(\sum_n\Bigr(\frac{S_{n^2}-\mathbb{E}(S_{n^2})}{n^2}\Bigr)^2\Bigr)=\sum_n\frac{\mathrm{Var}(S_{n^2})}{n^4}=\mathcal{O}\Bigr(\sum_n\frac{1}{n^2}\Bigr)<\infty.$$ It follows that $\sum_n\Bigr(\frac{S_{n^2}-\mathbb{E}(S_{n^2})}{n^2}\Bigr)^2<\infty$ almost surely, which implies that $\frac{S_{n^2}-\mathbb{E}(S_{n^2})}{n^2}\to0$ almost surely. This gives that $\frac{1}{n^2}S_{n^2}\to\ell$ almost surely. It remains to use the sandwich$$\frac{S_{n^2}}{n^2}\frac{n^2}{(n+1)^2}\leq\frac{S_k}{k}\leq\frac{S_{(n+1)^2}}{(n+1)^2}\frac{(n+1)^2}{n^2}$$ valid if $k$ is such that $n^2\leq k\leq (n+1)^2$ (here we use the non-negative nature of the $X$’s).

Note and further reading. I have learnt this proof recently from my friend Arnaud Guyader who found it in a paper (Theorem 5) by Bernard Delyon and François Portier. It is probably a classic however. It is possible to drop the assumption of being non-negative by using $X=X_+-X_-$, but this would require to modify the remaining assumptions, increasing the complexity, see for instance Theorem 6.2 in Chapter 3 of Erhan Çinlar book (and the blog post comments below). The proof above has the advantage of being quick and beautiful.


  1. yihong 2019-01-21

    Yes this type of subsequence argument can be found in e.g. Theorem 6.2 of E. Cinlar “Probability and Stochastics”

    Therein it was pointed out that for iid sequence one can apply the argument to X_i^+ and X_i^- separately so positivity can be assumed without loss of generality.

  2. Djalil Chafaï 2019-01-21

    Thanks for the $X_{\pm}$, I have just updated the post accordingly. However the proof in this book is less elegant.

  3. amic 2019-01-21

    Then you need something more if you do not suppose nonnegativity, right ? iid ?
    Because the assumptions on $S_{n,+}$ and $Var(S_{n,+}$ may not be satisfied…

  4. Djalil Chafaï 2019-01-22

    Yes, and in fact I prefer the initial version. So I reverted the post to its initial version!

  5. yihong 2019-01-22

    I did not see any essential difference. The “Borel-Cantelli” part is the same. Lemma 1.7 in the book is the sandwich bound of the ratios.

  6. Djalil Chafaï 2019-01-22

    The difference is essentially in aesthetics and kind of generality, I would say. Nevertheless, I have just added precise references to better suit readers sharing your way of thinking.

  7. Rodrigo Pena 2019-01-24

    To complement yihong’s answer, it’s Theorem 6.2 of *Chapter III* of Çinlar’s book (the theorems numbers are reset at eat chapter).

  8. Jerry 2019-04-22

    I don’t understand the Sandwich take X_1 X_2 .. to be deterministic

    2, 2, 2, 2, -3, -4, 0, 0, 1

    and case n=2, S_4/9=8/9 > S_6/6=1/6.

    Maybe you mean X_1 X_2 .. positive, but then the theorem is less interesting.

  9. Djalil Chafaï 2019-04-25

    Yes, thanks, I’ve fixed the post. Still it remains remarkably beautiful.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Syntax · Style · .