{"id":11074,"date":"2019-01-21T19:14:11","date_gmt":"2019-01-21T18:14:11","guid":{"rendered":"http:\/\/djalil.chafai.net\/blog\/?p=11074"},"modified":"2019-04-25T21:41:44","modified_gmt":"2019-04-25T19:41:44","slug":"law-of-large-numbers","status":"publish","type":"post","link":"https:\/\/djalil.chafai.net\/blog\/2019\/01\/21\/law-of-large-numbers\/","title":{"rendered":"Law of large numbers"},"content":{"rendered":"\r\n
\r\n
\"\"<\/figure>\r\n<\/div>\r\n\r\n\r\n\r\n

This post is devoted to a quick proof of a version of the law of large numbers for non-negative<\/strong> random variables in $\\mathrm{L}^2$. It does not require independence or same distribution. It is a variant of the famous one for independent random variables bounded in $\\mathrm{L}^4$.<\/p>\r\n\r\n\r\n\r\n

The statement.<\/strong> If $X_1,X_2,\\ldots$ are non-negative $\\mathrm{L}^2$ random variables on $(\\Omega,\\mathcal{A},\\mathbb{P})$ such that $\\lim_{n\\to\\infty}\\frac{1}{n}\\mathbb{E}(S_n)=\\ell\\in\\mathbb{R}$, and $\\mathrm{Var}(S_n)=\\mathcal{O}(n)$ where $S_n:=X_1+\\cdots+X_n$, then $$\\lim_{n\\to\\infty}\\frac{S_n}{n}=\\ell\\text{ almost surely}.$$<\/p>\r\n\r\n\r\n\r\n

A proof.<\/strong> We have $$\\mathbb{E}\\Bigr(\\sum_n\\Bigr(\\frac{S_{n^2}-\\mathbb{E}(S_{n^2})}{n^2}\\Bigr)^2\\Bigr)=\\sum_n\\frac{\\mathrm{Var}(S_{n^2})}{n^4}=\\mathcal{O}\\Bigr(\\sum_n\\frac{1}{n^2}\\Bigr)<\\infty.$$ It follows that $\\sum_n\\Bigr(\\frac{S_{n^2}-\\mathbb{E}(S_{n^2})}{n^2}\\Bigr)^2<\\infty$ almost surely, which implies that $\\frac{S_{n^2}-\\mathbb{E}(S_{n^2})}{n^2}\\to0$ almost surely. This gives that $\\frac{1}{n^2}S_{n^2}\\to\\ell$ almost surely. It remains to use the sandwich$$\\frac{S_{n^2}}{n^2}\\frac{n^2}{(n+1)^2}\\leq\\frac{S_k}{k}\\leq\\frac{S_{(n+1)^2}}{(n+1)^2}\\frac{(n+1)^2}{n^2}$$ valid if $k$ is such that $n^2\\leq k\\leq (n+1)^2$ (here we use the non-negative nature of the $X$'s).<\/p>\r\n\r\n\r\n\r\n

Note and further reading.<\/strong> I have learnt this proof recently from my friend Arnaud Guyader<\/a> who found it in a paper<\/a> (Theorem 5) by Bernard Delyon<\/a> and Fran\u00e7ois Portier<\/a>. It is probably a classic however. It is possible to drop the assumption of being non-negative by using $X=X_+-X_-$, but this would require to modify the remaining assumptions, increasing the complexity, see for instance Theorem 6.2 in Chapter 3 of Erhan \u00c7inlar<\/a> book<\/a> (and the blog post comments below). The proof above has the advantage of being quick and beautiful.<\/p>\r\n","protected":false},"excerpt":{"rendered":"

This post is devoted to a quick proof of a version of the law of large numbers for non-negative random variables in $\\mathrm{L}^2$. It does…<\/p>\n

Continue readingLaw of large numbers<\/span><\/a><\/div>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":235},"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/posts\/11074"}],"collection":[{"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/comments?post=11074"}],"version-history":[{"count":84,"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/posts\/11074\/revisions"}],"predecessor-version":[{"id":11464,"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/posts\/11074\/revisions\/11464"}],"wp:attachment":[{"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/media?parent=11074"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/categories?post=11074"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/tags?post=11074"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}