This post is about the self-normalized Central Limit Theorem (CLT) involving cumulants, discovered in the 1930s by Fréchet and Shohat. It is simple and useful.
This post is about the self-normalized Central Limit Theorem (CLT) involving cumulants, discovered in the 1930s by Fréchet and Shohat. It is simple and useful.
Cumulants. If $X$ is a real random variable with finite moments of all orders, its cumulants ${(C_k(X))}_{k\geq1}$ are giving by the expansion of \[ t\mapsto\log\mathbb{E}(\mathrm{e}^{\mathrm{i}tX})=\sum_{k=1}^\infty\frac{\mathrm{i}^kt^k}{k!}C_k(X) \] in a neighborhood of $0$. In particular \[ C_1(X)=\mathbb{E}(X) \quad\text{and}\quad C_2(X)=\mathrm{Var}(X)=\mathbb{E}(X^2)-\mathbb{E}(X)^2. \] More generally, for all $k$, there exists a universal polynomial that depends only on $k$ expressing $C_k(X)$ in terms of the moments of $X$ up to order $k$, and conversely.
The Fréchet-Shohat theorem. Let ${(X_n)}_{n\geq1}$ be a sequence of real random variables with finite moments of all order, and such that \[ C_k(X_n)=o_{n\to\infty}(\sqrt{C_2(X_n)}^k) \quad\text{for all $k\geq3$}. \] Then \[ \frac{X_n-\mathbb{E}(X_n)}{\sqrt{\mathrm{Var}(X_n)}} \xrightarrow[n\to\infty]{\mathrm{d}} \mathcal{N}(0,1). \] The condition is satisfied in particular when the variance blows up while the higher order cumulants remain bounded, as $n$ goes to infinity, more precisely \[ \lim_{n\to\infty}C_2(X_n)=+\infty \quad\text{while}\quad \sup_{n\geq1}C_k(X_n) < \infty \quad\text{for all $k\geq3$}. \]
Proof. If we define \[ Y_n=\frac{X_n-\mathbb{E}(X_n)}{\sqrt{\mathrm{Var}(X_n)}}=\frac{X_n-C_1(X_n)}{\sqrt{C_2(X_n)}}, \] then we get, by the basic translation-scaling properties of cumulants \[ C_k(Y_n) =\begin{cases} 0 & \text{if $k=1$} \\ 1 & \text{if $k=2$} \\ \displaystyle\frac{C_k(X_n)}{\sqrt{C_2(X_n)}^k} & \text{if $k\geq3$} \end{cases}. \] Thus $Y_n\to\mathcal{N}(0,1)$ as $n\to\infty$, in the sense of cumulants, hence in the sense of moments. Now the standard normal distribution $\mathcal{N}(0,1)$ is characterized by its moments, according to the Carleman condition, see [Feller, page 222], and therefore, the convergence in the sense of moments implies the convergence in distribution, see for instance [Billingsley, Theorem 30.2].
Usefulness. For certain stochastic models, it is possible to express the cumulants using combinatorics, paving the way to the Fréchet-Shohat theorem. This is for instance the case for determinantal point processes, as observed initially by Costin and Lebowitz, and further studied notably by Soshnikov, and by Rider and Virág.
Further reading.
- Patrick Billingsley
Probability and measure
John Wiley and Sons, third edition (1995) - William Feller
An introduction to probability theory and its applications - Volume II
John Wiley and Sons, second edition (1971) - René Maurice Fréchet and James Alexander Shohat
A proof of the generalized second-limit theorem in the theory of probability
Transactions of the American Mathematical Society 33:533-543 (1931) - Ovidiu Costin and Joel L. Lebowitz
Gaussian fluctuations in random matrices
Physical Review Letters 75 69-72 (1995) - Alexander Soshnikov
Gaussian limits for determinantal random point fields
The Annals of Probability 30 171-181 (2002) - Brian Rider and Balint Virág
The noise in the circular law and the Gaussian free field
International Mathematical Research Notices 2:rnm006 (2007)
About Fréchet and Shohat. René Maurice Fréchet (1878 - 1973) is a famous French mathematician mostly known by students for his contributions to general topology. But actually he also played an important role in the development of probability theory and mathematical statistics, on the theoretical side as well as on the practical side ! James Alexander Shohat (1886 - 1944) is an American-Russian mathematician, who is famous for his work on the moment problem, notably through his monograph on this subject written with Jacob David Tamarkin (1888 - 1945).