Libres pensées d'un mathématicien ordinaire

Yesterday, a colleague of mine asked during a dinner “is there an elementary way to construct martingales which are not Markov chains?” Let us show that the answer is positive, by using a recursive recipe. Let \( {{(f_n)}_{n\geq1}} \) be a sequence of functions where \( {f_n:\mathbb{R}^{n+1}\rightarrow\mathbb{R}} \). Let \( {{(\varepsilon_n)}_{n\geq1}} \) be a sequence of i.i.d. real random variables of zero mean, independent of a real random variable \( {X_0} \). We now define the sequence \( {{(X_n)}_{n\geq0}} \) by setting, for every \( {n\geq0} \),

\[ X_{n+1}:=f_{n+1}(X_0,\ldots,X_n,\varepsilon_{n+1}). \]

We may safely assume that our ingredients \( {X_0} \), \( {{(f_n)}_{n\geq1}} \), and \( {{(\varepsilon_n)}_{n\geq1}} \) are chosen in such a way that \( {X_n} \) is integrable for every \( {n\geq0} \). The stochastic process \( {{(X_n)}_{n\geq0}} \) is a martingale for its natural filtration as soon as for every \( {n\geq0} \) and \( {x_0,\ldots,x_n\in\mathbb{R}} \),

\[ \mathbb{E}(f_{n+1}(x_0,\ldots,x_n,\varepsilon_{n+1}))=x_n. \]

However, if for instance \( {f_{n+1}} \) depends on the first variable \( {x_0} \) for every \( {n\geq0} \) then \( {{(X_n)}_{n\geq0}} \) is not a Markov chain (of any order). This is clearly the case if we take

\[ f_{n+1}(x_0,\ldots,x_n,\varepsilon) =\varepsilon g_{n+1}(x_0,\ldots,x_n)+x_n \]

where \( {g_{n+1}:\mathbb{R}^{n+2}\rightarrow\mathbb{R}} \) depends on its first variable for every \( {n\geq0} \). This leads to the following simple example of a martingale which is not a Markov chain (of any order):

\[ X_{n+1}=\varepsilon_{n+1}X_0+X_n. \]

Another way to construct martingales which are not Markov chains (of any order) consists in perturbing a martingale. Namely, let \( {{(M_n)}_{n\geq0}} \) be a martingale. Let \( {{(\varepsilon_n)}_{n\geq1}} \) be a sequence of i.i.d. random variables of zero mean, independent of \( {{(M_n)}_{n\geq0}} \). Let \( {{(g_n)}_{n\geq1}} \) be a sequence of functions with \( {g_n:\mathbb{R}^n\rightarrow\mathbb{R}} \). Then one may define the sequence \( {{(Y_n)}_{n\geq0}} \) by \( {Y_0=M_0} \), and for every \( {n\geq0} \),

\[ Y_{n+1}=\varepsilon_{n+1}g_{n+1}(Y_0,\ldots,Y_n)+M_{n+1}. \]

The stochastic process \( {{(Y_n)}_{n\geq0}} \) is a martingale (one can easily guess the filtration), but is not a Markov chain (of any order) if \( {g_{n+1}} \) depends on the first variable.

Note: a Markov chain (of any order) is a stochastic recursive sequence of finite order, or equivalently an auto-regressive process of finite order (possibly nonlinear). In contrast, the martingale property does not put constraints on the order of recursion, while imposing a linear projection condition. If \( {{(M_n)}_{n\geq0}} \) and \( {{(M_n’)}_{n\geq0}} \) are two martingales for the same filtration then \( {{(M_n+M_n’)}_{n\geq0}} \) is a martingale. In contrast, the sum of two Markov chains is not a Markov chain in general (delicate counter examples are available).

Converse: the converse is well known. Namely, if \( {{(M_n)}_{n\geq0}} \) is a Markov chain with state space \( {E} \) and kernel \( {P} \), and if \( {f:E\rightarrow\mathbb{R}} \) is such that \( {f(M_n)} \) is integrable for every \( {n\geq0} \), then \( {{(f(M_n))}_{n\geq0}} \) is a martingale as soon as \( {f} \) is harmonic, i.e. \( {Pf=f} \). In particular, if \( {E=\mathbb{R}} \) then \( {{(M_n)}_{n\geq0}} \) is a martingale as soon as \( {P(x,\cdot)} \) has mean \( {x} \) for every \( {x} \).