 Yesterday, a colleague of mine asked during a dinner “is there an elementary way to construct martingales which are not Markov chains?” Let us show that the answer is positive, by using a recursive recipe. Let ${{(f_n)}_{n\geq1}}$ be a sequence of functions where ${f_n:\mathbb{R}^{n+1}\rightarrow\mathbb{R}}$. Let ${{(\varepsilon_n)}_{n\geq1}}$ be a sequence of i.i.d. real random variables of zero mean, independent of a real random variable ${X_0}$. We now define the sequence ${{(X_n)}_{n\geq0}}$ by setting, for every ${n\geq0}$,

$X_{n+1}:=f_{n+1}(X_0,\ldots,X_n,\varepsilon_{n+1}).$

We may safely assume that our ingredients ${X_0}$, ${{(f_n)}_{n\geq1}}$, and ${{(\varepsilon_n)}_{n\geq1}}$ are chosen in such a way that ${X_n}$ is integrable for every ${n\geq0}$. The stochastic process ${{(X_n)}_{n\geq0}}$ is a martingale for its natural filtration as soon as for every ${n\geq0}$ and ${x_0,\ldots,x_n\in\mathbb{R}}$,

$\mathbb{E}(f_{n+1}(x_0,\ldots,x_n,\varepsilon_{n+1}))=x_n.$

However, if for instance ${f_{n+1}}$ depends on the first variable ${x_0}$ for every ${n\geq0}$ then ${{(X_n)}_{n\geq0}}$ is not a Markov chain (of any order). This is clearly the case if we take

$f_{n+1}(x_0,\ldots,x_n,\varepsilon) =\varepsilon g_{n+1}(x_0,\ldots,x_n)+x_n$

where ${g_{n+1}:\mathbb{R}^{n+2}\rightarrow\mathbb{R}}$ depends on its first variable for every ${n\geq0}$. This leads to the following simple example of a martingale which is not a Markov chain (of any order):

$X_{n+1}=\varepsilon_{n+1}X_0+X_n.$

Another way to construct martingales which are not Markov chains (of any order) consists in perturbing a martingale. Namely, let ${{(M_n)}_{n\geq0}}$ be a martingale for some filtration, and let ${(X_n)_{n\geq0}}$ be a martingale constructed as above, using ingredients ${X_0}$ and ${{(\varepsilon_n)}_{n\geq1}}$ independent of ${(M_n)_{n\geq0}}$. Then one may define the sequence ${{(Y_n)}_{n\geq0}}$ by ${Y_n=M_n+X_n}$. The stochastic process ${{(Y_n)}_{n\geq0}}$ is a martingale (one can guess the filtration!), but is not a Markov chain (of any order) if ${g_{n+1}}$ depends on the first variable.

Note: a Markov chain (of any order) is a stochastic recursive sequence of finite order, or equivalently an auto-regressive process of finite order (possibly nonlinear). In contrast, the martingale property does not put constraints on the order of recursion, while imposing a linear projection condition. If ${{(M_n)}_{n\geq0}}$ and ${{(M_n’)}_{n\geq0}}$ are two martingales for the same filtration then ${{(M_n+M_n’)}_{n\geq0}}$ is a martingale. In contrast, the sum of two Markov chains is not a Markov chain in general (delicate counter examples are available).

Converse: the converse is well known. Namely, if ${{(M_n)}_{n\geq0}}$ is a Markov chain with state space ${E}$ and kernel ${P}$, and if ${f:E\rightarrow\mathbb{R}}$ is such that ${f(M_n)}$ is integrable for every ${n\geq0}$, then ${{(f(M_n))}_{n\geq0}}$ is a martingale as soon as ${f}$ is harmonic, i.e. ${Pf=f}$. In particular, if ${E=\mathbb{R}}$ then ${{(M_n)}_{n\geq0}}$ is a martingale as soon as ${P(x,\cdot)}$ has mean ${x}$ for every ${x}$.

1. Florent Benaych-Georges 2012-01-21

What is the law of the process you plotted some trajectories of ? Is that a random walk ?

2. arthur 2012-03-28

Nice,
I have to keep that example in mind,
thanks

3. muyee 2012-12-03

Dear Professor,

The post is very interesting. I have a problem on the construction of the Martingales which are not a Markov chain by perturbing a given martingale. I do not know why the function $g_{n+1}$ turns to be $g_{n}$, when checking the martingale property of $Y_{n+1}$. Can you give an explaination for me? Thank you very much!

Yours Sincerely,

guangyu

4. Djalil Chafaï 2012-12-03

Yes, it was a typo. Thanks. This is fixed now I think. Best.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Syntax · Style · Tracking & Privacy.