Yesterday, a colleague of mine asked during a dinner “is there an elementary way to construct martingales which are not Markov chains?” Let us show that the answer is positive, by using a recursive recipe. Let ${{(f_n)}_{n\geq1}}$ be a sequence of functions where ${f_n:\mathbb{R}^{n+1}\rightarrow\mathbb{R}}$. Let ${{(\varepsilon_n)}_{n\geq1}}$ be a sequence of i.i.d. real random variables of zero mean, independent of a real random variable ${X_0}$. We now define the sequence ${{(X_n)}_{n\geq0}}$ by setting, for every ${n\geq0}$,

$X_{n+1}:=f_{n+1}(X_0,\ldots,X_n,\varepsilon_{n+1}).$

We may safely assume that our ingredients ${X_0}$, ${{(f_n)}_{n\geq1}}$, and ${{(\varepsilon_n)}_{n\geq1}}$ are chosen in such a way that ${X_n}$ is integrable for every ${n\geq0}$. The stochastic process ${{(X_n)}_{n\geq0}}$ is a martingale for its natural filtration as soon as for every ${n\geq0}$ and ${x_0,\ldots,x_n\in\mathbb{R}}$,

$\mathbb{E}(f_{n+1}(x_0,\ldots,x_n,\varepsilon_{n+1}))=x_n.$

However, if for instance ${f_{n+1}}$ depends on the first variable ${x_0}$ for every ${n\geq0}$ then ${{(X_n)}_{n\geq0}}$ is not a Markov chain (of any order). This is clearly the case if we take

$f_{n+1}(x_0,\ldots,x_n,\varepsilon) =\varepsilon g_{n+1}(x_0,\ldots,x_n)+x_n$

where ${g_{n+1}:\mathbb{R}^{n+2}\rightarrow\mathbb{R}}$ depends on its first variable for every ${n\geq0}$. This leads to the following simple example of a martingale which is not a Markov chain (of any order):

$X_{n+1}=\varepsilon_{n+1}X_0+X_n.$

Another way to construct martingales which are not Markov chains (of any order) consists in perturbing a martingale. Namely, let ${{(M_n)}_{n\geq0}}$ be a martingale for some filtration, and let ${(X_n)_{n\geq0}}$ be a martingale constructed as above, using ingredients ${X_0}$ and ${{(\varepsilon_n)}_{n\geq1}}$ independent of ${(M_n)_{n\geq0}}$. Then one may define the sequence ${{(Y_n)}_{n\geq0}}$ by ${Y_n=M_n+X_n}$. The stochastic process ${{(Y_n)}_{n\geq0}}$ is a martingale (one can guess the filtration!), but is not a Markov chain (of any order) if ${g_{n+1}}$ depends on the first variable.

Note. A Markov chain (of any order) is a stochastic recursive sequence of finite order, or equivalently an auto-regressive process of finite order (possibly nonlinear). In contrast, the martingale property does not put constraints on the order of recursion, while imposing a linear projection condition. If ${{(M_n)}_{n\geq0}}$ and ${{(M_n’)}_{n\geq0}}$ are two martingales for the same filtration then ${{(M_n+M_n’)}_{n\geq0}}$ is a martingale. In contrast, the sum of two Markov chains is not a Markov chain in general, and various counter examples are available. Here is an elementary counter example provided by my friend Arnaud Guyader: if for all ${n\geq0}$ we set ${X_n=X}$ and ${Y_n=(-1)^nX}$ for some arbitrary random variable ${X}$ not identically zero, then ${S_n=X_n+Y_n=2X\mathbf{1}_{\{n\text{ even}\}}}$ does not define a Markov chain.

Converse. The converse is well known. Namely, if ${{(M_n)}_{n\geq0}}$ is a Markov chain with state space ${E}$ and kernel ${P}$, and if ${f:E\rightarrow\mathbb{R}}$ is such that ${f(M_n)}$ is integrable for every ${n\geq0}$, then ${{(f(M_n))}_{n\geq0}}$ is a martingale as soon as ${f}$ is harmonic, i.e. ${Pf=f}$. In particular, if ${E=\mathbb{R}}$ then ${{(M_n)}_{n\geq0}}$ is a martingale as soon as ${P(x,\cdot)}$ has mean ${x}$ for every ${x}$.

1. Florent Benaych-Georges 2012-01-21

What is the law of the process you plotted some trajectories of ? Is that a random walk ?

2. arthur 2012-03-28

Nice,
I have to keep that example in mind,
thanks

3. muyee 2012-12-03

Dear Professor,

The post is very interesting. I have a problem on the construction of the Martingales which are not a Markov chain by perturbing a given martingale. I do not know why the function $g_{n+1}$ turns to be $g_{n}$, when checking the martingale property of $Y_{n+1}$. Can you give an explaination for me? Thank you very much!

Yours Sincerely,

guangyu

4. Djalil Chafaï 2012-12-03

Yes, it was a typo. Thanks. This is fixed now I think. Best.

5. Maurizio Tiso 2022-09-08

Although it has been a long time, I would like to ask a question in case someone is still watching this webiste.

When we define X_{n+1} := X_n + \epsilon_{n+1}X_0 we are told that the \epsilon_n’s are independent of X_0.
I assume that the filtration is the one generated by F_n=\sigma(X_0, X_1, … , X_n).
Hence when we try to prove that it is a martingale, we use the fact that
E[X_{n+1}|F_n] = X_n + E[\epsilon_{n+1}X_0 |F_n] = X_n +X_0E[\epsilon_{n+1}|F_n]

And this is my question:
How do we know that \epsilon_{n+1} is independent of F_n if we were only told that the epsilon_n’s are independent of X_0 only?

Best Regards,
Maurizio

6. Djalil Chafaï 2022-09-08

Dear Maurizio, $\mathcal{F}_n$ is generated by $X_0$, $\varepsilon_1,\ldots,\varepsilon_n$, so it is independent of $\varepsilon_{n+1}$, because the random variables $X_0,\varepsilon_1,\ldots$ are all independent..

7. Vanessa 2022-10-03

Is it possible to have the MATLAB code you used to plot the graph? Thank you.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Syntax · Style · .