Press "Enter" to skip to content

Martingales which are not Markov chains

Martingales

Yesterday, a colleague of mine asked during a dinner “is there an elementary way to construct martingales which are not Markov chains?” Let us show that the answer is positive, by using a recursive recipe. Let \( {{(f_n)}_{n\geq1}} \) be a sequence of functions where \( {f_n:\mathbb{R}^{n+1}\rightarrow\mathbb{R}} \). Let \( {{(\varepsilon_n)}_{n\geq1}} \) be a sequence of i.i.d. real random variables of zero mean, independent of a real random variable \( {X_0} \). We now define the sequence \( {{(X_n)}_{n\geq0}} \) by setting, for every \( {n\geq0} \),

\[ X_{n+1}:=f_{n+1}(X_0,\ldots,X_n,\varepsilon_{n+1}). \]

We may safely assume that our ingredients \( {X_0} \), \( {{(f_n)}_{n\geq1}} \), and \( {{(\varepsilon_n)}_{n\geq1}} \) are chosen in such a way that \( {X_n} \) is integrable for every \( {n\geq0} \). The stochastic process \( {{(X_n)}_{n\geq0}} \) is a martingale for its natural filtration as soon as for every \( {n\geq0} \) and \( {x_0,\ldots,x_n\in\mathbb{R}} \),

\[ \mathbb{E}(f_{n+1}(x_0,\ldots,x_n,\varepsilon_{n+1}))=x_n. \]

However, if for instance \( {f_{n+1}} \) depends on the first variable \( {x_0} \) for every \( {n\geq0} \) then \( {{(X_n)}_{n\geq0}} \) is not a Markov chain (of any order). This is clearly the case if we take

\[ f_{n+1}(x_0,\ldots,x_n,\varepsilon) =\varepsilon g_{n+1}(x_0,\ldots,x_n)+x_n \]

where \( {g_{n+1}:\mathbb{R}^{n+2}\rightarrow\mathbb{R}} \) depends on its first variable for every \( {n\geq0} \). This leads to the following simple example of a martingale which is not a Markov chain (of any order):

\[ X_{n+1}=\varepsilon_{n+1}X_0+X_n. \]

Another way to construct martingales which are not Markov chains (of any order) consists in perturbing a martingale. Namely, let \( {{(M_n)}_{n\geq0}} \) be a martingale for some filtration, and let \( {(X_n)_{n\geq0}} \) be a martingale constructed as above, using ingredients \( {X_0} \) and \( {{(\varepsilon_n)}_{n\geq1}} \) independent of \( {(M_n)_{n\geq0}} \). Then one may define the sequence \( {{(Y_n)}_{n\geq0}} \) by \( {Y_n=M_n+X_n} \). The stochastic process \( {{(Y_n)}_{n\geq0}} \) is a martingale (one can guess the filtration!), but is not a Markov chain (of any order) if \( {g_{n+1}} \) depends on the first variable.

Note. A Markov chain (of any order) is a stochastic recursive sequence of finite order, or equivalently an auto-regressive process of finite order (possibly nonlinear). In contrast, the martingale property does not put constraints on the order of recursion, while imposing a linear projection condition. If \( {{(M_n)}_{n\geq0}} \) and \( {{(M_n’)}_{n\geq0}} \) are two martingales for the same filtration then \( {{(M_n+M_n’)}_{n\geq0}} \) is a martingale. In contrast, the sum of two Markov chains is not a Markov chain in general, and various counter examples are available. Here is an elementary counter example provided by my friend Arnaud Guyader: if for all \( {n\geq0} \) we set \( {X_n=X} \) and \( {Y_n=(-1)^nX} \) for some arbitrary random variable \( {X} \) not identically zero, then \( {S_n=X_n+Y_n=2X\mathbf{1}_{\{n\text{ even}\}}} \) does not define a Markov chain.

Converse. The converse is well known. Namely, if \( {{(M_n)}_{n\geq0}} \) is a Markov chain with state space \( {E} \) and kernel \( {P} \), and if \( {f:E\rightarrow\mathbb{R}} \) is such that \( {f(M_n)} \) is integrable for every \( {n\geq0} \), then \( {{(f(M_n))}_{n\geq0}} \) is a martingale as soon as \( {f} \) is harmonic, i.e. \( {Pf=f} \). In particular, if \( {E=\mathbb{R}} \) then \( {{(M_n)}_{n\geq0}} \) is a martingale as soon as \( {P(x,\cdot)} \) has mean \( {x} \) for every \( {x} \).

9 Comments

  1. Florent Benaych-Georges 2012-01-21

    What is the law of the process you plotted some trajectories of ? Is that a random walk ?

  2. arthur 2012-03-28

    Nice,
    I have to keep that example in mind,
    thanks

  3. muyee 2012-12-03

    Dear Professor,

    The post is very interesting. I have a problem on the construction of the Martingales which are not a Markov chain by perturbing a given martingale. I do not know why the function $g_{n+1}$ turns to be $g_{n}$, when checking the martingale property of $Y_{n+1}$. Can you give an explaination for me? Thank you very much!

    Yours Sincerely,

    guangyu

  4. Djalil Chafaï 2012-12-03

    Yes, it was a typo. Thanks. This is fixed now I think. Best.

  5. Maurizio Tiso 2022-09-08

    Although it has been a long time, I would like to ask a question in case someone is still watching this webiste.

    When we define X_{n+1} := X_n + \epsilon_{n+1}X_0 we are told that the \epsilon_n’s are independent of X_0.
    I assume that the filtration is the one generated by F_n=\sigma(X_0, X_1, … , X_n).
    Hence when we try to prove that it is a martingale, we use the fact that
    E[X_{n+1}|F_n] = X_n + E[\epsilon_{n+1}X_0 |F_n] = X_n +X_0E[\epsilon_{n+1}|F_n]

    And this is my question:
    How do we know that \epsilon_{n+1} is independent of F_n if we were only told that the epsilon_n’s are independent of X_0 only?

    Best Regards,
    Maurizio

  6. Djalil Chafaï 2022-09-08

    Dear Maurizio, $\mathcal{F}_n$ is generated by $X_0$, $\varepsilon_1,\ldots,\varepsilon_n$, so it is independent of $\varepsilon_{n+1}$, because the random variables $X_0,\varepsilon_1,\ldots$ are all independent..

  7. Vanessa 2022-10-03

    Is it possible to have the MATLAB code you used to plot the graph? Thank you.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Syntax · Style · .