Press "Enter" to skip to content

Martingales which are not Markov chains

Martingales

Yesterday, a colleague of mine asked during a dinner “is there an elementary way to construct martingales which are not Markov chains?” Let us show that the answer is positive, by using a recursive recipe. Let \( {{(f_n)}_{n\geq1}} \) be a sequence of functions where \( {f_n:\mathbb{R}^{n+1}\rightarrow\mathbb{R}} \). Let \( {{(\varepsilon_n)}_{n\geq1}} \) be a sequence of i.i.d. real random variables of zero mean, independent of a real random variable \( {X_0} \). We now define the sequence \( {{(X_n)}_{n\geq0}} \) by setting, for every \( {n\geq0} \),

\[ X_{n+1}:=f_{n+1}(X_0,\ldots,X_n,\varepsilon_{n+1}). \]

We may safely assume that our ingredients \( {X_0} \), \( {{(f_n)}_{n\geq1}} \), and \( {{(\varepsilon_n)}_{n\geq1}} \) are chosen in such a way that \( {X_n} \) is integrable for every \( {n\geq0} \). The stochastic process \( {{(X_n)}_{n\geq0}} \) is a martingale for its natural filtration as soon as for every \( {n\geq0} \) and \( {x_0,\ldots,x_n\in\mathbb{R}} \),

\[ \mathbb{E}(f_{n+1}(x_0,\ldots,x_n,\varepsilon_{n+1}))=x_n. \]

However, if for instance \( {f_{n+1}} \) depends on the first variable \( {x_0} \) for every \( {n\geq0} \) then \( {{(X_n)}_{n\geq0}} \) is not a Markov chain (of any order). This is clearly the case if we take

\[ f_{n+1}(x_0,\ldots,x_n,\varepsilon) =\varepsilon g_{n+1}(x_0,\ldots,x_n)+x_n \]

where \( {g_{n+1}:\mathbb{R}^{n+2}\rightarrow\mathbb{R}} \) depends on its first variable for every \( {n\geq0} \). This leads to the following simple example of a martingale which is not a Markov chain (of any order):

\[ X_{n+1}=\varepsilon_{n+1}X_0+X_n. \]

Another way to construct martingales which are not Markov chains (of any order) consists in perturbing a martingale. Namely, let \( {{(M_n)}_{n\geq0}} \) be a martingale for some filtration, and let \( {(X_n)_{n\geq0}} \) be a martingale constructed as above, using ingredients \( {X_0} \) and \( {{(\varepsilon_n)}_{n\geq1}} \) independent of \( {(M_n)_{n\geq0}} \). Then one may define the sequence \( {{(Y_n)}_{n\geq0}} \) by \( {Y_n=M_n+X_n} \). The stochastic process \( {{(Y_n)}_{n\geq0}} \) is a martingale (one can guess the filtration!), but is not a Markov chain (of any order) if \( {g_{n+1}} \) depends on the first variable.

Note: a Markov chain (of any order) is a stochastic recursive sequence of finite order, or equivalently an auto-regressive process of finite order (possibly nonlinear). In contrast, the martingale property does not put constraints on the order of recursion, while imposing a linear projection condition. If \( {{(M_n)}_{n\geq0}} \) and \( {{(M_n’)}_{n\geq0}} \) are two martingales for the same filtration then \( {{(M_n+M_n’)}_{n\geq0}} \) is a martingale. In contrast, the sum of two Markov chains is not a Markov chain in general (delicate counter examples are available).

Converse: the converse is well known. Namely, if \( {{(M_n)}_{n\geq0}} \) is a Markov chain with state space \( {E} \) and kernel \( {P} \), and if \( {f:E\rightarrow\mathbb{R}} \) is such that \( {f(M_n)} \) is integrable for every \( {n\geq0} \), then \( {{(f(M_n))}_{n\geq0}} \) is a martingale as soon as \( {f} \) is harmonic, i.e. \( {Pf=f} \). In particular, if \( {E=\mathbb{R}} \) then \( {{(M_n)}_{n\geq0}} \) is a martingale as soon as \( {P(x,\cdot)} \) has mean \( {x} \) for every \( {x} \).

6 Comments

  1. Florent Benaych-Georges 2012-01-21

    What is the law of the process you plotted some trajectories of ? Is that a random walk ?

  2. arthur 2012-03-28

    Nice,
    I have to keep that example in mind,
    thanks

  3. muyee 2012-12-03

    Dear Professor,

    The post is very interesting. I have a problem on the construction of the Martingales which are not a Markov chain by perturbing a given martingale. I do not know why the function $g_{n+1}$ turns to be $g_{n}$, when checking the martingale property of $Y_{n+1}$. Can you give an explaination for me? Thank you very much!

    Yours Sincerely,

    guangyu

  4. Djalil Chafaï 2012-12-03

    Yes, it was a typo. Thanks. This is fixed now I think. Best.

Leave a Reply

Your email address will not be published. Required fields are marked *