{"id":3867,"date":"2012-01-20T20:48:52","date_gmt":"2012-01-20T19:48:52","guid":{"rendered":"http:\/\/djalil.chafai.net\/blog\/?p=3867"},"modified":"2024-08-29T17:36:43","modified_gmt":"2024-08-29T15:36:43","slug":"martingales-which-are-not-markov-chains","status":"publish","type":"post","link":"https:\/\/djalil.chafai.net\/blog\/2012\/01\/20\/martingales-which-are-not-markov-chains\/","title":{"rendered":"Martingales which are not Markov chains"},"content":{"rendered":"

\"Martingales\"<\/a><\/p>\n

Yesterday, a colleague of mine asked during a dinner ``is there an elementary way to construct martingales which are not Markov chains?<\/em>'' Let us show that the answer is positive, by using a recursive recipe. Let \\( {{(f_n)}_{n\\geq1}} \\) be a sequence of functions where \\( {f_{n+1}:\\mathbb{R}^{n+1}\\rightarrow\\mathbb{R}} \\). Let \\( {{(\\varepsilon_n)}_{n\\geq1}} \\) be a sequence of i.i.d. real random variables of zero mean, independent of a real random variable \\( {X_0} \\). We now define the sequence \\( {{(X_n)}_{n\\geq0}} \\) by setting, for every \\( {n\\geq0} \\),<\/p>\n

\\[ X_{n+1}:=f_{n+1}(X_0,\\ldots,X_n,\\varepsilon_{n+1}). \\]<\/p>\n

We may safely assume that our ingredients \\( {X_0} \\), \\( {{(f_n)}_{n\\geq1}} \\), and \\( {{(\\varepsilon_n)}_{n\\geq1}} \\) are chosen in such a way that \\( {X_n} \\) is integrable for every \\( {n\\geq0} \\). The stochastic process \\( {{(X_n)}_{n\\geq0}} \\) is a martingale for its natural filtration as soon as for every \\( {n\\geq0} \\) and \\( {x_0,\\ldots,x_n\\in\\mathbb{R}} \\),<\/p>\n

\\[ \\mathbb{E}(f_{n+1}(x_0,\\ldots,x_n,\\varepsilon_{n+1}))=x_n. \\]<\/p>\n

However, if for instance \\( {f_{n+1}} \\) depends on the first variable \\( {x_0} \\) for every \\( {n\\geq0} \\) then \\( {{(X_n)}_{n\\geq0}} \\) is not a Markov chain (of any order). This is clearly the case if we take<\/p>\n

\\[ f_{n+1}(x_0,\\ldots,x_n,\\varepsilon) =\\varepsilon g_{n+1}(x_0,\\ldots,x_n)+x_n \\]<\/p>\n

where \\( {g_{n+1}:\\mathbb{R}^{n+1}\\rightarrow\\mathbb{R}} \\) depends on its first variable for every \\( {n\\geq0} \\). This leads to the following simple example of a martingale which is not a Markov chain (of any order):<\/p>\n

\\[ X_{n+1}=\\varepsilon_{n+1}X_0+X_n. \\]<\/p>\n

Another way to construct martingales which are not Markov chains (of any order) consists in perturbing a martingale. Namely, let \\( {{(M_n)}_{n\\geq0}} \\) be a martingale for some filtration, and let \\( {(X_n)_{n\\geq0}} \\) be a martingale constructed as above, using ingredients \\( {X_0} \\) and \\( {{(\\varepsilon_n)}_{n\\geq1}} \\) independent of \\( {(M_n)_{n\\geq0}} \\). Then one may define the sequence \\( {{(Y_n)}_{n\\geq0}} \\) by \\( {Y_n=M_n+X_n} \\). The stochastic process \\( {{(Y_n)}_{n\\geq0}} \\) is a martingale (one can guess the filtration!), but is not a Markov chain (of any order) if \\( {g_{n+1}} \\) depends on the first variable.<\/p>\n

Note.<\/b> A Markov chain (of any order) is a stochastic recursive sequence of finite order, or equivalently an auto-regressive process of finite order (possibly nonlinear). In contrast, the martingale property does not put constraints on the order of recursion, while imposing a linear projection condition. If \\( {{(M_n)}_{n\\geq0}} \\) and \\( {{(M_n')}_{n\\geq0}} \\) are two martingales for the same filtration then \\( {{(M_n+M_n')}_{n\\geq0}} \\) is a martingale. In contrast, the sum of two Markov chains is not a Markov chain in general, and various counter examples are available. Here is an elementary counter example provided by my friend Arnaud Guyader<\/a>: if for all \\( {n\\geq0} \\) we set \\( {X_n=X} \\) and \\( {Y_n=(-1)^nX} \\) for some arbitrary random variable \\( {X} \\) not identically zero, then \\( {S_n=X_n+Y_n=2X\\mathbf{1}_{\\{n\\text{ even}\\}}} \\) does not define a Markov chain.<\/p>\n

Converse.<\/b> The converse is well known. Namely, if \\( {{(M_n)}_{n\\geq0}} \\) is a Markov chain with state space \\( {E} \\) and kernel \\( {P} \\), and if \\( {f:E\\rightarrow\\mathbb{R}} \\) is such that \\( {f(M_n)} \\) is integrable for every \\( {n\\geq0} \\), then \\( {{(f(M_n))}_{n\\geq0}} \\) is a martingale as soon as \\( {f} \\) is harmonic, i.e. \\( {Pf=f} \\). In particular, if \\( {E=\\mathbb{R}} \\) then \\( {{(M_n)}_{n\\geq0}} \\) is a martingale as soon as \\( {P(x,\\cdot)} \\) has mean \\( {x} \\) for every \\( {x} \\).<\/p>\n","protected":false},"excerpt":{"rendered":"

Yesterday, a colleague of mine asked during a dinner ``is there an elementary way to construct martingales which are not Markov chains?'' Let us show…<\/p>\n

Continue readingMartingales which are not Markov chains<\/span><\/a><\/div>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":2643},"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/posts\/3867"}],"collection":[{"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/comments?post=3867"}],"version-history":[{"count":22,"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/posts\/3867\/revisions"}],"predecessor-version":[{"id":20494,"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/posts\/3867\/revisions\/20494"}],"wp:attachment":[{"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/media?parent=3867"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/categories?post=3867"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/djalil.chafai.net\/blog\/wp-json\/wp\/v2\/tags?post=3867"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}