Exercise. Recently a friend of mine asked the following basic question: suppose that ${H}$ is a ${n\times n}$ Hermitian matrix with ${>0}$ spectrum and suppose that ${P}$ is a ${n\times n}$ orthogonal projection matrix. Is it true that all the non-zero eigenvalues of the product ${HP}$ are above the least eigenvalue of ${H}$?

Quick proof (suggested by a colleague). Since ${P}$ is an orthogonal projection, we have

$P^*=P=P^2.$

Let ${x}$ be an eigenvector of ${HP}$ associated with a non zero eigenvalue ${\lambda}$, meaning that ${HPx=\lambda x\neq0}$. Note that ${Px\neq0}$ otherwise ${HPx=0}$. Now if ${\lambda_*}$ is the smallest non-zero eigenvalue of ${H}$ then ${\lambda_*>0}$ and

$\left\Vert Px\right\Vert_2^2\lambda_* \leq\left<Px,HPx\right> =\left<Px,\lambda x\right> =\lambda\left<Px,Px\right> =\lambda\left\Vert Px\right\Vert_2,$

which shows that ${\lambda}$ is real and ${\lambda_*\leq\lambda}$ as expected.

Comments. There exists a unitary matrix ${U}$ such that ${UPU^*}$ is diagonal with diagonal in ${\{0,1\}}$. If ${H}$ and ${P}$ commute, then ${UHU^*}$ is also diagonal, with diagonal elements in ${\mathbb{R}_+}$, and therefore ${UHPU^*}$ is diagonal with diagonal elements in ${\{0\}\cup\mathrm{spectrum}(H)}$. This shows that the problem has positive answer when ${H}$ and ${P}$ commute.

In the general situation, ${H}$ and ${P}$ do not commute, and the product ${HP}$ is not necessarily Hermitian even if ${H}$ and ${P}$ are Hermitian. We may use the fact that ${AB}$ and ${BA}$ have same spectrum. This is a consequence of the Sylvester determinant theorem which states that

$\det(I+AB)=\det(I+BA).$

As a consequence, if ${A}$ and ${B}$ are both Hermitian with ${\geq0}$ spectrum, then ${AB}$ has the spectrum of the Hermitian matrix ${B^{1/2}A^{1/2}A^{1/2}B^{1/2}}$. Therefore, this spectrum is real and ${\geq0}$, formed by the squared singular values of ${A^{1/2}B^{1/2}}$. In particular, if ${A=H}$ is Hermitian with ${\geq0}$ spectrum and if ${B=P}$ is the matrix of an orthogonal projection, then ${AB=HP}$ has a real ${\geq0}$ spectrum, formed by the squared singular values of ${H^{1/2}P^{1/2}=H^{1/2}P}$ since ${P^{1/2}=P}$ (${P}$ is an orthogonal projection).

Additional comments. If one denotes by ${s_1(M)\geq\cdots\geq s_n(M)\geq0}$ the singular values of a ${n\times n}$ matrix ${M}$, then the Courant-Fischer min-max variational formulas give, for any ${1\leq k\leq n}$,

$s_k(M) =\max_{V\in\mathcal{V}_k}\min_{\substack{v\in V\\\left\Vert v\right\Vert_2=1}}\left\Vert Mv\right\Vert_2 \left(=\min_{V\in\mathcal{V}_{n-k+1}}\max_{\substack{v\in V\\\left\Vert v\right\Vert_2=1}}\left\Vert Mv\right\Vert_2\right)$

where ${\mathcal{V}_k}$ is the set of sub-spaces with dimension ${k}$. Used with ${M=H^{1/2}P}$,

$s_k(H^{1/2}P)=\max_{V\in\mathcal{V}_k}\min_{\substack{v\in V\\\left\Vert v\right\Vert_2=1}} \left\Vert H^{1/2}Pv\right\Vert_2.$

If ${V\cap\mathrm{ker}(P)\neq\{0\}}$ then there exists ${v\in V}$ with ${\left\Vert v\right\Vert_2=1}$ and ${\left\Vert H^{1/2}Pv\right\Vert_2=0}$, hence

$s_k(H^{1/2}P) =\max_{\substack{V\in\mathcal{V}_k\\V\cap\mathrm{ker}(P)\neq\{0\}}} \min_{\substack{v\in V\\\left\Vert v\right\Vert_2=1}} \left\Vert H^{1/2}Pv\right\Vert_2.$

Now if ${k>d}$ where ${d=n-\mathrm{dim}(\mathrm{ker}(P))=\mathrm{dim}(\mathrm{Im}(P))}$, then for every ${V\in\mathcal{V}_k}$, we have ${V\cap\mathrm{ker}(P)\neq\{0\}}$, and thus ${s_k(H^{1/2}P)=0}$.

Let us suppose in contrast now that ${k\leq d}$. Then there exists ${V_*\in\mathcal{V}_k}$ such that ${V_*\subset\mathrm{Im}(P)\perp\mathrm{ker}(P)}$, and thus, for every ${v\in V_*}$ such that ${\left\Vert v\right\Vert_2=1}$, we have ${Pv=v}$, which gives then

$s_k(H^{1/2}P) \geq \min_{\substack{v\in V_*\\\left\Vert v\right\Vert_2=1}} \left\Vert H^{1/2}v\right\Vert_2 \geq \min_{\left\Vert v\right\Vert_2=1}\left\Vert H^{1/2}v\right\Vert_2 = s_n(H^{1/2})=s_n(H)^{1/2}.$

As a conclusion we have, still with ${d=\mathrm{dim}(\mathrm{Im}(P))}$,

$s_1(HP)\geq\cdots\geq s_d(HP)\geq s_n(H)$

$s_{d+1}(HP)=\cdots=s_n(HP)=0.$

Alternative. Actually, the result can be deduced quickly from the following general bound, which is also a consequence of Courant-Fischer formulas: if ${A}$ and ${D}$ are ${n\times n}$ with ${D}$ diagonal then for any ${1\leq k\leq n}$,

$s_n(D)s_k(A)\leq s_k(DA)\leq s_1(D)s_k(A).$

Indeed, if ${P}$ is an orthogonal projector and if ${H=UDU^*}$ is Hermitian with ${\geq0}$ spectrum, then the lower bound in the inequality above used with ${A=U^*P}$ gives

$s_n(H)s_k(P)=s_n(D)s_k(U^*P)\leq s_k(DU^*P)=s_k(UDU^*P)=s_k(HP).$

Now ${s_k(P)=1}$ if ${k\geq d}$, where ${d=n-\mathrm{dim}(\mathrm{ker}(P))=\mathrm{dim}(\mathrm{Im}(P))}$, which gives ${s_k(HP)\geq s_n(H)}$ if ${k\geq d}$. On the other hand, using the upper bound we get

$s_k(HP)=s_k(UDU^*P)=s_k(DU^*P)\leq s_1(D)s_k(U^*P)=s_1(H)s_k(P).$

Now if ${k<d}$ then ${s_k(P)=0}$, which gives ${s_k(HP)=0}$.

Further reading. Many other bounds are available, see for instance the books

Be First to Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Syntax · Style · Tracking & Privacy.