Exercise. Recently a friend of mine asked the following basic question: suppose that \( {H} \) is a \( {n\times n} \) Hermitian matrix with \( {>0} \) spectrum and suppose that \( {P} \) is a \( {n\times n} \) orthogonal projection matrix. Is it true that all the non-zero eigenvalues of the product \( {HP} \) are above the least eigenvalue of \( {H} \)?
Quick proof (suggested by a colleague). Since \( {P} \) is an orthogonal projection, we have
\[ P^*=P=P^2. \]
Let \( {x} \) be an eigenvector of \( {HP} \) associated with a non zero eigenvalue \( {\lambda} \), meaning that \( {HPx=\lambda x\neq0} \). Note that \( {Px\neq0} \) otherwise \( {HPx=0} \). Now if \( {\lambda_*} \) is the smallest non-zero eigenvalue of \( {H} \) then \( {\lambda_*>0} \) and
\[ \left\Vert Px\right\Vert_2^2\lambda_* \leq\left<Px,HPx\right> =\left<Px,\lambda x\right> =\lambda\left<Px,Px\right> =\lambda\left\Vert Px\right\Vert_2, \]
which shows that \( {\lambda} \) is real and \( {\lambda_*\leq\lambda} \) as expected.
Comments. There exists a unitary matrix \( {U} \) such that \( {UPU^*} \) is diagonal with diagonal in \( {\{0,1\}} \). If \( {H} \) and \( {P} \) commute, then \( {UHU^*} \) is also diagonal, with diagonal elements in \( {\mathbb{R}_+} \), and therefore \( {UHPU^*} \) is diagonal with diagonal elements in \( {\{0\}\cup\mathrm{spectrum}(H)} \). This shows that the problem has positive answer when \( {H} \) and \( {P} \) commute.
In the general situation, \( {H} \) and \( {P} \) do not commute, and the product \( {HP} \) is not necessarily Hermitian even if \( {H} \) and \( {P} \) are Hermitian. We may use the fact that \( {AB} \) and \( {BA} \) have same spectrum. This is a consequence of the Sylvester determinant theorem which states that
\[ \det(I+AB)=\det(I+BA). \]
As a consequence, if \( {A} \) and \( {B} \) are both Hermitian with \( {\geq0} \) spectrum, then \( {AB} \) has the spectrum of the Hermitian matrix \( {B^{1/2}A^{1/2}A^{1/2}B^{1/2}} \). Therefore, this spectrum is real and \( {\geq0} \), formed by the squared singular values of \( {A^{1/2}B^{1/2}} \). In particular, if \( {A=H} \) is Hermitian with \( {\geq0} \) spectrum and if \( {B=P} \) is the matrix of an orthogonal projection, then \( {AB=HP} \) has a real \( {\geq0} \) spectrum, formed by the squared singular values of \( {H^{1/2}P^{1/2}=H^{1/2}P} \) since \( {P^{1/2}=P} \) (\( {P} \) is an orthogonal projection).
Additional comments. If one denotes by \( {s_1(M)\geq\cdots\geq s_n(M)\geq0} \) the singular values of a \( {n\times n} \) matrix \( {M} \), then the Courant-Fischer min-max variational formulas give, for any \( {1\leq k\leq n} \),
\[ s_k(M) =\max_{V\in\mathcal{V}_k}\min_{\substack{v\in V\\\left\Vert v\right\Vert_2=1}}\left\Vert Mv\right\Vert_2 \left(=\min_{V\in\mathcal{V}_{n-k+1}}\max_{\substack{v\in V\\\left\Vert v\right\Vert_2=1}}\left\Vert Mv\right\Vert_2\right) \]
where \( {\mathcal{V}_k} \) is the set of sub-spaces with dimension \( {k} \). Used with \( {M=H^{1/2}P} \),
\[ s_k(H^{1/2}P)=\max_{V\in\mathcal{V}_k}\min_{\substack{v\in V\\\left\Vert v\right\Vert_2=1}} \left\Vert H^{1/2}Pv\right\Vert_2. \]
If \( {V\cap\mathrm{ker}(P)\neq\{0\}} \) then there exists \( {v\in V} \) with \( {\left\Vert v\right\Vert_2=1} \) and \( {\left\Vert H^{1/2}Pv\right\Vert_2=0} \), hence
\[ s_k(H^{1/2}P) =\max_{\substack{V\in\mathcal{V}_k\\V\cap\mathrm{ker}(P)\neq\{0\}}} \min_{\substack{v\in V\\\left\Vert v\right\Vert_2=1}} \left\Vert H^{1/2}Pv\right\Vert_2. \]
Now if \( {k>d} \) where \( {d=n-\mathrm{dim}(\mathrm{ker}(P))=\mathrm{dim}(\mathrm{Im}(P))} \), then for every \( {V\in\mathcal{V}_k} \), we have \( {V\cap\mathrm{ker}(P)\neq\{0\}} \), and thus \( {s_k(H^{1/2}P)=0} \).
Let us suppose in contrast now that \( {k\leq d} \). Then there exists \( {V_*\in\mathcal{V}_k} \) such that \( {V_*\subset\mathrm{Im}(P)\perp\mathrm{ker}(P)} \), and thus, for every \( {v\in V_*} \) such that \( {\left\Vert v\right\Vert_2=1} \), we have \( {Pv=v} \), which gives then
\[ s_k(H^{1/2}P) \geq \min_{\substack{v\in V_*\\\left\Vert v\right\Vert_2=1}} \left\Vert H^{1/2}v\right\Vert_2 \geq \min_{\left\Vert v\right\Vert_2=1}\left\Vert H^{1/2}v\right\Vert_2 = s_n(H^{1/2})=s_n(H)^{1/2}. \]
As a conclusion we have, still with \( {d=\mathrm{dim}(\mathrm{Im}(P))} \),
\[ s_1(HP)\geq\cdots\geq s_d(HP)\geq s_n(H) \]
\[ s_{d+1}(HP)=\cdots=s_n(HP)=0. \]
Alternative. Actually, the result can be deduced quickly from the following general bound, which is also a consequence of Courant-Fischer formulas: if \( {A} \) and \( {D} \) are \( {n\times n} \) with \( {D} \) diagonal then for any \( {1\leq k\leq n} \),
\[ s_n(D)s_k(A)\leq s_k(DA)\leq s_1(D)s_k(A). \]
Indeed, if \( {P} \) is an orthogonal projector and if \( {H=UDU^*} \) is Hermitian with \( {\geq0} \) spectrum, then the lower bound in the inequality above used with \( {A=U^*P} \) gives
\[ s_n(H)s_k(P)=s_n(D)s_k(U^*P)\leq s_k(DU^*P)=s_k(UDU^*P)=s_k(HP). \]
Now \( {s_k(P)=1} \) if \( {k\geq d} \), where \( {d=n-\mathrm{dim}(\mathrm{ker}(P))=\mathrm{dim}(\mathrm{Im}(P))} \), which gives \( {s_k(HP)\geq s_n(H)} \) if \( {k\geq d} \). On the other hand, using the upper bound we get
\[ s_k(HP)=s_k(UDU^*P)=s_k(DU^*P)\leq s_1(D)s_k(U^*P)=s_1(H)s_k(P). \]
Now if \( {k<d} \) then \( {s_k(P)=0} \), which gives \( {s_k(HP)=0} \).
Further reading. Many other bounds are available, see for instance the books
- Bhatia, Positive Definite Matrices
- Bhatia, Perturbation Bounds for Matrix Eigenvalues
- Horn and Johnson, Matrix Analysis
- Horn and Johnson, Topics in Matrix Analysis