Eigenvalue and Singular Value Inequalities

Victor Bystrov

Keywords

Matrix algebra, Covariance matrices, Eigenvalues, Singular values

Review Status

General review by COMISEF Wiki Admin, 15/01/2009

As eigenvalues of covariance matrices are often used to characterize multivariate dynamic models, we report some inequalities, which may be of use in the inference about eigenvalues.

Assume that we are interested in an $N$-dimensional stochastic process $X_{t}$, which is not observed. Instead, we observe $Y_{t}=X_{t}+\mathcal{E}_{t}$, $t=1,...,T$, where $\mathcal{E}_{t}$ is a vector of disturbances. We can write it in the matrix form, $Y=X+\mathcal{E}$, where $Y$, $X$ and $\mathcal{E}$ are $T\times N$ matrices.

The true covariance matrix $\Sigma =\mathbb{E}(X_{t}-\mathbb{E}X_{t})(X_{t}-\mathbb{E}X_{t})'$ is not observed. Instead, we observe an estimate $S = \Sigma + \Delta$, where $\Delta$ is a matrix of estimation errors (perturbation matrix). Using Weyl's inequalities, we can get bounds for the eigenvalues of $S$ and apply these bounds for the testing of assumptions about the true covariance matrix $\Sigma$.

Weyl's Inequalities (Eigenvalues). Let $\Sigma$ and $\Delta$ be $N\times N$ Hermitian matrices with eigenvalues $\lambda_{1}(\Sigma)\geq\lambda_{2}(\Sigma)\geq...\geq\lambda_{N}(\Sigma)$ and $\lambda_{1}(\Delta)\geq\lambda_{2}(\Delta)\geq...\geq\lambda_{N}(\Delta)$, respectively; let $\lambda_{1}(S)\geq\lambda_{2}(S)\geq...\geq\lambda_{N}(S)$ be eigenvalues of $S=(\Sigma+\Delta)$. Then

(1)
\begin{align} \lambda_{i}(S)\geq \left\{\begin{array}{lcl} \lambda_{i}(\Sigma)&+&\lambda_{N}(\Delta) \\ \lambda_{i+1}(\Sigma)&+&\lambda_{N-1}(\Delta) \\ &\vdots& \\ \lambda_{N}(\Sigma)&+&\lambda_{i}(\Delta) \\ \end{array}\right. \phantom{XXX} \lambda_{i}(S)\leq \left\{\begin{array}{lcl} \lambda_{i}(\Sigma)&+&\lambda_{1}(\Delta) \\ \lambda_{i-1}(\Sigma)&+&\lambda_{2}(\Delta) \\ &\vdots& \\ \lambda_{1}(\Sigma)&+&\lambda_{i}(\Delta) \\ \end{array}\right. \end{align}

There is a singular value version of Weyl's inequalities.

Weyl's Inequalities (Singular Values). Let $X$ and $\mathcal{E}$ be $(T \times N)$ matrices, $r=\min\{T,N\};$ $\sigma_{1}(X)\geq \sigma_{2}(X)\geq ... \geq \sigma_{r}(X)\geq 0$, $\sigma_{1}(\mathcal{E})\geq \sigma_{2}(\mathcal{E})\geq ... \geq \sigma_{r}(\mathcal{E})\geq 0$ and $\sigma_{1}(Y)\geq \sigma_{2}(Y)\geq ... \geq \sigma_{r}(Y)\geq 0$ singular values of $X$, $\mathcal{E}$ and $Y=X+\mathcal{E}$, respectively. Then

$\sigma_{i+j-1}(Y) \leq \sigma_{i}(X)+\sigma_{j}(\mathcal{E}),$ $1\leq i,j \leq r$ with $i+j \leq r+1$

The singular values of $Y$, $X$ and $\mathcal{E}$ are the square roots of the eigenvalues of cross-product matrices $Y'Y$, $X'X$ and $\mathcal{E}'\mathcal{E}$.

For the theory of the eigenvalue and singlular value inequalities see ,  and . For a particular application see .