Eigenvalue and Singular Value Inequalities

Author

Victor Bystrov

Keywords

Matrix algebra, Covariance matrices, Eigenvalues, Singular values

Review Status

General review by COMISEF Wiki Admin, 15/01/2009

As eigenvalues of covariance matrices are often used to characterize multivariate dynamic models, we report some inequalities, which may be of use in the inference about eigenvalues.

Assume that we are interested in an $N$-dimensional stochastic process $X_{t}$, which is not observed. Instead, we observe $Y_{t}=X_{t}+\mathcal{E}_{t}$, $t=1,...,T$, where $\mathcal{E}_{t}$ is a vector of disturbances. We can write it in the matrix form, $Y=X+\mathcal{E}$, where $Y$, $X$ and $\mathcal{E}$ are $T\times N$ matrices.

The true covariance matrix $\Sigma =\mathbb{E}(X_{t}-\mathbb{E}X_{t})(X_{t}-\mathbb{E}X_{t})'$ is not observed. Instead, we observe an estimate $S = \Sigma + \Delta$, where $\Delta$ is a matrix of estimation errors (perturbation matrix). Using Weyl's inequalities, we can get bounds for the eigenvalues of $S$ and apply these bounds for the testing of assumptions about the true covariance matrix $\Sigma$.

Weyl's Inequalities (Eigenvalues). Let $\Sigma$ and $\Delta$ be $N\times N$ Hermitian matrices with eigenvalues $\lambda_{1}(\Sigma)\geq\lambda_{2}(\Sigma)\geq...\geq\lambda_{N}(\Sigma)$ and $\lambda_{1}(\Delta)\geq\lambda_{2}(\Delta)\geq...\geq\lambda_{N}(\Delta)$, respectively; let $\lambda_{1}(S)\geq\lambda_{2}(S)\geq...\geq\lambda_{N}(S)$ be eigenvalues of $S=(\Sigma+\Delta)$. Then

(1)
\begin{align} \lambda_{i}(S)\geq \left\{\begin{array}{lcl} \lambda_{i}(\Sigma)&+&\lambda_{N}(\Delta) \\ \lambda_{i+1}(\Sigma)&+&\lambda_{N-1}(\Delta) \\ &\vdots& \\ \lambda_{N}(\Sigma)&+&\lambda_{i}(\Delta) \\ \end{array}\right. \phantom{XXX} \lambda_{i}(S)\leq \left\{\begin{array}{lcl} \lambda_{i}(\Sigma)&+&\lambda_{1}(\Delta) \\ \lambda_{i-1}(\Sigma)&+&\lambda_{2}(\Delta) \\ &\vdots& \\ \lambda_{1}(\Sigma)&+&\lambda_{i}(\Delta) \\ \end{array}\right. \end{align}

There is a singular value version of Weyl's inequalities.

Weyl's Inequalities (Singular Values). Let $X$ and $\mathcal{E}$ be $(T \times N)$ matrices, $r=\min\{T,N\};$ $\sigma_{1}(X)\geq \sigma_{2}(X)\geq ... \geq \sigma_{r}(X)\geq 0$, $\sigma_{1}(\mathcal{E})\geq \sigma_{2}(\mathcal{E})\geq ... \geq \sigma_{r}(\mathcal{E})\geq 0$ and $\sigma_{1}(Y)\geq \sigma_{2}(Y)\geq ... \geq \sigma_{r}(Y)\geq 0$ singular values of $X$, $\mathcal{E}$ and $Y=X+\mathcal{E}$, respectively. Then

$\sigma_{i+j-1}(Y) \leq \sigma_{i}(X)+\sigma_{j}(\mathcal{E}),$ $1\leq i,j \leq r$ with $i+j \leq r+1$

The singular values of $Y$, $X$ and $\mathcal{E}$ are the square roots of the eigenvalues of cross-product matrices $Y'Y$, $X'X$ and $\mathcal{E}'\mathcal{E}$.

For the theory of the eigenvalue and singlular value inequalities see [1], [2] and [3]. For a particular application see [4].

Internal Links

Concepts
Tutorials
Tips
Related Articles

External links

References
Weblinks
Selected Papers
[1] "Matrix Analysis", Rajendra Bahtia (Springer, 1996)

[2]"Matrix Theory", Joel N. Franklin (Dover Publications, 1993)

[3]"Topics on Matrix Theory", Roger A. Horn, Charles R. Johnson (Cambridge University Press,1994)

[4] "Determining the number of factors from empirical distribution of eigenvalues", Alexei Onatski, Working Paper

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License