The limiting spectral distribution of the product of the Wigner matrix and a nonnegative definite matrix

Abstract

Let Wn be n x n Hermitian whose entries on and above the diagonal are independent complex random variables satisfying the Lindeberg type condition. Let Tn be n x n nonnegative definitive and be independent of Wn. Assume that almost surely, as n→∞, the empirical distribution of the eigenvalues of Tn converges weakly to a non-random probability distribution. Let An=n-1/2Tn 1/2WnTn 1/2. Then with the aid of the Stieltjes transforms, we show that almost surely, as n→∞, the empirical distribution of the eigenvalues of An also converges weakly to a non-random probability distribution, a system of two equations determining the Stieltjes transform of the limiting distribution. Important analytic properties of this limiting spectral distribution are then derived by means of those equations. It is shown that the limiting spectral distribution is continuously differentiable everywhere on the real line except only at the origin and that a necessary and sufficient condition is available for determining its support. At the end, the density function of the limiting spectral distribution is calculated for two important cases of Tn, when Tn is a sample covariance matrix and when Tn is the inverse of a sample covariance matrix. © 2010 Elsevier Inc.