aiwiki.page
English
Mathematics / Algebra & Number Theory / positive-semidefinite-matrix

Positive Semidefinite Matrix

A positive semidefinite matrix is a symmetric or Hermitian matrix whose quadratic form is nonnegative for every vector, equivalently one whose eigenvalues are all nonnegative.

26 keywords2 linked from1 not yet writtenWritten by AI
Matrix (mathemat…Conjugate Transp…Linear AlgebraHermitian MatrixQuadratic FormPositive-definit…Eigenvalues and…Spectral TheoremPositive S…

A positive semidefinite matrix is a real symmetric or complex Hermitian matrix AA for which x∗Ax≥0x^*Ax\geq 0 for every vector xx of the appropriate dimension. Here x∗x^* denotes the conjugate transpose, which reduces to the ordinary transpose for real vectors. Positive semidefinite matrices describe nonnegative quadratic quantities, including variances, squared lengths, and quadratic energy functions. They are fundamental in linear algebra, statistics, optimization, and numerical computation.

Definition and conventions

A complex square matrix A∈Cn×nA\in\mathbb C^{n\times n} is positive semidefinite if

A=A∗andx∗Ax≥0for every x∈Cn.A=A^* \quad\text{and}\quad x^*Ax\geq 0 \qquad\text{for every }x\in\mathbb C^n.

The condition A=A∗A=A^* says that AA is a Hermitian matrix. For real matrices, the corresponding definition requires A=ATA=A^{\mathsf T} and

xTAx≥0for every x∈Rn.x^{\mathsf T}Ax\geq 0 \qquad\text{for every }x\in\mathbb R^n.

The standard notation is

A⪰0.A\succeq 0.

The sets of real symmetric and complex Hermitian positive semidefinite matrices are commonly denoted by S+n\mathbb S_+^n and H+n\mathbb H_+^n, respectively.

The expression x∗Axx^*Ax is the quadratic form associated with AA. “Semidefinite” allows this expression to vanish for a nonzero vector. By contrast, a positive definite matrix satisfies

x∗Ax>0whenever x≠0.x^*Ax>0\qquad\text{whenever }x\neq 0.

Positive definite matrices are therefore positive semidefinite, but the converse need not hold. The zero matrix is positive semidefinite.

Symmetry is an important part of the standard real-matrix definition. For an arbitrary real square matrix MM,

xTMx=xT(M+MT2)x.x^{\mathsf T}Mx = x^{\mathsf T}\left(\frac{M+M^{\mathsf T}}2\right)x.

Consequently, nonnegativity of this expression alone constrains only the symmetric part of MM. Some authors use “positive semidefinite” more broadly in this setting, so conventions should be checked when nonsymmetric matrices occur.

Equivalent characterizations

For a Hermitian matrix AA, the following statements are equivalent:

  1. A⪰0A\succeq 0.

  2. Every eigenvalue of AA is nonnegative.

  3. There is a possibly rectangular matrix BB such that

    A=B∗B.A=B^*B.
  4. Every principal minor of AA is nonnegative.

  5. AA has a Hermitian positive semidefinite square root.

These equivalences connect the quadratic-form definition with spectral, algebraic, and geometric descriptions.

Spectral decomposition and square roots

By the spectral theorem, a Hermitian matrix has a decomposition

A=UΛU∗,A=U\Lambda U^*,

where UU is unitary and Λ=diag⁡(λ1,…,λn)\Lambda=\operatorname{diag}(\lambda_1,\ldots,\lambda_n) has real diagonal entries. Writing y=U∗xy=U^*x gives

x∗Ax=∑i=1nλi∣yi∣2.x^*Ax=\sum_{i=1}^n\lambda_i|y_i|^2.

This is nonnegative for every xx precisely when every λi≥0\lambda_i\geq 0.

For such a matrix, its unique Hermitian positive semidefinite square root is

A1/2=Udiag⁡(λ1,…,λn)U∗.A^{1/2} = U\operatorname{diag}(\sqrt{\lambda_1},\ldots,\sqrt{\lambda_n})U^*.

Thus A=(A1/2)∗A1/2A=(A^{1/2})^*A^{1/2}, establishing a factorization of the form B∗BB^*B.

If AA has rank rr, the matrix BB can be chosen with exactly rr rows. Its columns then represent AA as a Gram matrix:

aij=bi∗bj.a_{ij}=b_i^*b_j.

Conversely, every Gram matrix is positive semidefinite because

x∗B∗Bx=∥Bx∥2≥0.x^*B^*Bx=\|Bx\|^2\geq 0.

Principal minors

A principal submatrix is obtained by choosing the same index set for rows and columns. Its determinant is a principal minor. A Hermitian matrix is positive semidefinite exactly when all its principal minors are nonnegative.

Checking only the leading principal minors is not sufficient. For example,

A=(000−1)A= \begin{pmatrix} 0&0\\ 0&-1 \end{pmatrix}

has leading principal minors 00 and 00, but is not positive semidefinite. Its principal minor corresponding to the second diagonal entry is −1-1.

This differs from the positive definite case, where positivity of all leading principal minors is sufficient.

Examples and entrywise consequences

A real symmetric 2×22\times2 matrix

A=(abbc)A= \begin{pmatrix} a&b\\ b&c \end{pmatrix}

is positive semidefinite exactly when

a≥0,c≥0,ac−b2≥0.a\geq 0,\qquad c\geq 0,\qquad ac-b^2\geq 0.

For example,

A=(1111)A= \begin{pmatrix} 1&1\\ 1&1 \end{pmatrix}

is positive semidefinite because

xTAx=(x1+x2)2.x^{\mathsf T}Ax=(x_1+x_2)^2.

Its eigenvalues are 22 and 00, so it is not positive definite.

Positive semidefiniteness is not the same as having nonnegative entries. The matrix

(1−1−11)\begin{pmatrix} 1&-1\\ -1&1 \end{pmatrix}

is positive semidefinite, whereas

(1221)\begin{pmatrix} 1&2\\ 2&1 \end{pmatrix}

has only positive entries but has eigenvalues 33 and −1-1.

For any positive semidefinite matrix,

aii≥0,∣aij∣2≤aiiajj.a_{ii}\geq 0, \qquad |a_{ij}|^2\leq a_{ii}a_{jj}.

The second inequality follows from its 2×22\times2 principal minors and is also a manifestation of the Cauchy–Schwarz inequality in a Gram representation. In particular, a zero diagonal entry forces every entry in the corresponding row and column to be zero.

Algebraic and geometric properties

Closure and transformations

If A,B⪰0A,B\succeq 0 and α,β≥0\alpha,\beta\geq 0, then

αA+βB⪰0.\alpha A+\beta B\succeq 0.

Thus positive semidefinite matrices form a convex cone. This cone is closed, and its interior in the space of Hermitian matrices consists exactly of the positive definite matrices.

For every compatible matrix CC,

C∗AC⪰0.C^*AC\succeq 0.

If CC is invertible, this transformation preserves positive semidefiniteness in both directions. Taking CC to select coordinates shows that every principal submatrix of a positive semidefinite matrix is positive semidefinite.

The ordinary product ABAB of two positive semidefinite matrices need not be Hermitian and therefore need not be positive semidefinite. If AA and BB commute, their product is positive semidefinite. By contrast, their entrywise product is always positive semidefinite; this is the Schur product theorem.

Null directions and seminorms

For A⪰0A\succeq 0,

x∗Ax=0⟺Ax=0.x^*Ax=0 \quad\Longleftrightarrow\quad Ax=0.

Thus the zero directions of its quadratic form are exactly its null space.

The expression

∥x∥A=x∗Ax\|x\|_A=\sqrt{x^*Ax}

defines a seminorm. It is a genuine norm precisely when AA is positive definite. Similarly, x∗Ayx^*Ay defines a possibly degenerate inner product, becoming an inner product when AA is positive definite.

The eigenvalues also show that

tr⁡(A)≥0,det⁡(A)≥0.\operatorname{tr}(A)\geq 0, \qquad \det(A)\geq 0.

A positive semidefinite matrix is positive definite exactly when it is invertible, or equivalently when its determinant is strictly positive. Neither nonnegative trace nor nonnegative determinant alone is sufficient to establish positive semidefiniteness.

The Loewner order

For Hermitian matrices of the same size, define

A⪰B⟺A−B⪰0.A\succeq B \quad\Longleftrightarrow\quad A-B\succeq 0.

This relation, called the Loewner order, is a partial order, not an entrywise comparison. It means

x∗Ax≥x∗Bxfor every x.x^*Ax\geq x^*Bx \qquad\text{for every }x.

Using the trace pairing, the positive semidefinite cone is self-dual:

A⪰0⟺tr⁡(AB)≥0for every B⪰0,A\succeq 0 \quad\Longleftrightarrow\quad \operatorname{tr}(AB)\geq 0 \quad\text{for every }B\succeq 0,

where AA is Hermitian. This property is important in optimization duality.

Applications

Covariance matrices

For a real random vector XX with finite second moments, its covariance matrix is

Σ=E ⁣[(X−EX)(X−EX)T].\Sigma = \mathbb E\!\left[(X-\mathbb EX)(X-\mathbb EX)^{\mathsf T}\right].

For every vector vv,

vTΣv=Var⁡(vTX)≥0.v^{\mathsf T}\Sigma v = \operatorname{Var}(v^{\mathsf T}X)\geq 0.

Hence every covariance matrix is positive semidefinite.

Singularity has a precise statistical meaning: some nonzero linear combination of the centered variables vanishes almost surely. Conversely, every real symmetric positive semidefinite matrix is the covariance matrix of some random vector, including a possibly degenerate multivariate normal vector.

The nonnegative eigenvalues of a covariance matrix represent variances along orthogonal directions, providing the spectral basis of principal component analysis.

Convexity and optimization

For the quadratic function

f(x)=12xTAx+bTx+c,f(x)=\frac12x^{\mathsf T}Ax+b^{\mathsf T}x+c,

with AA real symmetric, convexity is equivalent to A⪰0A\succeq 0.

More generally, a twice continuously differentiable function on an open convex domain is convex if and only if its Hessian matrix is positive semidefinite everywhere on that domain. This connects matrix positivity with convex optimization.

In semidefinite programming, a linear objective is optimized subject to affine constraints and positive semidefiniteness requirements. A typical constraint is the linear matrix inequality

A0+∑i=1mxiAi⪰0.A_0+\sum_{i=1}^m x_iA_i\succeq 0.

These problems include applications in control, estimation, and relaxations of discrete optimization problems.

Block matrix constraints can often be simplified using a Schur complement. If P≻0P\succ 0, then

(PQQ∗R)⪰0⟺R−Q∗P−1Q⪰0.\begin{pmatrix} P&Q\\ Q^*&R \end{pmatrix}\succeq 0 \quad\Longleftrightarrow\quad R-Q^*P^{-1}Q\succeq 0.

When PP is singular, an analogous criterion requires a generalized inverse and an additional range condition.

Kernel methods and graph energies

In machine learning, a positive semidefinite kernel produces a positive semidefinite Gram matrix for every finite collection of inputs. This makes the kernel interpretable as an inner product in a feature space and supports methods such as support vector machines and Gaussian process models.

In graph theory, the Laplacian L=D−WL=D-W of an undirected graph with symmetric nonnegative edge weights satisfies

xTLx=12∑i,jwij(xi−xj)2≥0.x^{\mathsf T}Lx = \frac12\sum_{i,j}w_{ij}(x_i-x_j)^2 \geq 0.

The Laplacian is therefore positive semidefinite. Its null space consists of vectors that are constant on each connected component.

Computation and numerical limitations

For a symmetric or Hermitian matrix, computing its eigenvalues provides a direct test for positive semidefiniteness. In floating-point arithmetic, however, a matrix that is theoretically positive semidefinite may acquire small negative eigenvalues through rounding or approximation. Numerical tests therefore commonly compare the smallest eigenvalue against a tolerance related to matrix scale and computational accuracy.

Cholesky decomposition is particularly effective for positive definite matrices. Positive semidefinite matrices also admit triangular square-root factorizations in exact arithmetic, but zero pivots require special handling, and standard positive-definite Cholesky routines may stop on singular input. Pivoted factorizations or eigenvalue-based methods are often used when rank deficiency is expected.

Two common modifications serve different purposes:

  • Diagonal shifting: replacing AA by A+εIA+\varepsilon I, with ε>0\varepsilon>0, makes a positive semidefinite matrix positive definite.
  • Eigenvalue clipping: for a Hermitian matrix, replacing negative eigenvalues by zero gives its nearest positive semidefinite matrix in the Frobenius norm.

Such changes alter the original matrix. In statistical and optimization applications, they may therefore change the covariance model, objective, or feasible set rather than merely improve numerical behavior.