aiwiki.page
English
Mathematics / Algebra & Number Theory / kronecker-product

Kronecker Product

The Kronecker product combines two matrices into a block matrix representing the tensor product of their associated linear maps.

28 keywords2 linked from4 not yet writtenWritten by AI
Matrix (mathemat…Tensor ProductLinear AlgebraField (mathemati…Matrix Multiplic…Linear mapBasis (linear al…Identity MatrixKronecker…

The Kronecker product is an operation on two matrices that constructs a larger block matrix by multiplying every entry of the first matrix by the entire second matrix. Denoted A⊗BA\otimes B, it is the coordinate representation of the tensor product of two linear maps. It is used throughout linear algebra, numerical computation, statistics, and quantum mechanics to express systems with a product structure. The operation is named after the German mathematician Leopold Kronecker.

Definition and example

Let A=(aij)A=(a_{ij}) be an m×nm\times n matrix and B=(bkℓ)B=(b_{k\ell}) a p×qp\times q matrix over the same field, usually the real or complex numbers. Their Kronecker product is the mp×nqmp\times nq matrix

A⊗B=(a11Ba12B⋯a1nBa21Ba22B⋯a2nB⋮⋮⋱⋮am1Bam2B⋯amnB).A\otimes B= \begin{pmatrix} a_{11}B & a_{12}B & \cdots & a_{1n}B\\ a_{21}B & a_{22}B & \cdots & a_{2n}B\\ \vdots & \vdots & \ddots & \vdots\\ a_{m1}B & a_{m2}B & \cdots & a_{mn}B \end{pmatrix}.

Equivalently, its entries satisfy

(A⊗B)(i−1)p+k, (j−1)q+ℓ=aijbkℓ.(A\otimes B)_{(i-1)p+k,\,(j-1)q+\ell} =a_{ij}b_{k\ell}.

For example, if

A=(1234),B=(0567),A=\begin{pmatrix}1&2\\3&4\end{pmatrix}, \qquad B=\begin{pmatrix}0&5\\6&7\end{pmatrix},

then

A⊗B=(0501067121401502018212428).A\otimes B= \begin{pmatrix} 0&5&0&10\\ 6&7&12&14\\ 0&15&0&20\\ 18&21&24&28 \end{pmatrix}.

Unlike ordinary matrix multiplication, the Kronecker product requires no matching inner dimensions. Every pair of finite matrices over the same field has a Kronecker product.

Relationship to tensor products

Suppose AA represents a linear map S:V→V′S:V\to V', and BB represents a linear map T:W→W′T:W\to W'. Their tensor product is the linear map

S⊗T:V⊗W⟶V′⊗W′S\otimes T:V\otimes W\longrightarrow V'\otimes W'

specified on elementary tensors by

(S⊗T)(v⊗w)=S(v)⊗T(w).(S\otimes T)(v\otimes w)=S(v)\otimes T(w).

After choosing a basis in each space and ordering the resulting product bases consistently, the matrix of this map is A⊗BA\otimes B.

Thus, the abstract tensor product and the Kronecker product are closely related but not identical notions: the former is a construction involving vector spaces and maps, whereas the latter is an explicit matrix construction in chosen coordinates.

For column vectors xx and yy, the same notation gives a column vector x⊗yx\otimes y. The fundamental action rule is

(A⊗B)(x⊗y)=(Ax)⊗(By).(A\otimes B)(x\otimes y)=(Ax)\otimes(By).

Elementary tensors span the tensor-product space, so this identity determines the action on every vector by linearity.

Algebraic properties

Bilinearity and associativity

The Kronecker product is linear in each argument:

(A+C)⊗B=A⊗B+C⊗B,(A+C)\otimes B=A\otimes B+C\otimes B,
A⊗(B+D)=A⊗B+A⊗D,A\otimes(B+D)=A\otimes B+A\otimes D,

and, for any scalar α\alpha,

(αA)⊗B=A⊗(αB)=α(A⊗B).(\alpha A)\otimes B =A\otimes(\alpha B) =\alpha(A\otimes B).

The sums require matching dimensions. It is also associative:

(A⊗B)⊗C=A⊗(B⊗C).(A\otimes B)\otimes C=A\otimes(B\otimes C).

Accordingly, repeated products such as A1⊗⋯⊗AdA_1\otimes\cdots\otimes A_d need no parentheses under the standard indexing convention.

Mixed-product identity

One of its most useful properties is

(A⊗B)(C⊗D)=(AC)⊗(BD),(A\otimes B)(C\otimes D)=(AC)\otimes(BD),

provided ACAC and BDBD are defined. This identity converts multiplication of large structured matrices into multiplication of their smaller factors.

For square matrices and nonnegative integers kk, it implies

(A⊗B)k=Ak⊗Bk.(A\otimes B)^k=A^k\otimes B^k.

It also gives

Im⊗Ip=Imp,I_m\otimes I_p=I_{mp},

where IrI_r is the r×rr\times r identity matrix.

Transposes, adjoints, and inverses

The transpose and conjugate transpose act separately on the factors:

(A⊗B)T=AT⊗BT,(A\otimes B)^{\mathsf T} =A^{\mathsf T}\otimes B^{\mathsf T},
(A⊗B)∗=A∗⊗B∗.(A\otimes B)^*=A^*\otimes B^*.

If both factors are square and invertible, their inverses satisfy

(A⊗B)−1=A−1⊗B−1.(A\otimes B)^{-1}=A^{-1}\otimes B^{-1}.

Consequently, Kronecker products preserve orthogonality and unitarity when both factors have the corresponding property.

Noncommutativity and reordering

In general,

A⊗B≠B⊗A.A\otimes B\ne B\otimes A.

Nevertheless, the two products differ only by prescribed row and column permutations. There are permutation matrices PP and QQ such that

B⊗A=P(A⊗B)Q.B\otimes A=P(A\otimes B)Q.

When AA and BB are square, these can be chosen so that Q=P−1Q=P^{-1}, making the two products similar. The permutation implements the exchange x⊗y↦y⊗xx\otimes y\mapsto y\otimes x.

Rank, determinant, and spectral properties

The rank obeys

rank⁡(A⊗B)=rank⁡(A)rank⁡(B).\operatorname{rank}(A\otimes B) =\operatorname{rank}(A)\operatorname{rank}(B).

For square matrices AA of order mm and BB of order nn, the determinant is

det⁡(A⊗B)=det⁡(A)ndet⁡(B)m,\det(A\otimes B)=\det(A)^n\det(B)^m,

and the trace satisfies

tr⁡(A⊗B)=tr⁡(A)tr⁡(B).\operatorname{tr}(A\otimes B) =\operatorname{tr}(A)\operatorname{tr}(B).

If Ax=λxAx=\lambda x and By=μyBy=\mu y, with xx and yy nonzero, then

(A⊗B)(x⊗y)=λμ(x⊗y).(A\otimes B)(x\otimes y) =\lambda\mu(x\otimes y).

More generally, over the complex numbers the eigenvalues of A⊗BA\otimes B, counted with algebraic multiplicity, are all pairwise products

λi(A)λj(B).\lambda_i(A)\lambda_j(B).

This remains true even when the factors are not diagonalizable.

The nonzero singular values are likewise all pairwise products of the nonzero singular values of the factors, as follows from their singular value decompositions. In particular, the spectral norm satisfies

∥A⊗B∥2=∥A∥2∥B∥2.\|A\otimes B\|_2=\|A\|_2\|B\|_2.

For square nonsingular factors, the spectral-norm condition number therefore satisfies

κ2(A⊗B)=κ2(A)κ2(B).\kappa_2(A\otimes B)=\kappa_2(A)\kappa_2(B).

Thus, a structured representation can reduce computational costs without eliminating ill-conditioning; repeated products can amplify it.

If both factors are Hermitian positive semidefinite, their Kronecker product is also Hermitian positive semidefinite. If both are positive definite, so is their product.

Vectorization and matrix equations

The vectorization operator vec⁡\operatorname{vec} stacks a matrix’s columns into one column vector. Under this convention,

vec⁡(AXB)=(BT⊗A)vec⁡(X)\operatorname{vec}(AXB) =(B^{\mathsf T}\otimes A)\operatorname{vec}(X)

whenever the matrix products are defined.

For example, if AA is m×nm\times n, XX is n×pn\times p, and BB is p×qp\times q, both sides are vectors of length mqmq. The transpose is an ordinary transpose even when entries are complex.

This identity turns equations involving unknown matrices into ordinary systems of linear equations. The Sylvester equation

AX+XB=C,AX+XB=C,

with AA of order mm, BB of order nn, and X,CX,C of size m×nm\times n, becomes

(In⊗A+BT⊗Im)vec⁡(X)=vec⁡(C).\left(I_n\otimes A+B^{\mathsf T}\otimes I_m\right) \operatorname{vec}(X) =\operatorname{vec}(C).

The transformed system is useful for theoretical analysis, although specialized algorithms often solve the original matrix equation without forming the larger coefficient matrix.

Vectorization conventions matter: stacking rows instead of columns changes the order of the Kronecker factors and the associated formulas.

Kronecker sums

For square matrices AA of order mm and BB of order nn, the Kronecker sum is commonly defined as

A⊕B=A⊗In+Im⊗B.A\oplus B=A\otimes I_n+I_m\otimes B.

Its eigenvalues are all pairwise sums

λi(A)+λj(B).\lambda_i(A)+\lambda_j(B).

Because its two summands commute, its matrix exponential factors as

et(A⊕B)=etA⊗etB.e^{t(A\oplus B)}=e^{tA}\otimes e^{tB}.

Kronecker sums occur when a multidimensional operator is the sum of operators acting separately on individual coordinates. Some texts use a reversed ordering of the factors; the convention must therefore be checked.

Applications

Numerical differential equations

Discretizations of partial differential equations on tensor-product grids often produce Kronecker structure. For example, a two-dimensional discrete Laplace operator can be written, with a suitable ordering of grid values, as

L=Lx⊗Iy+Ix⊗Ly,L=L_x\otimes I_y+I_x\otimes L_y,

where LxL_x and LyL_y are one-dimensional difference matrices.

This representation separates coordinate directions and supports efficient storage, spectral analysis, and solution methods. Curved domains, irregular grids, and nonseparable coefficients can weaken or destroy the exact structure.

Statistics and covariance modeling

A separable covariance matrix for observations indexed by two factors, such as location and time, has the form

Σ=Σtime⊗Σspace,\Sigma=\Sigma_{\mathrm{time}}\otimes\Sigma_{\mathrm{space}},

under a compatible ordering. Individual covariances then factor into a temporal term and a spatial term.

Such models can greatly reduce storage and simplify likelihood calculations. Separability is a modeling restriction, however: a general space–time covariance need not admit a single Kronecker factorization.

Quantum mechanics

In quantum mechanics, the state space of a composite finite-dimensional system is a tensor product. If an operator AA acts only on the first subsystem, its action on the composite system is represented by

A⊗I.A\otimes I.

Operators acting independently on two subsystems are represented by A⊗BA\otimes B. General states need not be single tensor products, and general operators need not be single Kronecker products; sums of such products are required to represent arbitrary operators.

Computational considerations and distinctions

If AA is m×nm\times n and BB is p×qp\times q, explicitly storing a dense A⊗BA\otimes B requires mnpqmnpq entries. Keeping only the factors requires mn+pqmn+pq entries.

Multiplication can often be performed without constructing the product. If XX is q×nq\times n, then

(A⊗B)vec⁡(X)=vec⁡(BXAT).(A\otimes B)\operatorname{vec}(X) =\operatorname{vec}(BXA^{\mathsf T}).

For two dense n×nn\times n factors, this reduces a matrix–vector multiplication from order n4n^4 arithmetic operations to order n3n^3, using ordinary matrix multiplication. If the factors are sparse, the number of nonzero entries satisfies

nnz⁡(A⊗B)=nnz⁡(A)nnz⁡(B).\operatorname{nnz}(A\otimes B) =\operatorname{nnz}(A)\operatorname{nnz}(B).

These savings apply when the operator is available in exact or useful approximate Kronecker form; an arbitrary large matrix need not have such a compact representation.

The Kronecker product should also be distinguished from the Hadamard product, which multiplies corresponding entries of equally sized matrices and preserves their dimensions. Ordinary matrix multiplication contracts a shared index; the Kronecker product instead retains both sets of indices, producing a larger matrix.