In linear algebra, the column space of a matrix is the set of all linear combinations of its column vectors. For an matrix over a field , it is a linear subspace of , commonly denoted by . Equivalently, it is the set of all vectors as ranges over . The column space describes which outputs a matrix can produce and which right-hand sides make a linear system solvable.
Definition and interpretation
Write the columns of as , so that
Then
Here, denotes the linear span. Since
the same definition can be written as
The column space contains the zero vector and is closed under vector addition and scalar multiplication. These properties make it a vector space in its own right.
The matrix defines a linear map
Its column space is precisely the image of this map. The image is an intrinsic property of a linear map; its description as a column space arises after choosing bases and representing the map by a matrix.
For real matrices, the geometry is particularly direct. A column space of dimension one is a line through the origin; one of dimension two is a plane through the origin. The column space always lies in the space associated with the matrix’s rows: an matrix has column space in , not generally in .
Dimension, rank, and bases
The dimension of the column space is the rank of the matrix:
Consequently,
Although all columns together span the column space, they need not form a basis. A basis can be obtained by selecting a maximal linearly independent subset of the columns.
Two important cases are:
- Full column rank: . All columns are independent and form a basis of the column space. This requires .
- Full row rank: . The column space equals the entire codomain . This requires .
For a square matrix, the column space is the entire ambient space exactly when an inverse exists.
Finding a basis by elimination
Gaussian elimination identifies pivot columns in an echelon form of . The columns of the original matrix in those pivot positions form a basis for .
This distinction is essential: elementary row operations generally change the column space. If row reduction produces
where is invertible, then
The spaces have equal dimensions, but need not be the same subspace. Row operations preserve the linear dependence relations among the columns, which is why pivot positions still identify a basis in the original matrix.
By contrast, multiplying on the right by an invertible matrix preserves the column space:
Such multiplication replaces the columns by invertible combinations of themselves without changing their span.
Example
Consider the real matrix
Its columns are
Since , the second column adds no new direction. The first and third columns are independent, so
Thus the column space is the plane
in , and has rank two.
The vector belongs to this plane, whereas does not. Consequently, has a solution, but has none.
Solvability and the null space
is consistent if and only if
Equivalently, adjoining as an extra column does not increase the rank:
The column space determines which outputs are attainable. The null space,
determines whether different inputs produce the same output. If is one solution of , every solution has the form
A consistent system therefore has a unique solution exactly when the null space is .
These two spaces are related by the rank–nullity theorem:
Orthogonality and least squares
Over or , equipped with the standard inner product, the column space satisfies
where denotes the orthogonal complement and is the conjugate transpose. For real matrices, .
Indeed, a vector is orthogonal to every column of exactly when . Hence
These orthogonality statements require an inner-product setting; the basic definition of column space does not.
This decomposition underlies least squares. If does not belong to the column space, no exact solution exists, but the problem
finds the attainable output nearest to . The fitted vector
is the orthogonal projection of onto , and its residual satisfies
The fitted vector is unique even when the coefficient vector is not.
If has full column rank, the projection matrix is
Alternatively, if the columns of form an orthonormal basis of the column space, then
In linear regression, the column space of the design matrix is the set of all possible fitted response vectors. Under ordinary least squares, different coefficient vectors can produce the same fit when the design matrix has dependent columns.
Numerical computation
In exact arithmetic, pivot columns determine a basis. With floating-point arithmetic, nearly dependent columns require more care: small perturbations can change the exact rank, so a numerical computation often identifies an effective column space using a tolerance.
A QR decomposition, often with column pivoting, can provide an orthonormal basis for that effective space. The singular value decomposition
gives another characterization: if has exact rank , the left singular vectors corresponding to its nonzero singular values form an orthonormal basis of .
Treating sufficiently small singular values as zero yields an approximation to the column space rather than necessarily its exact algebraic value. The chosen tolerance depends on scale, arithmetic precision, and uncertainty in the matrix entries.