Linear algebra is the branch of mathematics concerned with vector spaces, linear maps, and their representation through matrices. It investigates structures in which objects can be added and multiplied by scalars, and transformations that preserve these operations. Its central problems include solving linear equations, describing independent directions, and understanding transformations through invariant properties. The subject combines abstract reasoning with computational methods, connecting algebraic formulas to geometric interpretations. (linear.axler.net)
Vector spaces and coordinates
A vector space consists of vectors together with addition and scalar multiplication satisfying specified axioms. Scalars belong to a field, commonly the real or complex numbers. Familiar examples include coordinate vectors, but polynomials and suitable collections of functions also form vector spaces. Thus, a vector need not be an arrow in physical space: its defining feature is participation in these algebraic operations. (linear.axler.net)
A linear combination has the form [ a_1v_1+\cdots+a_kv_k. ] The span of a collection comprises all its linear combinations. Linear independence means that a combination equals zero only when every coefficient is zero. A basis is an independent collection spanning the entire space; every vector has a unique coordinate expression relative to it. The number of basis vectors in a finite-dimensional space is its dimension. Coordinates depend on the chosen basis, whereas the underlying vector and dimension do not. (linear.axler.net)
Linear maps and matrices
A map (T:V\to W) is linear when [ T(au+bv)=aT(u)+bT(v) ] for all vectors (u,v) and scalars (a,b). Once bases are selected for finite-dimensional domain and codomain spaces, a matrix represents the map. Its columns are the coordinates of the images of the domain basis vectors. Matrix multiplication corresponds to composition of linear maps; changing bases changes the representing matrix without changing the map itself. (linear.axler.net)
A linear subspace is a subset that is itself a vector space under the inherited operations. Two important subspaces associated with a map are its kernel, consisting of vectors sent to zero, and its image, consisting of all possible outputs. For a matrix, the image is its column space, whose dimension is the rank. The rank–nullity theorem states that the dimensions of the kernel and image sum to the dimension of the domain. (github.com)
Linear equations and invertibility
A system of linear equations can be written [ Ax=b, ] where (A) contains coefficients, (x) contains unknowns, and (b) contains specified values. A solution exists exactly when (b) belongs to the column space of (A). If (x_0) is one solution, all solutions are (x_0+z), where (Az=0). This separates the question of existence from the question of uniqueness. (github.com)
Gaussian elimination uses elementary row operations to produce an echelon form, revealing pivots, free variables, and inconsistent equations. A square matrix is invertible precisely when its rank equals its size, equivalently when its kernel is zero. Its inverse then gives the unique solution (x=A^{-1}b), although computational methods generally solve the system without explicitly forming that inverse. (web.mit.edu)
The determinant assigns a scalar to a square matrix and vanishes exactly when the matrix is singular. For real matrices acting on Euclidean space, its absolute value describes the factor by which volumes are scaled; its sign indicates whether orientation is preserved or reversed. Determinants therefore connect solvability with geometry. (web.mit.edu)
Orthogonality and approximation
An inner product supplies notions of length, angle, and orthogonality. In real coordinate space, the standard inner product is (u^{T}v). Orthonormal bases simplify computations because their vectors have unit length and are mutually perpendicular. Orthogonal projection identifies the nearest vector in a subspace and provides the geometric foundation of least-squares approximation. (ocw.mit.edu)
When (Ax=b) is inconsistent, least squares minimizes (|Ax-b|_2^2). At a minimizer, the residual is orthogonal to every column of (A), yielding the normal equations (A^{T}Ax=A^{T}b). This is fundamental to ordinary least squares in statistical modeling. QR factorization and singular value decomposition provide alternative solution methods with numerical advantages over directly forming the normal equations. (arxiv.org)
Eigenvalues and decompositions
Eigenvalues and eigenvectors describe nonzero vectors satisfying (Av=\lambda v): the transformation acts on such a vector by scalar multiplication. If a square matrix has a basis of eigenvectors, diagonalization expresses it as (A=PDP^{-1}), with (D) diagonal. This simplifies powers of matrices and the analysis of repeated transformations. Not every square matrix is diagonalizable. (linear.axler.net)
The spectral theorem guarantees an orthonormal eigenbasis for real symmetric matrices and, over complex spaces, for normal matrices. The singular value decomposition applies more generally to every real or complex rectangular matrix: [ A=U\Sigma V^{*}, ] where (U,V) are unitary and (\Sigma) contains nonnegative singular values. It exposes rank and supports low-rank approximation and minimum-norm least-squares solutions. (linear.axler.net)
Computation and applications
Numerical linear algebra studies algorithms under finite-precision arithmetic, addressing sensitivity to input perturbations, rounding errors, and efficient matrix factorizations. A mathematically valid formula may be computationally unreliable when a problem is ill-conditioned. (arxiv.org)
Applications include fitting models in statistics, principal component analysis, and other forms of machine learning. Linear algebra also analyzes systems of differential equations, networks, and Markov chains. In each setting, the same structural questions recur: which outputs are attainable, which directions are independent, and how transformations behave under iteration. (arxiv.org)