A matrix is a rectangular array of entries arranged in rows and columns. Its entries are usually numbers, although other mathematical objects may also be used. Matrices are central objects in linear algebra, providing a compact notation for systems of linear equations and a computational representation of linear transformations. Their significance lies not merely in storing values but in the operations defined on them, especially addition and multiplication. (deeplearningbook.org)
Notation and dimensions
A matrix with (m) rows and (n) columns has size, or shape, (m\times n). It is commonly written (A=(a_{ij})), where (a_{ij}) denotes the entry in row (i), column (j). For example,
[ A=\begin{pmatrix} 1&2&3\ 4&5&6 \end{pmatrix} ]
is a (2\times3) matrix. Two matrices are equal precisely when their shapes and corresponding entries agree. Entries commonly belong to the real numbers or complex numbers. A matrix with one column is a column vector; one with one row is a row vector. If (m=n), the matrix is square. (deeplearningbook.org)
The transpose (A^{\mathsf T}) exchanges rows and columns, so that ((A^{\mathsf T}){ij}=a{ji}). For complex matrices, the conjugate transpose (A^*) additionally replaces each entry by its complex conjugate. These operations are important when expressing inner products and orthogonality. (deeplearningbook.org)
Matrix operations
Matrices of the same shape can be added entry by entry. Scalar multiplication multiplies every entry by the same scalar:
[ (A+B){ij}=a{ij}+b_{ij}, \qquad (cA){ij}=ca{ij}. ]
Together, these operations make the matrices of a fixed shape over a field into a vector space. (deeplearningbook.org)
Matrix multiplication follows a different rule. If (A) is (m\times n) and (B) is (n\times p), their product (AB) is (m\times p), with
[ (AB){ij}=\sum{k=1}^{n}a_{ik}b_{kj}. ]
Thus, each output entry is the dot product of a row of (A) with a column of (B). The matching inner dimensions are essential: some pairs of matrices cannot be multiplied in a given order. (math.mit.edu)
Multiplication is associative and distributes over addition, but generally is not commutative: (AB) need not equal (BA), even when both products exist. Ordinary matrix multiplication also differs from entrywise multiplication. The identity matrix (I_n), whose diagonal entries are one and other entries zero, satisfies (AI_n=A) for any matrix with (n) columns. (deeplearningbook.org)
Linear transformations and equations
Once bases have been chosen, a linear transformation between finite-dimensional vector spaces can be represented by a matrix. An (m\times n) matrix acts on an (n)-component column vector through (x\mapsto Ax). Its columns are the images of the input basis vectors, and (Ax) is their linear combination with coefficients supplied by (x). Matrix multiplication represents composition: (ABx) applies (B) first, then (A). (math.mit.edu)
A system of linear equations can therefore be written as
[ Ax=b. ]
Here (A) contains the coefficients, (x) the unknowns, and (b) the specified outputs. The system has a solution exactly when (b) lies in the span of the columns of (A). Gaussian elimination uses elementary row operations to simplify the system while preserving its solutions. (math.mit.edu)
A square matrix has a matrix inverse if there is a matrix (A^{-1}) satisfying (AA^{-1}=A^{-1}A=I). In that case, (Ax=b) has the unique solution (x=A^{-1}b) for every (b). A square matrix without an inverse is called singular. (math.mit.edu)
Rank, determinants, and eigenstructure
The rank of a matrix is the dimension of its column space; it also equals the dimension of its row space. Rank measures the number of independent directions represented by the matrix. For an (m\times n) matrix, it cannot exceed (\min(m,n)). The nullspace consists of vectors satisfying (Ax=0), and its dimension plus the rank equals (n). (math.mit.edu)
The determinant assigns a scalar to a square matrix. A matrix is invertible exactly when its determinant is nonzero. For a real square matrix, the determinant’s absolute value describes the factor by which the associated transformation scales volume; its sign distinguishes preservation from reversal of orientation. For a (2\times2) matrix,
[ \det\begin{pmatrix}a&b\c&d\end{pmatrix}=ad-bc. ] (ocw.mit.edu)
For a square matrix, eigenvalues and eigenvectors satisfy (Av=\lambda v), with (v\ne0). They identify vectors whose images are scalar multiples of themselves. If a matrix has a basis of eigenvectors, it can be written (A=PDP^{-1}), where (D) is diagonal. This simplifies matrix powers and many evolution problems. (live.ocw.mit.edu)
Special matrices and factorization
A diagonal matrix has zero entries away from its main diagonal. A triangular matrix has zeros either above or below that diagonal. A real symmetric matrix satisfies (A^{\mathsf T}=A); it has real eigenvalues and an orthonormal basis of eigenvectors. A real orthogonal matrix satisfies (Q^{\mathsf T}Q=I) and preserves lengths and angles. (live.ocw.mit.edu)
Matrix factorization expresses a matrix using simpler factors. LU factorization is associated with elimination, while QR factorization uses orthonormal columns and a triangular factor. The singular value decomposition writes a real matrix as (A=U\Sigma V^{\mathsf T}). Unlike eigenvector diagonalization, it applies to rectangular matrices as well as square ones. (live.ocw.mit.edu)
Applications and computation
In machine learning, matrices organize observations and parameters and support transformations between representations. In statistics, they express multivariable calculations, including least-squares estimation. Factorizations help solve these problems and reveal structure that is not immediately visible in individual entries. (deeplearningbook.org)
Numerical matrix computation distinguishes exact mathematical properties from behavior under rounding error. Numerical linear algebra studies methods for solving linear systems, least-squares problems, and eigenvalue problems, together with their accuracy and sensitivity. Implementations such as LAPACK provide specialized routines for these tasks, including different methods for general, symmetric, and banded matrices. (netlib.org)