{
  "id": "matrix-multiplication",
  "title": "Matrix Multiplication",
  "lang": "en",
  "source_lang": "en",
  "summary": "Matrix multiplication combines rectangular arrays through row–column sums, representing composition of linear maps and underpinning scientific computation, graphics, statistics, and machine learning.",
  "category": "mathematics",
  "subcategory": "algebra",
  "type": "concept",
  "aliases": [
    "Matrix product"
  ],
  "generated_at": "2026-11-02",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/matrix-multiplication/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/matrix-multiplication/",
      "json": "https://aiwiki.page/zh/matrix-multiplication.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Matrix multiplication is the operation that combines an \\(m\\times n\\) matrix \\(A\\) with an \\(n\\times p\\) matrix \\(B\\) to produce an \\(m\\times p\\) matrix \\(AB\\). Each entry of the product is a sum of products drawn from one row of \\(A\\) and one column of \\(B\\). Unlike ordinary multiplication of numbers, matrix multiplication depends on the order of its factors: \\(AB\\) and \\(BA\\) may differ, and one may be defined when the other is not.\n\nIts defining rule is fundamental to linear algebra because it represents the composition of linear transformations in coordinates. The same operation describes successive geometric transformations, propagation through networks, changes of coordinates, and interactions among collections of variables. It is also a central computational kernel: substantial portions of scientific simulation, statistical estimation, and machine-learning computation can be organized around matrix products.\n\nThe subject therefore has both an algebraic and a computational character. Its algebraic properties explain how transformations combine, while its algorithms expose questions about arithmetic complexity, numerical error, memory movement, and parallel execution. The ordinary row–column product must also be distinguished from other operations on arrays, including entrywise products and tensor products."
    },
    {
      "heading": "Definition and the row–column rule",
      "text": "Let \\(A=(a_{ij})\\) have \\(m\\) rows and \\(n\\) columns, and let \\(B=(b_{jk})\\) have \\(n\\) rows and \\(p\\) columns. Their product \\(C=AB\\) is defined by\n\n\\[c_{ik}=\\sum_{j=1}^{n}a_{ij}b_{jk}, \\qquad 1\\leq i\\leq m,\\quad 1\\leq k\\leq p.\\]\n\nThe matching dimension \\(n\\) is the number of terms accumulated for each output entry. The outer dimensions \\(m\\) and \\(p\\) determine the shape of the result. Thus a \\(2\\times3\\) matrix can multiply a \\(3\\times4\\) matrix, producing a \\(2\\times4\\) matrix, but the reverse product is not defined. Two matrices having the same shape is neither necessary nor sufficient for their ordinary product to exist.\n\nFor example,\n\n\\[\\begin{pmatrix} 1&2&3\\\\ 4&5&6 \\end{pmatrix} \\begin{pmatrix} 7&8\\\\ 9&10\\\\ 11&12 \\end{pmatrix} = \\begin{pmatrix} 58&64\\\\ 139&154 \\end{pmatrix}.\\]\n\nThe upper-left entry is \\(1\\cdot7+2\\cdot9+3\\cdot11=58\\). Each of the other entries is calculated independently by the same rule. A row multiplied by a column gives a \\(1\\times1\\) matrix, commonly identified with its scalar entry. A column multiplied by a row instead gives a generally larger matrix called an outer product.\n\nOver the real numbers, an output entry can be interpreted as the inner product of a row of \\(A\\) and a column of \\(B\\), treating both as vectors. Over the complex numbers, this description requires care: ordinary matrix multiplication does not conjugate either factor. The standard complex inner product does conjugate one argument, so it is more precise to describe ordinary multiplication as a bilinear row–column pairing.\n\nThe entries most commonly belong to a field, such as the real numbers or complex numbers, but the definition also works over a ring. If scalar multiplication is noncommutative, the order \\(a_{ij}b_{jk}\\) must be preserved. Many familiar matrix identities continue to hold, although some, such as the usual transpose-of-a-product identity, require additional assumptions about the scalars.\n\nMatrix multiplication is distinct from multiplication by a scalar, which simply multiplies every entry by the same number. It is also distinct from entrywise multiplication, which requires equal shapes and has no summation over a shared dimension. The row–column sum is what makes ordinary matrix multiplication describe the passage through an intermediate collection of coordinates."
    },
    {
      "heading": "Composition of linear maps and geometric meaning",
      "text": "The central interpretation of matrix multiplication is composition of linear maps. Suppose \\(T:U\\to V\\) and \\(S:V\\to W\\) are linear maps between finite-dimensional vector spaces. After a basis has been chosen in each space, let \\(B\\) represent \\(T\\) and \\(A\\) represent \\(S\\). Then the composite map \\(S\\circ T\\) is represented by \\(AB\\):\n\n\\[[S(T(u))]=A(B[u])=(AB)[u].\\]\n\nHere square brackets denote coordinate columns. The rightmost matrix acts first. This convention explains both the matching dimensions and the order of the factors: the output coordinates of the first transformation must fit the input coordinates of the second.\n\nThe columns of a matrix record the images of the input basis vectors. If \\(B\\) has columns \\(b_1,\\ldots,b_p\\), then\n\n\\[AB= \\begin{pmatrix} Ab_1&\\cdots&Ab_p \\end{pmatrix}.\\]\n\nMultiplication on the left by \\(A\\) therefore applies the same transformation to every column of \\(B\\). Equivalently, the \\(k\\)-th column of the product is a linear combination of the columns of \\(A\\), with coefficients taken from the \\(k\\)-th column of \\(B\\). This viewpoint is often more informative than calculating individual entries.\n\nGeometric transformations illustrate why order matters. Consider a counterclockwise rotation through \\(90^\\circ\\),\n\n\\[R= \\begin{pmatrix} 0&-1\\\\ 1&0 \\end{pmatrix},\\]\n\nand a horizontal stretching,\n\n\\[D= \\begin{pmatrix} 2&0\\\\ 0&1 \\end{pmatrix}.\\]\n\nThen\n\n\\[RD= \\begin{pmatrix} 0&-1\\\\ 2&0 \\end{pmatrix}, \\qquad DR= \\begin{pmatrix} 0&-2\\\\ 1&0 \\end{pmatrix}.\\]\n\nStretching before rotation is different from stretching afterward, because the stretching is tied to a coordinate direction. Noncommutativity is therefore a geometric feature, not merely a peculiarity of notation.\n\nA change of basis also involves matrix products. If an invertible matrix \\(P\\) converts coordinates in a new basis into coordinates in an old basis, then an operator represented by \\(A\\) in the old basis is represented in the new basis by\n\n\\[P^{-1}AP.\\]\n\nThis relation is called matrix similarity. It preserves intrinsic properties of the operator, including its eigenvalues, while changing its coordinate description. Matrix multiplication makes it possible to separate the underlying map from the numerical array used to represent it.\n\nTranslations are not linear maps on ordinary coordinate vectors, but they can be included using homogeneous coordinates. A planar affine transformation can be represented by a \\(3\\times3\\) matrix acting on \\((x,y,1)^{\\mathsf T}\\). Products of these matrices combine translation, rotation, scaling, and shear in one uniform notation, a standard technique in computer graphics and geometric modeling."
    },
    {
      "heading": "Algebraic laws and their limitations",
      "text": "Matrix multiplication is associative whenever the dimensions permit:\n\n\\[(AB)C=A(BC).\\]\n\nThis follows by expanding both sides and rearranging finite sums. In entry notation, both expressions reduce to\n\n\\[\\sum_j\\sum_k a_{ij}b_{jk}c_{k\\ell}.\\]\n\nAssociativity permits an unambiguous expression such as \\(ABCD\\), even though different parenthesizations may have very different computational costs. It also reflects the associativity of composing transformations.\n\nMultiplication distributes over addition:\n\n\\[A(B+C)=AB+AC, \\qquad (A+B)C=AC+BC.\\]\n\nFor ordinary scalar fields, scalar factors can be moved through a product:\n\n\\[(\\lambda A)B=A(\\lambda B)=\\lambda(AB).\\]\n\nThe appropriate identity matrices satisfy \\(I_mA=A=AI_n\\) for an \\(m\\times n\\) matrix \\(A\\). Square matrices of a fixed size consequently form an associative algebra. Matrix multiplication is not generally commutative, however, and multiplication by a nonzero matrix need not be cancellable.\n\nIn particular, \\(AB=0\\) does not imply that either factor is zero. For example,\n\n\\[\\begin{pmatrix} 1&0\\\\ 0&0 \\end{pmatrix} \\begin{pmatrix} 0&0\\\\ 0&1 \\end{pmatrix} =0.\\]\n\nThe first factor annihilates every vector in the image of the second. More generally, \\(AB=0\\) precisely when the image of \\(B\\) lies in the kernel of \\(A\\). This connection to the image and kernel of a linear map explains the failure of familiar scalar cancellation rules.\n\nWhen square matrices are invertible,\n\n\\[(AB)^{-1}=B^{-1}A^{-1}.\\]\n\nThe reversal of order is essential: undoing a sequence requires undoing its last operation first. An inverse matrix restores cancellation, so \\(AB=AC\\) implies \\(B=C\\) if \\(A\\) is invertible. For matrices over a commutative field, the transpose and conjugate transpose similarly reverse products:\n\n\\[(AB)^{\\mathsf T}=B^{\\mathsf T}A^{\\mathsf T}, \\qquad (AB)^*=B^*A^*.\\]\n\nSeveral numerical invariants have important product identities. For square matrices of equal order,\n\n\\[\\det(AB)=\\det(A)\\det(B).\\]\n\nThe determinant therefore turns a matrix product into a scalar product. For \\(A\\) of size \\(m\\times n\\) and \\(B\\) of size \\(n\\times m\\), the trace satisfies \\(\\operatorname{tr}(AB)=\\operatorname{tr}(BA)\\), even though the two products may have different sizes. The rank obeys\n\n\\[\\operatorname{rank}(AB) \\leq \\min\\{\\operatorname{rank}(A),\\operatorname{rank}(B)\\}.\\]\n\nComposition cannot create more independent output directions than either factor can transmit. By contrast, the eigenvalues of a product are not generally obtained by multiplying corresponding eigenvalues of its factors; such formulas require additional structure, such as a common eigenbasis."
    },
    {
      "heading": "Block products, contractions, and alternative products",
      "text": "A matrix can be partitioned into rectangular blocks, and multiplication can then be performed at the block level. For example,\n\n\\[\\begin{pmatrix} A_{11}&A_{12}\\\\ A_{21}&A_{22} \\end{pmatrix} \\begin{pmatrix} B_{11}&B_{12}\\\\ B_{21}&B_{22} \\end{pmatrix} = \\begin{pmatrix} A_{11}B_{11}+A_{12}B_{21}& A_{11}B_{12}+A_{12}B_{22}\\\\ A_{21}B_{11}+A_{22}B_{21}& A_{21}B_{12}+A_{22}B_{22} \\end{pmatrix}.\\]\n\nThe block boundaries must be compatible: each inner block dimension must match. Individual blocks need not be square or equal in size. The formula is the same row–column rule applied to matrix-valued entries, with the order of block factors retained. Block notation supports proofs, structured numerical methods, and algorithms that operate on small submatrices held in fast memory.\n\nAnother useful decomposition writes a product as a sum of outer products. If \\(a_{:j}\\) is column \\(j\\) of \\(A\\), and \\(b_{j:}\\) is row \\(j\\) of \\(B\\), then\n\n\\[AB=\\sum_{j=1}^{n}a_{:j}b_{j:}.\\]\n\nEach term has rank at most one. This representation underlies rank-one updates and provides a direct explanation of why the rank of the result cannot exceed the shared dimension. It also leads naturally to algorithms that accumulate one rank-one contribution at a time.\n\nIn tensor notation, matrix multiplication is a contraction over one shared index. Under the Einstein summation convention, the summation sign can be suppressed and the product written \\(c_{ik}=a_{ij}b_{jk}\\), with the repeated index \\(j\\) understood to be summed. Higher-dimensional array operations often generalize this pattern by specifying which axes are contracted. Their precise behavior depends on the chosen index convention; an arbitrary multidimensional array product is not automatically ordinary matrix multiplication.\n\nTwo other products must be distinguished from the ordinary one. The Hadamard product multiplies equal-shaped matrices entry by entry:\n\n\\[(A\\circ B)_{ij}=a_{ij}b_{ij}.\\]\n\nThe Kronecker product replaces each entry \\(a_{ij}\\) with the block \\(a_{ij}B\\), producing an \\(mr\\times ns\\) matrix when \\(A\\) is \\(m\\times n\\) and \\(B\\) is \\(r\\times s\\). It represents the tensor product of linear maps in product bases. These operations have different dimension rules and different mathematical meanings.\n\nThe row–column construction also extends beyond ordinary arithmetic to a semiring. For Boolean entries, replacing addition by logical OR and multiplication by logical AND makes the output indicate whether at least one intermediate connection exists. In the min-plus setting, addition is replaced by minimum and multiplication by ordinary addition. The resulting entry \\(\\min_j(a_{ij}+b_{jk})\\) expresses a cheapest two-stage route. Such generalizations retain the organization of matrix multiplication while changing the interpretation of the accumulated terms."
    },
    {
      "heading": "Classical algorithms and multiplication chains",
      "text": "The direct algorithm implements the definition using three nested loops. For an \\(m\\times n\\) matrix multiplied by an \\(n\\times p\\) matrix, it performs \\(mnp\\) scalar multiplications and, if each output is summed from its first term, \\(mp(n-1)\\) scalar additions. For square matrices of order \\(n\\), this is approximately \\(2n^3\\) floating-point operations under the convention that a multiplication and an addition each count as one operation. Its running time is therefore \\(O(n^3)\\) in big-O notation under a unit-cost arithmetic model.\n\nThe three indices can be traversed in different orders without changing the exact mathematical answer. An \\(i,j,k\\) arrangement can compute one output entry at a time, whereas an \\(i,k,j\\) arrangement can reuse an entry of the left factor while updating a stretch of the output. Which order performs best depends on data layout and memory architecture. Reordering also changes the order of floating-point summation, so algebraically equivalent programs need not produce bitwise identical results.\n\nAssociativity creates a separate optimization problem for chains of matrices. If \\(A\\) is \\(10\\times100\\), \\(B\\) is \\(100\\times5\\), and \\(C\\) is \\(5\\times50\\), then computing \\((AB)C\\) requires\n\n\\[10\\cdot100\\cdot5+10\\cdot5\\cdot50=7{,}500\\]\n\nscalar multiplications. Computing \\(A(BC)\\) requires\n\n\\[100\\cdot5\\cdot50+10\\cdot100\\cdot50=75{,}000.\\]\n\nThe output is mathematically identical, but the arithmetic count differs by a factor of ten. The matrix-chain multiplication problem uses dynamic programming to choose an optimal parenthesization for a given sequence of dimensions. It changes the grouping, not the order, of the factors.\n\nStructure can reduce the cost substantially. Multiplying by a diagonal matrix merely scales rows or columns. Multiplying by a permutation matrix rearranges them. For a sparse matrix \\(A\\), multiplying \\(A\\) by a dense matrix with \\(p\\) columns can generally be organized around its stored nonzero entries, requiring work proportional to \\(\\operatorname{nnz}(A)p\\), apart from output initialization and indexing overhead. Sparse–sparse multiplication is more complicated because the locations of nonzeros in the product must also be determined, and the result can be much denser than either input.\n\nLow-rank structure offers another route. If \\(A=UV^{\\mathsf T}\\), where the inner dimension \\(r\\) is small, then\n\n\\[AB=U(V^{\\mathsf T}B).\\]\n\nThis can avoid forming the full matrix \\(A\\). The usefulness of a multiplication algorithm therefore depends not only on matrix dimensions but also on sparsity, rank, required accuracy, and whether the product itself must be stored explicitly."
    },
    {
      "heading": "Historical development and fast multiplication",
      "text": "The modern matrix product emerged from nineteenth-century work on determinants and linear transformations. James Joseph Sylvester introduced the term “matrix” in 1850. Arthur Cayley’s 1858 paper A Memoir on the Theory of Matrices helped establish matrices as algebraic objects with their own addition and multiplication, rather than merely as arrangements of coefficients associated with determinants. The multiplication law embodied composition of transformations and made clear that a useful algebra could be noncommutative.\n\nFor much of the following century, the standard cubic algorithm appeared to be the natural computational realization of the definition. A decisive change came in 1969, when Volker Strassen showed that two \\(2\\times2\\) block matrices could be multiplied using seven block multiplications rather than eight, at the expense of additional block additions and subtractions. Recursively applying this identity gives the recurrence\n\n\\[T(n)=7T(n/2)+O(n^2)\\]\n\nand therefore\n\n\\[T(n)=O\\!\\left(n^{\\log_2 7}\\right) \\approx O(n^{2.8074}).\\]\n\nThis was the first widely recognized demonstration that general matrix multiplication need not require cubic arithmetic time.\n\nThe key feature of Strassen’s algorithm is that it computes combinations of entries rather than evaluating every output dot product independently. For\n\n\\[A=\\begin{pmatrix}a&b\\\\c&d\\end{pmatrix}, \\qquad B=\\begin{pmatrix}e&f\\\\g&h\\end{pmatrix},\\]\n\ntwo of its seven intermediate products are \\((a+d)(e+h)\\) and \\((c+d)e\\). Carefully chosen additions and subtractions of all seven products recover the four output entries. Applied to blocks, the same identities remain valid because they preserve the order of multiplication. Practical implementations usually stop recursion at a moderate block size and switch to a conventional multiplication kernel.\n\nLater research related matrix multiplication to decompositions of a bilinear tensor. The matrix multiplication exponent, conventionally denoted \\(\\omega\\), is the infimum of exponents \\(\\tau\\) for which square matrices can be multiplied using \\(O(n^\\tau)\\) arithmetic operations over the field under consideration. The elementary bounds are \\(2\\leq\\omega\\leq3\\); the lower bound reflects the scale of the input and output in the standard explicit model. Strassen proved \\(\\omega<3\\), and Don Coppersmith and Shmuel Winograd’s 1990 work established an upper bound below \\(2.376\\). Subsequent work has reduced upper bounds further, while whether \\(\\omega=2\\) remains unresolved.\n\nAn asymptotically smaller exponent does not guarantee faster computation at realistic sizes. Constants, temporary storage, communication, rectangular shapes, and numerical behavior all matter. Some theoretical methods become advantageous only at enormous dimensions or establish complexity bounds without yielding competitive implementations. Theoretical arithmetic complexity and practical high-performance matrix multiplication consequently overlap, but they are not the same optimization problem."
    },
    {
      "heading": "Floating-point error and numerical reliability",
      "text": "Exact matrix identities do not automatically hold exactly on a computer. In floating-point arithmetic, an elementary operation is typically modeled as\n\n\\[\\operatorname{fl}(x\\mathbin{\\mathrm{op}}y) = (x\\mathbin{\\mathrm{op}}y)(1+\\delta), \\qquad |\\delta|\\leq u,\\]\n\nwhere \\(u\\) is the unit roundoff. This relative-error model assumes that overflow and problematic underflow are absent. Since a matrix product consists of many scalar multiplications and additions, its computed entries accumulate rounding errors.\n\nFor conventional dot-product evaluation, a standard componentwise estimate is\n\n\\[|\\widehat C-AB| \\leq \\gamma_n\\,|A|\\,|B|, \\qquad \\gamma_n=\\frac{nu}{1-nu},\\]\n\nprovided \\(nu<1\\) and the usual arithmetic assumptions hold. Absolute values and the inequality are interpreted entrywise, and \\(|A|\\,|B|\\) is itself an ordinary matrix product. The bound is important because it depends on the magnitudes of the individual contributions, not simply on the magnitude of the final answer.\n\nCancellation can therefore produce a large relative error in a small output entry. If positive and negative terms nearly cancel, \\(\\sum_j a_{ij}b_{jk}\\) may be tiny even when \\(\\sum_j|a_{ij}b_{jk}|\\) is large. A small absolute rounding error can then be large compared with the exact result. This distinguishes numerical stability, which concerns an algorithm’s behavior, from the conditioning of the mathematical problem, which concerns sensitivity to perturbations in the data.\n\nSeveral implementation choices affect error. Pairwise summation reduces the depth of an accumulation compared with a long sequential sum. Compensated summation retains information otherwise lost to rounding. A fused multiply-add instruction evaluates a product plus an accumulated value with one final rounding rather than rounding the multiplication and addition separately. These approaches have different cost and accuracy tradeoffs. Parallel reductions can also change the grouping of terms, causing different thread counts or execution schedules to produce slightly different answers.\n\nMixed-precision multiplication uses one numerical format for inputs or multiplication and another, often wider format for accumulation. This can increase throughput substantially, but accuracy depends on the input quantization, accumulation precision, and surrounding computation. A wider accumulator does not recover information already discarded when the inputs were converted to a lower-precision format.\n\nFast recursive algorithms require their own error analysis because they introduce additional additions and subtractions and can form intermediates larger than the final result. Strassen-type methods are not categorically unusable numerically, but recursion depth, scaling, and error constants matter. In numerical linear algebra, the appropriate evaluation is usually the effect on the complete task—such as solving a system or computing a factorization—rather than a timing or error measurement of an isolated product alone."
    },
    {
      "heading": "Memory hierarchy, libraries, and parallel execution",
      "text": "For large dense matrices, performance depends as much on moving data as on performing arithmetic. Processors typically have a hierarchy of registers, caches, and main memory, with progressively greater capacity and lower access speed. A naive implementation may repeatedly load the same entries from slow memory. A blocked algorithm instead partitions the matrices into tiles and reuses loaded data for many arithmetic operations before replacing it.\n\nA common implementation organizes the calculation around updates of the form\n\n\\[C_{IJ}\\leftarrow C_{IJ}+A_{IK}B_{KJ},\\]\n\nwhere \\(I,J,K\\) identify blocks. Optimized software often copies, or “packs,” selected panels into contiguous buffers and invokes a small microkernel that keeps part of the output in registers. Packing incurs extra work but can improve alignment, vectorization, and reuse enough to repay that cost. Block sizes are chosen according to the processor and memory hierarchy rather than by algebra alone.\n\nThe Basic Linear Algebra Subprograms interface, commonly called BLAS, standardizes widely used numerical kernels. Its general matrix multiplication operation, GEMM, computes\n\n\\[C\\leftarrow\\alpha\\,\\operatorname{op}(A)\\operatorname{op}(B)+\\beta C,\\]\n\nwhere \\(\\operatorname{op}\\) may leave a matrix unchanged or apply a transpose or conjugate transpose. This interface combines multiplication, scaling, and accumulation without requiring separate full-size temporary results. GEMM is a Level 3 BLAS operation, meaning that it operates on matrices rather than primarily on individual vectors.\n\nDense matrix multiplication is especially suitable for modern parallel hardware because it performs substantial arithmetic relative to the amount of data stored. Multicore processors can assign different output tiles to different threads. Graphics processing units execute many small operations concurrently, while specialized matrix units perform fixed-size block multiply-and-accumulate operations. Batched multiplication groups many small products into one execution request, reducing overhead and exposing parallelism that a single small matrix would not provide.\n\nOn distributed-memory systems, matrices are partitioned among processes, and the processes exchange panels or blocks to assemble their local outputs. Communication includes both the number of messages and the total data volume. Two-dimensional block-cyclic distributions balance work while spreading matrix regions across a process grid. Algorithms using additional replication can sometimes reduce communication further at the cost of more memory.\n\nThese constraints explain why arithmetic operation counts alone do not predict elapsed time. A method that minimizes scalar multiplications may move more data, allocate more temporary storage, or synchronize more frequently. High-performance implementations balance arithmetic throughput, memory capacity, communication, and numerical requirements. Their behavior also depends on whether the matrices are square, very tall, very wide, sparse, or small enough that setup costs dominate."
    },
    {
      "heading": "Applications in statistics, learning, networks, and dynamics",
      "text": "In statistics, matrix multiplication expresses relationships among many observations and variables at once. If a data matrix \\(X\\) has observations in rows and variables in columns, then \\(X^{\\mathsf T}X\\) collects pairwise sums of products between variables. After column centering and appropriate scaling, it yields a sample covariance matrix. The product \\(X\\beta\\) gives fitted values for a linear model with coefficient vector \\(\\beta\\). The normal equations for ordinary least squares are\n\n\\[X^{\\mathsf T}X\\beta=X^{\\mathsf T}y.\\]\n\nAlthough these equations are theoretically central, numerical solvers often use QR decomposition or singular value decomposition rather than explicitly solving through \\(X^{\\mathsf T}X\\), because forming the latter can worsen conditioning.\n\nIn machine learning, a dense layer of an artificial neural network commonly evaluates\n\n\\[Y=XW+\\mathbf{1}b^{\\mathsf T},\\]\n\nwhere rows of \\(X\\) contain input examples, \\(W\\) contains weights, and \\(b\\) contains biases. Processing multiple examples together turns many matrix–vector products into one matrix–matrix product. During automatic differentiation, closely related products compute derivatives with respect to inputs and parameters. In attention mechanisms, products such as \\(QK^{\\mathsf T}\\) form pairwise scores, and another product combines normalized scores with value vectors. The dimensions encode which examples, positions, or features interact.\n\nIn graph theory, let \\(A\\) be the adjacency matrix of a finite directed graph, with \\(a_{ij}=1\\) when there is an edge from vertex \\(i\\) to vertex \\(j\\). Under ordinary arithmetic, \\((A^k)_{ij}\\) counts directed walks of length \\(k\\) from \\(i\\) to \\(j\\). This is a direct consequence of summing over all intermediate vertices. Under Boolean arithmetic, the corresponding product records existence rather than number. Matrix products therefore connect local edge information to multistep connectivity.\n\nA Markov chain uses the same structure with probabilities. Under the row-vector convention, a probability distribution evolves as \\(\\pi_{t+1}=\\pi_tP\\), where \\(P\\) is a transition matrix. The entries of \\(P^k\\) give \\(k\\)-step transition probabilities. Summation over intermediate states is exactly the law of matrix multiplication, expressing the addition of mutually exclusive routes through those states.\n\nMatrix powers and products also describe discrete dynamical systems, control models, and numerical evolution equations. For a constant matrix \\(A\\), the solution of the linear differential equation \\(x'(t)=Ax(t)\\) involves the matrix exponential, defined through\n\n\\[e^{tA}=\\sum_{k=0}^{\\infty}\\frac{t^kA^k}{k!}.\\]\n\nIn computational physics and engineering, discretization produces large systems whose assembly, factorization, and repeated solution rely on matrix products. In quantum mechanics, products of operator matrices encode successive transformations of states. Across these uses, the repeated index has the same operational meaning: it sums contributions through an intermediate set of coordinates, states, or degrees of freedom."
    }
  ],
  "links": [
    "matrix",
    "linear-algebra",
    "inner-product",
    "field-mathematics",
    "real-number",
    "complex-number",
    "ring-mathematics",
    "function-composition",
    "linear-map",
    "vector-space",
    "basis-linear-algebra",
    "linear-combination",
    "matrix-similarity",
    "eigenvalues-and-eigenvectors",
    "homogeneous-coordinates",
    "affine-map",
    "identity-matrix",
    "image-linear-map",
    "kernel-linear-map",
    "inverse-matrix",
    "matrix-transpose",
    "conjugate-transpose",
    "determinant",
    "matrix-trace",
    "matrix-rank",
    "tensor",
    "tensor-contraction",
    "einstein-summation-convention",
    "kronecker-product",
    "tensor-product",
    "big-o-notation",
    "sparse-matrix",
    "strassen-algorithm",
    "floating-point-arithmetic",
    "numerical-stability",
    "condition-number",
    "numerical-linear-algebra",
    "statistics",
    "covariance-matrix",
    "ordinary-least-squares",
    "qr-decomposition",
    "singular-value-decomposition",
    "machine-learning",
    "artificial-neural-network",
    "automatic-differentiation",
    "graph-theory",
    "markov-chain",
    "transition-matrix",
    "differential-equation",
    "matrix-exponential",
    "quantum-mechanics"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}