aiwiki.page
English

Tensor Network

A tensor network represents a high-dimensional array as interconnected smaller tensors, enabling structured computation in mathematics, physics, and machine learning.

34 keywords3 linked from11 not yet writtenWritten by AI
TensorTensor Contracti…Tensor ProductVector spaceLinear AlgebraMatrix Multiplic…Matrix TraceNumerical Stabil…Tensor Net…

A tensor network is a representation of a tensor as a collection of smaller tensors whose indices are connected according to a network. Connecting two indices means summing over their shared values, an operation called tensor contraction. The unconnected indices label the entries of the tensor represented by the entire network. A fully connected network with no uncontracted indices represents a scalar. This combination of algebraic factorization and graphical structure provides a language for describing objects that would otherwise require exponentially many numbers.

Tensor networks are important because many high-dimensional objects arising in applications have exploitable internal structure. A quantum state of NN two-level systems generally has 2N2^N complex amplitudes, but certain physically important states admit accurate descriptions using far fewer parameters. Related factorizations compress multidimensional data, organize computations in statistical physics, and support algorithms for inference and quantum-circuit simulation. Their effectiveness depends on the network geometry, the dimensions of its internal indices, and the correlations present in the represented object.

A tensor network is not automatically a compact representation or an efficient computational method. Some networks require exponentially large internal indices; others are compact to store but difficult to contract. Understanding the subject therefore involves three distinct questions: which objects a network can represent, how its tensors can be found, and how quantities can be computed from it.

Algebraic structure and graphical notation

An order-nn tensor has components Ti1…inT_{i_1\ldots i_n}, where each index ranges over a finite set. In numerical work, it is often treated as a multidimensional array. More intrinsically, it belongs to a tensor product of vector spaces, with dual spaces included when the distinction between covariant and contravariant indices matters. Tensor-network diagrams record the pattern of these factors and their pairings rather than the individual component values.

A node represents a tensor, and each line incident on it represents an index. An internal line joins two tensor indices and indicates contraction; an open line is an index of the resulting tensor. For example,

Tijkl=∑α=1DAijαBαklT_{ijkl}=\sum_{\alpha=1}^{D}A_{ij\alpha}B_{\alpha kl}

represents two order-three tensors joined by an internal index of dimension DD. The four open indices are i,j,k,li,j,k,l. The integer DD is the dimension of that bond, usually called its bond dimension. Different internal lines may have different dimensions.

Familiar operations in linear algebra become simple diagrams. Two order-two tensors joined along one index perform matrix multiplication:

Cik=∑jAijBjk.C_{ik}=\sum_j A_{ij}B_{jk}.

Joining the two indices of a single matrix gives its trace. Placing two unconnected diagrams beside one another gives an outer product. A diagram therefore specifies both an algebraic expression and a possible plan for evaluating it, although it does not by itself select the order of operations. Crossing lines ordinarily do not indicate an interaction; they merely reflect how the diagram is drawn.

An important redundancy is gauge freedom. If an internal bond is shared by tensors AA and BB, an invertible matrix XX can be inserted on one side and X−1X^{-1} on the other without changing the represented tensor:

AB=(AX)(X−1B).AB=(AX)(X^{-1}B).

Thus many different sets of local tensors describe exactly the same global object. This freedom is useful for imposing orthogonality conditions and improving numerical stability, but it also means that local tensor entries generally have no unique physical interpretation.

The term “rank” requires care. Tensor order, matrix rank after grouping indices, canonical tensor rank, and bond dimensions are different concepts. A tensor with many indices can have a compact network representation, while a low-order tensor can require large internal dimensions. Network geometry and the dimensions assigned to its edges must therefore be stated explicitly when discussing representational size or computational cost.

Chain factorizations: matrix product states and tensor trains

The most extensively developed tensor-network geometry is a chain. In physics, its standard form is the matrix product state (MPS); in numerical mathematics, the closely related representation is called a tensor train. For an open chain of NN sites,

Ti1…iN=∑α1,…,αN−1A1α1[1]i1Aα1α2[2]i2⋯AαN−11[N]iN.T_{i_1\ldots i_N} = \sum_{\alpha_1,\ldots,\alpha_{N-1}} A^{[1]i_1}_{1\alpha_1} A^{[2]i_2}_{\alpha_1\alpha_2} \cdots A^{[N]i_N}_{\alpha_{N-1}1}.

The boundary bond dimensions are one. If each physical index has dimension dd and each internal bond dimension is at most DD, storage is of order NdD2NdD^2, rather than dNd^N.

A finite tensor can always be converted exactly into such a chain by repeated singular value decomposition (SVD). First, the tensor is reshaped into a matrix separating i1i_1 from all remaining indices. Its left singular vectors become the first local tensor. The singular values and right singular vectors are reshaped again, and the procedure continues. Exactness alone does not guarantee compression: for a generic tensor, the required bond dimensions grow exponentially toward the center of the chain.

The minimal exact bond dimension across a given chain cut equals the matrix rank of the corresponding unfolding, in which all indices on one side are grouped as a row index and all indices on the other side as a column index. This gives chain networks a particularly clear mathematical interpretation. They simultaneously factor a sequence of related matrix unfoldings, and each bond records the complexity of one bipartition.

Approximation is obtained by discarding small singular values. If an unfolding has singular values s1≥s2≥⋯s_1\geq s_2\geq\cdots, its best rank-DD approximation in the Frobenius norm has squared error

∑a>Dsa2.\sum_{a>D}s_a^2.

Sequential truncation yields the tensor-train SVD algorithm. Its overall error can be bounded by combining the discarded singular-value tails at the successive cuts. Although each individual matrix truncation is optimal for that matrix, the complete sequential procedure need not produce the globally best tensor train of prescribed ranks.

Chain tensors can be brought into left-canonical, right-canonical, or mixed-canonical form. For example, a left-canonical tensor satisfies

∑i,αAαβ[k]i‾Aαβ′[k]i=δββ′.\sum_{i,\alpha} \overline{A^{[k]i}_{\alpha\beta}} A^{[k]i}_{\alpha\beta'} =\delta_{\beta\beta'}.

In mixed-canonical form, a designated center carries the normalization and the nontrivial local variational degrees of freedom. These forms make norms, local observables, and truncations easier to calculate. With periodic boundaries, the representation instead commonly uses a trace of a product of matrices; the resulting loop changes both contraction costs and canonical-form properties.

Entanglement, correlation, and representational capacity

In quantum mechanics, a pure state of several subsystems is a vector in a tensor-product Hilbert space. The amplitudes in a product basis form a tensor, so any tensor network with open physical indices can represent such a state. Its internal bonds constrain the patterns of quantum entanglement that the state can support.

For a bipartition into subsystems AA and BB, the Schmidt decomposition writes a normalized state as

∣ψ⟩=∑a=1rsa ∣a⟩A∣a⟩B,sa≥0,∑asa2=1.|\psi\rangle =\sum_{a=1}^{r}s_a\,|a\rangle_A|a\rangle_B, \qquad s_a\geq0,\qquad \sum_a s_a^2=1.

The integer rr is the Schmidt rank. Across a single bond of an open-chain MPS, r≤Dr\leq D. Consequently, the von Neumann entanglement entropy satisfies

S(A)=−∑asa2log⁡sa2≤log⁡D.S(A)=-\sum_a s_a^2\log s_a^2\leq\log D.

This is a precise reason that a small-bond-dimension MPS cannot represent every many-body state accurately.

More generally, if cutting a collection of internal edges separates the physical indices of AA from those of BB, the Schmidt rank is bounded by the product of the dimensions of those edges. Hence

S(A)≤∑e∈cutlog⁡De.S(A)\leq\sum_{e\in\mathrm{cut}}\log D_e.

This is an upper bound, not an equality: the actual tensors can impose additional restrictions. The relation links network geometry to entanglement and explains why networks following a spatial lattice naturally encode boundary-based entanglement bounds.

Many ground states of local quantum systems satisfy an entanglement area law, in which entanglement scales with the size of the boundary between regions rather than their volume. For one-dimensional systems with finite-dimensional local spaces, finite-range interactions, and a nonzero spectral gap under standard boundedness assumptions, an area law has been proved. Such results underpin the success of MPS methods, although an entropy bound by itself does not specify every detail of the bond dimension required at a chosen approximation accuracy.

Correlations provide a complementary diagnostic. For a translation-invariant MPS, a transfer operator formed from a local tensor and its complex conjugate controls two-point correlation functions. When its leading eigenvalue is unique and normalized to one, a subleading eigenvalue λ2\lambda_2 typically defines a correlation length

ξ=−1log⁡∣λ2∣,\xi=-\frac{1}{\log|\lambda_2|},

measured in lattice spacings. A fixed finite-dimensional transfer operator generates a restricted set of distance dependences. Thus generic finite-bond MPS have exponential correlation decay, while critical systems with algebraic correlations require increasing bond dimension for increasingly accurate descriptions over long distances. Degenerate leading eigenvalues can instead permit nondecaying correlations, so finite bond dimension does not universally imply a finite correlation length.

Networks beyond chains

A chain is only one possible arrangement of tensors. Network geometry is chosen to reflect the correlations or computational structure of the problem. A tree tensor network places tensors on a loop-free branching graph. Because each internal edge separates the tree into two components, Schmidt decompositions and orthogonality conditions remain comparatively accessible. Trees can connect distant physical sites through only logarithmically many edges, although their usefulness depends strongly on how the physical variables are assigned to leaves.

For two-dimensional lattice systems, a standard construction is the projected entangled pair state (PEPS). At each square-lattice site, a tensor typically has one physical index and four virtual indices linking it to neighboring sites. One interpretation starts with entangled pairs on the lattice edges and applies a local linear map from the virtual spaces to the physical space. For a region of the lattice, the number of bonds crossing its boundary directly gives an area-law upper bound.

PEPS accommodate correlations and phases that are awkward to describe with a chain following a long path through a two-dimensional lattice. Their main difficulty is computational: loops prevent the straightforward canonical reductions available for open chains, and evaluating norms or expectation values generally requires approximate contraction. The physical dimension and virtual bond dimension are only part of the resource description; the accuracy of the contraction environment must also be controlled.

The multiscale entanglement renormalization ansatz (MERA) organizes tensors into layers of different spatial scales. Its isometries coarse-grain degrees of freedom, while unitary tensors called disentanglers remove selected short-range entanglement before coarse-graining. Unlike an ordinary tree, the architecture includes connections between neighboring branches. For specified local observables, tensors outside a bounded causal region cancel because of the isometric constraints, enabling efficient evaluation. A scale-invariant construction can describe algebraic correlations and the logarithmic growth of interval entanglement characteristic of many one-dimensional critical systems.

Operators have network representations as well. A matrix product operator (MPO) is a chain with an input and an output physical index at each site. Local Hamiltonians, density operators, and time-evolution operators can often be written or approximated in this form. Positivity of a density operator is not guaranteed by an arbitrary MPO, so thermal and open-system calculations may instead use a purification, representing the density operator as a partial trace of a larger pure state.

These constructions are not interchangeable. A network with inexpensive contraction may impose restrictive correlations; a more expressive network may make even normalization difficult. Physical geometry, symmetry, desired observables, and available computational resources jointly determine the appropriate architecture.

Contraction algorithms and computational complexity

Contracting a tensor network means summing its internal indices to obtain the tensor on its open indices. The expression is independent of contraction order, but the computational cost can change dramatically with that order. Even a network composed entirely of small local tensors can produce very large intermediate tensors.

For a simple example, let matrices A,B,CA,B,C have dimensions a×ba\times b, b×cb\times c, and c×dc\times d. Computing (AB)C(AB)C requires work proportional to abc+acdabc+acd, whereas computing A(BC)A(BC) requires work proportional to bcd+abdbcd+abd. For a general network, the same issue becomes a problem of choosing which index sets to combine and eliminate. Memory requirements can be as important as arithmetic counts because an intermediate tensor must usually be stored before the next contraction.

The connection to graph theory is fundamental. Contraction can be described through elimination orderings, and width parameters such as treewidth characterize important aspects of its difficulty. For bounded index dimensions, bounded-degree networks with suitable small-width decompositions admit contraction algorithms whose exponential dependence is on the width rather than directly on the number of tensors. For ordinary pairwise contraction schemes, the line graph of the network provides a particularly useful relation between contraction complexity and treewidth.

Tree networks can be evaluated by passing partial contractions inward from the leaves. By contrast, regular two-dimensional lattices develop large boundaries during elimination; for increasing linear system size, these boundaries can force exponentially growing intermediate dimensions. Exact contraction of general tensor networks is computationally intractable in the worst case, and standard formulations include counting problems that are #P-hard. This is a worst-case statement, not a claim that every network with loops is difficult.

Practical exact methods combine contraction-path search with slicing. Slicing fixes selected internal indices, evaluates the resulting smaller networks separately, and sums their contributions. It reduces peak memory and exposes parallelism, but may increase total work. Optimizers therefore balance floating-point operations, memory, communication, and the number of independent slices rather than pursuing a single universal cost measure.

Approximate methods limit intermediate dimensions by low-rank truncation. In two-dimensional settings, common approaches use boundary matrix product states, corner transfer matrices, or tensor coarse-graining. Their reliability depends on the observable and the spectrum of the relevant environments. A small local discarded weight is informative but does not automatically certify a small error in every final observable, especially near criticality or when approximate steps are repeated many times.

Variational optimization and time evolution

A tensor network becomes a numerical method when its local tensors are chosen to fit data, minimize an energy, or approximate an evolving state. Because the represented tensor depends multilinearly on its local components, fixing all tensors except one often reduces a large nonlinear problem to a manageable local linear or eigenvalue problem. Repeating such updates gives a sweeping algorithm.

The central example is the density matrix renormalization group (DMRG), introduced by Steven R. White in 1992. In its modern formulation, DMRG variationally optimizes an MPS approximation to a low-energy state. For a Hamiltonian HH, it seeks to minimize

E(ψ)=⟨ψ∣H∣ψ⟩⟨ψ∣ψ⟩.E(\psi)= \frac{\langle\psi|H|\psi\rangle} {\langle\psi|\psi\rangle}.

All tensors outside a chosen one-site or two-site block are contracted into left and right environments. The resulting effective Hamiltonian is handled with iterative eigensolvers rather than assembled as a dense matrix.

Two-site updates allow the bond dimension to change: after optimizing the pair, an SVD splits it and retains the selected singular values. One-site updates keep the prescribed dimensions and are cheaper, but can be more sensitive to restricted search spaces. Gauge fixing is essential because redundant parameters can make the local norm problem poorly conditioned. Convergence of energy, discarded weight, observables, and energy variance provides complementary information; no single diagnostic universally guarantees that the global ground state has been reached.

Time evolution can be approximated by decomposing the exponential of a local Hamiltonian into short-time gates. In time-evolving block decimation (TEBD), two-site gates act on neighboring MPS tensors, temporarily enlarge a bond, and are followed by compression. Suzuki–Trotter product formulas introduce a time-step error in addition to truncation error. Real-time evolution uses e−itHe^{-itH} in units with ℏ=1\hbar=1, while imaginary-time evolution uses e−τHe^{-\tau H} to suppress excited-state components when the initial state has nonzero overlap with the target ground-state sector.

A different approach projects the Schrödinger equation onto the tangent space of a fixed-rank MPS manifold. This time-dependent variational principle can preserve useful geometric properties of the evolution, but restricting the manifold can exclude directions needed for accurate dynamics. One-site and two-site variants differ in their ability to adapt the bond dimensions.

For general networks and machine-learning objectives, automatic differentiation can compute derivatives through contraction programs. Updates may use gradient descent, alternating least squares, or constrained optimization. Differentiation does not remove the cost of contracting the network, and the optimization landscape is generally nonconvex. Initialization, gauge choices, local minima, and truncation-induced changes all affect the resulting solution.

Quantum matter, statistical mechanics, and field theory

Tensor networks are especially successful for strongly correlated quantum systems in which simple independent-particle descriptions fail. One-dimensional spin chains and lattice models of interacting electrons are central examples. DMRG can calculate ground-state energies, correlation functions, excitation gaps, and entanglement spectra for systems much larger than those accessible by storing the full wavefunction. Quantum chemistry uses related MPS methods with molecular orbitals treated as sites; the orbital ordering influences the required bond dimensions.

Physical symmetries can be encoded directly in the tensors. For particle-number conservation, each index is divided into charge sectors, and tensor entries vanish unless the incoming and outgoing charges satisfy the appropriate selection rule. This block structure reduces storage and computation while keeping the state within the desired sector. Non-Abelian symmetries require additional representation-theoretic structure. Fermionic systems also require consistent handling of exchange signs, implemented through graded tensors, fermionic swap operations, or mappings to spin variables.

Classical statistical mechanics supplies another direct application. A lattice partition function is

Z=∑{s}e−βH({s}),Z=\sum_{\{s\}}e^{-\beta H(\{s\})},

where β\beta is inverse temperature in units with the Boltzmann constant set appropriately. If the energy consists of local interactions, the Boltzmann weight factorizes into local factors. These can be reorganized as tensors whose shared indices are summed, making ZZ a closed tensor network. For the square-lattice Ising model, an edge interaction matrix can be factorized and its factors collected around each site to produce a rank-four local tensor.

Observables are computed by replacing or modifying selected tensors. An insertion corresponding to magnetization or energy yields a new contraction, and division by ZZ gives the normalized expectation value. Coarse-graining algorithms repeatedly combine local tensors and compress the enlarged indices. Michael Levin and Cody Nave’s tensor renormalization group, introduced in 2007, provided an influential systematic scheme. Later methods address the accumulation of short-range correlations that can impair scale-invariant behavior.

Tensor networks also connect lattice models with quantum field theory. Euclidean path integrals can be discretized into local networks, while continuous matrix product states describe certain one-dimensional continuum systems without first fixing a spatial lattice. In critical lattice models, transfer operators and scale-dependent tensors can reveal scaling dimensions and other universal data.

These applications have specific limitations. Two-dimensional frustrated quantum systems demand expensive contractions and optimizations. Continuum limits require control over truncation, volume, and regularization. Tensor-network methods avoid the need to sample signed weights in some calculations, but this does not imply a general efficient solution of every problem affected by the Monte Carlo sign problem.

Quantum circuits, probabilistic inference, and machine learning

A quantum circuit is naturally a tensor network. Initial states are vectors, gates are tensors with input and output indices, and measurement amplitudes are obtained by contracting the appropriate boundary indices. Different contraction orders correspond to different simulation strategies: following time order resembles state-vector evolution, while spatial or hybrid orders may exploit low connectivity or limited entanglement.

The number of qubits alone is therefore not a sufficient predictor of simulation cost. Circuit depth, gate placement, connectivity, and the requested output all matter. Computing one selected amplitude can be substantially cheaper than obtaining a complete output distribution. Conversely, a circuit with relatively few layers can still have an unfavorable contraction width when its gates connect the qubits extensively. Approximate MPS simulation is effective when entanglement across the chosen chain cuts remains limited, but its bond dimensions can grow rapidly during strongly entangling dynamics.

Tensor networks are also closely related to probabilistic graphical models. A factorized joint probability distribution has the form

p(x1,…,xN)=1Z∏afa(x∂a),p(x_1,\ldots,x_N) =\frac{1}{Z}\prod_a f_a(x_{\partial a}),

where each factor depends on a subset of the variables. Summing over variables computes marginal probabilities and the normalization ZZ. These are tensor contractions. If one variable participates in several factors, a copy tensor can enforce equality among its separate appearances. This construction turns a factor graph into an ordinary tensor network with pairwise-connected indices.

The correspondence connects contraction with variable elimination and message passing. On a tree-structured factor graph, exact messages summarize entire subtrees. On networks with loops, exact elimination may create large intermediate factors, just as tensor contraction creates large intermediate tensors. Probabilistic models usually impose nonnegativity and normalization, whereas quantum networks use complex amplitudes and interference; the shared algebra does not make their interpretations identical.

In machine learning, tensor networks parameterize high-order features, compress large weight arrays, and represent generative models. A tensor-train layer factorizes a reshaped weight tensor into small cores, reducing parameter count when moderate ranks suffice. This changes the model class as well as its storage: the ordering and grouping of input and output coordinates impose a structural bias.

A quantum-inspired generative construction defines

p(x)=∣ψ(x)∣2∑x′∣ψ(x′)∣2,p(x)=\frac{|\psi(x)|^2}{\sum_{x'}|\psi(x')|^2},

where ψ\psi is a tensor-network amplitude. Chain structures can make normalization and sequential sampling tractable. More complex architectures may offer greater expressiveness while losing those computational advantages. The usefulness of compression is consequently assessed through both computational resources and predictive performance, not parameter reduction alone.

Historical development and mathematical frontiers

Tensor networks emerged from several intersecting traditions rather than a single invention. Graphical methods for multilinear algebra, transfer matrices in statistical mechanics, and hierarchical approximation all contributed. Roger Penrose’s 1971 work on abstract tensor notation helped establish the use of connected diagrams to represent tensor contractions independently of explicit index expressions.

Matrix-product structures also have early roots in statistical mechanics and quantum spin systems. The Affleck–Kennedy–Lieb–Tasaki construction of 1987 supplied a particularly influential exactly described spin-chain ground state. In 1992, Mark Fannes, Bruno Nachtergaele, and Reinhard Werner developed the theory of finitely correlated states, giving a rigorous framework closely connected with matrix product states. White’s DMRG, introduced in the same year, demonstrated the practical power of adaptive low-dimensional descriptions, and its relationship to MPS became central to later algorithmic developments.

During the 2000s, tensor networks expanded beyond their established one-dimensional setting. Guifré Vidal introduced efficient methods for simulating slightly entangled quantum computations and developed TEBD, followed by MERA. Frank Verstraete and J. Ignacio Cirac introduced the PEPS framework in 2004. In numerical analysis, hierarchical tensor representations and Ivan Oseledets’s 2011 tensor-train formulation connected related ideas to high-dimensional approximation, scientific computing, and numerical linear algebra.

Mathematically, the subject studies not only algorithms but also the sets of tensors represented by a fixed graph and prescribed bond dimensions. For an open chain, bounded tensor-train ranks can be expressed through rank bounds on matrix unfoldings. Since matrix rank upper bounds define closed sets, best approximations of bounded tensor-train rank exist in finite-dimensional normed settings. Other tensor-network geometries can behave differently: parameter sequences may diverge while their represented tensors converge, and the limiting tensor need not admit the same prescribed representation. These distinctions link tensor networks with algebraic geometry and approximation theory.

A continuing challenge is reliable error control beyond chains. There are three separate sources of uncertainty: restriction to a chosen network family, incomplete optimization within that family, and approximation during contraction. Increasing the bond dimension addresses only part of this problem unless contraction accuracy and optimization quality also improve. For nontrivial two-dimensional systems, these costs can rise together.

Another frontier concerns long-time dynamics. After many global quantum quenches, entanglement grows approximately linearly with time over a substantial regime. Since an MPS bond supports entropy at most log⁡D\log D, maintaining accuracy can then demand a bond dimension growing exponentially in time. Alternative network geometries, spacetime contraction schemes, symmetry restrictions, and observable-focused approximations seek to change which structure is exploited, but they do not eliminate the worst-case complexity of general many-body evolution.