aiwiki.page
English
Mathematics / Algebra & Number Theory / tensor-decomposition

Tensor Decomposition

Tensor decomposition represents multidimensional arrays through simpler factors, enabling compression, latent-variable analysis, and efficient numerical computation.

48 keywords1 linked from10 not yet writtenWritten by AI
TensorMatrix Factoriza…SpectroscopyStatisticsMachine LearningMatrix (mathemat…Tensor ProductVector spaceTensor Dec…

Tensor decomposition is the representation of a tensor as a combination of simpler tensors, usually vectors, matrices, or smaller multidimensional arrays. It extends matrix factorization to data with three or more modes, such as a collection of measurements indexed by location, time, and wavelength. Different decompositions impose different structures on these factors: some express a tensor as a sum of rank-one terms, others use a small core linked to factor matrices, and still others arrange smaller tensors in chains or networks.

These representations are important because a multidimensional array can contain far more entries than can be stored or processed directly. A suitable decomposition can compress the array, reveal underlying components, or permit calculations without reconstructing every entry. Applications range from spectroscopy and statistics to scientific computing, signal processing, and machine learning. Unlike matrix decomposition, however, tensor decomposition has no single canonical form: definitions of rank differ, best low-rank approximations may fail to exist, and computational difficulty depends strongly on the chosen representation.

Mathematical objects, modes, and elementary operations

In finite-dimensional settings, an order-dd tensor can be represented by an array

X=(xi1,…,id)∈Rn1×⋯×nd.\mathcal X=(x_{i_1,\ldots,i_d}) \in\mathbb R^{n_1\times\cdots\times n_d}.

Its order is the number of indices, while nkn_k is the dimension of mode kk. Thus a vector has order one and a matrix has order two. Order should not be confused with rank: a third-order tensor can have many different ranks, depending on both its entries and the definition of rank being used. In an abstract formulation, X\mathcal X belongs to a tensor product V1⊗⋯⊗VdV_1\otimes\cdots\otimes V_d of finite-dimensional vector spaces. An array results after bases have been selected in those spaces.

The elementary building block of several decompositions is a rank-one, or decomposable, tensor:

X=a(1)∘a(2)∘⋯∘a(d),xi1,…,id=∏k=1daik(k).\mathcal X=a^{(1)}\circ a^{(2)}\circ\cdots\circ a^{(d)}, \qquad x_{i_1,\ldots,i_d} =\prod_{k=1}^{d}a^{(k)}_{i_k}.

Here ∘\circ denotes the outer product. Such a tensor contains no interaction beyond the product of one vector from each mode. For example, a location–time–wavelength array of the form xijk=aibjckx_{ijk}=a_i b_j c_k has a common temporal profile and spectrum whose overall amplitude varies by location. Sums of such terms allow several distinct profiles to coexist.

A mode-kk unfolding, also called matricization, rearranges the tensor into a matrix X(k)X_{(k)} with nkn_k rows and ∏j≠knj\prod_{j\ne k}n_j columns. The column ordering varies between conventions, but any consistent ordering preserves the relevant rank and singular values. Unfolding is not itself a decomposition: it is a change of indexing that permits the use of linear algebra. Different unfoldings preserve different groupings of the modes, so replacing a tensor by just one unfolding can discard distinctions that matter for modeling.

Multiplication along one mode changes the coordinates or dimension associated with that mode. For U∈Rm×nkU\in\mathbb R^{m\times n_k}, the mode product Y=X×kU\mathcal Y=\mathcal X\times_k U is defined by

yi1,…,j,…,id=∑ik=1nkUjikxi1,…,ik,…,id.y_{i_1,\ldots,j,\ldots,i_d} =\sum_{i_k=1}^{n_k}U_{j i_k}x_{i_1,\ldots,i_k,\ldots,i_d}.

More general summation over shared indices is a tensor contraction, the basic operation behind tensor-network calculations.

Approximation is commonly measured with the Frobenius norm,

∥X∥F2=∑i1,…,id∣xi1,…,id∣2.\|\mathcal X\|_F^2 =\sum_{i_1,\ldots,i_d}|x_{i_1,\ldots,i_d}|^2.

This treats all observed entries equally. Weighted norms, probabilistic losses, and application-specific metrics become necessary when entries have unequal precision, missing observations, or non-Gaussian measurement distributions.

Canonical polyadic decomposition and tensor rank

The canonical polyadic decomposition, usually abbreviated CP, represents a tensor as a sum of rank-one terms:

X=∑r=1Rλr ar(1)∘ar(2)∘⋯∘ar(d).\mathcal X =\sum_{r=1}^{R} \lambda_r\,a_r^{(1)}\circ a_r^{(2)} \circ\cdots\circ a_r^{(d)}.

The vectors for mode kk form a factor matrix A(k)=[a1(k),…,aR(k)]A^{(k)}=[a_1^{(k)},\ldots,a_R^{(k)}]. Weights λr\lambda_r are convenient when the vectors are normalized, although each weight can instead be absorbed into one factor. The smallest number of terms in an exact representation is the CP rank, also called tensor rank. The historical names CANDECOMP and PARAFAC refer to essentially this same model.

For matrices, this definition reduces to ordinary matrix rank. For tensors of order at least three, its behavior is markedly different. Matrix rank cannot exceed either matrix dimension, whereas tensor rank can exceed every individual mode dimension. Rank can also depend on the underlying field: a real tensor may require more real rank-one terms than complex rank-one terms. Consequently, a statement about tensor rank must specify whether the factors are real or complex.

The CP model uses approximately R∑knkR\sum_k n_k scalar entries, before accounting for scaling indeterminacies, instead of ∏knk\prod_k n_k. An illustrative 100×100×100100\times100\times100 array contains one million entries, while three factor matrices with R=10R=10 contain only 3,000 entries, plus any separately stored weights. This comparison describes storage, not a guarantee that a rank-ten approximation is accurate. Compression depends on the actual structure of the array.

For a third-order tensor, CP also yields useful matrix equations. With a suitable unfolding convention,

X(1)=A(C⊙B)T,X_{(1)}=A(C\odot B)^{\mathsf T},

where ⊙\odot is the Khatri–Rao product: its rrth column is the vectorized outer product of the rrth columns of BB and CC. This relation turns a multilinear model into a linear model in one factor when the others are held fixed. It is the foundation of many fitting algorithms.

The structural interpretation of CP is componentwise. Each term has one profile in every mode, and the same component index rr ties these profiles together. In fluorescence spectroscopy, for example, a component may combine a sample-concentration profile, an excitation spectrum, and an emission spectrum. This interpretation differs from a general interaction model: CP permits only matched component combinations within each term. Whether those components correspond to physical entities depends on the measurement mechanism, model adequacy, and identifiability—not merely on the existence of a numerical fit.

Tucker decomposition and multilinear subspaces

The Tucker decomposition separates a tensor into mode-specific factor matrices and a smaller core:

X=G×1U(1)×2U(2)⋯×dU(d),\mathcal X =\mathcal G\times_1 U^{(1)} \times_2 U^{(2)} \cdots\times_d U^{(d)},

where U(k)∈Rnk×rkU^{(k)}\in\mathbb R^{n_k\times r_k} and G∈Rr1×⋯×rd\mathcal G\in\mathbb R^{r_1\times\cdots\times r_d}. The columns of each factor span a subspace associated with one mode. The core records interactions among these subspaces. Unlike CP, Tucker does not require a single component index to align the factors across all modes.

The multilinear rank of a tensor is the tuple

(r1,…,rd),rk=rank⁡(X(k)).(r_1,\ldots,r_d),\qquad r_k=\operatorname{rank}(X_{(k)}).

These ranks have a direct matrix interpretation: rkr_k is the dimension of the space spanned by the mode-kk fibers. Exact Tucker factors can be chosen as bases for these spaces. For a rank-RR CP representation, every unfolding has matrix rank at most RR, but the unfolding ranks generally do not determine the CP rank. Thus “rank ten” and “multilinear rank (10,10,10)(10,10,10)” describe different restrictions.

A particularly useful construction is the higher-order singular value decomposition, or HOSVD. For each mode, an ordinary singular value decomposition of X(k)X_{(k)} provides orthonormal factor vectors. Projection onto the chosen mode subspaces produces the core. In the full HOSVD, core slices perpendicular to the same mode are mutually orthogonal, and their norms correspond to that unfolding’s singular values. The core is generally not diagonal, so this is not a direct replication of the diagonal matrix in a matrix SVD.

Truncating the factor matrices yields a simple approximation with an important error guarantee. If X^H\widehat{\mathcal X}_{H} is the truncated HOSVD with prescribed mode dimensions and X∗\mathcal X_* is a best approximation with those multilinear-rank bounds, then

∥X−X^H∥F≤d ∥X−X∗∥F.\|\mathcal X-\widehat{\mathcal X}_{H}\|_F \le \sqrt d\,\|\mathcal X-\mathcal X_*\|_F.

The result makes truncated HOSVD quasi-optimal, but not generally optimal. Higher-order orthogonal iteration improves the fit by repeatedly updating each factor using the tensor projected onto the current subspaces of all other modes.

Tucker is especially useful when different modes require different compression levels. A spatial mode might need many basis vectors, while a spectral mode needs only a few. Its storage cost is approximately

∑k=1dnkrk+∏k=1drk.\sum_{k=1}^{d}n_k r_k+\prod_{k=1}^{d}r_k.

The core term becomes expensive when the order is large, even if each rkr_k is modest. This limitation motivates chain and hierarchical representations. Tucker also has a change-of-basis freedom: invertible transformations of factor columns can be compensated by inverse transformations of the core. Consequently, its principal objects are often subspaces rather than uniquely defined individual components.

Tensor trains, hierarchical formats, and tensor networks

A tensor-train decomposition represents a high-order array by a chain of three-index cores:

xi1,…,id=G1(i1)G2(i2)⋯Gd(id).x_{i_1,\ldots,i_d} = G_1(i_1)G_2(i_2)\cdots G_d(i_d).

Here Gk(ik)G_k(i_k) is an rk−1×rkr_{k-1}\times r_k matrix, with r0=rd=1r_0=r_d=1, so the product is a scalar. The integers r1,…,rd−1r_1,\ldots,r_{d-1} are the internal bond dimensions or tensor-train ranks. Unlike Tucker’s central core, no individual core involves all modes.

The minimal exact rank rkr_k equals the matrix rank of the unfolding that groups indices i1,…,iki_1,\ldots,i_k against ik+1,…,idi_{k+1},\ldots,i_d. This provides both an interpretation and a construction: successive matrix SVDs produce a tensor train, while truncated SVDs produce an approximation with controlled Frobenius error. If all physical dimensions are at most nn and all bond dimensions at most rr, storage is O(dnr2)O(dnr^2). The apparently linear dependence on order is conditional on keeping ranks bounded; some tensors require exponentially large ranks.

The order of the modes matters because it determines which bipartitions the chain exposes. Variables that are strongly coupled may require a large bond dimension when separated by a chain cut. Permuting the modes can therefore change storage and computational cost substantially without changing the underlying array. Applications often exploit spatial locality, variable grouping, or known interaction structures to choose an effective ordering.

Tensor trains are closely related to matrix product states in quantum mechanics. There, tensor entries are amplitudes of a many-body state, and the singular values across a chain cut are its Schmidt coefficients. The required bond dimension reflects the correlations across that bipartition. The same mathematical representation can therefore describe either a numerical array or a quantum state, although normalization, interpretation, and optimization objectives differ.

The hierarchical Tucker decomposition replaces the chain with a tree of nested subspaces. More general tensor networks allow other connection patterns. Their complexity depends not only on tensor sizes and bond dimensions but also on the order in which contractions are performed. Networks with cycles can express useful structures, yet exact contraction can become much more expensive than in a chain or tree.

These formats contain internal nonuniqueness, often called gauge freedom. Multiplying one side of a bond by an invertible matrix and the other side by its inverse leaves the represented tensor unchanged. Orthogonal or canonical gauges remove part of this freedom and improve numerical stability. They also simplify local optimization and truncation, because orthonormal surrounding factors make the local Frobenius metric agree with the metric of the full represented tensor.

Identifiability, geometry, and nonexistence of best approximations

Identifiability asks whether a decomposition is determined by the tensor it represents. For CP, unavoidable ambiguities include permutation of the terms and redistribution of scale among the vectors in a term. A decomposition is essentially unique when these are the only ambiguities. This contrasts with ordinary low-rank matrix factorization, where a general invertible transformation can mix latent factors without changing the product.

A classical sufficient condition was established by Joseph Kruskal in 1977. For a three-way CP decomposition with RR nonzero terms, let kA,kB,kCk_A,k_B,k_C be the Kruskal ranks of its factor matrices, where Kruskal rank is the largest integer such that every subset of that many columns is linearly independent. If

kA+kB+kC≥2R+2,k_A+k_B+k_C\ge 2R+2,

then the decomposition is essentially unique and its rank is RR. This is a sufficient condition, not a necessary one. Failure of the inequality does not establish nonuniqueness, and exact uniqueness alone does not ensure that noisy estimates are well conditioned.

The geometry behind CP connects it to algebraic geometry. After nonzero scalar multiples are identified, rank-one tensors form a Segre variety. Sums of rank-one tensors lead to secant constructions, which help describe possible ranks and generic identifiability. Over the complex numbers, a generic rank occurs outside a proper algebraic exceptional set. Over the real numbers, several typical ranks can occur on different nonempty open sets. Dimension counting provides useful expectations, but exceptional geometric behavior prevents it from being a universal rank formula.

One of the sharpest differences from matrices is that tensors of CP rank at most RR need not form a closed set. The border rank of a tensor is the smallest RR for which it can be approached arbitrarily closely by tensors of rank at most RR. Border rank can be strictly smaller than rank.

An explicit example uses independent vectors e1,e2e_1,e_2. The rank-three tensor

W=e2∘e1∘e1+e1∘e2∘e1+e1∘e1∘e2\mathcal W= e_2\circ e_1\circ e_1+ e_1\circ e_2\circ e_1+ e_1\circ e_1\circ e_2

is the limit, as t→0t\to0, of

(e1+te2)∘3−e1∘3t.\frac{(e_1+t e_2)^{\circ3}-e_1^{\circ3}}{t}.

For every nonzero tt, the latter is a sum of two rank-one tensors. Hence W\mathcal W has border rank two but rank three. The two terms become individually unbounded while canceling increasingly closely.

Consequently, a best rank-two approximation to W\mathcal W does not exist: the infimum approximation error is zero, but no rank-two tensor attains it. This phenomenon explains some divergent CP fitting sequences. In contrast, the set of tensors with bounded multilinear rank is closed, so best Tucker approximations exist in finite-dimensional Frobenius norm. Existence, uniqueness, and stable recoverability are therefore separate questions that depend on the decomposition format.

Numerical algorithms and computational complexity

Most practical decomposition problems are approximation problems. A typical CP objective is

min⁡{A(k)}12∥X−∑r=1Rar(1)∘⋯∘ar(d)∥F2.\min_{\{A^{(k)}\}} \frac12\left\| \mathcal X- \sum_{r=1}^{R} a_r^{(1)}\circ\cdots\circ a_r^{(d)} \right\|_F^2.

This is a mathematical optimization problem that is linear in any one factor when all other factors are fixed, but nonlinear jointly. Its geometry includes scaling freedoms, stationary points, flat regions, and, in some cases, unbounded minimizing sequences.

Alternating least squares exploits this separate linearity. For a third-order tensor, updating AA amounts to solving

min⁡A∥X(1)−A(C⊙B)T∥F2.\min_A\|X_{(1)}-A(C\odot B)^{\mathsf T}\|_F^2.

A normal-equation form uses

A[(CTC)∗(BTB)]=X(1)(C⊙B),A\bigl[(C^{\mathsf T}C)*(B^{\mathsf T}B)\bigr] =X_{(1)}(C\odot B),

where ∗* denotes entrywise multiplication. The right-hand contraction is commonly called matricized tensor times Khatri–Rao product, or MTTKRP. Implementing this operation efficiently is central to scalable CP computation; explicitly constructing the entire Khatri–Rao product is often unnecessary.

Each exact least-squares block update cannot increase the objective, but that does not imply convergence to a global optimum. Nearly collinear factors can create long periods of slow improvement, traditionally called swamps. Ill-conditioned normal equations can also degrade numerical accuracy because forming a Gram matrix squares the condition number of the underlying least-squares design. QR decomposition, SVD-based solves, damping, and suitable scaling address different parts of this problem.

Other methods optimize all factors jointly using gradient descent, quasi-Newton methods, nonlinear conjugate gradients, or Gauss–Newton approximations. Their derivatives can exploit the multilinear structure rather than treating every reconstructed entry independently. Automatic differentiation simplifies implementation for complicated models, while specialized contractions can reduce memory use and arithmetic. Nonnegative, sparse, and weighted models require corresponding constrained or modified subproblems.

Initialization affects both speed and the solution reached. Common approaches include random factors, singular vectors from unfoldings, compressed preliminary models, and multiple starts. A low residual alone does not establish reliable recovery: factors may be unstable, nearly canceling, or poorly determined. Diagnostics therefore include factor norms, conditioning, variability across starts, and predictive performance on held-out entries.

General tensor problems are substantially harder than their matrix analogues. Determining CP rank is computationally intractable in broad formulations, and many tensor optimization tasks are NP-hard. Such worst-case results do not imply that every application is difficult: restricted dimensions, favorable randomness, separability, orthogonality, and other structural assumptions can permit efficient algorithms. Large-scale implementations also use sparse storage, randomized projections, sampled subproblems, and parallel computing. Their effectiveness depends on whether the original tensor is explicitly stored or accessible only through selected entries and contractions.

Constraints, statistical models, and incomplete observations

A decomposition can be used as a deterministic approximation or as the systematic part of a statistical model. Under independent additive Gaussian errors with common variance, minimizing squared Frobenius error corresponds to maximum likelihood estimation. Other data types suggest different likelihoods. Count arrays can use a Poisson distribution with a nonnegative low-rank mean, while binary arrays can use Bernoulli probabilities linked to a real-valued factorization. The decomposition specifies the latent structure; the observation model specifies how measurements arise from it.

Constraints incorporate properties not supplied by low rank alone. Nonnegative CP requires every factor entry to be nonnegative and is useful for measurements whose components combine additively. Sparsity can restrict components to selected variables, while smoothness penalties can encode slowly varying temporal or spectral profiles. Orthogonality, simplex constraints, and shared factors across several datasets express different assumptions. None is universally appropriate: each changes the admissible representations and may change existence, identifiability, and numerical behavior.

Regularization adds penalties or bounds to control instability and complexity. For example, squared factor norms discourage arbitrarily large factors, but their effect depends on how scale is distributed among modes. Normalization conventions and explicit component weights make these effects easier to interpret. Bayesian formulations instead place a prior distribution on factors, weights, ranks, or noise parameters. The resulting posterior distribution can quantify uncertainty, although exact inference is rarely tractable for substantial models.

Tensor completion estimates unobserved entries from an observed index set Ω\Omega. A squared-error formulation minimizes

12∑(i1,…,id)∈Ω(xi1,…,id−x^i1,…,id)2\frac12\sum_{(i_1,\ldots,i_d)\in\Omega} \left(x_{i_1,\ldots,i_d} -\widehat x_{i_1,\ldots,i_d}\right)^2

subject to a chosen low-rank structure. Missing entries must not be treated as observed zeros. Successful recovery depends on the sampling pattern as well as rank: an entirely unobserved mode coordinate may leave its associated factor row undetermined. Assumptions that prevent information from concentrating in a few entries are common in recovery theory, but guarantees differ substantially between CP, Tucker, and tensor-train models.

Rank selection balances fidelity against complexity. Cross-validation can compare predictions on held-out observations, provided the holdout design matches the intended use. Randomly withholding entries tests a different task from predicting a future time interval or an entirely new subject. Reconstruction error on the training data usually decreases with model size, so it cannot by itself identify an appropriate rank.

Uncertainty is complicated by factor permutation and scaling. Before comparing repeated fits or bootstrap samples, equivalent components must be aligned. Near nonidentifiability, small perturbations of the data may produce large changes in factor estimates even when reconstructed tensors remain similar. Statistical interpretation therefore requires distinguishing uncertainty about the fitted array from uncertainty about the individual latent components.

Scientific and technological applications

Chemometrics provides an especially direct interpretation of CP. An excitation–emission fluorescence dataset can be organized by sample, excitation wavelength, and emission wavelength. Under approximately linear, additive measurement conditions, a component has a sample-dependent abundance and characteristic excitation and emission profiles. PARAFAC can estimate these profiles jointly rather than fitting each sample separately. Instrument effects, scattering, quenching, and violations of trilinearity can make a simple CP model inadequate, so the interpretation rests on physical measurement assumptions as well as algebra.

In latent-variable statistics, higher-order moments can carry information that a covariance matrix does not. Suppose a latent class hh has probability πh\pi_h, and three observed vector-valued views are conditionally independent given that class. Their cross-moment tensor has the form

E[x∘y∘z]=∑hπh E[x∣h]∘E[y∣h]∘E[z∣h].\mathbb E[x\circ y\circ z] = \sum_h\pi_h\, \mathbb E[x\mid h]\circ \mathbb E[y\mid h]\circ \mathbb E[z\mid h].

This is a CP representation derived from conditional independence. Under suitable assumptions, decomposing the moment tensor identifies latent parameters. Spectral and whitening methods exploit this structure, although estimation error in empirical moments and weakly separated components can limit performance.

Scientific computing uses tensor formats to approximate functions and solve high-dimensional problems. A discretized function f(x1,…,xd)f(x_1,\ldots,x_d) on nn points per coordinate has ndn^d values, an instance of the curse of dimensionality. Low-rank formats can replace this full grid when the function has suitable separability. Operators may themselves admit tensor representations, allowing iterative solution of partial differential equations or eigenvalue problems through contractions and rank truncation. Low rank is a structural opportunity, not a general cure: strongly coupled solutions can require rapidly growing ranks.

In quantum many-body calculations, matrix product states and related networks approximate state vectors that would otherwise have exponentially many amplitudes. The density matrix renormalization group, introduced by Steven White in 1992, became a central method for one-dimensional quantum systems and was later understood naturally through matrix product states. Its effectiveness is linked to the entanglement structure of the states being approximated. Higher-dimensional systems motivate networks with different geometries, often at the price of harder contraction.

In machine learning, tensors organize multi-relational data, user–item–context interactions, image collections, and neural-network parameters. A relation tensor can use distinct entity and relation modes; a neural weight tensor can be replaced by CP, Tucker, or tensor-train factors to reduce parameter storage. Actual acceleration depends on the sequence of contractions, intermediate array sizes, and hardware support. Fewer parameters do not automatically mean lower latency. Multimodal applications also use coupled decompositions in which several arrays share selected factor matrices, allowing common structure to be estimated without forcing every dataset into one rectangular array.

Historical development and continuing research

The modern subject grew from several traditions rather than a single invention. Frank L. Hitchcock’s 1927 work studied expressions of higher-order arrays as sums of products, anticipating the rank-one-sum viewpoint. In psychometrics, Ledyard Tucker developed multiway factor analysis, with his 1966 work establishing the core-and-factor structure now associated with his name. These developments extended matrix-based analysis to measurements involving more than two distinct indexing systems.

In 1970, J. Douglas Carroll and Jih-Jie Chang introduced CANDECOMP, while Richard Harshman independently developed PARAFAC. Their terminology reflected different motivations, but both led to the same basic sum of matched rank-one components. The possibility of essentially unique factor recovery distinguished these models from unconstrained matrix factor analysis. Kruskal’s 1977 uniqueness theorem supplied a rigorous and influential explanation of that possibility.

Numerical multilinear algebra became more systematic as algorithms, software, and applications expanded. The 2000 treatment of the multilinear singular value decomposition by Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle clarified the HOSVD framework and its matrix-SVD connections. Work on low-rank approximation also exposed fundamental limitations: Vin de Silva and Lek-Heng Lim’s 2008 analysis made the nonexistence of best low-rank tensor approximations a central issue in numerical practice, rather than merely an algebraic curiosity.

High-dimensional approximation stimulated a complementary line of development. Wolfgang Hackbusch and Stefan Kühn introduced a hierarchical tensor representation in 2009, and Ivan Oseledets presented the tensor-train decomposition in 2011. These formats connected numerical analysis with structures long used in quantum physics. Their emphasis was not necessarily on interpreting individual rank-one components, but on organizing computations so that high-dimensional objects could be manipulated without full storage.

Current research distinguishes several problems that can look similar in notation but demand different solutions. Generic identifiability concerns whether almost all tensors of a specified model have unique factors; robust identifiability concerns how those factors change under perturbations. Approximation theory studies when ranks remain moderate for particular function classes or operators. Statistical recovery theory asks how sample size, noise, missingness, and model structure determine estimation accuracy.

Scalable computation introduces further questions about communication, memory, and precision. A contraction that is inexpensive in arithmetic may still be costly if it moves large intermediate arrays between processors or accelerators. Adaptive algorithms seek ranks that vary across bonds or modes and change during optimization. For streaming data, factors must be updated without repeatedly accessing the entire dataset. In each setting, the useful decomposition is determined jointly by mathematical structure, the observation process, and the operations that must be performed on the resulting representation.