{
  "id": "t-distributed-stochastic-neighbor-embedding",
  "title": "t-distributed stochastic neighbor embedding",
  "lang": "en",
  "source_lang": "en",
  "summary": "t-distributed stochastic neighbor embedding is a nonlinear dimensionality-reduction method that visualizes high-dimensional data by preserving local similarity relationships in a low-dimensional representation.",
  "category": "computer-science",
  "subcategory": "artificial-intelligence",
  "type": "concept",
  "aliases": [
    "t-SNE",
    "t-distributed stochastic neighbour embedding"
  ],
  "generated_at": "2026-11-03",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/t-distributed-stochastic-neighbor-embedding/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/t-distributed-stochastic-neighbor-embedding/",
      "json": "https://aiwiki.page/zh/t-distributed-stochastic-neighbor-embedding.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "t-distributed stochastic neighbor embedding (t-SNE) is a dimensionality-reduction method used primarily for visualizing high-dimensional datasets in two or three dimensions. It represents similarities between observations as probabilities and finds a low-dimensional arrangement whose similarity probabilities approximate those of the original data. Introduced by Laurens van der Maaten and Geoffrey Hinton in 2008, it is widely used in machine learning and exploratory data analysis. Its principal strength is revealing local neighborhood structure; distances between separated groups, group sizes, and apparent gaps require more cautious interpretation."
    },
    {
      "heading": "Background and purpose",
      "text": "High-dimensional observations may describe images, documents, gene-expression profiles, or representations learned by a model. Each observation can contain hundreds or thousands of numerical features, making direct visualization difficult.\n\nt-SNE belongs to unsupervised learning because its standard formulation does not require class labels. It is commonly grouped with manifold-learning techniques, although it does not require an explicit mathematical model of an underlying manifold.\n\nThe method builds on stochastic neighbor embedding (SNE), introduced by Hinton and Sam Roweis in 2002. SNE expresses neighborhood relationships probabilistically and optimizes a low-dimensional representation to reproduce them. t-SNE combines symmetric high-dimensional affinities with a heavy-tailed distribution in the embedding space. These choices produce a simpler optimization formulation and help address the crowding problem: moderately separated high-dimensional points cannot all be accommodated at corresponding distances in a much lower-dimensional space.\n\nUnlike principal component analysis (PCA), which constructs linear directions that successively maximize projected variance, t-SNE directly optimizes the positions of individual observations. Its axes therefore do not correspond to fixed combinations of the original features."
    },
    {
      "heading": "Mathematical formulation",
      "text": "Let the input dataset contain \\(n\\) observations,\n\n\\[x_1,\\ldots,x_n\\in\\mathbb{R}^{D},\\]\n\nand let their embedded coordinates be\n\n\\[y_1,\\ldots,y_n\\in\\mathbb{R}^{d},\\]\n\nwhere \\(d\\) is usually 2 or 3.\n\nHigh-dimensional affinities\n\nFor each observation \\(x_i\\), t-SNE defines a conditional probability that another observation \\(x_j\\) is its neighbor:\n\n\\[p_{j\\mid i} = \\frac{ \\exp\\left(-\\lVert x_i-x_j\\rVert^2/(2\\sigma_i^2)\\right) }{ \\sum_{k\\ne i} \\exp\\left(-\\lVert x_i-x_k\\rVert^2/(2\\sigma_i^2)\\right) }, \\qquad j\\ne i.\\]\n\nSelf-affinities are set to zero. The bandwidth \\(\\sigma_i\\) is adjusted separately for each observation, allowing neighborhood scales to adapt to differences in local density.\n\nThese conditional probabilities are symmetrized:\n\n\\[p_{ij} = \\frac{p_{j\\mid i}+p_{i\\mid j}}{2n}, \\qquad p_{ii}=0.\\]\n\nThe resulting affinities satisfy\n\n\\[\\sum_{i\\ne j}p_{ij}=1.\\]\n\nThe standard formulation uses squared Euclidean distances. Implementations can also work with other distance measures or precomputed dissimilarities, which change the meaning of the neighborhoods being preserved.\n\nPerplexity\n\nThe bandwidth for each observation is usually selected to match a user-specified perplexity:\n\n\\[\\operatorname{Perp}(P_i)=2^{H(P_i)},\\]\n\nwhere\n\n\\[H(P_i) = -\\sum_{j\\ne i}p_{j\\mid i}\\log_2 p_{j\\mid i}.\\]\n\nPerplexity can be interpreted as an effective number of neighbors. It is not a hard neighbor count: affinities remain weighted probabilities.\n\nSmall perplexities emphasize very local relationships, while larger values incorporate broader neighborhoods. Values between approximately 5 and 50 are common starting points, but suitable scales depend on dataset size and structure. The requested perplexity must be smaller than the number of observations; the theoretical maximum for a conditional distribution over the other observations is \\(n-1\\).\n\nLow-dimensional affinities\n\nIn the embedding, t-SNE uses a kernel based on a Student’s t-distribution with one degree of freedom:\n\n\\[q_{ij} = \\frac{ \\left(1+\\lVert y_i-y_j\\rVert^2\\right)^{-1} }{ \\sum_{k\\ne l} \\left(1+\\lVert y_k-y_l\\rVert^2\\right)^{-1} }, \\qquad q_{ii}=0.\\]\n\nThis kernel has heavier tails than a Gaussian. Consequently, dissimilar observations can be positioned relatively far apart without their low-dimensional affinity becoming as small as it would under a Gaussian kernel. This helps make room for local neighborhoods and reduces crowding.\n\nObjective function\n\nThe embedding minimizes the Kullback–Leibler divergence from the high-dimensional affinity distribution \\(P\\) to the low-dimensional distribution \\(Q\\):\n\n\\[C = D_{\\mathrm{KL}}(P\\parallel Q) = \\sum_{i\\ne j} p_{ij}\\log\\frac{p_{ij}}{q_{ij}}.\\]\n\nThis loss function is asymmetric. A pair with large \\(p_{ij}\\) is strongly penalized if its embedded affinity \\(q_{ij}\\) becomes very small. Pairs with very small \\(p_{ij}\\) contribute comparatively little directly to the loss, although all points remain coupled through the normalization of \\(Q\\).\n\nThe objective therefore emphasizes retaining important neighbors rather than preserving every pairwise distance. It does not guarantee exact neighborhood preservation or faithful global geometry.\n\nFor the standard formulation, the gradient is\n\n\\[\\frac{\\partial C}{\\partial y_i} = 4\\sum_{j\\ne i} (p_{ij}-q_{ij}) \\frac{y_i-y_j}{1+\\lVert y_i-y_j\\rVert^2}.\\]\n\nThis can be interpreted in terms of attractive interactions associated with high-dimensional affinities and repulsive interactions arising from the embedding distribution."
    },
    {
      "heading": "Optimization and parameters",
      "text": "t-SNE solves a nonconvex optimization problem, typically using gradient descent with momentum or related update rules. Different initial coordinates and optimization settings can lead to different layouts.\n\nImportant settings include:\n\n- Perplexity: controls the scale of the high-dimensional neighborhood model.\n- Initialization: coordinates may begin as small random values or as a scaled PCA projection. PCA initialization can improve reproducibility and retain more broad-scale structure, but it does not guarantee global fidelity.\n- Learning rate: controls update size. Poor choices can produce slow convergence or distorted arrangements. Numerical conventions differ between software packages.\n- Early exaggeration: temporarily multiplies the attractive affinities by a factor greater than one. This strengthens attraction early in optimization and can encourage separated groups to form.\n- Iteration count and stopping criteria: influence whether the optimization has settled sufficiently for interpretation.\n- Random seed: controls stochastic elements such as initialization and can affect the resulting arrangement.\n\nEarly exaggeration is an optimization phase rather than a different probability model for the final embedding. Later iterations normally optimize the unexaggerated objective.\n\nA fixed random seed can make a particular computational setup more reproducible, but it does not establish that its layout is uniquely determined by the data. Comparing layouts across parameter choices can reveal which visual relationships are stable."
    },
    {
      "heading": "Computational methods",
      "text": "The exact method evaluates interactions among all pairs of observations. Its pairwise calculations require approximately \\(O(n^2)\\) time per optimization iteration, and explicitly storing dense affinities requires \\(O(n^2)\\) memory. These costs limit its use on large datasets.\n\nSeveral approximations improve scalability:\n\n- Sparse affinities: high-dimensional affinities are approximated using a restricted set of nearby observations, often found through approximate nearest-neighbor search.\n- Barnes–Hut t-SNE: approximates long-range repulsive interactions by grouping distant points in a spatial tree. In typical low-dimensional settings, this reduces the force-calculation cost toward \\(O(n\\log n)\\).\n- Interpolation-based t-SNE: methods such as FIt-SNE interpolate interactions onto a grid and use fast Fourier transforms to accelerate the repulsive calculation.\n- Hardware acceleration: implementations can exploit parallel computing and graphics processing units.\n\nThese approaches approximate parts of the calculation rather than simply executing the exact method faster. Their accuracy and runtime depend on implementation settings, data size, and the dimensionality of the embedding."
    },
    {
      "heading": "Applications and preprocessing",
      "text": "t-SNE is commonly used to explore:\n\n- image features and representations from deep-learning models;\n- document vectors and word embeddings;\n- single-cell gene-expression measurements;\n- other multivariate scientific datasets.\n\nIn representation learning, a t-SNE plot can show whether observations with shared labels occupy nearby regions. Labels may be used to color the result without participating in the embedding calculation. Such visual separation is descriptive, not an independent measure of predictive performance.\n\nPreprocessing strongly affects the result because it determines the distances from which affinities are constructed. Feature scaling, normalization, feature selection, and distance-metric choice can all change neighborhoods substantially.\n\nFor high-dimensional dense data, PCA is often applied before t-SNE to reduce noise and computational cost. This also changes the information available to t-SNE: structure discarded during preprocessing cannot be recovered by the embedding."
    },
    {
      "heading": "Interpretation and limitations",
      "text": "Local structure is not global geometry\n\nNearby points in a t-SNE plot are intended to reflect neighborhood relationships in the chosen input representation. Distances between widely separated groups, however, generally cannot be interpreted as proportional to their original dissimilarities.\n\nA large visual gap does not establish that two populations are sharply separated in the original space. Conversely, neighboring groups need not be one another’s closest groups under the original distance measure.\n\nGroup sizes and densities can be misleading\n\nBecause t-SNE adapts bandwidths locally, it can expand dense regions and compress sparse ones. The area occupied by a group is therefore not a reliable measure of its original variance, density, or intrinsic extent.\n\nShapes, empty spaces, and relative arrangements also depend on perplexity, optimization, sampling, and preprocessing. Some apparent groups may reflect these choices rather than robust divisions in the data.\n\nVisualization is not clustering\n\nt-SNE produces coordinates, not cluster assignments. Visually distinct islands are not by themselves evidence of discrete categories, and continuous data can appear fragmented under some settings.\n\nApplying k-means clustering or another clustering method to t-SNE coordinates analyzes the geometry of the embedding, which may differ substantially from the original geometry. Quantitative conclusions require evidence beyond visual separation.\n\nCoordinates have no inherent semantic axes\n\nTranslations, rotations, and reflections preserve the pairwise distances used by the objective. Consequently, the horizontal and vertical axes do not normally have intrinsic meanings. Their directions cannot usually be interpreted as individual features, principal components, or ordered biological variables.\n\nNew observations require additional machinery\n\nStandard t-SNE optimizes coordinates for a particular dataset and does not automatically learn a reusable mapping from input features to embedding coordinates. Adding observations and rerunning it can move previously embedded points.\n\nSome implementations support placement of new observations relative to a fixed reference embedding. Parametric t-SNE instead learns an explicit mapping, often with an artificial neural network. These extensions have different properties from the original nonparametric method."
    },
    {
      "heading": "Comparison with related methods",
      "text": "PCA provides a linear projection with defined directions of explained variance and a straightforward transformation for new observations. It can miss nonlinear neighborhood relationships that t-SNE makes visible.\n\nUniform Manifold Approximation and Projection (UMAP) also emphasizes neighborhood relationships but uses a different mathematical construction and objective. It often supports efficient transformation of new observations. Its plots, like t-SNE plots, do not automatically provide faithful global distances.\n\nOther manifold-learning methods, such as Isomap and locally linear embedding, preserve different structural properties and make different assumptions about the data. Their outputs answer different geometric questions rather than providing interchangeable versions of the same map.\n\nEvaluation can include neighborhood-overlap measures, trustworthiness scores, stability across runs, and comparison with distances or labels in the original representation. A visually striking layout alone is not a quantitative validation of the underlying structure."
    }
  ],
  "links": [
    "dimensionality-reduction",
    "data-visualization",
    "machine-learning",
    "unsupervised-learning",
    "manifold-learning",
    "principal-component-analysis",
    "perplexity",
    "students-t-distribution",
    "kullback-leibler-divergence",
    "loss-function",
    "mathematical-optimization",
    "gradient-descent",
    "learning-rate",
    "parallel-computing",
    "graphics-processing-unit",
    "deep-learning",
    "word-embedding",
    "gene-expression",
    "representation-learning",
    "feature-scaling",
    "k-means-clustering",
    "artificial-neural-network",
    "umap"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}