{
  "id": "spectral-clustering",
  "title": "Spectral Clustering",
  "lang": "en",
  "source_lang": "en",
  "summary": "Spectral clustering partitions data using eigenvectors of a similarity graph, revealing groups that may not be separable by simple geometric boundaries in the original feature space.",
  "category": "computer-science",
  "subcategory": "artificial-intelligence",
  "type": "concept",
  "aliases": [
    "Graph spectral clustering",
    "spectral graph clustering"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/spectral-clustering/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/spectral-clustering/",
      "json": "https://aiwiki.page/zh/spectral-clustering.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Spectral clustering is a family of unsupervised learning methods that groups observations by analyzing the eigenvectors of a matrix representing their similarities. Rather than assigning clusters directly in the original feature space, it constructs a weighted graph, maps its vertices into a spectral representation, and partitions that representation, often with k-means clustering. The approach can identify nonconvex groups, such as curved or interlocking shapes, when the graph captures their underlying connectivity. Its results depend strongly on how similarities and graph connections are defined."
    },
    {
      "heading": "Mathematical foundations",
      "text": "Representing data as a graph\n\nGiven observations \\(x_1,\\ldots,x_n\\), spectral clustering constructs an undirected, weighted graph whose vertices represent observations and whose edge weights represent similarity. These weights form a symmetric affinity matrix \\(W\\), with entries \\(w_{ij}\\geq 0\\). A missing edge has weight zero. Self-weights \\(w_{ii}\\) are commonly set to zero.\n\nOne common affinity is the Gaussian similarity:\n\n\\[w_{ij} = \\exp\\left(-\\frac{\\lVert x_i-x_j\\rVert^2}{2\\sigma^2}\\right), \\qquad i\\ne j,\\]\n\nwhere \\(\\sigma>0\\) sets the distance scale. Small values emphasize local relationships; large values make distant observations more similar.\n\nCommon graph constructions include:\n\n- Fully connected graphs: every pair of distinct observations receives a similarity weight.\n- Nearest-neighbor graphs: each observation connects to a specified number of nearby observations. Because the neighbor relation can be asymmetric, the graph is usually symmetrized by retaining either the union or intersection of neighbor relations.\n- Neighborhood-radius graphs: an edge exists only when the distance between two observations is below a threshold.\n\nThe neighbor count used to construct the graph is distinct from the desired number of clusters. A nearest-neighbor graph also does not make spectral clustering a form of nearest-neighbor classification: it specifies relationships among unlabeled observations rather than predicting labels from labeled neighbors.\n\nAffinities may instead come from network connections, document similarities, or domain-specific measurements. Ordinary spectral clustering assumes a symmetric, nonnegative affinity matrix; directed graphs and signed similarities require modified formulations.\n\nGraph Laplacians\n\nThe degree of vertex \\(i\\) is\n\n\\[d_i=\\sum_{j=1}^{n}w_{ij}.\\]\n\nLet \\(D=\\operatorname{diag}(d_1,\\ldots,d_n)\\). The unnormalized graph Laplacian is\n\n\\[L=D-W.\\]\n\nTwo widely used normalized forms are\n\n\\[L_{\\mathrm{sym}} = I-D^{-1/2}WD^{-1/2}\\]\n\nand\n\n\\[L_{\\mathrm{rw}} = I-D^{-1}W.\\]\n\nThese definitions assume positive degrees; isolated vertices require separate handling or an explicit convention.\n\nFor a real vector \\(f\\),\n\n\\[f^\\mathsf{T}Lf = \\frac{1}{2}\\sum_{i,j}w_{ij}(f_i-f_j)^2.\\]\n\nThus, low values of this quadratic form correspond to functions that vary little across strongly connected vertices. This is the central mechanism of spectral clustering: eigenvectors associated with small Laplacian eigenvalues provide coordinates that change slowly within strongly connected groups.\n\nThe eigenvalues and eigenvectors also encode exact connectivity. The multiplicity of the zero eigenvalue equals the number of connected components. For the unnormalized Laplacian, component-indicator vectors span the zero eigenspace. For \\(L_{\\mathrm{sym}}\\), the corresponding vectors are degree-weighted indicators. When components are weakly connected rather than completely disconnected, small nonzero eigenvalues can indicate approximate cluster structure."
    },
    {
      "heading": "Connection to graph partitioning",
      "text": "Spectral clustering can be derived as a relaxation of discrete graph partitioning problems.\n\nFor disjoint vertex sets \\(A\\) and \\(B\\), define\n\n\\[\\operatorname{cut}(A,B) = \\sum_{i\\in A,\\;j\\in B}w_{ij},\\]\n\nand define the volume of a set by\n\n\\[\\operatorname{vol}(A)=\\sum_{i\\in A}d_i.\\]\n\nMinimizing cut weight alone can favor separating a single weakly connected vertex. Balanced objectives counter this tendency by accounting for cluster size or volume.\n\nFor a partition into \\(k\\) nonempty sets \\(A_1,\\ldots,A_k\\), two important objectives are\n\n\\[\\operatorname{RatioCut} = \\sum_{r=1}^{k} \\frac{\\operatorname{cut}(A_r,\\overline{A_r})}{|A_r|}\\]\n\nand\n\n\\[\\operatorname{Ncut} = \\sum_{r=1}^{k} \\frac{\\operatorname{cut}(A_r,\\overline{A_r})} {\\operatorname{vol}(A_r)}.\\]\n\nHere \\(\\overline{A_r}\\) denotes the complement of \\(A_r\\). Ratio cut balances by the number of vertices; normalized cut balances by total incident edge weight.\n\nImportant balanced-cut formulations are NP-hard. Spectral methods replace discrete cluster indicators with real-valued vectors, producing tractable relaxations. In particular, the ratio-cut relaxation leads to\n\n\\[\\min_{H^\\mathsf{T}H=I} \\operatorname{tr}(H^\\mathsf{T}LH),\\]\n\nwhose solutions use eigenvectors associated with the \\(k\\) smallest eigenvalues of \\(L\\).\n\nThe normalized-cut relaxation can be expressed as\n\n\\[\\min_{H^\\mathsf{T}DH=I} \\operatorname{tr}(H^\\mathsf{T}LH),\\]\n\nleading to the generalized eigenproblem\n\n\\[Lu=\\lambda Du.\\]\n\nIt can equivalently be transformed into an ordinary symmetric eigenproblem involving \\(L_{\\mathrm{sym}}\\).\n\nThese relaxations explain the method’s computational structure, but they do not generally recover the exact optimum of the original discrete objective. Converting continuous eigenvectors into cluster labels introduces a further approximation."
    },
    {
      "heading": "Main algorithmic variants",
      "text": "A typical spectral clustering procedure has four stages:\n\n- Construct the affinity graph.\n- Compute a Laplacian or related normalized matrix.\n- Extract a small set of eigenvectors to form an embedding.\n- Convert embedded coordinates into cluster labels.\n\nThe choice of matrix, eigenvectors, normalization, and final partitioning rule distinguishes the variants.\n\nUnnormalized spectral clustering\n\nCompute eigenvectors corresponding to the \\(k\\) smallest eigenvalues of \\(L\\), arrange them as columns of \\(U\\in\\mathbb{R}^{n\\times k}\\), and cluster the rows of \\(U\\).\n\nThis variant follows the ratio-cut relaxation. It can behave poorly when graph degrees vary greatly, because its size-balancing objective does not account for those degree differences.\n\nRandom-walk normalized clustering\n\nSolve \\(Lu=\\lambda Du\\), or equivalently use the right eigenvectors of \\(L_{\\mathrm{rw}}\\) associated with its smallest eigenvalues. Cluster the rows of the resulting eigenvector matrix.\n\nThis formulation is closely associated with normalized cut. For a two-way partition, a threshold on a nontrivial eigenvector can be used instead of a separate multidimensional clustering step.\n\nSymmetric normalized clustering\n\nA widely used procedure forms the first \\(k\\) eigenvectors of \\(L_{\\mathrm{sym}}\\), places them in the columns of \\(U\\), and normalizes each nonzero row:\n\n\\[y_i=\\frac{U_{i,:}}{\\lVert U_{i,:}\\rVert}.\\]\n\nK-means then partitions the row vectors \\(y_i\\). This row normalization is a characteristic part of the Ng–Jordan–Weiss formulation.\n\nAn equivalent formulation uses the eigenvectors associated with the largest eigenvalues of \\(D^{-1/2}WD^{-1/2}\\), since this matrix equals \\(I-L_{\\mathrm{sym}}\\).\n\nFor standard multiway normalized clustering, the \\(k\\) eigenvectors generally include the trivial eigenvector. Other spectral embedding tasks may discard it; the two conventions should not be treated as interchangeable."
    },
    {
      "heading": "Interpretation and related methods",
      "text": "Random walks\n\nThe matrix\n\n\\[P=D^{-1}W\\]\n\nis a transition matrix for a random walk on the graph. For a connected undirected graph, its stationary distribution is proportional to vertex degree.\n\nA useful cluster is a region in which a random walk tends to remain before crossing into another region. Normalized cut measures the tendency to leave clusters relative to their volumes. This interpretation connects spectral clustering to diffusion-based methods and the study of slowly mixing processes.\n\nGeometry and representation\n\nSpectral clustering is also a form of representation learning: it replaces the original coordinates with graph-derived coordinates. Two observations can become close in this representation because they belong to the same connected structure, even if their original coordinates are far apart.\n\nFor example, two interlocking crescent-shaped groups may be difficult to separate with k-means in the original plane. A local similarity graph can strongly connect points along each crescent while creating few connections between them. The spectral representation can then make the groups easier to separate.\n\nThis relationship places spectral clustering near manifold learning, particularly Laplacian eigenmaps. Their goals differ: manifold learning typically emphasizes a useful low-dimensional representation, whereas spectral clustering uses such a representation to produce a partition.\n\nRelationship to kernel methods\n\nSpectral clustering is related to kernel methods because both can operate through pairwise similarity matrices. Certain graph-cut objectives are equivalent to appropriately constructed weighted kernel k-means objectives. Nevertheless, a nonnegative graph affinity matrix is not necessarily a positive semidefinite kernel matrix, and spectral clustering is not simply ordinary k-means applied to an arbitrary kernel."
    },
    {
      "heading": "Parameter selection and numerical behavior",
      "text": "Similarity scale and preprocessing\n\nGraph construction is often more consequential than the choice among standard spectral variants. Distance-based affinities depend on units, irrelevant variables, and the chosen metric. Feature scaling or a domain-specific similarity measure can substantially change the resulting graph.\n\nAn excessively local graph may fragment into disconnected pieces. An overly dense graph, or an excessively large Gaussian bandwidth, may blur meaningful separation. Locally scaled affinities use neighborhood-dependent bandwidths to accommodate varying sampling density, but introduce their own assumptions.\n\nNumber of clusters\n\nThe number of clusters is often supplied in advance. A common heuristic examines the ordered Laplacian eigenvalues\n\n\\[0=\\lambda_1\\leq\\lambda_2\\leq\\cdots\\]\n\nand looks for a substantial gap between \\(\\lambda_k\\) and \\(\\lambda_{k+1}\\). Such an eigengap can support a \\(k\\)-cluster interpretation, but is not a universal selection rule. Noise, overlapping groups, uneven density, and graph parameters can all change the spectrum.\n\nStability and label assignment\n\nEigenvectors have arbitrary signs, and an eigenspace with repeated eigenvalues admits different orthonormal bases. These ambiguities do not inherently change Euclidean distances between embedded rows under a common orthogonal transformation. However, a small gap between selected and unselected eigenvalues can make the selected subspace sensitive to perturbations.\n\nThe final k-means stage can produce different results under different initializations. Thresholding, discretization methods, and alternative rounding procedures provide other ways to assign labels."
    },
    {
      "heading": "Computational cost and scalability",
      "text": "Spectral clustering can be expensive because it operates on relationships among observations rather than only their individual feature vectors.\n\nA dense affinity matrix requires \\(O(n^2)\\) storage, and straightforward pairwise-distance construction requires quadratic work in the number of observations, with additional dependence on feature dimension. A full dense eigendecomposition has approximately \\(O(n^3)\\) time complexity.\n\nPractical implementations usually compute only a few eigenvectors. Sparse neighborhood graphs and iterative eigensolvers can reduce storage and computation considerably. Their cost depends on graph sparsity, spectral gaps, numerical tolerances, and the number of requested eigenvectors; there is no single runtime bound that describes all implementations.\n\nLarge-scale approaches include:\n\n- sparse graph construction and approximate neighbor search;\n- landmark and Nyström approximations;\n- graph coarsening and multilevel partitioning;\n- distributed eigensolvers;\n- learned approximations to spectral embeddings.\n\nThese methods can alter the effective graph or embedding. Their accuracy therefore depends on whether they preserve the structures relevant to the partition."
    },
    {
      "heading": "Applications and limitations",
      "text": "Spectral clustering has been used in image segmentation, document grouping, network community detection, motion segmentation, and the analysis of relational data. In computer vision, affinities can combine appearance and spatial proximity. In document analysis, they can represent similarity between sparse text vectors or learned embeddings.\n\nIts main limitations are:\n\n- Dependence on similarity design: a partition reflects the chosen graph, not an objective grouping independent of representation.\n- Sensitivity to noise and degree imbalance: outliers, weak connections, and uneven sampling can distort eigenvectors or create unintended partitions.\n- Limited scalability: affinity construction and eigenvector computation can dominate memory and runtime.\n- Transductive structure: standard formulations cluster the observations used to build the graph. Assigning new observations requires an extension, such as an out-of-sample embedding or a separately trained prediction rule.\n- Hard partitioning: standard variants assign each observation to one group, which may not represent overlapping or hierarchical structure.\n- No universal shape guarantee: the ability to separate some nonconvex groups does not imply reliable recovery of arbitrary shapes or densities."
    },
    {
      "heading": "Historical development",
      "text": "Spectral clustering developed from spectral graph theory and numerical graph partitioning. Miroslav Fiedler’s work in the 1970s established important connections between graph connectivity and Laplacian eigenvalues. The eigenvector associated with the second-smallest Laplacian eigenvalue became central to spectral bisection.\n\nIn computer vision, Jianbo Shi and Jitendra Malik’s normalized-cut work, published in 2000, connected graph partitioning with perceptual image segmentation. Andrew Ng, Michael Jordan, and Yair Weiss subsequently presented an influential normalized spectral clustering formulation in the early 2000s. These developments helped establish spectral clustering as a standard approach within machine learning, linking graph-cut objectives, spectral embeddings, and conventional clustering procedures."
    }
  ],
  "links": [
    "unsupervised-learning",
    "k-means-clustering",
    "graph-theory",
    "matrix",
    "k-nearest-neighbors-algorithm",
    "graph-laplacian",
    "eigenvalues-and-eigenvectors",
    "np-hardness",
    "random-walk",
    "representation-learning",
    "manifold-learning",
    "kernel-method",
    "feature-scaling",
    "heuristic",
    "time-complexity",
    "image-segmentation",
    "computer-vision",
    "machine-learning"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}