{
  "id": "dirichlet-distribution",
  "title": "Dirichlet Distribution",
  "lang": "en",
  "source_lang": "en",
  "summary": "The Dirichlet distribution is a family of continuous probability distributions over vectors of nonnegative proportions that sum to one, widely used in Bayesian modeling.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "Multivariate beta distribution",
    "Dirichlet family"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/dirichlet-distribution/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/dirichlet-distribution/",
      "json": "https://aiwiki.page/zh/dirichlet-distribution.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "The Dirichlet distribution is a family of continuous probability distributions for vectors of proportions whose components are nonnegative and sum to one. It generalizes the beta distribution from two components to any finite number of components. Its parameters control both the expected proportions and the concentration of probability around them. In Bayesian inference, it is especially important as a conjugate prior for categorical and multinomial probabilities."
    },
    {
      "heading": "Definition and density",
      "text": "Let \\(K\\geq 2\\) be an integer, and let\n\n\\[\\boldsymbol{\\alpha}=(\\alpha_1,\\ldots,\\alpha_K), \\qquad \\alpha_i>0.\\]\n\nA random vector \\(\\mathbf X=(X_1,\\ldots,X_K)\\) has a Dirichlet distribution, written\n\n\\[\\mathbf X\\sim\\operatorname{Dir}(\\boldsymbol{\\alpha}),\\]\n\nif its probability density is\n\n\\[f(\\mathbf x;\\boldsymbol{\\alpha}) = \\frac{1}{B(\\boldsymbol{\\alpha})} \\prod_{i=1}^{K}x_i^{\\alpha_i-1},\\]\n\non the interior of the probability simplex\n\n\\[\\Delta^{K-1} = \\left\\{ \\mathbf x\\in\\mathbb R^K: x_i\\geq0,\\quad \\sum_{i=1}^{K}x_i=1 \\right\\}.\\]\n\nThe normalizing constant is the multivariate beta function,\n\n\\[B(\\boldsymbol{\\alpha}) = \\frac{\\prod_{i=1}^{K}\\Gamma(\\alpha_i)} {\\Gamma(\\alpha_0)}, \\qquad \\alpha_0=\\sum_{i=1}^{K}\\alpha_i,\\]\n\nwhere \\(\\Gamma\\) is the gamma function.\n\nThe density is conventionally taken with respect to the coordinates \\(x_1,\\ldots,x_{K-1}\\), with\n\n\\[x_K=1-\\sum_{i=1}^{K-1}x_i.\\]\n\nIt is not a density with respect to \\(K\\)-dimensional volume: the simplex lies in a \\((K-1)\\)-dimensional hyperplane. A density defined instead relative to Euclidean surface measure requires a constant adjustment.\n\nFor strictly positive parameters, all components are strictly positive almost surely. Boundary points have probability zero, even when the density becomes unbounded as a component approaches zero.\n\nWhen \\(K=2\\), the first component satisfies\n\n\\[X_1\\sim\\operatorname{Beta}(\\alpha_1,\\alpha_2), \\qquad X_2=1-X_1.\\]"
    },
    {
      "heading": "Parameter interpretation and shape",
      "text": "A useful parameterization separates the mean vector from the total concentration:\n\n\\[\\alpha_i=s\\,m_i, \\qquad m_i>0,\\quad \\sum_i m_i=1,\\quad s=\\alpha_0.\\]\n\nThe vector \\(\\mathbf m\\) specifies the expected proportions, while \\(s\\) controls their dispersion. Holding \\(\\mathbf m\\) fixed, increasing \\(s\\) concentrates the distribution around \\(\\mathbf m\\); decreasing \\(s\\) spreads probability toward the boundary.\n\nThe individual parameters determine boundary behavior:\n\n- If \\(\\alpha_i>1\\), the density tends to zero as \\(x_i\\) approaches zero while the other coordinates remain away from their boundaries.\n- If \\(\\alpha_i=1\\), the corresponding power of \\(x_i\\) is zero.\n- If \\(0<\\alpha_i<1\\), the density diverges as \\(x_i\\) approaches zero.\n\nA symmetric Dirichlet distribution has \\(\\alpha_i=a\\) for every component. Its mean is the uniform vector \\((1/K,\\ldots,1/K)\\).\n\n- For \\(a=1\\), the distribution is uniform over the simplex.\n- For \\(a>1\\), it favors relatively balanced vectors.\n- For \\(0<a<1\\), it favors vectors near the boundary, often with a few large components and many small ones.\n\n“Uniform over the simplex” does not mean that each coordinate is uniformly distributed. For \\(K>2\\), a coordinate under \\(\\operatorname{Dir}(1,\\ldots,1)\\) has distribution \\(\\operatorname{Beta}(1,K-1)\\).\n\nThe distinction between \\(a\\) and the total concentration \\(Ka\\) matters when comparing models with different numbers of categories."
    },
    {
      "heading": "Moments and dependence",
      "text": "Writing \\(m_i=\\alpha_i/\\alpha_0\\), the mean and variance are\n\n\\[\\mathbb E[X_i]=m_i,\\]\n\n\\[\\operatorname{Var}(X_i) = \\frac{\\alpha_i(\\alpha_0-\\alpha_i)} {\\alpha_0^2(\\alpha_0+1)} = \\frac{m_i(1-m_i)}{\\alpha_0+1}.\\]\n\nFor distinct components,\n\n\\[\\operatorname{Cov}(X_i,X_j) = -\\frac{\\alpha_i\\alpha_j} {\\alpha_0^2(\\alpha_0+1)} = -\\frac{m_i m_j}{\\alpha_0+1}.\\]\n\nThus every pair of distinct components has negative covariance. The covariance matrix can be written compactly as\n\n\\[\\operatorname{Cov}(\\mathbf X) = \\frac{\\operatorname{diag}(\\mathbf m)-\\mathbf m\\mathbf m^\\mathsf T} {\\alpha_0+1}.\\]\n\nIt is singular because the coordinates sum exactly to one.\n\nMore generally, for real exponents \\(r_i\\) satisfying \\(\\alpha_i+r_i>0\\),\n\n\\[\\mathbb E\\left[\\prod_{i=1}^{K}X_i^{r_i}\\right] = \\frac{\\Gamma(\\alpha_0)} {\\Gamma\\!\\left(\\alpha_0+\\sum_i r_i\\right)} \\prod_{i=1}^{K} \\frac{\\Gamma(\\alpha_i+r_i)}{\\Gamma(\\alpha_i)}.\\]\n\nThe expected log proportions are\n\n\\[\\mathbb E[\\log X_i] = \\psi(\\alpha_i)-\\psi(\\alpha_0),\\]\n\nwhere \\(\\psi(t)=d\\log\\Gamma(t)/dt\\) is the digamma function. These expectations appear in parameter estimation and variational inference.\n\nIf every \\(\\alpha_i>1\\), the density has a unique interior mode:\n\n\\[x_i^{\\mathrm{mode}} = \\frac{\\alpha_i-1}{\\alpha_0-K}.\\]\n\nThis formula is not a general mode formula when some parameters are at most one. In those cases, density maxima or divergent limiting values can occur at the boundary, and modes may be nonunique."
    },
    {
      "heading": "Marginals, aggregation, and conditioning",
      "text": "The Dirichlet family has several closure properties that make it convenient for modeling proportions.\n\nMarginal distributions\n\nEach coordinate has a beta distribution:\n\n\\[X_i\\sim \\operatorname{Beta}(\\alpha_i,\\alpha_0-\\alpha_i).\\]\n\nMore generally, for a nonempty proper subset \\(A\\) of the component indices,\n\n\\[\\sum_{i\\in A}X_i \\sim \\operatorname{Beta}\\left( \\sum_{i\\in A}\\alpha_i, \\sum_{i\\notin A}\\alpha_i \\right).\\]\n\nAggregation\n\nIf \\(A_1,\\ldots,A_J\\) partition the component indices and\n\n\\[Y_j=\\sum_{i\\in A_j}X_i,\\]\n\nthen\n\n\\[(Y_1,\\ldots,Y_J) \\sim \\operatorname{Dir}\\left( \\sum_{i\\in A_1}\\alpha_i,\\ldots, \\sum_{i\\in A_J}\\alpha_i \\right).\\]\n\nMerging categories therefore corresponds to adding their parameters.\n\nNormalized subvectors\n\nFor a subset \\(A\\), let \\(S_A=\\sum_{i\\in A}X_i\\). The normalized subvector satisfies\n\n\\[\\left(\\frac{X_i}{S_A}\\right)_{i\\in A} \\sim \\operatorname{Dir}\\bigl((\\alpha_i)_{i\\in A}\\bigr).\\]\n\nIt is independent of \\(S_A\\) and of the components outside \\(A\\). This strong independence structure is often described through the Dirichlet distribution’s neutrality properties."
    },
    {
      "heading": "Gamma construction and simulation",
      "text": "A fundamental construction uses independent gamma random variables. Let\n\n\\[Y_i\\sim\\operatorname{Gamma}(\\alpha_i,\\text{rate}=\\beta), \\qquad i=1,\\ldots,K,\\]\n\nindependently, with a common rate \\(\\beta>0\\). Then\n\n\\[X_i=\\frac{Y_i}{\\sum_jY_j}\\]\n\nhas distribution \\(\\operatorname{Dir}(\\boldsymbol{\\alpha})\\).\n\nMoreover,\n\n\\[T=\\sum_iY_i\\sim\\operatorname{Gamma}(\\alpha_0,\\text{rate}=\\beta),\\]\n\nand \\(T\\) is independent of \\(\\mathbf X\\). The common rate disappears on normalization.\n\nThis gives a standard simulation method: generate independent gamma variates and normalize their sum. For very small shape parameters, implementations require care because tiny gamma variates can underflow in floating-point arithmetic.\n\nAn alternative is a finite stick-breaking construction. Independently draw\n\n\\[V_i\\sim\\operatorname{Beta} \\left(\\alpha_i,\\sum_{j=i+1}^{K}\\alpha_j\\right), \\qquad i=1,\\ldots,K-1,\\]\n\nand set\n\n\\[X_i=V_i\\prod_{j<i}(1-V_j), \\qquad X_K=\\prod_{j<K}(1-V_j).\\]\n\nEach \\(V_i\\) determines the fraction of the remaining mass allocated to component \\(i\\)."
    },
    {
      "heading": "Bayesian inference for categorical data",
      "text": "Suppose \\(\\mathbf p=(p_1,\\ldots,p_K)\\) is an unknown category-probability vector. Given counts \\(n_1,\\ldots,n_K\\), with \\(N=\\sum_i n_i\\), the multinomial distribution gives a likelihood proportional to\n\n\\[L(\\mathbf p)\\propto\\prod_i p_i^{n_i}.\\]\n\nIf the prior distribution is\n\n\\[\\mathbf p\\sim\\operatorname{Dir}(\\alpha_1,\\ldots,\\alpha_K),\\]\n\nmultiplication by this likelihood yields the posterior distribution\n\n\\[\\mathbf p\\mid\\mathbf n \\sim \\operatorname{Dir}(\\alpha_1+n_1,\\ldots,\\alpha_K+n_K).\\]\n\nThis is the conjugacy relationship.\n\nThe posterior mean is\n\n\\[\\mathbb E[p_i\\mid\\mathbf n] = \\frac{\\alpha_i+n_i}{\\alpha_0+N}.\\]\n\nFor \\(N>0\\), it can also be written as\n\n\\[\\frac{\\alpha_0}{\\alpha_0+N}\\frac{\\alpha_i}{\\alpha_0} + \\frac{N}{\\alpha_0+N}\\frac{n_i}{N},\\]\n\na weighted average of the prior mean and the observed frequency. This motivates interpreting the \\(\\alpha_i\\) as prior pseudocounts, although that interpretation depends on which posterior summary is being considered.\n\nFor a new observation from a categorical distribution, the posterior predictive probability is\n\n\\[\\Pr(Z_{\\mathrm{new}}=i\\mid\\mathbf n) = \\frac{\\alpha_i+n_i}{\\alpha_0+N}.\\]\n\nFor example, a \\(\\operatorname{Dir}(1,1,1)\\) prior and observed counts \\((8,1,1)\\) give a \\(\\operatorname{Dir}(9,2,2)\\) posterior and predictive probabilities \\((9/13,2/13,2/13)\\).\n\nIntegrating out \\(\\mathbf p\\) gives the Dirichlet–multinomial distribution:\n\n\\[\\Pr(\\mathbf n) = \\frac{N!}{\\prod_i n_i!} \\frac{\\Gamma(\\alpha_0)}{\\Gamma(\\alpha_0+N)} \\prod_i \\frac{\\Gamma(\\alpha_i+n_i)}{\\Gamma(\\alpha_i)}.\\]\n\nIt allows greater count variability than a multinomial model with fixed probabilities.\n\nThe all-ones prior is uniform over the simplex, but it is not uniquely noninformative. For the multinomial model, the Jeffreys prior is \\(\\operatorname{Dir}(1/2,\\ldots,1/2)\\). Taking every parameter to zero does not produce a proper Dirichlet distribution."
    },
    {
      "heading": "Estimating the parameters",
      "text": "Given independent observed proportion vectors \\(\\mathbf x^{(1)},\\ldots,\\mathbf x^{(M)}\\) with strictly positive components, the log-likelihood is\n\n\\[\\ell(\\boldsymbol{\\alpha}) = M\\left[ \\log\\Gamma(\\alpha_0)-\\sum_i\\log\\Gamma(\\alpha_i) \\right] + \\sum_i(\\alpha_i-1)\\sum_{m=1}^{M}\\log x_i^{(m)}.\\]\n\nThe maximum-likelihood equations are\n\n\\[\\psi(\\alpha_i)-\\psi(\\alpha_0) = \\frac{1}{M}\\sum_{m=1}^{M}\\log x_i^{(m)}.\\]\n\nThey generally require numerical solution, using methods such as Newton’s method or fixed-point iteration. The constraints \\(\\alpha_i>0\\) must be preserved.\n\nObserved exact zeros require separate treatment: the ordinary Dirichlet model assigns them probability zero, and their logarithms are undefined. Zeros caused by rounding or finite sampling differ from structural zeros that represent genuinely impossible components."
    },
    {
      "heading": "Applications and limitations",
      "text": "Dirichlet distributions are used for uncertain class probabilities, mixture weights, transition probabilities, and compositional proportions. In latent Dirichlet allocation, they commonly provide priors for document–topic and topic–word probability vectors. The Dirichlet process extends the finite-partition structure: probabilities assigned to the sets in any finite measurable partition have a Dirichlet distribution.\n\nDespite its tractability, the family imposes substantial restrictions:\n\n- Negative pairwise covariance. Every pair of distinct components is negatively correlated. The sum-to-one constraint alone does not require this; other distributions on a simplex can permit positive association between selected components.\n- Restricted dispersion structure. Once the mean is fixed, a single total concentration controls all component variances.\n- No exact zeros. Small parameters can generate very small proportions, but not positive probability at zero.\n- Limited multimodal flexibility. A single Dirichlet distribution cannot describe arbitrary clusters of compositions centered at several unrelated interior locations.\n\nAlternatives include mixtures of Dirichlet distributions, explicit models for structural zeros, and the logistic-normal distribution, which transforms a multivariate normal vector into proportions and supports richer dependence structures."
    }
  ],
  "links": [
    "probability-distribution",
    "beta-distribution",
    "bayesian-inference",
    "conjugate-prior",
    "probability-density-function",
    "simplex",
    "gamma-function",
    "hyperplane",
    "almost-surely",
    "expected-value",
    "variance",
    "covariance",
    "covariance-matrix",
    "variational-inference",
    "floating-point-arithmetic",
    "prior-distribution",
    "posterior-distribution",
    "posterior-predictive-distribution",
    "maximum-likelihood-estimation",
    "newtons-method"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}