{
  "id": "covariance",
  "title": "Covariance",
  "lang": "en",
  "source_lang": "en",
  "summary": "Covariance measures how two random variables vary together through the expected product of their deviations from their means.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "Population covariance",
    "sample covariance"
  ],
  "generated_at": "2026-10-04",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/covariance/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/covariance/",
      "json": "https://aiwiki.page/zh/covariance.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Covariance is a numerical measure of joint variability in probability theory and statistics. It describes how the deviations of two random variables from their respective means vary together. Positive covariance indicates that deviations in the same direction predominate, while negative covariance indicates that deviations in opposite directions predominate. Unlike a standardized correlation coefficient, its magnitude depends on the variables’ measurement scales. (online.stat.psu.edu)"
    },
    {
      "heading": "Definition and interpretation",
      "text": "For real-valued random variables \\(X\\) and \\(Y\\), with means \\(\\mu_X\\) and \\(\\mu_Y\\), covariance is defined by\n\n\\[\\operatorname{Cov}(X,Y) =\\mathbb E[(X-\\mu_X)(Y-\\mu_Y)].\\]\n\nHere, \\(\\mathbb E\\) denotes expected value. Expanding the product gives the equivalent identity\n\n\\[\\operatorname{Cov}(X,Y) =\\mathbb E[XY]-\\mathbb E[X]\\mathbb E[Y].\\]\n\nThe expectation is taken over the variables’ joint probability distribution, rather than their separate marginal distributions alone. For discrete variables it is a probability-weighted sum; for continuous variables with a joint density, it is a double integral. (online.stat.psu.edu)\n\nFinite second moments, \\(\\mathbb E[X^2]<\\infty\\) and \\(\\mathbb E[Y^2]<\\infty\\), guarantee that covariance is finite. Each centered product is positive when both variables are above their means or both are below them, and negative when one is above its mean and the other below. Covariance averages these contributions, including their magnitudes. Zero covariance therefore means that the positive and negative contributions balance, not that the variables have no relationship. (ocw.mit.edu)\n\nCovariance has units equal to the product of the variables’ units. Changing a measurement from meters to centimeters multiplies its covariance with an unchanged second variable by 100. Consequently, an unstandardized covariance cannot by itself provide a scale-independent assessment of association. (ocw.mit.edu)"
    },
    {
      "heading": "Algebraic properties",
      "text": "Covariance is symmetric and reduces to variance when both arguments are the same:\n\n\\[\\operatorname{Cov}(X,Y)=\\operatorname{Cov}(Y,X), \\qquad \\operatorname{Cov}(X,X)=\\operatorname{Var}(X).\\]\n\nFor constants \\(a,b,c,d\\),\n\n\\[\\operatorname{Cov}(aX+b,cY+d) =ac\\,\\operatorname{Cov}(X,Y).\\]\n\nThus, adding constants does not change covariance, while multiplication rescales it and can reverse its sign. Covariance is also additive in each argument:\n\n\\[\\operatorname{Cov}(X+Y,Z) =\\operatorname{Cov}(X,Z)+\\operatorname{Cov}(Y,Z).\\]\n\nThese properties explain its usefulness for studying linear combinations of random quantities. In particular,\n\n\\[\\operatorname{Var}(X+Y) =\\operatorname{Var}(X)+\\operatorname{Var}(Y) +2\\operatorname{Cov}(X,Y).\\]\n\nThe covariance term accounts for the contribution of joint variation to the variance of a sum. (ocw.mit.edu)"
    },
    {
      "heading": "Covariance, correlation, and independence",
      "text": "When both standard deviations are finite and positive, Pearson’s population correlation is\n\n\\[\\rho_{XY} =\\frac{\\operatorname{Cov}(X,Y)}{\\sigma_X\\sigma_Y}.\\]\n\nThis normalization removes measurement units. The Cauchy–Schwarz inequality gives\n\n\\[|\\operatorname{Cov}(X,Y)|\\leq\\sigma_X\\sigma_Y,\\]\n\nso correlation lies between \\(-1\\) and \\(1\\). Equality corresponds to an exact affine relationship almost surely, provided both variances are positive. If either variable has zero variance, covariance is zero but this correlation formula is undefined. (online.stat.psu.edu)\n\nStatistical independence implies zero covariance when the required expectations exist. The converse generally fails. As an illustrative calculation, let \\(X\\) take the values \\(-1,0,1\\), each with probability \\(1/3\\), and let \\(Y=X^2\\). Then\n\n\\[\\mathbb E[X]=0,\\qquad \\mathbb E[XY]=\\mathbb E[X^3]=0,\\]\n\ngiving zero covariance even though \\(Y\\) is determined by \\(X\\). This is a nonlinear dependence that covariance does not detect. The example illustrates the general distinction between independence and uncorrelatedness. (ocw.mit.edu)\n\nAn important exception occurs for jointly Gaussian variables: within a multivariate normal distribution, zero covariance between two components implies their independence. Merely having individually normal marginal distributions is insufficient for this conclusion. (probabilitycourse.com)"
    },
    {
      "heading": "Estimation from observations",
      "text": "For \\(n>1\\) paired observations \\((x_i,y_i)\\), the usual sample covariance is\n\n\\[s_{XY} =\\frac{1}{n-1} \\sum_{i=1}^{n}(x_i-\\bar x)(y_i-\\bar y),\\]\n\nwhere \\(\\bar x\\) and \\(\\bar y\\) are sample means. Under independent, identically distributed sampling of the pairs, with finite second moments, this estimator has zero bias for population covariance. The denominator \\(n-1\\) applies the same Bessel’s correction used in sample variance. (online.stat.psu.edu)\n\nDividing the centered sum by \\(n\\) instead gives the covariance of the empirical distribution assigning equal probability to every observed pair. This convention is useful descriptively, but under the sampling assumptions above its expectation is \\((n-1)/n\\) times the population covariance. The denominator must therefore be specified when reporting or comparing calculations. (stats.libretexts.org)"
    },
    {
      "heading": "Covariance matrices and applications",
      "text": "For a random vector \\(\\mathbf X=(X_1,\\ldots,X_p)^{\\mathsf T}\\), the covariance matrix collects all pairwise covariances:\n\n\\[\\Sigma_{ij}=\\operatorname{Cov}(X_i,X_j), \\qquad \\Sigma=\\mathbb E[ (\\mathbf X-\\boldsymbol\\mu) (\\mathbf X-\\boldsymbol\\mu)^{\\mathsf T}].\\]\n\nIts diagonal entries are variances, and its off-diagonal entries are covariances. This matrix is symmetric and positive semidefinite because, for every real vector \\(\\mathbf a\\),\n\n\\[\\mathbf a^{\\mathsf T}\\Sigma\\mathbf a =\\operatorname{Var}(\\mathbf a^{\\mathsf T}\\mathbf X)\\geq0.\\]\n\nFor a fixed matrix \\(A\\), a transformed vector has covariance \\(A\\Sigma A^{\\mathsf T}\\). (web.mit.edu)\n\nIn simple linear regression with an intercept, the ordinary least squares slope is \\(s_{XY}/s_X^2\\), provided the predictor’s sample variance is positive. Covariance therefore determines the slope’s sign and contributes directly to its magnitude. (online.stat.psu.edu)\n\nIn principal component analysis, the covariance matrix’s eigenvectors identify mutually orthogonal directions of variation, while the corresponding eigenvalues give projected variances. This supports dimensionality reduction in machine learning. Because covariance depends on scale, PCA based on raw covariance can differ substantially from PCA based on standardized variables or a correlation matrix. (online.stat.psu.edu)"
    }
  ],
  "links": [
    "probability",
    "statistics",
    "random-variable",
    "correlation",
    "expected-value",
    "joint-probability-distribution",
    "integral",
    "variance",
    "linear-combination",
    "standard-deviation",
    "statistical-independence",
    "multivariate-normal-distribution",
    "estimator-bias",
    "bessels-correction",
    "covariance-matrix",
    "matrix",
    "linear-regression",
    "ordinary-least-squares",
    "principal-component-analysis",
    "eigenvalues-and-eigenvectors",
    "dimensionality-reduction",
    "machine-learning"
  ],
  "sources": [
    {
      "title": "online.stat.psu.edu/stat505/Lesson01",
      "url": "https://online.stat.psu.edu/stat505/Lesson01"
    },
    {
      "title": "online.stat.psu.edu/stat414/Lesson18",
      "url": "https://online.stat.psu.edu/stat414/Lesson18"
    },
    {
      "title": "ocw.mit.edu/courses/18-05-introduction-to-probability-and-statistics-spring-202…",
      "url": "https://ocw.mit.edu/courses/18-05-introduction-to-probability-and-statistics-spring-2022/mit18_05_s22_lec07.pdf"
    },
    {
      "title": "probabilitycourse.com/chapter6/6_1_5_random_vectors.php",
      "url": "https://www.probabilitycourse.com/chapter6/6_1_5_random_vectors.php"
    },
    {
      "title": "stats.libretexts.org/Bookshelves/Probability_Theory/Probability_Mathematical_St…",
      "url": "https://stats.libretexts.org/Bookshelves/Probability_Theory/Probability_Mathematical_Statistics_and_Stochastic_Processes_%28Siegrist%29/06%3A_Random_Samples/6.07%3A_Sample_Correlation_and_Regression"
    },
    {
      "title": "web.mit.edu/18.06/www/Spring21/Lecture%20notes.pdf",
      "url": "https://web.mit.edu/18.06/www/Spring21/Lecture%20notes.pdf"
    },
    {
      "title": "online.stat.psu.edu/stat501/Lesson01",
      "url": "https://online.stat.psu.edu/stat501/Lesson01"
    },
    {
      "title": "online.stat.psu.edu/stat505/Lesson11",
      "url": "https://online.stat.psu.edu/stat505/Lesson11"
    }
  ],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}