Correlation is a statistical association between variables: their values exhibit a pattern of variation together. In statistics, the term often refers specifically to the strength and direction of a linear relationship, measured by Pearson’s correlation coefficient. Other correlation measures describe associations between rankings or monotonic patterns. Correlation characterizes observed or probabilistic relationships; by itself, it does not establish causation. (online.stat.psu.edu)
Pearson correlation
For two random variables (X) and (Y) with finite, strictly positive variances, the population Pearson correlation coefficient is
[ \rho_{X,Y} =\frac{\operatorname{Cov}(X,Y)}{\sigma_X\sigma_Y} =\frac{\mathbb E[(X-\mu_X)(Y-\mu_Y)]}{\sigma_X\sigma_Y}. ]
Here, (\mu_X) and (\mu_Y) are expected values, (\operatorname{Cov}(X,Y)) is their covariance, and (\sigma_X,\sigma_Y) are their standard deviations. Normalizing covariance removes dependence on measurement units. If either variable has zero variance, Pearson correlation is undefined. (online.stat.psu.edu)
For (n) paired observations, the sample coefficient is
[ r= \frac{\sum_{i=1}^{n}(x_i-\bar x)(y_i-\bar y)} {\sqrt{\sum_{i=1}^{n}(x_i-\bar x)^2 \sum_{i=1}^{n}(y_i-\bar y)^2}}. ]
The observations must be paired: each (x_i) corresponds to the same case as (y_i). The sample coefficient estimates a population relationship but also serves as a descriptive statistic for the observed dataset. Computing it does not require normally distributed data, although conventional tests and intervals impose additional assumptions. (docs.scipy.org)
Interpretation and mathematical properties
Pearson correlation lies between (-1) and (+1). Positive values indicate that larger values of one variable tend to accompany larger values of the other; negative values indicate the opposite direction. The absolute value measures the strength of linear association. Values of (+1) and (-1) represent exact increasing and decreasing linear relationships, respectively. A value of zero indicates absence of linear correlation, not absence of every possible relationship. (online.stat.psu.edu)
Correlation is symmetric: exchanging (X) and (Y) leaves its value unchanged. Adding a constant or multiplying either variable by a positive constant also leaves Pearson correlation unchanged; multiplying just one variable by a negative constant reverses its sign. Consequently, changing measurement units through positive linear conversions does not alter it. Unlike a regression slope, correlation has no units and does not designate an explanatory or response variable. (web.stanford.edu)
Statistical independence implies zero correlation whenever the coefficient exists, but the converse generally fails. As a mathematical example, let (X) be uniformly distributed on ([-1,1]) and (Y=X^2). Symmetry gives zero covariance, although (Y) is completely determined by (X). For variables with a joint multivariate normal distribution, however, zero covariance does imply independence. (online.stat.psu.edu)
Rank correlation and partial correlation
Spearman’s rank correlation measures monotonic association: whether one variable generally increases or decreases as the other increases, without requiring a straight-line relationship. Its sample form is Pearson correlation applied to the observations’ ranks, with tied observations usually assigned average ranks. Ranking therefore changes the question from association between numerical magnitudes to association between relative positions. (docs.scipy.org)
Kendall’s rank correlation, commonly denoted (\tau), compares concordant and discordant pairs. A pair is concordant when both variables order its observations in the same direction, and discordant when their orders disagree. Without ties,
[ \tau=\frac{C-D}{\binom n2}, ]
where (C) and (D) count concordant and discordant pairs. Variants such as tau-b adjust for ties. Spearman and Kendall coefficients describe related aspects of rank association but are not numerically interchangeable. (docs.scipy.org)
Partial correlation measures the linear association remaining after specified additional variables have been accounted for. In its usual sample form, it is the correlation between residuals from separate linear regressions of the two variables on the control variables. Statistical adjustment does not, by itself, establish a causal interpretation. (online.stat.psu.edu)
Statistical inference
Statistical hypothesis testing can assess whether an observed sample correlation is compatible with a population correlation of zero. For independent observation pairs drawn from a bivariate normal population, under (H_0:\rho=0),
[ t=r\sqrt{\frac{n-2}{1-r^2}} ]
follows Student’s t-distribution with (n-2) degrees of freedom. The resulting p-value describes extremeness under the null model, rather than the magnitude or practical importance of the association. (online.stat.psu.edu)
A confidence interval expresses uncertainty about the population coefficient. One common approximate method uses the Fisher transformation,
[ z=\frac12\log\left(\frac{1+r}{1-r}\right). ]
Under the usual normal-population assumptions, its approximate standard error is (1/\sqrt{n-3}). Interval endpoints on the transformed scale are converted back to the correlation scale. Thus, identical observed coefficients can have substantially different precision at different sample sizes. (online.stat.psu.edu)
Limitations and relationship to regression
A scatter plot reveals features that a single coefficient can conceal, including curvature, clusters, and unusual observations. Pearson correlation is sensitive to outliers, which can strengthen, weaken, or reverse an apparent association. A strong nonlinear relationship may nevertheless have a small Pearson coefficient. (online.stat.psu.edu)
Confounding can produce associations through other variables, and controlling relevant variables can change an association’s magnitude or direction. Causal inference therefore requires evidence and assumptions beyond correlation alone, including consideration of alternative explanations and study design. Statistical significance does not remove these interpretive limitations. (online.stat.psu.edu)
In simple regression fitted by ordinary least squares with an intercept, the coefficient of determination satisfies (R^2=r^2). It represents the proportion of observed response variation accounted for by the fitted line. Squaring removes direction: equally strong positive and negative correlations yield the same (R^2). This identity belongs to that specific regression setting, not to arbitrary predictive models. (online.stat.psu.edu)