Exam PMultivariate probabilityFree to read

Covariance, correlation and sums

Covariance measures joint movement, correlation rescales it to [−1, 1], and the variance of a sum is where both actually get used.

The formulas

Covariance

$$\operatorname{Cov}(X,Y) = E[XY] - E[X]E[Y]$$

Correlation

$$\rho = \dfrac{\operatorname{Cov}(X,Y)}{\sigma_X\sigma_Y}, \quad -1 \le \rho \le 1$$

Variance of a sum

$$\operatorname{Var}(aX+bY) = a^{2}\sigma_X^{2} + b^{2}\sigma_Y^{2} + 2ab\operatorname{Cov}(X,Y)$$

Bilinearity

$$\operatorname{Cov}(aX+b,\,cY+d) = ac\operatorname{Cov}(X,Y)$$

Independence

$$X \perp Y \Rightarrow \operatorname{Cov}(X,Y)=0$$

the converse is false

Where it comes from

1. Expanding $E[(X-\mu_X)(Y-\mu_Y)]$ and using linearity gives $E[XY] - \mu_X\mu_Y$.
2. Dividing by $\sigma_X\sigma_Y$ makes the measure scale-free, and Cauchy-Schwarz forces $|\rho|\le 1$.
3. $\operatorname{Var}(X+Y) = \operatorname{Cov}(X+Y, X+Y)$, and bilinearity expands it into the three-term formula.

Worked example

This answer is recomputed from the site’s own interest-theory and probability functions every time the test suite runs, so the page and the mathematics cannot drift apart.

Memory hooks

• For a DIFFERENCE the cross term is −2ab·Cov. Subtracting flips the sign, not the variance terms.
• Zero correlation does not mean independent - the standard counterexample is Y = X² with X symmetric about 0.

Traps

• Dividing the covariance by the variances instead of the standard deviations.
• Dropping the covariance term when the variables are merely uncorrelated in the question's wording but not stated independent.

Related

• Joint, marginal and conditional distributions
• Expectation, variance and moments
• Sums, the central limit theorem and normal approximation

Drill this: the Exam P question bank has original questions on this topic, and today’s free round is open to everyone.