Skip to content

Sample variance and the estimated mean

Textbook

For observations $x_1,\ldots,x_n$, the sample mean is $\bar x=\frac1n\sum_i x_i$. The quantity $v_n=\frac1n\sum_i(x_i-\bar x)^2$ is the variance of the empirical distribution assigning each observation weight $1/n$. It describes the spread of that data set.

For independent, identically distributed observations with finite second moment and $n>1$, $s^2=\frac1{n-1}\sum_i(x_i-\bar x)^2$ is an unbiased estimator of the source variance: $\mathbb E[s^2]=\operatorname{Var}(X)$. Unbiasedness concerns repeated sampling, not the accuracy of each individual answer.

The correction accounts for estimating the center from the same data: $\sum_i(x_i-\bar x)=0$, so the last deviation is determined by the others. Under these assumptions, $\mathbb E[\sum_i(X_i-\bar X)^2]=(n-1)\operatorname{Var}(X)$, explaining the denominator.

For 2, 4, and 6, the squared deviations sum to 8. The empirical variance is $8/3$ and the corrected estimate is 4. With $n=1$, the former is zero and the latter undefined. Library terminology varies; the result depends on the formula and degrees-of-freedom parameter.

Changing the denominator leaves the center and sum of squares untouched; it changes the role of the result. For $n>1$, $s^2=\frac{n}{n-1}v_n$. The corrected estimate is therefore at least as large as the empirical variance, but need not be closer to the unknown source variance in a particular sample. If all observations coincide, both quantities are zero. That single data set does not establish that the source always produces the same value.

Definitions and notation: Mathematics for Machine Learning.

Related concepts: Variance: the mean squared deviation, Arithmetic mean of observations.

In plain language

Three measured boards support two different questions: their own spread and the spread of boards in the warehouse. Describing the observed three gives each squared deviation weight one third. When estimating the source variance, the mean has already adapted to the same observations, and the correction accounts for that adaptation under the stated probability assumptions. “Minus one” in the denominator serves a particular estimation task rather than entering a formula beauty contest. The correction neither guarantees accuracy for one triple nor replaces missing information about the warehouse.

Example

The constructed sample 2, 4, 6 has mean 4. One sum of squares gives two answers for two different tasks.

Sample variance and the estimated mean

Deviations

The sample mean, 4, is subtracted from each observation.

Step 1 of 4: Deviations

Read all steps
  1. Deviations

    The sample mean, 4, is subtracted from each observation.

    Deviations
    −2current02current
  2. Sum of squares

    Squares 4, 0, and 4 sum to 8.

    Squares
    404
    Total
    8included
  3. Description of the data

    Division by the count 3 gives the empirical variance.

    Numerator; denominator; result
    83current8/3included
  4. Source variance estimate

    Under independent, identically distributed sampling, the denominator is 2.

    Numerator; denominator; result
    82current4included

The shared numerator is: $(-2)^2+0^2+2^2=8$. Dividing by three describes these three numbers’ mean squared deviation; dividing by two serves the stated estimation task under its assumptions. A common error is to average signed deviations first and then square the resulting zero. Order matters: every deviation is squared separately, preventing positive and negative deviations from canceling before spread can be measured.

The goal determines the calculation: describing the observed data or estimating the source variance.

The original text of this free article is published under CC BY-NC-ND 4.0.

Previous / next

Previous
Variance: the mean squared deviation
Next
Covariance: deviations together
  • Natural Numbers: Counting Discrete Objects

    Which values count discrete objects, and why the zero convention must be stated.

    Read article
    Math
  • Integers: Counting on Both Sides of Zero

    Positive and negative integers, zero, and the limits of integer division.

    Read article
    Math
  • Rational Numbers: Ratios of Integers

    Numbers as ratios of integers: equivalent fractions and the nonzero denominator requirement.

    Read article
    Math