Expected value: the mean of a distribution

Textbook

For a finite-valued random variable $X$, expected value is $\mathbb E[X]=\sum_x xP(X=x)$. The sum runs over distinct possible values, each multiplied by its probability. Other notation includes $E[X]$, $E(X)$, and $M(X)$.

For equally likely values 2, 4, and 6, the expectation is 4. For a fair coin taking values 0 and 1, it is $1/2$, even though a toss never produces that value. Expectation describes a distribution; it does not promise the next observation.

For a function of the variable, $\mathbb E_{X\sim D}[g(X)]$ states which distribution supplies the averaging weights. In particular, $\mathbb E[\mathbf1\{A\}]=P(A)$. This connects a classifier’s error probability with the expected indicator of error.

For an infinite set of values, a finite expectation requires absolute integrability, $\mathbb E[|X|]<\infty$. With density $p_X$, the expectation is $\mathbb E[X]=\int_{-\infty}^{+\infty}x\,p_X(x)\,dx$ under that condition. A density at a point is not the probability of that value. A particular sample’s mean is a different object and can differ from expectation.

In a finite model, nonnegative weights sum to one, so expectation lies between the smallest and largest possible values. Increasing every value by $c$ increases expectation by $c$, because the extra contributions total $c\sum_xP(X=x)=c$. Equal probabilities are a special case in which the weighted sum becomes the ordinary arithmetic mean of the list of possible values; unequal probabilities require their actual weights.

Definitions and notation: Mathematics for Computer Science; Mathematics for Machine Learning.

Related concept: classification error.

Related concepts: Random variables: a number for each outcome, Arithmetic mean of observations.