Random Variables and Expected Value

A random variable is one of the most powerful ideas in all of mathematics: it lets us take the messy, verbal world of chance ("the die might land on a 6", "the customer might not show up") and turn it into a number we can add, average, and reason about. Once uncertainty wears the clothing of numbers, the full machinery of algebra and calculus becomes available to it. Expected value — the long-run average of a random variable — is the single number that most often answers the practical question hiding inside any gamble, insurance policy, or business decision: on average, what should I expect?

In this guide we build random variables from scratch, distinguish the discrete from the continuous case, define probability distributions carefully, and then develop the two summary numbers that dominate applied probability: the expected value (center) and the variance (spread). We finish with the law of large numbers, the theorem that finally justifies why "expected value" deserves the word "expected".

Learning Objectives

• Define a random variable and distinguish discrete from continuous types.
• Read and use probability mass functions (PMFs) and probability density functions (PDFs).
• Compute the expected value of a discrete and a continuous random variable.
• Compute variance and standard deviation, and interpret them as spread.
• Apply linearity of expectation to combine random variables quickly.
• State and interpret the law of large numbers and connect it to real averages.

Quick Answer

A random variable $X$ is a rule that assigns a number to each outcome of a random experiment. If it takes isolated values (0, 1, 2, ...) it is discrete and is described by a PMF $p(x) = P(X = x)$; if it takes values over a continuum it is continuous and is described by a PDF $f(x)$, where probabilities are areas under the curve. The expected value $E[X]$ is the probability-weighted average of the values: $\sum x\,p(x)$ in the discrete case and $\int x\,f(x)\,dx$ in the continuous case. The variance $\operatorname{Var}(X) = E[(X-\mu)^2]$ measures how far outcomes typically fall from the mean, and its square root is the standard deviation. The law of large numbers guarantees that the average of many independent repetitions converges to $E[X]$.

Where It Came From

Expected value was not born in a university — it was born at a gambling table. In 1654 a French writer and gambler, the Chevalier de Méré, posed a puzzle that had frustrated players for over a century: the problem of points. Two players stake equal money on a game of several rounds; whoever first wins a set number of rounds takes the whole pot. But suppose the game is interrupted before anyone wins — say the score is 2–1 in a race to 3. How should the pot be divided fairly?

The naive answers (split evenly, or split in ratio 2:1 by rounds won) both felt wrong, and no one could say why. De Méré took the question to Blaise Pascal, who in turn began a now-famous 1654 correspondence with Pierre de Fermat. Their key insight was revolutionary: fairness should depend not on the rounds already played but on the probabilities of each player eventually winning if the game had continued. Each player should receive the pot in proportion to his expected share — the sum of each possible prize weighted by the chance of receiving it. This is precisely the modern definition of expected value.

A few years later, in 1657, the Dutch scientist Christiaan Huygens read of the correspondence and wrote the first published treatise on probability, De ratiociniis in ludo aleae ("On Reasoning in Games of Chance"), in which expected value is the central organizing concept. The need that forced the idea into existence was therefore intensely practical: how to price uncertainty fairly — a need that today drives insurance, finance, and every decision made under risk.

Random Variables: Turning Outcomes into Numbers

Formally, a random variable is a function from the sample space (the set of all possible outcomes) to the real numbers. Roll two dice: the outcome is a pair like (3, 5), but we usually care about a number derived from it — say $X =$ the sum. Then $X$ can be $2, 3, \ldots, 12$, and each value has a probability.

Discrete random variables take values you can list, even if the list is infinite (0, 1, 2, ...). Examples: the number of heads in 10 coin flips, the number of emails you receive today, the sum of two dice.

Continuous random variables take any value in an interval — you cannot list them. Examples: a person's exact height, the time until a lightbulb fails, the temperature at noon. For these, the probability of any single exact value is zero (there are infinitely many); we can only speak of the probability of landing in a range.

Worked example: the distribution of a dice sum

Roll two fair dice and let $X$ be the sum. There are $6 \times 6 = 36$ equally likely outcomes. Counting the ways to make each sum:

$x$	2	3	4	5	6	7	8	9	10	11	12
$P(X=x)$	$\tfrac{1}{36}$	$\tfrac{2}{36}$	$\tfrac{3}{36}$	$\tfrac{4}{36}$	$\tfrac{5}{36}$	$\tfrac{6}{36}$	$\tfrac{5}{36}$	$\tfrac{4}{36}$	$\tfrac{3}{36}$	$\tfrac{2}{36}$	$\tfrac{1}{36}$

Adding these gives $\frac{36}{36} = 1$, as every valid PMF must. The peak at 7 is why 7 is the most common roll in games like craps.

Probability Distributions: PMFs and PDFs

A probability mass function (PMF) $p(x) = P(X = x)$ describes a discrete variable. It must satisfy two rules: every value is non-negative, $p(x) \ge 0$, and they sum to one, $\sum_x p(x) = 1$.

A probability density function (PDF) $f(x)$ describes a continuous variable. Here $f(x)$ is not a probability — it is a density, and probability is the area under the curve:

$P(a \le X \le b) = \int_a^b f(x)\,dx$

A valid PDF satisfies $f(x) \ge 0$ everywhere and $\int_{-\infty}^{\infty} f(x)\,dx = 1$ (total area one). Because area at a single point is zero, $P(X = a) = 0$ for any continuous variable — a crucial and often surprising fact.

Worked example: a continuous density

Suppose the waiting time $X$ (in minutes) at a counter has density $f(x) = \tfrac{1}{5}e^{-x/5}$ for $x \ge 0$ (an exponential distribution with mean 5). What is the probability you wait between 2 and 6 minutes?

$P(2 \le X \le 6) = \int_2^6 \tfrac{1}{5}e^{-x/5}\,dx = \left[-e^{-x/5}\right]_2^6 = e^{-2/5} - e^{-6/5}$

Numerically, $e^{-0.4} \approx 0.6703$ and $e^{-1.2} \approx 0.3012$, so the probability is about $0.6703 - 0.3012 = 0.369$, roughly a 37% chance.

Expected Value: The Center of Gravity

The expected value is the probability-weighted average of all possible values. It is the "balance point" of the distribution.

Discrete: $E[X] = \sum_x x\,p(x)$ Continuous: $E[X] = \int_{-\infty}^{\infty} x\,f(x)\,dx$

Worked example: expected value of one die

A fair six-sided die:

$E[X] = 1\cdot\tfrac{1}{6} + 2\cdot\tfrac{1}{6} + 3\cdot\tfrac{1}{6} + 4\cdot\tfrac{1}{6} + 5\cdot\tfrac{1}{6} + 6\cdot\tfrac{1}{6} = \frac{21}{6} = 3.5$

Notice $3.5$ is not a value the die can show — the expected value is a long-run average, not a prediction of any single roll.

Worked example: a fair game and a casino edge

You pay $1 to play a game: you win $3 (net +$2) with probability $\tfrac{1}{4}$ and lose your dollar (net −$1) with probability $\tfrac{3}{4}$. The expected net gain is

$E[\text{gain}] = (+2)\cdot\tfrac{1}{4} + (-1)\cdot\tfrac{3}{4} = \tfrac{2}{4} - \tfrac{3}{4} = -\tfrac{1}{4} = -0.25$

On average you lose 25 cents per play. This negative expectation, called the house edge, is exactly how casinos profit — every game is engineered so $E[\text{gain}] < 0$ for the player.

Linearity of expectation

One of the most useful facts in all of probability: for any random variables and constants,

$E[aX + bY + c] = a\,E[X] + b\,E[Y] + c$

This holds even when $X$ and $Y$ are dependent. Example: the expected sum of two dice is simply $3.5 + 3.5 = 7$ — no need to use the 11-row table above.

Variance: How Spread Out Is It?

Two distributions can share the same mean but feel completely different — one tightly clustered, one wildly spread. Variance captures that spread by averaging the squared distance from the mean $\mu = E[X]$:

$\operatorname{Var}(X) = E[(X-\mu)^2] = E[X^2] - \big(E[X]\big)^2$

The second form (the "computational formula") is usually easier. The standard deviation $\sigma = \sqrt{\operatorname{Var}(X)}$ returns to the original units, which is why it is the more interpretable measure.

Worked example: variance of one die

We know $\mu = 3.5$. First compute $E[X^2]$:

$E[X^2] = \frac{1^2 + 2^2 + 3^2 + 4^2 + 5^2 + 6^2}{6} = \frac{91}{6} \approx 15.167$

Then

$\operatorname{Var}(X) = 15.167 - (3.5)^2 = 15.167 - 12.25 = 2.917$

so $\sigma = \sqrt{2.917} \approx 1.71$. A typical roll lands about 1.7 away from the mean of 3.5 — consistent with values spread across 1 to 6.

The Law of Large Numbers

Why should a single number computed from probabilities describe reality? The law of large numbers (LLN) is the answer. It states that if you independently repeat an experiment many times and average the results, that sample average converges to the true expected value:

$\bar{X}_n = \frac{X_1 + X_2 + \cdots + X_n}{n} \longrightarrow E[X] \quad \text{as } n \to \infty$

Roll a die 10 times and your average might be 4.1; roll it 10,000 times and it will hover extremely close to 3.5. The LLN is what makes casinos, insurers, and pollsters confident: over enough trials, the average is essentially guaranteed.

Warning — the gambler's fallacy: the LLN does not say that a run of low rolls must be "balanced" by high rolls. Each roll is independent; the average converges because early deviations are diluted by the growing number of trials, not corrected by future outcomes.

Real-World Applications

• Insurance pricing: An insurer sets a premium above the expected payout $E[\text{claim}]$. The LLN guarantees that across millions of policies, total claims stay near the expected total, so the small per-policy margin becomes reliable profit.
• Finance and expected return: The expected value of a portfolio's return drives investment decisions, while variance (and its square root, volatility) measures risk. Modern portfolio theory is literally the study of trading off $E[X]$ against $\operatorname{Var}(X)$.
• A/B testing and analytics: Conversion rate is the expected value of a 0/1 (Bernoulli) variable; sample size requirements come directly from variance and the LLN.
• Quality control: Expected number of defects per batch and its variance determine inspection and tolerance limits in manufacturing.
• Machine learning: Loss functions are expected values over the data distribution; training minimizes an estimate of that expectation.

Common Mistakes

1. Thinking the expected value is the most likely outcome. A die's expected value is 3.5, which can never occur. Correction: expected value is a long-run average, not a mode or a prediction of a single trial.

2. Treating a PDF value $f(x)$ as a probability. For continuous variables, $f(x)$ can even exceed 1 (a narrow tall density). Correction: only areas $\int_a^b f(x)\,dx$ are probabilities; the probability of any exact single value is 0.

3. Believing in the gambler's fallacy. After five reds at roulette, black is not "due". Correction: independent trials have no memory; the LLN works by dilution over many trials, not by compensating for past results.

4. Forgetting variance is in squared units. Reporting a variance of "2.917 points" is misleading. Correction: take the square root to get the standard deviation ($\approx 1.71$) for an interpretable spread in the original units.

Comparison and Connections

Concept	Discrete case	Continuous case
Distribution	PMF $p(x) = P(X=x)$	PDF $f(x)$, area = probability
Normalization	$\sum p(x) = 1$	$\int f(x)\,dx = 1$
Expected value	$\sum x\,p(x)$	$\int x\,f(x)\,dx$
$P(X = a)$	Can be positive	Always 0

Expected value vs. variance: $E[X]$ tells you where the distribution sits; $\operatorname{Var}(X)$ tells you how tightly it clusters there. Both are needed — a bet with high expected value but enormous variance may still be unwise.

Expected value vs. median: the median splits probability in half; the expected value is the balance point. For skewed distributions (e.g. income) they differ sharply — a few billionaires pull the mean far above the median.

Practice Questions

Recall

State the two conditions a valid PMF must satisfy. Answer: Every value satisfies $p(x) \ge 0$, and all values sum to one, $\sum_x p(x) = 1$.

Understanding

For a continuous random variable, why is $P(X = 3) = 0$ even though 3 is a possible value? Answer: Probability is the area under the density over an interval; a single point has width zero, so its area — and thus its probability — is zero. Probabilities exist only over ranges.

Application

A lottery ticket costs $2. You win $500 with probability $\tfrac{1}{1000}$ and nothing otherwise. What is the expected net gain per ticket? Answer: $E = (500-2)\cdot\tfrac{1}{1000} + (-2)\cdot\tfrac{999}{1000} = 0.498 - 1.998 = -1.50$. On average you lose $1.50 per ticket.

Analysis

Two games have the same expected gain of $0. Game A pays ±$1 on a coin flip; Game B pays +$1000 or −$1000 on a coin flip. How do they differ, and which would a risk-averse person prefer? Answer: They share $E[X] = 0$ but differ enormously in variance: Game A has variance 1, Game B has variance 1,000,000. A risk-averse person prefers Game A because the outcomes are far more predictable, even though the average payoff is identical.

FAQ

Is expected value the same as the average? They are closely related. The expected value is a theoretical average weighted by probabilities; the sample average (mean) is what you compute from actual data. The law of large numbers says the sample average converges to the expected value.

Can the expected value be negative or a non-achievable number? Yes to both. A losing bet has negative expected value, and a die's expected value (3.5) is a number the die can never show. It is a summary, not a prediction.

What's the difference between a PMF and a PDF? A PMF gives actual probabilities at discrete points and its values sum to 1. A PDF gives a density for continuous variables; you integrate it over a range to get a probability, and its total area is 1.

Why do we square the deviations in variance instead of just averaging them? Plain deviations from the mean always sum to zero (positives cancel negatives). Squaring makes them all positive and penalizes large deviations more heavily. Taking the square root at the end (standard deviation) restores the original units.

Does the law of large numbers mean I'll eventually break even in a losing game? No — quite the opposite. If each play has negative expected value, the LLN guarantees your average loss per play converges to that negative number, so your total losses grow steadily. The LLN protects the house, not the player.

Quick Revision

• Random variable: assigns a number to each outcome; discrete (listable) or continuous (interval).
• PMF: $p(x) = P(X=x)$, $\sum p(x) = 1$. PDF: $f(x) \ge 0$, $\int f(x)\,dx = 1$, probability = area.
• Expected value: $E[X] = \sum x\,p(x)$ or $\int x\,f(x)\,dx$ — the balance point.
• Linearity: $E[aX + bY + c] = a\,E[X] + b\,E[Y] + c$, always.
• Variance: $\operatorname{Var}(X) = E[X^2] - (E[X])^2$; standard deviation $\sigma = \sqrt{\operatorname{Var}(X)}$.
• Fair die: $E[X] = 3.5$, $\operatorname{Var}(X) \approx 2.917$, $\sigma \approx 1.71$.
• Law of large numbers: sample average $\to E[X]$ as $n \to \infty$. Beware the gambler's fallacy.

Related Topics

Prerequisites

• Probability Basics — sample spaces, events, and how probabilities are assigned.

Related Topics

• Probability Distributions (binomial, normal, Poisson) — named families of random variables.
• Descriptive Statistics — mean, variance, and standard deviation computed from data.

Next Topics

• The Normal Distribution and the Central Limit Theorem — why averages become bell-shaped.
• Statistics and Probability overview: ../index.md