Probability Distributions

Roll a die and you get a number, but you don't know which one until you roll. Measure the height of a randomly chosen adult and you get a value, but not a predictable one. A probability distribution is the complete description of how likely each possible outcome is — the blueprint that turns "I don't know what I'll get" into "here is exactly how the uncertainty is shaped." It is the single most important object in all of statistics, because once you know the distribution, you can compute averages, spreads, risks, and confidence.

This page builds the idea from the ground up: first the notion of a random variable, then two distributions that run the world — the binomial (for counting successes) and the normal bell curve (for measurements and averages). Along the way you will learn the famous 68-95-99.7 rule, how z-scores let you compare apples to oranges, and the deep reason the bell curve keeps showing up no matter what you started with.

Learning Objectives

By the end of this page, you should be able to:

• Define a random variable and distinguish discrete from continuous ones.
• Compute probabilities, the mean, and the standard deviation of a binomial distribution.
• Describe the normal distribution and apply the 68-95-99.7 rule.
• Convert any value to a z-score and use it to find probabilities and compare data.
• Explain intuitively why the central limit theorem makes the normal distribution appear so often.

Quick Answer

A random variable assigns a number to each outcome of a random process; its distribution lists how much probability sits on each value. The binomial distribution counts successes in $n$ independent yes/no trials, each with success probability $p$; it has mean $np$ and standard deviation $\sqrt{np(1-p)}$. The normal distribution is the symmetric bell curve described entirely by its mean $\mu$ and standard deviation $\sigma$; roughly 68% of values fall within $1\sigma$ of the mean, 95% within $2\sigma$, and 99.7% within $3\sigma$. A z-score, $z = (x-\mu)/\sigma$, rescales any value into "number of standard deviations from the mean," making different distributions directly comparable. The normal curve appears everywhere because of the central limit theorem: sums and averages of many small independent influences pile up into a bell shape regardless of the original ingredients.

Where It Came From

The bell curve was born from a very practical headache: measurement error. In the 1700s and early 1800s, astronomers repeatedly measured the position of a star or planet and got slightly different numbers every time. Which measurement was right? The natural answer was to average them — but why should averaging work, and how much should you trust the result?

Abraham de Moivre took the first big step around 1733. Studying games of chance, he wanted to approximate binomial probabilities for large numbers of coin flips, which were brutal to compute by hand. He discovered that the jagged binomial histogram, when $n$ is large, is beautifully approximated by a smooth continuous curve — the shape we now call normal. He had essentially found the bell curve as a computational shortcut, though he did not fully grasp its universal significance.

The person whose name stuck was Carl Friedrich Gauss. Around 1809, working on predicting the orbit of the dwarf planet Ceres from noisy telescope data, Gauss asked: what distribution of errors makes the plain average the "most probable" true value? He showed the answer is precisely the bell curve, which is why it is often called the Gaussian distribution. Pierre-Simon Laplace then connected the dots with an early central limit theorem, explaining why errors — each a sum of many tiny independent disturbances — should be normally distributed. The need was real and urgent: navigation, astronomy, and surveying all depended on squeezing reliable answers out of imperfect measurements. The normal distribution was the tool that made "average your data and trust it" into rigorous mathematics.

Random Variables: Turning Chance into Numbers

A random variable is a rule that attaches a number to every outcome of a random experiment. Flip three coins; let $X$ be the number of heads. The outcomes are things like HTH, but $X$ turns each into a number: HTH becomes $2$. Random variables come in two flavors:

• Discrete: takes separate, countable values (0, 1, 2, 3 heads). We describe it with a probability mass function giving $P(X = k)$ for each value.
• Continuous: takes any value in a range (a height like 172.4183… cm). We describe it with a probability density, and probabilities correspond to areas under a curve.

Two summary numbers describe any distribution's center and spread. The expected value (mean) is the long-run average:

$\mu = E(X) = \sum_k k \cdot P(X = k)$

The variance measures spread, and the standard deviation $\sigma = \sqrt{\text{Var}(X)}$ puts it back in the original units.

Worked Example: Three Coin Flips

Let $X$ be the number of heads in three fair flips. Each outcome has probability $1/8$. Counting outcomes: $P(X=0)=1/8$, $P(X=1)=3/8$, $P(X=2)=3/8$, $P(X=3)=1/8$. Check: they sum to $8/8 = 1$. The mean is

$\mu = 0\cdot\tfrac18 + 1\cdot\tfrac38 + 2\cdot\tfrac38 + 3\cdot\tfrac18 = \frac{0+3+6+3}{8} = \frac{12}{8} = 1.5.$

So on average you get 1.5 heads — sensible, since heads and tails are symmetric.

The Binomial Distribution: Counting Successes

The three-coin example is a special case of the binomial distribution, which arises whenever you have:

1. A fixed number $n$ of trials,
2. Each trial independent of the others,
3. Two outcomes per trial ("success"/"failure"),
4. The same success probability $p$ every trial.

If $X$ counts successes, then

$P(X = k) = \binom{n}{k} p^k (1-p)^{n-k},$

where $\binom{n}{k} = \dfrac{n!}{k!\,(n-k)!}$ counts the number of ways to arrange $k$ successes among $n$ trials. The mean and standard deviation are wonderfully simple:

$\mu = np, \qquad \sigma = \sqrt{np(1-p)}.$

Worked Example: A Multiple-Choice Guess

A quiz has 10 questions, each with 4 choices. A student guesses randomly, so $p = 0.25$. What is the probability of getting exactly 3 correct?

$P(X=3) = \binom{10}{3}(0.25)^3(0.75)^7.$

Compute the pieces: $\binom{10}{3} = 120$; $(0.25)^3 = 0.015625$; $(0.75)^7 \approx 0.1334839$. Multiply:

$P(X=3) \approx 120 \times 0.015625 \times 0.1334839 \approx 0.2503.$

So about a 25% chance of exactly 3 correct. The expected score is $\mu = 10 \times 0.25 = 2.5$ correct, with spread $\sigma = \sqrt{10 \times 0.25 \times 0.75} = \sqrt{1.875} \approx 1.37$. Passing (say 6 or more) by luck is very unlikely, which is exactly why guessing rarely works.

The Normal Distribution and the 68-95-99.7 Rule

The normal distribution is the continuous bell curve, symmetric about its mean $\mu$, with spread controlled by $\sigma$. Its density is

$f(x) = \frac{1}{\sigma\sqrt{2\pi}}\, e^{-\frac{(x-\mu)^2}{2\sigma^2}},$

but you rarely use this formula directly. What you use constantly is the empirical rule for how probability is distributed around the mean:

Range around mean	Approx. probability inside
$\mu \pm 1\sigma$	68%
$\mu \pm 2\sigma$	95%
$\mu \pm 3\sigma$	99.7%

Because the curve is symmetric, the leftover probability splits equally into the two tails. Beyond $2\sigma$ lies only about 5%, split as 2.5% in each tail.

Worked Example: Test Scores

Suppose IQ-style scores are normal with $\mu = 100$ and $\sigma = 15$. What fraction of people score between 85 and 115? That range is exactly $\mu \pm 1\sigma$, so about 68%. What about above 130? That is $\mu + 2\sigma$; 95% lie within $\pm 2\sigma$, leaving 5% in the two tails combined, so 2.5% score above 130. This kind of instant estimate — no calculator needed — is the everyday power of the rule.

Z-Scores: A Universal Ruler

Different normal distributions have different means and spreads, which makes raw values hard to compare. The z-score fixes this by rescaling:

$z = \frac{x - \mu}{\sigma}.$

It answers "how many standard deviations above or below the mean is this value?" A z-score of $+2$ always means "2 standard deviations above average," whether we are talking about heights, test scores, or blood pressure. This process, called standardization, converts any normal distribution into the standard normal with mean 0 and standard deviation 1, letting you read one universal table (or one calculator function) for all problems.

Worked Example: Comparing Two Students

Anaya scored 82 on a test with $\mu = 70$, $\sigma = 8$. Ben scored 88 on a different test with $\mu = 80$, $\sigma = 12$. Who did better relative to their class?

$z_{\text{Anaya}} = \frac{82-70}{8} = \frac{12}{8} = 1.5, \qquad z_{\text{Ben}} = \frac{88-80}{12} = \frac{8}{12} \approx 0.67.$

Anaya is 1.5 standard deviations above her class; Ben only 0.67. Even though Ben's raw score is higher, Anaya outperformed her peers by more. From a standard normal table, $z = 1.5$ corresponds to roughly the 93rd percentile, while $z = 0.67$ is about the 75th percentile.

Why the Normal Distribution Appears So Often

Here is the deep idea: the central limit theorem (CLT) says that if you add up (or average) many independent random influences, none of them dominating, the result is approximately normal — regardless of the shape of the individual pieces.

Think about a person's height. It is the combined effect of thousands of genes, nutrition during childhood, sleep, and more. Each factor nudges height up or down a little. Add up thousands of tiny independent nudges and the bell curve emerges automatically. The same logic explains measurement error (many tiny disturbances), and why the binomial itself starts to look normal for large $n$ — a binomial count is a sum of many independent 0/1 trials, exactly the CLT recipe. De Moivre's discovery was the first sighting of this phenomenon.

Worked Example: Averaging Dice

A single die roll is not normal — it is flat, each face equally likely, mean 3.5. But roll five dice and average them. The average clusters strongly near 3.5, extreme averages (all 1s or all 6s) are rare, and the histogram of many such averages is already a rough bell shape. Roll thirty dice and it becomes strikingly normal. Nothing about a die is bell-shaped, yet averaging manufactures the bell curve. That is the CLT turning uniform ingredients into a normal result — and why the normal distribution is nature's default for anything built from many small parts.

Real-World Applications

• Quality control: Factories set tolerance limits at $\mu \pm 3\sigma$; parts beyond that signal a machine fault. "Six Sigma" management is literally named after standard deviations.
• Medicine: Growth charts, cholesterol ranges, and lab "reference intervals" are z-score bands; a result flagged "high" is usually beyond $2\sigma$.
• Finance: Portfolio risk models describe returns with means and standard deviations; "value at risk" is a tail probability of a (roughly) normal curve.
• Elections and polling: The margin of error in a poll comes from the binomial/normal spread of a sample proportion.
• Standardized testing: SAT, GRE, and IQ scores are deliberately scaled to normal distributions so percentiles are meaningful.
• Manufacturing defects and genetics: Counting defective items or inherited traits in fixed samples is textbook binomial.

Common Mistakes

1. Confusing probability with certainty in the empirical rule. Misconception: "99.7% within $3\sigma$ means nothing is ever beyond $3\sigma$." Wrong — about 0.3% of values do fall outside, and in large populations that is many people. Correction: the rule gives probabilities, not hard limits; rare tail events happen.

2. Applying the binomial when trials aren't independent or $p$ changes. Misconception: "Any counting problem is binomial." If you draw cards without replacement, the probability shifts each draw, so it is not binomial (it is hypergeometric). Correction: check all four conditions — fixed $n$, independence, two outcomes, constant $p$.

3. Forgetting to standardize before using z-tables. Misconception: "I can look up the raw score directly." Z-tables are for the standard normal only. Correction: always compute $z = (x-\mu)/\sigma$ first, then look up $z$.

4. Thinking everything is normal. Misconception: "All data is bell-shaped." Incomes, city sizes, and waiting times are famously skewed. Correction: the CLT makes averages and sums normal, not necessarily the raw data itself.

Comparison and Connections

Feature	Binomial	Normal
Type	Discrete (counts)	Continuous (measurements)
Parameters	$n$, $p$	$\mu$, $\sigma$
Mean	$np$	$\mu$
Std. deviation	$\sqrt{np(1-p)}$	$\sigma$
Shape	Bars; symmetric when $p=0.5$	Smooth symmetric bell
Typical use	"How many successes?"	"How large a measurement?"

The two connect through the CLT: for large $n$, a binomial with mean $np$ and standard deviation $\sqrt{np(1-p)}$ is well approximated by a normal with those same parameters — exactly de Moivre's original insight. The normal is the continuous limit of the discrete binomial.

Practice Questions

Recall

State the mean and standard deviation of a binomial distribution with $n = 50$ and $p = 0.2$. Answer: $\mu = np = 50 \times 0.2 = 10$; $\sigma = \sqrt{50 \times 0.2 \times 0.8} = \sqrt{8} \approx 2.83$.

Understanding

Heights are normal with $\mu = 170$ cm and $\sigma = 7$ cm. What percentage of people are between 156 cm and 184 cm? Answer: That range is $170 \pm 14 = \mu \pm 2\sigma$, so about 95%.

Application

A basketball player makes 80% of free throws. In 6 attempts, find the probability she makes exactly 5. Answer: $P(X=5) = \binom{6}{5}(0.8)^5(0.2)^1 = 6 \times 0.32768 \times 0.2 = 0.3932$, about 39%.

Analysis

A factory's bolt lengths are normal with $\mu = 50$ mm, $\sigma = 0.5$ mm. Bolts outside $49$–$51$ mm are rejected. What fraction is rejected, and why might the true rate be higher than the rule suggests? Answer: $49$–$51$ mm is $\mu \pm 2\sigma$, so about 95% pass and 5% are rejected. The true rate could be higher if the process drifts (mean shifts) or the data has heavier tails than a perfect normal, so the empirical rule underestimates the tails.

FAQ

Q: What's the difference between a probability mass function and a density function? A mass function gives actual probabilities at discrete points ($P(X=2)$ makes sense). A density function applies to continuous variables, where the probability of any exact value is zero and you instead measure areas over intervals.

Q: Why can't a continuous variable have a probability at a single point? Because there are infinitely many possible values, each single value carries zero probability. Only ranges (areas under the curve) have positive probability. Asking "P(height = exactly 170.000… cm)" is like asking for the width of a single line.

Q: When can I use the normal to approximate the binomial? A common rule of thumb is when both $np \ge 10$ and $n(1-p) \ge 10$. Then the binomial is symmetric enough for the bell curve to fit well.

Q: Is the central limit theorem why averages are more reliable than single measurements? Yes. Averaging $n$ values shrinks the standard deviation of the average by a factor of $\sqrt{n}$ and pushes its distribution toward normal, so averages cluster tightly around the true value — the mathematical justification astronomers needed.

Q: Do I need to memorize the normal density formula? Almost never. Exams and real work rely on the 68-95-99.7 rule, z-scores, and tables or software. Understanding what $\mu$ and $\sigma$ do matters far more than the exponential formula.

Quick Revision

• Random variable: assigns numbers to outcomes; discrete (counts) or continuous (measurements).
• Mean: $\mu = \sum k\,P(X=k)$; std. deviation $\sigma = \sqrt{\text{Var}}$.
• Binomial: $P(X=k) = \binom{n}{k}p^k(1-p)^{n-k}$; $\mu = np$, $\sigma = \sqrt{np(1-p)}$.
• Normal: bell curve set by $\mu$, $\sigma$; symmetric.
• 68-95-99.7 rule: within $1\sigma$, $2\sigma$, $3\sigma$ of the mean.
• Z-score: $z = (x-\mu)/\sigma$ = standard deviations from the mean; standardize before using tables.
• CLT: sums/averages of many small independent influences become normal — why the bell curve is everywhere.
• History: de Moivre (~1733) found it approximating the binomial; Gauss (~1809) tied it to measurement error and the average.

Related Topics

Prerequisites

• Probability Basics
• Descriptive Statistics

Related Topics

• Conditional Probability and Bayes
• Exponents and Logarithms

Next Topics

• Statistical Inference