Probability Basics

Probability is the mathematics of uncertainty — a way to attach a precise number to statements like "this is likely," "that is a long shot," or "it's a coin flip." Instead of arguing about hunches, probability lets us reason about chance with the same rigor we bring to arithmetic. Every time a doctor quotes a survival rate, a weather app says "70% chance of rain," an insurer sets a premium, or a poker player decides to call a bet, they are using the ideas on this page.

What makes probability beautiful is that a handful of simple rules — outcomes, a scale from 0 to 1, and three or four ways to combine events — are enough to handle situations that feel impossibly tangled. Master these foundations and the rest of statistics, from Bayes' theorem to the normal distribution, becomes a natural extension rather than a new mystery.

Learning Objectives

By the end of this page, you should be able to:

• Describe a random experiment using its sample space and identify events as subsets of it.
• Explain why every probability is a number between $0$ and $1$, and compute probabilities for equally likely outcomes.
• Apply the complement rule, the addition rule (OR), and the multiplication rule (AND) correctly.
• Distinguish clearly between mutually exclusive and independent events — the single most common source of student error.
• Solve multi-step probability problems and recognize which rule each step requires.

Quick Answer

A probability is a number from $0$ (impossible) to $1$ (certain) that measures how likely an event is. The sample space $S$ is the set of all possible outcomes of an experiment, and an event is any subset of $S$. When outcomes are equally likely, $P(\text{event}) = \dfrac{\text{favorable outcomes}}{\text{total outcomes}}$. Three rules do most of the work: the complement rule $P(\text{not }A) = 1 - P(A)$; the addition rule $P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$; and the multiplication rule $P(A \text{ and } B) = P(A)\times P(B\mid A)$. Two events are mutually exclusive if they cannot both happen, and independent if one occurring does not change the probability of the other — these are different ideas, not synonyms.

Where It Came From

For most of human history, chance was treated as the domain of fate or the gods, not something you could calculate. Dice games were thousands of years old, yet no one had a systematic theory of how to bet on them. The turning point came in 1654, when a French nobleman and gambler, the Chevalier de Méré, posed a puzzle to the mathematician Blaise Pascal. The most famous was the "problem of points": suppose two players are partway through a game of chance, each having staked money, and the game is interrupted before either wins. How should the pot be divided fairly, given the score at the moment play stopped?

This is subtler than it sounds. Dividing the pot by the current score ignores the fact that a player who is behind still has some chance to come back. Pascal exchanged a now-legendary series of letters with Pierre de Fermat, and between them they cracked it — not by looking at the score, but by counting all the equally likely ways the game could have finished and giving each player the share of the pot proportional to their probability of eventual victory. That idea — reduce a problem to a set of equally likely outcomes and count — is the seed of the entire subject.

The Pascal–Fermat correspondence of 1654 is generally regarded as the birth of probability theory. Within decades, Christiaan Huygens wrote the first textbook, Jacob Bernoulli proved the first law of large numbers, and probability grew from a tool for settling gambling disputes into the mathematical backbone of insurance, physics, genetics, and modern data science. The motivation was intensely practical — how to bet and share stakes fairly — but the framework it produced turned out to describe uncertainty everywhere.

The Sample Space and Events

Every probability problem starts by pinning down the experiment — any process with an uncertain outcome — and its sample space $S$, the set of all possible outcomes. Getting this right is half the battle.

• Tossing one coin: $S = \{H, T\}$.
• Rolling one die: $S = \{1, 2, 3, 4, 5, 6\}$.
• Tossing two coins: $S = \{HH, HT, TH, TT\}$ — note there are four outcomes, not three, because $HT$ and $TH$ are distinct.

An event is any subset of the sample space — a collection of outcomes we care about. For a single die roll, "rolling an even number" is the event $E = \{2, 4, 6\}$, and "rolling more than $4$" is $\{5, 6\}$.

Worked Example: Building a Sample Space

Problem. Two fair dice (one red, one blue) are rolled. What is the sample space, and what is the event "the sum is $7$"?

Each die shows $1$ through $6$ independently, so the sample space is every ordered pair $(\text{red}, \text{blue})$:

$|S| = 6 \times 6 = 36 \text{ equally likely outcomes.}$

The event "sum is $7$" is $\{(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)\}$ — exactly $6$ outcomes. So

$P(\text{sum} = 7) = \frac{6}{36} = \frac{1}{6} \approx 0.167.$

Listing outcomes as ordered pairs is what keeps them equally likely; a beginner who lists only unordered sums ($2$ through $12$) wrongly treats them as equally likely and gets nonsense.

Probability as a Number from 0 to 1

Probability is measured on a fixed scale:

• $P(A) = 0$ means $A$ is impossible.
• $P(A) = 1$ means $A$ is certain.
• Everything else lives strictly between, with larger numbers meaning "more likely."

Two rules anchor the whole system (Kolmogorov's axioms, in plain form): every probability satisfies $0 \le P(A) \le 1$, and the probabilities of all outcomes in the sample space add up to exactly $1$.

When — and only when — every outcome is equally likely, we can compute probability by counting:

$P(A) = \frac{\text{number of outcomes in } A}{\text{number of outcomes in } S}.$

Worked Example: Equally Likely Outcomes

Problem. A standard deck has $52$ cards. Draw one at random. Find the probability it is (a) a queen, (b) a heart, (c) a face card (J, Q, K).

Every card is equally likely, so we just count favorable cards.

• (a) Four queens: $P = \dfrac{4}{52} = \dfrac{1}{13} \approx 0.077$.
• (b) Thirteen hearts: $P = \dfrac{13}{52} = \dfrac{1}{4} = 0.25$.
• (c) Twelve face cards ($3$ per suit $\times\ 4$ suits): $P = \dfrac{12}{52} = \dfrac{3}{13} \approx 0.231$.

The "equally likely" assumption is doing real work here. It holds for a well-shuffled deck; it would fail badly for something like "which language a random web page is written in," where you must estimate probabilities from data instead of counting.

The Complement Rule

The complement of $A$, written $A^c$ or "not $A$," is everything in the sample space outside $A$. Since $A$ and $A^c$ together cover all outcomes:

$P(A^c) = 1 - P(A).$

This little rule is a workhorse. Whenever "at least one" appears, computing the complement ("none") is usually far easier than the direct count.

Worked Example: "At Least One"

Problem. Roll a fair die $3$ times. What is the probability of getting at least one six?

Directly counting "one, two, or three sixes" is messy. The complement — "no sixes at all" — is easy. On each roll, $P(\text{not }6) = \frac{5}{6}$, and the rolls are independent, so:

$P(\text{no sixes}) = \left(\frac{5}{6}\right)^3 = \frac{125}{216} \approx 0.579.$

Therefore:

$P(\text{at least one six}) = 1 - \frac{125}{216} = \frac{91}{216} \approx 0.421.$

The Addition Rule (OR)

The addition rule answers "what is the probability of $A$ or $B$?" — meaning at least one of them happens. The general formula is:

$P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B).$

Why subtract the last term? Because when you add $P(A)$ and $P(B)$, any outcome in both events gets counted twice. Subtracting $P(A \text{ and } B)$ removes the double-count. (This is the probability version of the inclusion–exclusion principle.)

If $A$ and $B$ are mutually exclusive — they cannot both happen — then $P(A \text{ and } B) = 0$ and the rule simplifies to $P(A \text{ or } B) = P(A) + P(B)$.

Worked Example: Overlapping Events

Problem. Draw one card from a standard deck. Find $P(\text{king or heart})$.

• $P(\text{king}) = \dfrac{4}{52}$.
• $P(\text{heart}) = \dfrac{13}{52}$.
• $P(\text{king and heart})$: there is exactly one king of hearts, so $\dfrac{1}{52}$.

$P(\text{king or heart}) = \frac{4}{52} + \frac{13}{52} - \frac{1}{52} = \frac{16}{52} = \frac{4}{13} \approx 0.308.$

Forgetting to subtract the king of hearts would give $\frac{17}{52}$ — counting that one card twice. Contrast this with "king or queen," which are mutually exclusive (no card is both), so there we simply add: $\frac{4}{52} + \frac{4}{52} = \frac{8}{52}$.

The Multiplication Rule (AND)

The multiplication rule answers "what is the probability of $A$ and $B$ both happening?" The general form uses conditional probability $P(B \mid A)$, read "the probability of $B$ given that $A$ has occurred":

$P(A \text{ and } B) = P(A) \times P(B \mid A).$

If the events are independent — knowing that $A$ happened tells you nothing about $B$ — then $P(B \mid A) = P(B)$, and the rule simplifies to:

$P(A \text{ and } B) = P(A) \times P(B).$

Worked Example: Independent vs Dependent Draws

Problem. A bag holds $5$ red and $3$ blue marbles ($8$ total). Draw two marbles. Find $P(\text{both red})$ (a) with replacement, (b) without replacement.

(a) With replacement (independent). The first marble goes back, so the second draw faces the same bag:

$P(\text{both red}) = \frac{5}{8} \times \frac{5}{8} = \frac{25}{64} \approx 0.391.$

(b) Without replacement (dependent). After drawing one red, only $4$ reds remain out of $7$ marbles:

$P(\text{both red}) = \frac{5}{8} \times \frac{4}{7} = \frac{20}{56} = \frac{5}{14} \approx 0.357.$

The second scenario has a lower probability because removing a red marble makes the next red slightly less likely. Recognizing whether draws are independent is exactly what tells you which version of the rule to use.

Real-World Applications

• Medicine and public health. Diagnostic tests are described by probabilities (sensitivity, specificity). Combining a test result with disease prevalence — a multiplication-and-complement calculation — tells a doctor how much to trust a positive result.
• Insurance and finance. Premiums are set from the probability and cost of claims. The whole industry is applied probability descended directly from the actuarial tables Bernoulli and Halley helped invent.
• Engineering and reliability. A system with independent components fails only if enough parts fail; "at least one backup works" is a complement calculation that keeps aircraft and data centers safe.
• Genetics. Punnett squares are equally-likely sample spaces. The chance a child inherits a recessive condition is a direct count of favorable gene combinations.
• Everyday decisions. Weather forecasts, sports betting odds, spam filters, and even the "shuffle" on your music app all rest on the same rules of counting, complements, and combining events.

Common Mistakes

Mistake 1: Confusing "mutually exclusive" with "independent." Many students treat these as the same, but they are almost opposites. Mutually exclusive means the events cannot both happen ($P(A \text{ and } B) = 0$). Independent means one happening does not change the other's probability ($P(B\mid A) = P(B)$). If two events are mutually exclusive and both have nonzero probability, then knowing $A$ happened tells you $B$ cannot happen — so they are strongly dependent. The correction: check the definitions separately; do not assume one implies the other.

Mistake 2: Adding probabilities of overlapping events. Writing $P(A \text{ or } B) = P(A) + P(B)$ when $A$ and $B$ can overlap double-counts the shared outcomes (like the king of hearts above). The correction: always use the full rule $P(A) + P(B) - P(A \text{ and } B)$, and only drop the last term after confirming the events are mutually exclusive.

Mistake 3: The "gambler's fallacy." Believing that after five heads in a row, tails is "due." For independent coin tosses, each flip is still $50/50$ — the coin has no memory. The correction: independent events do not compensate for past results. (What is true is that long-run frequencies settle near the true probability, but that happens by swamping early streaks with future data, not by reversing them.)

Comparison and Connections

The two big pitfalls sit side by side here:

Property	Mutually Exclusive	Independent
Plain meaning	Cannot both happen	One doesn't affect the other
Key equation	$P(A \text{ and } B) = 0$	$P(A \text{ and } B) = P(A)\,P(B)$
Example	Rolling a $2$ vs a $5$ on one die	Rolling a die and flipping a coin
If $A$ happens, then $B$…	is impossible	is just as likely as before
Relationship	These are different ideas; two events with nonzero probability cannot be both

Probability also connects outward: conditional probability and Bayes' theorem generalize the multiplication rule; random variables and distributions take these outcome-counts and turn them into functions; and descriptive statistics provides the observed frequencies we often use to estimate probabilities when counting is impossible.

Practice Questions

Recall

State the complement rule and the general addition rule. Answer: $P(A^c) = 1 - P(A)$; and $P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$.

Understanding

A die is rolled once. Are the events "even number" and "greater than $4$" mutually exclusive? Independent? Answer: Even $= \{2,4,6\}$, greater than $4 = \{5,6\}$. They overlap at $\{6\}$, so not mutually exclusive. Check independence: $P(\text{even}) = \frac12$, $P(> 4) = \frac13$, product $= \frac16$; and $P(\text{even and} > 4) = P(\{6\}) = \frac16$. They match, so the events are independent.

Application

A jar has $4$ green and $6$ yellow sweets. You take two without replacement. Find $P(\text{one of each color})$. Answer: Two ways: green-then-yellow or yellow-then-green. $P(GY) = \frac{4}{10}\times\frac{6}{9} = \frac{24}{90}$ and $P(YG) = \frac{6}{10}\times\frac{4}{9} = \frac{24}{90}$. Total $= \frac{48}{90} = \frac{8}{15} \approx 0.533$.

Analysis

A test is $99\%$ accurate on healthy people (so it wrongly flags $1\%$). If $500$ healthy people are tested independently, what is the probability that at least one is wrongly flagged? Answer: Use the complement. $P(\text{no false alarms}) = (0.99)^{500}$. Since $(0.99)^{500} \approx 0.0066$, we get $P(\text{at least one}) \approx 1 - 0.0066 = 0.993$. Even a very accurate test produces false alarms once applied at scale — a crucial lesson in interpreting screening.

FAQ

Can a probability be greater than $1$ or negative? No. By definition every probability lies in $[0, 1]$. If your calculation gives $1.4$ or $-0.2$, you have made an error — often adding overlapping events or forgetting to subtract an intersection.

What's the real difference between "and" and "or" problems? "And" (both happen) usually means multiply — the event gets more specific and less likely. "Or" (at least one happens) usually means add (then subtract the overlap) — the event gets broader and more likely. Watching whether the probability should go up or down is a good sanity check.

How do I know if two events are independent? Formally, $A$ and $B$ are independent when $P(A \text{ and } B) = P(A)\times P(B)$. Practically, ask: "Does knowing $A$ happened change my estimate of $B$?" Drawing with replacement keeps draws independent; without replacement makes them dependent.

Why do we assume outcomes are "equally likely" so often? Because when they are, probability reduces to simple counting — the Pascal–Fermat insight. Well-designed dice, coins, and shuffled cards genuinely produce equally likely outcomes. For real-world events (weather, disease), we drop the assumption and estimate probabilities from data instead.

Is probability the same as the odds I see in betting? Related but different. Odds compare favorable to unfavorable outcomes; probability compares favorable to total. Odds of "$3$ to $1$ against" mean $3$ unfavorable for every $1$ favorable, i.e. probability $\frac{1}{4}$.

Quick Revision

• Sample space $S$: set of all outcomes. Event: a subset of $S$.
• Scale: $0 \le P(A) \le 1$; all outcome probabilities sum to $1$.
• Equally likely: $P(A) = \dfrac{\text{favorable}}{\text{total}}$.
• Complement: $P(A^c) = 1 - P(A)$ — use it for "at least one."
• Addition (OR): $P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$; drop the last term only if mutually exclusive.
• Multiplication (AND): $P(A \text{ and } B) = P(A)\times P(B\mid A)$; equals $P(A)\times P(B)$ only if independent.
• Mutually exclusive ($P(A\text{ and }B)=0$) $\ne$ independent ($P(B\mid A)=P(B)$).
• Gambler's fallacy: independent trials have no memory.
• Born in the 1654 Pascal–Fermat letters on the gambling "problem of points."

Related Topics

Prerequisites

• Descriptive Statistics — frequencies and proportions that feed probability estimates.
• Statistics and Probability overview — where this fits in the wider subfield.

Related Topics

• Conditional Probability and Bayes — generalizes the multiplication rule and updating beliefs.
• Distributions — probability spread across the values of a random variable.

Next Topics

• Statistical Inference — using probability to draw conclusions from data.
• Applications of Calculus — continuous probability relies on integration.