Conditional Probability and Bayes' Theorem

Most probabilities you care about in real life are conditional. You rarely ask "what is the chance it rains?" in a vacuum — you ask given that the sky is grey and the forecast said 70%. You rarely ask "does this patient have the disease?" without first knowing their test came back positive. Conditional probability is the mathematics of updating what you believe when new information arrives, and Bayes' theorem is the exact rule for doing that update. It is one of the highest-leverage ideas in all of mathematics: it powers spam filters, medical diagnosis, machine learning, courtroom reasoning, and the everyday skill of not fooling yourself with a scary-sounding statistic.

This page will teach you not just the formulas but the intuition — especially why a positive result on a very accurate test can still mean you probably don't have the disease. That single counterintuitive fact, once it clicks, will change how you read the news.

Learning Objectives

By the end of this page, you should be able to:

• Define and compute conditional probability $P(A \mid B)$ from a table, a tree, or raw counts.
• State the multiplication rule and use it to build probability trees.
• Decide whether two events are independent and explain what independence really means.
• State Bayes' theorem and apply it to update a prior belief in light of evidence.
• Work the classic medical-test problem and explain base-rate neglect to someone else.
• Recognise Bayesian reasoning in real applications like spam filtering and diagnostics.

Quick Answer

The conditional probability of $A$ given $B$ is $P(A \mid B) = \dfrac{P(A \cap B)}{P(B)}$ — the fraction of the $B$-world in which $A$ also happens. Two events are independent when knowing one tells you nothing about the other, i.e. $P(A \mid B) = P(A)$, equivalently $P(A \cap B) = P(A)\,P(B)$. Bayes' theorem reverses a conditional: $P(A \mid B) = \dfrac{P(B \mid A)\,P(A)}{P(B)}$. It lets you convert "probability of the evidence given the hypothesis" into "probability of the hypothesis given the evidence." The famous lesson is that when a condition is rare (low base rate), even an accurate test produces mostly false positives — so a positive result may still leave the true probability low.

Where It Came From

Before the 18th century, probability was mostly about forward questions: given a fair die, what is the chance of rolling two sixes? The gamblers' correspondence between Pascal and Fermat (1654) had cracked those. But the harder, more human question runs backwards: I have observed some data — what does it tell me about the hidden cause? If I draw several white balls from an urn of unknown composition, what should I now believe about the urn?

The Reverend Thomas Bayes (c. 1701–1761), an English Presbyterian minister and amateur mathematician, worked out a special case of exactly this inverse problem. He never published it; his friend Richard Price found the essay among his papers and read it to the Royal Society in 1763, two years after Bayes' death. The motivation was partly theological-philosophical — a rebuttal to Hume on whether evidence (miracles, or the regularity of nature) could justify belief in a cause.

The person who turned this seed into a general, powerful tool was Pierre-Simon Laplace. Working independently from around 1774, Laplace stated the theorem in its modern general form, gave it the interpretation of updating probabilities, and used it on real scientific problems — estimating the masses of planets, analysing birth-rate data (he used it to argue boys are born slightly more often than girls), and quantifying measurement error. For over a century the method was simply called "inverse probability."

Bayes' theorem spent much of the 20th century out of fashion, dismissed by some statisticians as too subjective. Its dramatic comeback came from computing and artificial intelligence: cheap computation made Bayesian updating practical at scale. Today it underlies spam filters (the "Bayesian filter" of the early 2000s), speech recognition, medical decision support, and the probabilistic models at the heart of machine learning. The minister's unpublished note became one of the engines of the information age.

Conditional Probability: Narrowing the World

When you condition on $B$, you throw away every outcome where $B$ did not happen and treat $B$ as your new universe. Within that smaller world, you ask how often $A$ also occurs:

$$P(A \mid B) = \frac{P(A \cap B)}{P(B)}, \qquad \text{provided } P(B) > 0.$$

Rearranging gives the multiplication rule:

$$P(A \cap B) = P(B)\,P(A \mid B) = P(A)\,P(B \mid A).$$

Worked example: a deck of cards

Draw one card from a standard 52-card deck. Let $A$ = "the card is a King" and $B$ = "the card is a face card (J, Q, K)."

There are 12 face cards, so $P(B) = 12/52$. The Kings are all face cards, and there are 4 of them, so $P(A \cap B) = 4/52$. Then:

$$P(A \mid B) = \frac{4/52}{12/52} = \frac{4}{12} = \frac{1}{3}.$$

Once you know the card is a face card, one-third of the possibilities are Kings. Notice the unconditional probability was only $P(A) = 4/52 = 1/13$. The information "it's a face card" pushed the probability up — that is conditioning at work.

Worked example: reading a two-way table

Suppose a school surveys 200 students about whether they walk to school and whether they live within 1 km.

	Walks	Does not walk	Total
Within 1 km	70	10	80
Farther than 1 km	30	90	120
Total	100	100	200

What is $P(\text{walks} \mid \text{within 1 km})$? Restrict to the 80 nearby students; 70 of them walk:

$$P(\text{walks} \mid \text{within 1 km}) = \frac{70}{80} = 0.875.$$

Compare $P(\text{walks} \mid \text{farther}) = 30/120 = 0.25$. Distance clearly matters — the two conditionals differ, which already tells you these events are not independent.

Independence: When Information Doesn't Help

Two events are independent if learning that one happened does not change the probability of the other:

$$P(A \mid B) = P(A) \quad\Longleftrightarrow\quad P(A \cap B) = P(A)\,P(B).$$

The product form is the cleaner test because it is symmetric and works even when a probability is zero.

Worked example: two dice

Roll two fair dice. Let $A$ = "first die is 6" and $B$ = "second die is odd." Then $P(A) = 1/6$, $P(B) = 3/6 = 1/2$, and because the dice don't influence each other, $P(A \cap B) = 1/6 \times 1/2 = 1/12$. Since $P(A \cap B) = P(A)P(B)$, the events are independent — as physics demands.

Now a trap: let $C$ = "the sum is 7." Is $A$ independent of $C$? We have $P(A) = 1/6$ and $P(C) = 6/36 = 1/6$. The only way to get a 6 on the first die and a sum of 7 is (6, 1), so $P(A \cap C) = 1/36$. Check: $P(A)P(C) = 1/6 \times 1/6 = 1/36$. They match, so $A$ and $C$ are independent — a surprising result that shows independence is a numerical fact, not something you can always eyeball. (Try $C$ = "sum is 6" instead and you'll find dependence.)

A key warning: independent is not the same as mutually exclusive. Mutually exclusive events cannot both happen ($P(A \cap B) = 0$), which — if both have positive probability — actually makes them strongly dependent, because knowing one occurred tells you the other did not.

Bayes' Theorem: Reversing the Conditional

Often you can easily measure $P(B \mid A)$ but you actually want $P(A \mid B)$. A test tells you $P(\text{positive} \mid \text{disease})$, but the patient wants $P(\text{disease} \mid \text{positive})$. Bayes' theorem flips it. Starting from the two forms of the multiplication rule, $P(A \cap B)$ equals both $P(B \mid A)P(A)$ and $P(A \mid B)P(B)$, so:

$$P(A \mid B) = \frac{P(B \mid A)\,P(A)}{P(B)}.$$

Usually you expand the denominator with the law of total probability, splitting on whether $A$ happened or not (writing $\lnot A$ for "not $A$"):

$$P(B) = P(B \mid A)\,P(A) + P(B \mid \lnot A)\,P(\lnot A).$$

The vocabulary: $P(A)$ is the prior (belief before evidence), $P(B \mid A)$ is the likelihood (how well the hypothesis predicts the evidence), and $P(A \mid B)$ is the posterior (updated belief). Bayes' theorem is a machine that turns prior + evidence into posterior.

The classic medical-test example

A disease affects 1 in 1000 people. A test is 99% sensitive (if you have the disease, it's positive 99% of the time) and 95% specific (if you're healthy, it's negative 95% of the time, so it gives a 5% false-positive rate). You test positive. What is the probability you actually have the disease?

Let $D$ = disease, $+$ = positive test. We know:

• Prior: $P(D) = 0.001$, so $P(\lnot D) = 0.999$.
• Likelihood: $P(+ \mid D) = 0.99$.
• False positive: $P(+ \mid \lnot D) = 0.05$.

Total probability of a positive test:

$$P(+) = (0.99)(0.001) + (0.05)(0.999) = 0.00099 + 0.04995 = 0.05094.$$

Now Bayes:

$$P(D \mid +) = \frac{(0.99)(0.001)}{0.05094} = \frac{0.00099}{0.05094} \approx 0.0194.$$

Only about 1.9%. Despite a positive result on a genuinely accurate test, you very probably do not have the disease.

The same numbers as a headcount (why it works)

Imagine 100,000 people. About $100,000 \times 0.001 = 100$ have the disease; of those, $99$ test positive. The other $99,900$ are healthy, but $5\%$ of them — about $4,995$ — test positive anyway. So the positive group holds $99 + 4,995 = 5,094$ people, of whom only $99$ are truly sick:

$$\frac{99}{5,094} \approx 0.0194.$$

The false positives from the huge healthy majority swamp the true positives from the tiny sick minority. This is why doctors retest and why screening rare conditions is genuinely hard.

Base-Rate Neglect: The Human Blind Spot

Base-rate neglect is the tendency to focus on the test's accuracy ("99%!") while ignoring how rare the condition is (the base rate, $P(D) = 0.001$). People routinely guess the answer above is around 95% or 99% when it is under 2%. Psychologists Daniel Kahneman and Amos Tversky documented this bias in the 1970s: our intuition latches onto the vivid likelihood and forgets the prior.

The cure is to always ask two questions, not one: How accurate is the evidence? and How common was the hypothesis to begin with? When the base rate is tiny, the prior dominates, and strong-sounding evidence can still leave you unconvinced.

Real-World Applications

• Medicine: interpreting screening tests (mammograms, HIV tests, COVID rapid tests). Positive predictive value depends on local prevalence — the very calculation above — which is why the same test means different things in high- vs low-prevalence settings.
• Spam filtering: early spam filters computed $P(\text{spam} \mid \text{words in email})$ from $P(\text{word} \mid \text{spam})$ using a "naive Bayes" model, learning as you flag messages. Your inbox is a working Bayesian classifier.
• Machine learning and AI: naive Bayes classifiers, Bayesian networks, and Bayesian inference underlie everything from document classification to probabilistic robotics and A/B testing.
• Law and forensics: DNA match probabilities must be combined with a prior (how many people could plausibly be the source) — ignoring the base rate is the notorious "prosecutor's fallacy."
• Search and rescue / reliability: Bayesian search theory located the lost submarine USS Scorpion (1968) and Air France 447's wreckage by continually updating a probability map as searches came up empty.
• Everyday reasoning: deciding how much a surprising rumour, a symptom, or a warning light should shift your beliefs.

Common Mistakes

1. Confusing $P(A \mid B)$ with $P(B \mid A)$ (the inversion fallacy). These are generally different. $P(\text{positive} \mid \text{disease}) = 0.99$ but $P(\text{disease} \mid \text{positive}) \approx 0.02$. Fix: never swap the direction of a conditional without running it through Bayes' theorem and the base rate.

2. Ignoring the base rate. Reasoning from the test's accuracy alone gives wildly wrong answers when the condition is rare. Fix: always include the prior $P(A)$ and expand the denominator with the law of total probability.

3. Treating "independent" and "mutually exclusive" as the same. Mutually exclusive events with positive probability are actually dependent — knowing one happened forces the other to zero. Fix: independence means $P(A \cap B) = P(A)P(B)$; mutual exclusivity means $P(A \cap B) = 0$. They almost never coincide.

4. Assuming independence without checking. Multiplying probabilities ($P(A)P(B)$) only works when events truly are independent. Correlated risks (two houses on the same floodplain) are not. Fix: verify the product rule numerically or justify independence physically before using it.

Comparison and Connections

Concept	Formula	Question it answers
Joint probability	$P(A \cap B)$	Chance both happen
Conditional probability	$P(A \mid B) = \frac{P(A \cap B)}{P(B)}$	Chance of $A$ within the $B$-world
Independence	$P(A \cap B) = P(A)P(B)$	Does $B$ tell us anything about $A$?
Bayes' theorem	$P(A \mid B) = \frac{P(B \mid A)P(A)}{P(B)}$	Flip a known conditional using a prior

Bayes' theorem is not a new law — it is a direct consequence of the conditional-probability definition plus the law of total probability. It sits downstream of basic probability and feeds directly into distributions and statistical inference, where "Bayesian" methods form an entire school of thought that competes with the "frequentist" approach.

Practice Questions

Recall

State Bayes' theorem and name the prior, likelihood, and posterior.

Answer: $P(A \mid B) = \frac{P(B \mid A)P(A)}{P(B)}$. Prior $= P(A)$; likelihood $= P(B \mid A)$; posterior $= P(A \mid B)$.

Understanding

A bag has 3 red and 2 blue marbles. You draw two without replacement. What is the probability the second is red given the first was red?

Answer: After removing one red, 2 red and 2 blue remain (4 total). So $P(\text{2nd red} \mid \text{1st red}) = 2/4 = 0.5$. This differs from the unconditional first-draw probability of $3/5$, showing the draws are dependent.

Application

A test for a condition present in 2% of a population is 90% sensitive and has a 10% false-positive rate. Find $P(\text{condition} \mid \text{positive})$.

Answer: $P(+) = (0.90)(0.02) + (0.10)(0.98) = 0.018 + 0.098 = 0.116$. Then $P(\text{cond} \mid +) = 0.018 / 0.116 \approx 0.155$, about 15.5% — still low because of the modest base rate.

Analysis

Using the Application scenario, suppose someone who tested positive is retested and is positive again (assume the two tests are conditionally independent given true status). Roughly how does the probability change, and why?

Guidance: Use the first posterior, $0.155$, as the new prior. Then $P(+ \text{again}) = (0.90)(0.155) + (0.10)(0.845) = 0.1395 + 0.0845 = 0.224$, and the updated posterior is $0.1395 / 0.224 \approx 0.62$. Two independent positives push belief from 15% to about 62% — evidence compounds, which is exactly why confirmatory testing is standard practice.

FAQ

Q: If the test is 99% accurate, why isn't a positive result 99% likely to be right? Because "99% accurate" describes $P(\text{positive} \mid \text{disease})$, not $P(\text{disease} \mid \text{positive})$. When the disease is rare, the small pool of true positives is outnumbered by false positives drawn from the large healthy population. The base rate, not just the accuracy, controls the answer.

Q: What exactly is a "prior," and isn't choosing one subjective? The prior is your probability for the hypothesis before seeing the new evidence — often an established base rate like disease prevalence. Sometimes it is estimated from data (objective), sometimes it reflects informed judgement (subjective). Critics dislike the subjectivity; supporters point out that with enough evidence, reasonable priors converge to the same posterior.

Q: Does $P(A \mid B) = P(B \mid A)$ ever hold? Only when $P(A) = P(B)$ (you can see this from Bayes' theorem). In general they are different, and assuming they're equal is the single most common probability error in real-world reasoning.

Q: How is independence different from having a correlation of zero? Independence is stronger. Independent events always have zero correlation, but two variables can have zero linear correlation while still being dependent in a nonlinear way. For events (yes/no outcomes), though, $P(A \cap B) = P(A)P(B)$ is the complete definition.

Q: Why is Bayes' theorem so central to modern AI? Learning from data is updating beliefs from evidence, which is precisely what Bayes' theorem formalises. Spam filters, recommendation systems, and probabilistic models all repeatedly compute posteriors as new data arrives. It scales beautifully once computers do the arithmetic.

Q: Can I use Bayes' theorem when there are more than two hypotheses? Yes. Replace the two-term denominator with a sum over all mutually exclusive hypotheses: $P(H_i \mid E) = \frac{P(E \mid H_i)P(H_i)}{\sum_j P(E \mid H_j)P(H_j)}$. This is how classifiers pick among many categories.

Quick Revision

• Conditional: $P(A \mid B) = \dfrac{P(A \cap B)}{P(B)}$ — restrict to the $B$-world.
• Multiplication rule: $P(A \cap B) = P(A)P(B \mid A) = P(B)P(A \mid B)$.
• Independence: $P(A \cap B) = P(A)P(B)$, equivalently $P(A \mid B) = P(A)$. Not the same as mutually exclusive.
• Bayes: $P(A \mid B) = \dfrac{P(B \mid A)P(A)}{P(B)}$, with $P(B) = P(B \mid A)P(A) + P(B \mid \lnot A)P(\lnot A)$.
• Prior → likelihood → posterior. Base rate matters as much as accuracy.
• Rare condition + accurate test = mostly false positives. Beware base-rate neglect and the $P(A \mid B)$ vs $P(B \mid A)$ swap.

Related Topics

Prerequisites

• Probability Basics
• Descriptive Statistics

Related Topics

• Distributions
• Statistics and Probability overview

Next Topics

• Statistical Inference
• Economics (Bayesian reasoning in decision-making)