Statistics and Probability
Statistics and probability are the mathematics of uncertainty and data. Probability works forwards — given a known process (a fair die, a coin), it predicts how likely each outcome is. Statistics works backwards — given the outcomes you actually observed (a survey, an experiment), it infers what the underlying process must be like. Together they are how modern science, medicine, business, and machine learning turn messy real-world data into trustworthy conclusions.
Learning Objectives
By the end of this subfield, you should be able to:
- Summarize a dataset using measures of center (mean, median, mode) and spread (range, variance, standard deviation)
- Apply the basic rules of probability, including independence and conditional probability
- Recognize common distributions, especially the normal ("bell curve") distribution
- Interpret Bayes' theorem and why updating on evidence matters
- Distinguish correlation from causation and avoid common statistical fallacies
Quick Answer
Descriptive statistics compress a pile of numbers into a few meaningful summaries — the mean (average) tells you the center, the standard deviation tells you how spread out the data is. Probability assigns each possible outcome a number between 0 (impossible) and 1 (certain). Inferential statistics uses a sample to draw conclusions about a whole population, always attaching a measure of how confident you can be. The normal distribution — the bell curve — appears so often (heights, measurement errors, test scores) that it underlies much of statistical reasoning.
Where It Came From
Probability has a memorable origin: gambling. In 1654 a French nobleman and gambler, the Chevalier de Méré, asked the mathematician Blaise Pascal how to divide the stakes of an interrupted dice game fairly. Pascal's correspondence with Pierre de Fermat to solve this "problem of points" laid the foundations of probability theory. The need was concrete: games of chance were everywhere, and no one had a principled way to reason about the odds.
Statistics grew from a different need — the state (the words share a root). Governments needed to count populations, tax citizens, and plan armies, so they collected data on births, deaths, and wealth. In 17th-century London, John Graunt analyzed mortality records to study the plague, effectively inventing demography. The two streams merged in the 1800s and early 1900s: Gauss formalized the normal distribution while analyzing astronomical measurement errors, and figures like Ronald Fisher built the modern toolkit of experimental design and statistical inference — largely to improve agricultural experiments. The pattern repeats through this subject: each method was invented to answer a pressing practical question — fair bets, public health, reliable measurement, better crops — long before it became abstract theory. Today the same mathematics powers clinical trials, opinion polls, quality control, and every machine-learning model.
Topics at a Glance
| Topic | What You'll Learn | Key Concepts |
|---|---|---|
| Descriptive Statistics | Summarizing data | Mean, median, mode, variance, standard deviation |
| Probability Basics | Measuring likelihood | Sample space, events, addition & multiplication rules |
| Conditional Probability & Bayes | Updating beliefs on evidence | Independence, conditional probability, Bayes' theorem |
| Distributions | Patterns of random outcomes | Normal, binomial, the bell curve, z-scores |
| Inference | From sample to population | Sampling, confidence intervals, hypothesis testing |
Learning Path
Real-World Applications
- Medicine: clinical trials use hypothesis testing to decide whether a drug truly works or just got lucky.
- Business and quality control: manufacturers monitor variation with standard deviation to keep products within spec.
- Polling and elections: confidence intervals quantify the "margin of error" you hear reported.
- Machine learning and AI: nearly every model is, at heart, a probabilistic or statistical estimator.
Key Terms
| Term | Definition | Related Concept |
|---|---|---|
| Mean | The arithmetic average | Center, expected value |
| Standard deviation | A measure of how spread out data is | Variance, normal curve |
| Probability | A number from 0 to 1 measuring likelihood | Events, randomness |
| Conditional probability | Probability of A given that B occurred | Bayes' theorem |
| Normal distribution | The symmetric bell-shaped curve | Z-scores, inference |
Common Pitfall
Correlation is not causation. Two things that rise together (ice-cream sales and drowning deaths) may share a hidden cause (hot weather) rather than one causing the other. Much of statistical training is about not being fooled by patterns that are real but misleading.
Quick Revision
- Mean = sum ÷ count; median = middle value; mode = most frequent.
- Standard deviation measures spread; a small SD means data clusters near the mean.
- Probability is between 0 and 1; probabilities of all outcomes sum to 1.
- Normal distribution: ~68% of data within 1 SD, ~95% within 2 SD of the mean.
- Correlation ≠ causation.
Related Topics
Prerequisites: Algebra (fractions, functions).
Related: used throughout Economics, Psychology, and data science.
Next: inferential statistics, regression, and the probability foundations of machine learning.