Random Variables

Learning Objectives

By the end of this page, you should be able to:

• Define a random variable and distinguish discrete from continuous types
• Compute probabilities using a probability mass function (PMF) for discrete variables
• Interpret a probability density function (PDF) for continuous variables
• Calculate the expected value and variance of a simple discrete random variable
• Explain why random variables are essential to modeling uncertainty in algorithms and machine learning
• Identify common mistakes students make when interpreting PMFs and PDFs

Quick Answer

A random variable is a rule that assigns a number to every possible outcome of a random process — instead of saying "the outcome was heads," you can say "$X = 1$," turning uncertain events into something you can do arithmetic on. Random variables come in two flavors: discrete (countable outcomes, like the number of heads in 10 coin flips) and continuous (any value in a range, like a person's height). Each type has its own way of describing probabilities — a probability mass function (PMF) for discrete variables, a probability density function (PDF) for continuous ones. Random variables matter in computer science because they're how you formally reason about algorithm runtimes under random input, model uncertainty in machine learning predictions, and simulate real-world systems.

What Is a Random Variable?

A random variable is a function that assigns a real number to every outcome of a random experiment. It's not "random" in the sense of being unpredictable chaos — it's a precise mathematical object that turns qualitative outcomes into quantitative ones so you can apply algebra, calculus, and statistics to them.

Example: Flip a coin three times. Let $X$ = "the number of heads." The random variable $X$ maps each of the 8 possible outcomes (HHH, HHT, ..., TTT) to a number: HHH → 3, HHT → 2, and so on, down to TTT → 0.

Why it matters: Once an outcome is represented as a number, you can compute averages, variances, and probabilities using standard mathematical tools — you couldn't directly average "heads" and "tails," but you can average the numbers 0 through 3.

Common misunderstanding: Students often think a random variable is the outcome itself. It's actually a function that maps outcomes to numbers — the underlying randomness lives in the experiment (the coin flips), while the random variable is just the numerical lens you're using to describe it.

Discrete Random Variables and the PMF

A discrete random variable can take only specific, countable values (like 0, 1, 2, 3, ...). Its behavior is described by a probability mass function (PMF), which assigns a probability to each possible value:

$P(X = x) = f(x), \qquad \sum_{\text{all } x} f(x) = 1$

Worked numeric example: Let $X$ be the outcome of rolling a fair six-sided die. Every face is equally likely, so the PMF is:

x	P(X = x)
1	1/6
2	1/6
3	1/6
4	1/6
5	1/6
6	1/6

Check: $6 \times \frac{1}{6} = 1$, confirming the probabilities sum to 1 as required.

Expected value: The expected value (mean) of a discrete random variable is the probability-weighted average of its possible values:

$E[X] = \sum_x x \cdot P(X = x)$

For the die: $E[X] = 1(\tfrac{1}{6}) + 2(\tfrac{1}{6}) + 3(\tfrac{1}{6}) + 4(\tfrac{1}{6}) + 5(\tfrac{1}{6}) + 6(\tfrac{1}{6}) = \frac{21}{6} = 3.5$.

Notice that 3.5 isn't even a possible outcome of a single roll — that's expected, since $E[X]$ represents the long-run average over many rolls, not a value $X$ can actually take.

Real-world example: In load testing a server, you might model "number of failed requests out of 100" as a discrete random variable. Its expected value tells engineers the average failure count they should plan capacity around.

Continuous Random Variables and the PDF

A continuous random variable can take any value within a range (like height, temperature, or the time between server requests). Because there are infinitely many possible values, the probability of hitting any exact value is technically 0 — instead, probabilities are described over intervals using a probability density function (PDF).

$P(a \le Y \le b) = \int_a^b f(y)\, dy$

The PDF itself, $f(y)$, is not a probability — it's a density. You only get an actual probability by integrating (finding the area under the curve) over some range.

Worked example: Suppose the time between requests to a server, $Y$, follows a distribution where $P(1 \le Y \le 2)$ (seconds) corresponds to the area under the PDF curve between 1 and 2. If that area works out to 0.3, then there's a 30% chance the next request arrives between 1 and 2 seconds after the previous one — even though $P(Y = 1.5 \text{ exactly})$ is 0.

Common misunderstanding: Students often think $f(y)$ itself is a probability and expect it to be at most 1. A PDF's height can exceed 1 (for example, if a distribution is tightly concentrated in a narrow interval); what must be true is that the total area under the entire curve equals 1, and that any interval's area is between 0 and 1.

Why it matters: This distinction — mass for discrete, density for continuous — is why you sum PMFs but integrate PDFs. Getting this backwards is one of the most common exam mistakes in probability.

Random Variables in Computer Science

Algorithm analysis: Randomized algorithms (like randomized quicksort) have runtimes that are random variables. Computing the expected runtime — rather than worst-case — often gives a more realistic performance picture, and relies directly on the $E[X]$ formula above.

Machine learning: A classifier's predicted probability of an email being spam is treated as (or derived from) a random variable's distribution. Continuous random variables underlie models like Gaussian Naive Bayes, which assumes numerical features follow a normal distribution.

import numpy as np

# Simulating a discrete random variable: number of heads in 10 coin flips
trials = np.random.binomial(n=10, p=0.5, size=100000)
print("Expected value (theoretical):", 10 * 0.5)   # 5.0
print("Sample mean (empirical):", trials.mean())    # close to 5.0

Why it matters: Random variables give you a rigorous way to reason about systems with built-in uncertainty — network latency, cache hit rates, user behavior — instead of only handling the "average case" informally.

Key Terms

Term	Definition
Random variable	A function that assigns a real number to each outcome of a random experiment
Discrete random variable	A random variable that takes only countable, specific values
Continuous random variable	A random variable that can take any value within a range
Probability mass function (PMF)	A function giving $P(X = x)$ for each value of a discrete random variable
Probability density function (PDF)	A function whose area under a given interval gives the probability a continuous variable falls in that interval
Expected value ($E[X]$)	The probability-weighted average of a random variable's possible values
Variance	A measure of how spread out a random variable's values are around its expected value
Cumulative distribution function (CDF)	A function giving $P(X \le x)$, defined for both discrete and continuous variables

Common Mistakes

1. Misconception: "A random variable is the outcome of the experiment itself." Why it's wrong: This confuses the raw outcome (like "heads") with the numerical mapping applied to it. Correct: A random variable is a function that converts outcomes into numbers, e.g., mapping "heads" to 1 and "tails" to 0 — the randomness comes from the experiment, while the random variable is the rule for numbering the results.

2. Misconception: "For a continuous random variable, $f(y)$ (the PDF value) is a probability, so it must be between 0 and 1." Why it's wrong: A PDF describes density, not probability directly, and density values can exceed 1 when a distribution is concentrated in a narrow range. Correct: Only the area under the PDF over an interval (an integral) gives a probability, and the total area under the entire curve must equal 1. The height of the curve at a single point has no direct probability interpretation.

3. Misconception: "The expected value is the most likely outcome." Why it's wrong: Expected value is a probability-weighted average, which may not even be a value the random variable can take. Correct: As shown with the die example, $E[X] = 3.5$ even though you can never actually roll a 3.5 — expected value describes the long-run average over many repetitions, not the single most probable outcome (that's the "mode").

Comparison and Connections

Concept	Discrete Random Variable	Continuous Random Variable
Possible values	Countable (0, 1, 2, ...)	Any value in an interval
Probability tool	PMF: $P(X = x)$	PDF: $P(a \le Y \le b) = \int_a^b f(y)dy$
Sum/Integral condition	$\sum P(X=x) = 1$	$\int_{-\infty}^{\infty} f(y)\,dy = 1$
Probability of exact value	Can be nonzero	Always 0
Typical CS example	Number of failed requests	Server response time

Practice Questions

Recall

1. What is the difference between a discrete and a continuous random variable? Answer: A discrete random variable takes only specific, countable values; a continuous random variable can take any value within a range.
2. What must be true of the sum of all probabilities in a PMF? Answer: They must sum to exactly 1, since one of the possible outcomes must occur.

Understanding 3. Why is the probability of a continuous random variable taking any exact single value always 0? Answer guidance: Because there are infinitely many possible values in any continuous range, and probability is the area under the PDF curve — the "area" of a single point (zero width) is always 0, no matter how tall the PDF is there. 4. Why doesn't the expected value need to be a value the random variable can actually take? Answer guidance: Expected value is a weighted average across all possible outcomes, similar to how a class average (like 3.5 out of 5 questions correct on average) need not match any individual student's actual score. It summarizes the distribution's center, not a specific achievable outcome.

Application 5. A discrete random variable $X$ represents the number of defective items in a batch of 3, with PMF: $P(X=0)=0.5$, $P(X=1)=0.3$, $P(X=2)=0.15$, $P(X=3)=0.05$. Compute $E[X]$. Answer: $E[X] = 0(0.5) + 1(0.3) + 2(0.15) + 3(0.05) = 0 + 0.3 + 0.3 + 0.15 = 0.75$ defective items on average. 6. A server's response time is modeled as a continuous random variable. If $P(0 \le Y \le 1\text{s}) = 0.4$ and $P(1 \le Y \le 2\text{s}) = 0.35$, what is $P(Y \ge 2\text{s})$, assuming response time is always at least 0? Answer: $P(Y \ge 2) = 1 - 0.4 - 0.35 = 0.25$, since total probability across all possible response times must equal 1.

Analysis 7. Compare how you would verify that a PMF is valid versus verifying that a PDF is valid. What's structurally similar and what's different? Answer guidance: Both require total probability to equal 1 and all individual probabilities/densities to be non-negative. The difference is mechanical: a PMF check sums a finite (or countable) list of values, while a PDF check requires integrating a continuous function over its entire domain — reflecting the underlying difference between counting and measuring area. 8. Explain why randomized quicksort's expected runtime analysis is more useful in practice than only knowing its worst-case runtime. Answer guidance: Randomized quicksort's worst case ($O(n^2)$) occurs only for specific unlucky pivot choices, which become vanishingly rare with random pivoting. Since the runtime is a random variable, computing $E[X]$ (which works out to $O(n\log n)$) reflects the overwhelmingly typical behavior across random inputs, giving a far more realistic performance expectation than the rare worst case.

FAQ

Q: Is a random variable always a number between 0 and 1? A: No — that confuses a random variable with a probability. A random variable can take any real value (like 0 to 100 defective items, or -40°C to 50°C temperature); it's the probabilities associated with those values that must fall between 0 and 1.

Q: Why do we need both a PMF and a PDF instead of one universal tool? A: Discrete variables have gaps between possible values, so you can meaningfully assign probability to each one individually (a PMF). Continuous variables have no gaps — infinitely many values exist in any interval — so probability has to be spread out as a density and measured via area (a PDF).

Q: What's the relationship between expected value and variance? A: Expected value tells you the center (average) of a random variable's distribution. Variance tells you how spread out the values are around that center — a low variance means outcomes cluster near $E[X]$, while high variance means outcomes are widely scattered.

Q: Can a random variable be both discrete and continuous? A: Not typically in introductory contexts — a variable is usually classified as one or the other based on whether its possible outcomes are countable or form a continuum. (Mixed distributions exist in advanced probability but are outside typical intro coursework.)

Q: How do random variables connect to machine learning models? A: Model outputs (like classification probabilities) and inputs (like sensor readings) are often treated as random variables. Many algorithms, such as Naive Bayes, explicitly assume a particular distribution (e.g., Gaussian) for continuous features to compute predictions.

Quick Revision

• A random variable is a function mapping outcomes of a random experiment to real numbers.
• Discrete random variables take countable values; continuous random variables take any value in a range.
• PMF: $P(X=x) = f(x)$, with $\sum f(x) = 1$ for all possible values.
• PDF: probability is the area under the curve over an interval, $\int_a^b f(y)\,dy$, with total area = 1.
• For continuous variables, $P(Y = \text{exact value}) = 0$ always.
• Expected value $E[X] = \sum x \cdot P(X=x)$ is a probability-weighted average, not necessarily an achievable outcome.
• Variance measures spread of a random variable's values around its expected value.
• A PDF's height can exceed 1; only the total area must equal 1.
• Randomized algorithm analysis uses expected value to describe typical, not worst-case, performance.
• Machine learning models often treat inputs, outputs, and errors as random variables with assumed distributions.

Related Topics

Prerequisites: Probability Theory, basic set theory, basic calculus (for continuous variables)

Related Topics: Probability distributions (Binomial, Poisson, Normal), expected value and variance, statistical inference

Next Topics: Hypothesis testing, machine learning fundamentals (as covered in later modules)