Chi-Square Tests

Imagine you roll a die 120 times and get a 6 only 12 times when you expected about 20. Is the die loaded, or is this just the ordinary luck of the draw? Or suppose a survey shows that men and women seem to prefer different phone brands — is that a real difference in the population, or noise in your sample? Chi-square tests are the standard tool for answering exactly this kind of question: they measure whether the counts you actually observed differ from the counts you would expect under some hypothesis by more than random chance can comfortably explain.

The beauty of the chi-square ($\chi^2$) family is that it works with categorical data — colors, brands, blood types, yes/no answers — where you cannot compute a meaningful average. Instead of comparing means, you compare frequencies, and a single elegant formula turns the total mismatch into one number you can look up in a table. This page teaches you the two most important versions and, just as importantly, why they work.

Learning Objectives

• Distinguish the chi-square goodness-of-fit test from the test of independence.
• Compute expected counts from a hypothesis or from a contingency table's margins.
• Calculate the Pearson $\chi^2$ statistic and interpret it against degrees of freedom.
• Build and read a contingency table and state precise hypotheses.
• Decide whether to reject the null hypothesis using a critical value or p-value.
• Recognize the assumptions and common pitfalls that invalidate the test.

Quick Answer

A chi-square test compares observed frequencies ($O$) to expected frequencies ($E$) using the statistic $\chi^2 = \sum \frac{(O - E)^2}{E}.$ A large value means the observed data are far from what the null hypothesis predicts. The goodness-of-fit test checks whether one categorical variable follows a claimed distribution; degrees of freedom are $k - 1$ (categories minus one). The test of independence checks whether two categorical variables in a contingency table are related; degrees of freedom are $(r-1)(c-1)$. You compare your computed $\chi^2$ to a critical value from the chi-square distribution (or find a p-value); if $\chi^2$ exceeds the critical value, you reject the null. The test assumes independent observations and expected counts of at least 5 in most cells.

Where It Came From

Before 1900, statisticians had no rigorous way to judge whether a set of counts agreed with a theory. People eyeballed tables and argued. The breakthrough came from Karl Pearson, working at University College London, who in 1900 published the paper "On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling." The clumsy title hides a revolutionary idea.

Pearson's motivating problem was concrete. Scientists of his era — including gamblers, biologists studying inheritance, and astronomers checking measurement errors — kept asking the same question in different clothing: my data fell into these categories in these proportions; is that consistent with the distribution my theory predicts, or is the discrepancy too large to be chance? Pearson wanted a single, general number that summarized the total deviation and whose behavior under pure randomness was known mathematically, so that "surprising" could be defined precisely rather than felt intuitively.

His insight was that each cell's squared deviation, scaled by its expected size, could be summed, and that this sum follows a predictable distribution — the chi-square distribution — when the null hypothesis is true. Dividing by $E$ matters: a deviation of 10 is enormous where you expected 5 but trivial where you expected 5000. Scaling puts every category on a fair footing. Later, R. A. Fisher refined the crucial detail of degrees of freedom in the 1920s, correcting Pearson's original count when parameters are estimated from the data. Together their work gave us the first widely used hypothesis test for categorical data, and it remains a workhorse of science more than a century later.

Observed vs Expected Counts and the Statistic

Every chi-square test rests on two lists of numbers:

• Observed counts ($O$) — what your data actually show.
• Expected counts ($E$) — what the null hypothesis predicts on average for a sample of your size.

The Pearson statistic aggregates the mismatch:

$\chi^2 = \sum \frac{(O_i - E_i)^2}{E_i}$

If observed and expected agree perfectly, every term is 0 and $\chi^2 = 0$. The worse the fit, the larger $\chi^2$ grows. Because we square the deviations, over- and under-counts do not cancel, and large individual gaps are penalized heavily.

Worked example (goodness-of-fit). A candy company claims its bags are $30\%$ red, $30\%$ blue, $20\%$ green, and $20\%$ yellow. You open a bag of 200 candies and count: red 50, blue 55, green 45, yellow 50. Does the bag match the claim?

Expected counts for $n = 200$: red $0.30 \times 200 = 60$, blue $60$, green $0.20 \times 200 = 40$, yellow $40$.

Color	$O$	$E$	$(O-E)^2/E$
Red	50	60	$100/60 = 1.667$
Blue	55	60	$25/60 = 0.417$
Green	45	40	$25/40 = 0.625$
Yellow	50	40	$100/40 = 2.500$

Summing: $\chi^2 = 1.667 + 0.417 + 0.625 + 2.500 = 5.209$.

Degrees of freedom $= k - 1 = 4 - 1 = 3$. The critical value at significance level $\alpha = 0.05$ is $7.815$. Since $5.209 < 7.815$, we fail to reject the null: the data are consistent with the company's claimed color distribution.

The Goodness-of-Fit Test

Use goodness-of-fit when you have one categorical variable and a hypothesized distribution for it. The steps are always the same:

1. State $H_0$ (the data follow the claimed distribution) and $H_a$ (they do not).
2. Compute expected counts: $E_i = n \times p_i$, where $p_i$ is the hypothesized probability of category $i$.
3. Compute $\chi^2 = \sum (O-E)^2/E$.
4. Find degrees of freedom: $df = k - 1$ (subtract one more for each parameter you estimated from the data).
5. Compare to the critical value, or compute a p-value, and conclude.

Worked example (uniform die). You suspect a die is fair and roll it 60 times, getting: 1→8, 2→13, 3→9, 4→7, 5→11, 6→12. Under fairness each face is expected $60/6 = 10$ times.

$\chi^2 = \frac{(8-10)^2 + (13-10)^2 + (9-10)^2 + (7-10)^2 + (11-10)^2 + (12-10)^2}{10}$

Numerators: $4 + 9 + 1 + 9 + 1 + 4 = 28$, so $\chi^2 = 28/10 = 2.8$. With $df = 5$, the $0.05$ critical value is $11.070$. Because $2.8 < 11.070$, there is no evidence the die is unfair.

Contingency Tables and the Test of Independence

When you have two categorical variables measured on the same subjects, you cross-tabulate them into a contingency table (rows = categories of one variable, columns = categories of the other). The test of independence asks: are the two variables related, or independent?

The clever part is computing expected counts. Under independence, the expected count of a cell is:

$E = \frac{(\text{row total}) \times (\text{column total})}{\text{grand total}}$

This follows directly from the multiplication rule for independent events: if row and column classifications are independent, the probability of landing in a given cell is the product of the marginal probabilities, and multiplying by $n$ gives the expected frequency.

Worked example. A researcher surveys 200 people about whether they exercise regularly and whether they sleep well:

	Sleeps well	Sleeps poorly	Row total
Exercises	60	20	80
Does not	50	70	120
Col total	110	90	200

Expected counts:

• Exercises & sleeps well: $(80 \times 110)/200 = 44$
• Exercises & sleeps poorly: $(80 \times 90)/200 = 36$
• Does not & sleeps well: $(120 \times 110)/200 = 66$
• Does not & sleeps poorly: $(120 \times 90)/200 = 54$

Now the statistic:

$\chi^2 = \frac{(60-44)^2}{44} + \frac{(20-36)^2}{36} + \frac{(50-66)^2}{66} + \frac{(70-54)^2}{54}$

Term by term: $256/44 = 5.818$; $256/36 = 7.111$; $256/66 = 3.879$; $256/54 = 4.741$. Sum: $\chi^2 = 21.549$.

Degrees of freedom $= (r-1)(c-1) = (2-1)(2-1) = 1$. The $0.05$ critical value is $3.841$. Since $21.549 \gg 3.841$, we reject independence: exercise and sleep quality are significantly associated in this sample.

Real-World Applications

• Genetics. Testing whether offspring ratios match Mendelian predictions (e.g. a 9:3:3:1 dihybrid cross) is a classic goodness-of-fit application.
• Medicine and epidemiology. Contingency tables compare treatment vs placebo against recovered vs not, testing whether outcome depends on treatment.
• Marketing. Firms test whether product preference is independent of age group or region before targeting campaigns.
• Quality control. Manufacturers check whether defect counts across shifts or machines match an expected uniform rate.
• Social science and polling. Researchers test associations between demographic categories and opinions, such as voting preference by education level.

Common Mistakes

1. Using percentages or proportions instead of raw counts. Why wrong: the $\chi^2$ formula is calibrated for actual frequencies; feeding it percentages destroys the dependence on sample size and inflates or deflates the statistic arbitrarily. Correction: always compute $\chi^2$ from raw counts, never from rates.

2. Ignoring the expected-count rule. Why wrong: the chi-square distribution is only an approximation to the true distribution of the statistic, and it breaks down when expected cell counts are small (a common rule: no cell below 5, and all above 1). Correction: combine sparse categories or use Fisher's exact test for small samples.

3. Miscounting degrees of freedom. Why wrong: using $df = k$ instead of $k-1$, or forgetting to subtract for estimated parameters, points you to the wrong critical value and can flip your conclusion. Correction: goodness-of-fit uses $k - 1 - (\text{parameters estimated})$; independence uses $(r-1)(c-1)$.

Comparison and Connections

Chi-square tests are often confused with each other and with other tests. The table clarifies:

Test	Data	Question	Degrees of freedom
Goodness-of-fit	One categorical variable	Does it match a claimed distribution?	$k - 1$
Test of independence	Two categorical variables, one sample	Are the variables associated?	$(r-1)(c-1)$
Test of homogeneity	Same table structure, several populations sampled	Do groups share the same distribution?	$(r-1)(c-1)$

Independence and homogeneity use identical arithmetic but differ in study design (one sample cross-classified vs several fixed-size samples). Chi-square differs from the t-test and ANOVA, which compare means of numerical data; chi-square compares frequencies of categories. The $\chi^2$ distribution itself is also the sum of squared standard normal variables, which is why it appears throughout statistics, including in variance estimation.

Practice Questions

Recall

State the Pearson chi-square formula and explain what $O$ and $E$ stand for. Answer: $\chi^2 = \sum (O-E)^2/E$, where $O$ is the observed count in a category and $E$ is the expected count under the null hypothesis.

Understanding

Why do we divide each squared deviation by $E$ rather than just summing $(O-E)^2$? Answer: Dividing by $E$ scales each deviation relative to how large it was expected to be, so a gap of 10 counts as major where 5 were expected but minor where 5000 were expected. It puts categories of different sizes on a comparable footing.

Application

A geneticist expects a 3:1 ratio of tall to short plants. In 160 plants she observes 108 tall and 52 short. Compute $\chi^2$ and test at $\alpha = 0.05$ ($df = 1$, critical value $3.841$). Answer: Expected: tall $120$, short $40$. $\chi^2 = (108-120)^2/120 + (52-40)^2/40 = 144/120 + 144/40 = 1.2 + 3.6 = 4.8$. Since $4.8 > 3.841$, reject the 3:1 hypothesis.

Analysis

Your contingency test gives $\chi^2 = 9.5$ with $df = 4$ (critical value $9.488$ at $\alpha = 0.05$). A colleague says the result is "clearly significant." How should you respond? Answer: The result barely exceeds the critical value, so it is significant at $0.05$ but only marginally. The p-value is just under $0.05$, meaning the evidence is weak; one should report the p-value, check expected-count assumptions, and avoid overstating the strength of the association.

FAQ

Is the chi-square test one-tailed or two-tailed? The chi-square test is inherently one-tailed on the right. Any deviation — in either direction — makes the squared terms positive and pushes $\chi^2$ up, so we only ever reject in the upper tail.

What if my expected counts are less than 5? The approximation becomes unreliable. Combine adjacent categories, collect more data, or use Fisher's exact test (especially for 2×2 tables), which computes exact probabilities.

Does a significant result mean the variables cause each other? No. Chi-square detects association, not causation. A significant test of independence tells you the variables are statistically related, but a confounding factor could drive both.

What is the difference between the test of independence and homogeneity? The math is identical. Independence starts from one sample cross-classified by two variables; homogeneity starts from several separate samples compared on one variable. The interpretation and sampling design differ, not the computation.

How large does $\chi^2$ have to be to reject the null? There is no universal threshold — it depends on the degrees of freedom. Look up the critical value for your $df$ and chosen $\alpha$, or compute the p-value. More degrees of freedom require a larger $\chi^2$ to reach significance.

Quick Revision

• Statistic: $\chi^2 = \sum (O-E)^2 / E$; always uses raw counts.
• Goodness-of-fit: $E_i = n p_i$; $df = k - 1$.
• Independence: $E = (\text{row total} \times \text{column total}) / \text{grand total}$; $df = (r-1)(c-1)$.
• Reject $H_0$ when $\chi^2 >$ critical value (right tail only).
• Assumptions: independent observations, expected counts $\geq 5$ in (nearly) all cells.
• Detects association, not causation; introduced by Karl Pearson in 1900.

Related Topics

Prerequisites

• Statistics and Probability overview

Related Topics

• Hypothesis Testing (significance levels, p-values, and critical values)
• Probability Distributions (the chi-square distribution)

Next Topics

• Analysis of Variance (ANOVA) for comparing means across groups
• Correlation and regression for numerical associations