Statistical Analysis in Psychology

Learning Objectives

By the end of this topic, you should be able to:

• Calculate and interpret the mean, median, mode, and standard deviation of a dataset
• Distinguish descriptive statistics from inferential statistics and explain what each is used for
• Explain the logic of hypothesis testing, including the null and alternative hypotheses
• Select the appropriate statistical test (t-test, ANOVA, correlation, regression) for a given research scenario
• Interpret a correlation coefficient in terms of strength and direction
• Explain why statistical significance does not automatically mean practical importance

Quick Answer

Statistical analysis is how psychologists turn raw numbers into evidence. Descriptive statistics (mean, median, mode, standard deviation) summarize what a dataset looks like. Inferential statistics (t-tests, ANOVA, correlation, regression) let researchers go further — testing whether a pattern observed in a sample is likely to reflect something real in the broader population, or could just be due to chance. Every inferential test is built on the same underlying logic: assume there's no real effect (the null hypothesis), then ask how surprising the actual data would be if that assumption were true. If the data would be very surprising, researchers reject the null hypothesis in favor of a real effect. This logic — not the formulas themselves — is what matters most for understanding psychological research.

Descriptive Statistics: Summarizing the Data

Before testing anything, researchers need to describe what the data looks like.

Measures of central tendency capture the "typical" value:

• Mean — the arithmetic average; sensitive to outliers.
• Median — the middle value when data is ordered; resistant to outliers.
• Mode — the most frequent value; the only measure usable with categorical data.

Measures of variability capture how spread out the data is:

• Range — the simplest measure: highest minus lowest value.
• Standard deviation — roughly, the average distance of each data point from the mean; a small standard deviation means scores cluster tightly around the mean, a large one means they're spread widely.

Worked example. Exam scores: 85, 90, 75, 80, 95.

• Mean = (85+90+75+80+95) / 5 = 85
• Median = 85 (the middle value when ordered: 75, 80, 85, 90, 95)
• Mode = none (all values are unique)
• Standard deviation: squared deviations from the mean are 0, 25, 100, 25, 100 → variance = 250/5 = 50 → SD = √50 ≈ 7.07

Notice mean and median are identical here because the scores are roughly symmetric — when a dataset is skewed by outliers, mean and median diverge, and the median becomes the more honest "typical value."

Inferential Statistics: Testing Hypotheses

Descriptive statistics describe a sample. Inferential statistics let researchers generalize from that sample to a population, and they always start with two competing claims:

• Null hypothesis (H₀) — there is no real effect or relationship; any pattern in the sample is due to chance.
• Alternative hypothesis (H₁) — there is a real effect or relationship.

A statistical test calculates a p-value: the probability of observing data this extreme (or more extreme) if the null hypothesis were actually true. By convention, psychology uses p < .05 as the threshold for statistical significance — meaning there's less than a 5% chance the observed pattern is a fluke, assuming no real effect exists. Crucially, a significant result doesn't prove H₁ is true; it means the evidence is strong enough that chance alone is an unsatisfying explanation.

t-tests

A t-test compares the means of exactly two groups to determine whether their difference is statistically significant. Example: comparing exam scores between Class A and Class B to see whether one teaching method outperforms another. If the calculated p-value is below .05, the difference is unlikely to be due to chance alone.

ANOVA

Analysis of Variance (ANOVA) extends the t-test's logic to three or more groups at once, testing whether at least one group differs significantly from the others. Example: comparing anxiety reduction across three therapy methods. Running separate t-tests on every pair of groups instead would inflate the chance of a false positive — ANOVA solves this by testing all groups in a single analysis.

Correlation

Correlation measures the strength and direction of a linear relationship between two variables, summarized by the correlation coefficient r, which ranges from −1 to +1.

• r near +1 — strong positive relationship (both variables increase together).
• r near −1 — strong negative relationship (one increases as the other decreases).
• r near 0 — little to no linear relationship.

Example: r = 0.8 between study hours and exam scores indicates a strong positive relationship — but, as covered in earlier chapters, correlation never by itself establishes causation.

Regression

Regression builds on correlation by using one or more variables to actually predict the value of another. Example: predicting a student's exam score from hours studied. Regression produces an equation that can generate a specific predicted score for a new value of the predictor, which correlation alone cannot do.

Why Significance Isn't the Whole Story

A result can be statistically significant yet practically trivial — with a large enough sample, even a tiny, meaningless difference can produce p < .05. This is why researchers increasingly report effect size (how large the difference or relationship actually is) alongside significance. A drug that "significantly" reduces symptoms by an amount patients can't even notice is not clinically meaningful, no matter how small the p-value. Reading research critically means always asking both "is this significant?" and "is this big enough to matter?"

Real-World Applications

Statistical literacy lets clinicians judge whether a new treatment's reported benefit is real or a small, hyped-up effect. It lets policy analysts decide whether a program's outcomes justify its cost. It lets any consumer of research headlines ("Study finds X causes Y!") ask the right follow-up questions: was this correlational or experimental, what was the effect size, and could this be a false positive from testing many things at once? Every psychology student eventually needs enough statistical fluency to read a results section and know what it actually supports.

Key Terms

Term	Definition	Related Concept
Mean	The arithmetic average of a dataset	Central Tendency
Median	The middle value in an ordered dataset	Central Tendency, Outlier Resistance
Mode	The most frequently occurring value	Central Tendency, Categorical Data
Standard Deviation	A measure of how spread out data points are from the mean	Variability
Null Hypothesis (H₀)	The assumption that no real effect or relationship exists	Alternative Hypothesis
Alternative Hypothesis (H₁)	The claim that a real effect or relationship exists	Null Hypothesis
P-value	The probability of the observed data (or more extreme) if H₀ were true	Statistical Significance
Statistical Significance	A result unlikely to have occurred by chance (conventionally p < .05)	P-value
t-test	A test comparing the means of two groups	ANOVA
ANOVA	A test comparing the means of three or more groups	t-test
Correlation Coefficient (r)	A number from −1 to +1 indicating strength/direction of a linear relationship	Regression
Regression	A technique for predicting one variable's value from another	Correlation
Effect Size	A measure of the magnitude of a difference or relationship, independent of sample size	Statistical Significance

Common Mistakes

Misconception: A statistically significant result (p < .05) proves the effect is real and important. Why it's wrong: Significance only tells you the result is unlikely to be due to chance; it says nothing about how large or practically meaningful the effect is, especially with large samples where even trivial effects reach significance. Correct understanding: Always evaluate significance alongside effect size — a significant but tiny effect may have no real-world importance, while a large, meaningful effect in a small study might not reach significance at all.

---

Misconception: The mean is always the best measure of "typical" performance. Why it's wrong: The mean is heavily influenced by extreme values (outliers). In a skewed dataset — like income, where a few very high earners pull the average up — the mean can misrepresent what's typical for most people. Correct understanding: The median is more resistant to outliers and often better represents "typical" in skewed distributions; researchers report both when a dataset might be skewed.

---

Misconception: A strong correlation coefficient (e.g., r = 0.8) means one variable causes the other. Why it's wrong: Correlation coefficients quantify the strength of a relationship, not its cause — a third variable or reverse causation can produce a strong r just as easily as a true causal link. Correct understanding: Only a properly controlled experiment with random assignment can support causal claims; a high r is an invitation to investigate further, not proof of causation.

Comparison and Connections

Technique	Compares	Number of Groups/Variables	Answers
t-test	Two group means	2 groups	Do these two groups differ significantly?
ANOVA	Three or more group means	3+ groups	Does at least one group differ significantly?
Correlation	Two continuous variables	2 variables	How strongly and in what direction are they related?
Regression	Predictor(s) and an outcome	1+ predictors, 1 outcome	What value of Y do we predict given X?

Practice Questions

Recall

1. Define mean, median, and mode, and state which is best suited to categorical data. Answer guidance: Mean = average; median = middle value; mode = most frequent value; mode is the only one usable for categorical (non-numeric) data.

2. What does a p-value of .03 mean in the context of a hypothesis test? Answer guidance: There's a 3% probability of observing data this extreme (or more extreme) if the null hypothesis were true — below the conventional .05 threshold, so the result is considered statistically significant.

Understanding

3. Explain why researchers use ANOVA instead of running multiple t-tests when comparing three or more groups. Answer guidance: Running multiple pairwise t-tests inflates the probability of a false positive (Type I error) across all the comparisons; ANOVA tests all groups in one analysis, controlling this error rate.

4. Why is standard deviation a more informative statistic than the mean alone? Answer guidance: The mean only describes central tendency; standard deviation describes spread, so two datasets can share the same mean but look very different (tightly clustered vs. widely scattered) — SD captures that difference.

Application

5. A researcher finds that a new therapy produces a statistically significant reduction in anxiety scores (p = .04) in a sample of 5,000 patients, but the average improvement is just half a point on a 100-point scale. How should this result be interpreted? Answer guidance: Statistically significant but likely not practically meaningful — the large sample size makes even a trivial effect reach significance; effect size should be reported and considered before recommending the therapy.

6. A psychologist wants to know if hours of sleep predict next-day mood scores, and wants to generate a specific predicted mood score for a given amount of sleep. Which technique should they use, and why not just correlation? Answer guidance: Regression, because it produces a predictive equation for specific values; correlation only tells you the strength/direction of the relationship, not a predicted score.

Analysis

7. A dataset of household incomes has a mean of $95,000 but a median of $62,000. Analyze what this discrepancy tells you about the distribution and which statistic better represents a "typical" household. Answer guidance: The large gap suggests a right-skewed distribution with a small number of very high earners pulling the mean up; the median better represents the typical household since it resists the influence of outliers.

8. Two studies test the same drug: Study A (n=20) finds a large effect that is not statistically significant; Study B (n=2000) finds a tiny effect that is statistically significant. Analyze what conclusions can and cannot be drawn from each. Answer guidance: Study A may be underpowered — the true effect could be real but the sample too small to detect it reliably; Study B demonstrates the effect is probably real but may be too small to matter practically. Neither study alone tells the full story about both reality and practical importance.

FAQ

Why is p < .05 the standard cutoff, and is it arbitrary? It's a convention established early in the history of statistics (largely credited to Ronald Fisher) rather than a law of nature — there's nothing magical about 5%. Critics have long argued it encourages a rigid "significant vs. not significant" mindset rather than treating evidence as a continuum. Some fields now use stricter thresholds or, increasingly, report effect sizes and confidence intervals alongside or instead of a bare significance verdict.

What's the difference between descriptive and inferential statistics, in plain terms? Descriptive statistics summarize the data you actually collected — the mean age of your sample, for instance. Inferential statistics use that sample data to make a claim about a larger population you didn't fully measure — for example, inferring that a drug likely works better than placebo across all patients, not just the ones in your study. Descriptive statistics describe; inferential statistics generalize.

Can a correlation ever be used to make causal claims? Not on its own, no matter how strong it is — this is one of the most repeated warnings in psychology precisely because the temptation is so strong. However, correlational data can support causal claims when combined with additional evidence: a plausible mechanism, longitudinal data showing the right time order, and ideally an experimental replication that manipulates the variable directly.

Why does sample size matter so much for statistical significance? Larger samples reduce random sampling error, making it easier to detect real effects reliably — but this cuts both ways. With a large enough sample, even a genuinely tiny and unimportant effect will eventually reach statistical significance, which is exactly why effect size matters alongside the p-value.

Do I need to memorize the formulas for these tests? For most introductory psychology courses, understanding when to use each test and how to interpret its output (a p-value, an r value, a predicted score) matters far more than memorizing the underlying computational formula, since statistical software does the calculation in real research. Focus study time on matching test to scenario and interpreting results correctly.

Quick Revision

• Descriptive statistics summarize a sample; inferential statistics generalize from a sample to a population
• Mean = average (sensitive to outliers); median = middle value (resistant to outliers); mode = most frequent value
• Standard deviation measures spread around the mean
• Null hypothesis (H₀) = no real effect; alternative hypothesis (H₁) = a real effect exists
• P-value = probability of the observed data if H₀ were true; p < .05 is the conventional significance threshold
• t-test compares two group means; ANOVA compares three or more group means
• ANOVA avoids the inflated false-positive risk of running many separate t-tests
• Correlation coefficient r (−1 to +1) shows strength and direction of a linear relationship, not causation
• Regression predicts a specific value of one variable from another
• Statistical significance does not equal practical importance — always check effect size too
• Large samples can make even trivial effects statistically significant
• Only controlled experiments with random assignment can support causal claims, not correlation or regression

Related Topics

Prerequisites

• Introduction to Research Methods
• Experimental Design
• Basic Mathematics/Algebra

Related Topics

• Data Collection Methods
• Qualitative Research

Next Topics

• Qualitative Research (thematic analysis as an alternative to numeric analysis)
• Research Ethics (data integrity and reporting standards)