Week 10: Assumptions + Diagnostics
Date: October 29, 2026
Today…
Is the model telling the truth?
One function, six panels
Each panel answers one question. Let’s take them one at a time.
1. Posterior predictive check
Question: If the model were true, would it produce data that look like ours?
- Green = the real outcome. Blue = outcomes simulated from the fitted model.
- Good: green sits inside the blue
- Bad: green has a peak, skew, or floor the blue lines miss
Here: close. Depression scores are a little more peaked than the model expects, but nothing alarming.
2. Linearity
Question: Is the straight line the right shape?
- Residuals against fitted values (the Week 7 plot)
- Good: the green line is flat at zero
- Bad: a curve. The model is too low at the ends and too high in the middle, or the reverse
Here: flat. ✅
3. Homogeneity of variance
Question: Is the model equally wrong for everyone?
- \(\sqrt{|\text{standardized residual}|}\) against fitted values
- Good: flat green line, even band
- Bad: a line that clearly rises or falls (the funnel from Week 7)
Here: a gentle downward drift. The model misses a little less for kids with high predicted depression. Hold that thought.
4. Influential observations
Question: Is any single person running the model?
- Leverage (how unusual someone’s predictors are) against residual (how badly the model misses them)
- Good: every point inside the dashed contours
- Bad: a labeled point outside them. That case has a large Cook’s distance
Here: nobody close. ✅
5. Normality of residuals
Question: Are the leftovers roughly bell-shaped?
- A Q-Q plot, “detrended” so a perfect fit is a flat line
- Good: dots hug the line, maybe wandering a little at the very ends
- Bad: an S-shape (skew) or tails that peel away (outliers, heavy tails)
Here: fine. ✅ And with n = 400, the Central Limit Theorem covers a lot of sins here.
6. Collinearity
Question: Are the predictors stepping on each other? (Week 8)
- VIF = variance inflation factor. 1 = no overlap at all
- Rough guide: < 5 fine, 5–10 worth a look, > 10 a problem
Here: VIF = 1.02. Sleep and peer stress barely overlap (r = −.13). ✅
Every panel is unhappy
Why: streams grow by multiplying, not adding
Code
p1 <- spotify %>%
ggplot(aes(x = streams_m)) +
geom_histogram(bins = 40, fill = "grey60", color = "white") +
labs(x = "Streams (millions)", y = "Songs", title = "Raw") +
theme_bw(base_size = 15)
p2 <- spotify %>%
ggplot(aes(x = log10(streams))) +
geom_histogram(bins = 40, fill = "grey60", color = "white") +
labs(x = "log10(streams)", y = "Songs", title = "Logged") +
theme_bw(base_size = 15)
p1 + p2
A song going from 1M to 2M streams is the same kind of change as 500M to 1,000M: it doubled. Counts like this (streams, income, reaction times, followers, reported drinks) usually behave better on a log scale.
Same data, different story
Flat lines, an even band, a bell-shaped PP check, nobody near the contours. Except one point. We’ll get to it.
⚠️ The tests answer the wrong question
A significance test asks “can I detect any departure at all?” That depends on n, not on whether the departure matters.
Code
set.seed(640)
# Small sample, badly skewed errors
p_small <- replicate(1000, {
x <- rnorm(15); y <- x + (rexp(15) - 1)
shapiro.test(resid(lm(y ~ x)))$p.value
})
# Huge sample, nearly normal errors (t with 10 df)
p_big <- replicate(200, {
x <- rnorm(5000); y <- x + rt(5000, df = 10)
shapiro.test(resid(lm(y ~ x)))$p.value
})
x <- rnorm(15); y <- x + (rexp(15) - 1)
q1 <- tibble(r = resid(lm(y ~ x))) %>%
ggplot(aes(sample = r)) + stat_qq() + stat_qq_line() +
labs(title = paste0("n = 15, badly skewed\nflagged in ",
round(100 * mean(p_small < .05)), "% of samples")) +
theme_bw(base_size = 14)
x <- rnorm(5000); y <- x + rt(5000, df = 10)
q2 <- tibble(r = resid(lm(y ~ x))) %>%
ggplot(aes(sample = r)) + stat_qq(alpha = .2) + stat_qq_line() +
labs(title = paste0("n = 5,000, nearly normal\nflagged in ",
round(100 * mean(p_big < .05)), "% of samples")) +
theme_bw(base_size = 14)
q1 + q2
Small samples: real problems slip through. Large samples: harmless wiggles get flagged. Look at the plots. Use the tests as a second opinion, never as the verdict.
💻 Your Turn: Name that violation (if time)
Tip
~10 min · Pairs
Each panel comes from a different model. For each one: which assumption, what’s wrong, what would you do?
Code
set.seed(1029)
n <- 250
x <- runif(n, 1, 10)
mA <- lm(y ~ x, data = tibble(x, y = 2 + 3 * x - 0.3 * x^2 + rnorm(n, 0, 1.5)))
mB <- lm(y ~ x, data = tibble(x, y = exp(0.3 * x + rnorm(n, 0, 0.45))))
dC <- tibble(x = c(x[1:39], 25), y = c(2 + 0.5 * x[1:39] + rnorm(39, 0, 1.5), -4))
mC <- lm(y ~ x, data = dC)
x1 <- rnorm(n); x2 <- x1 + rnorm(n, 0, 0.2); x3 <- rnorm(n)
mD <- lm(y ~ x1 + x2 + x3, data = tibble(x1, x2, x3, y = x1 + x3 + rnorm(n)))
pA <- plot(check_model(mA, check = "linearity", panel = FALSE))[[1]] + labs(title = "A")
pB <- plot(check_model(mB, check = "qq", panel = FALSE))[[1]] + labs(title = "B")
pC <- plot(check_model(mC, check = "outliers", panel = FALSE))[[1]] + labs(title = "C")
pD <- plot(check_model(mD, check = "vif", panel = FALSE))[[1]] + labs(title = "D")
(pA + pB) / (pC + pD)
