Beyond Normal Linear Models

The normal model’s assumptions

\[y_i \sim \mathrm{N}(\mu_i, \sigma^2)\]

Two assumptions embedded here:

1. Residuals are normally distributed
2. Variance \(\sigma^2\) is constant across all observations (homoskedasticity)

Both frequently fail in practice.

The problem with outliers

• The normal distribution has thin tails
• Outliers are very improbable under normality
• The model tries hard to accommodate them, distorting all other estimates
• Solution: heavier-tailed residual distribution

Student-t regression

Replace the normal with a Student-t:

\[y_i \sim \mathrm{t}(\nu, \mu_i, \sigma^2)\]

• \(\nu\): degrees of freedom (estimated from data)
• Small \(\nu\): heavy tails, robust to outliers
• As \(\nu \to \infty\): approaches the normal
• In brms: family = student()

Fitting a robust regression

M_12 <- brm(
  bf(weight ~ height + gender + race,
     sigma ~ height + gender + race),
  family = student(),
  save_pars = save_pars(all = TRUE),
  data = weight_df
)

Distributional regression

What if variance is not constant?

• Model \(\sigma\) as a function of predictors
• Use bf() to specify a formula for sigma
• The log link keeps \(\sigma\) positive

The model now estimates: how does variability in the outcome change with the predictors?

The bf() syntax

bf(
  weight ~ height + gender + race,   # model for the mean
  sigma ~ height + gender + race     # model for the variance
)

Separate coefficients are estimated for both equations.

Posterior predictive checks

pp_check(M_10)   # normal model
pp_check(M_12)   # Student-t with distributional sigma

• Draw datasets from the posterior predictive distribution
• Overlay on the observed data density
• Poor fit: simulated data look unlike real data

Comparing models

waic(M_12, M_10)

Lower WAIC: better estimated out-of-sample predictive accuracy. The pp_check tells you why one model fits better.

Summary

• Normal linear models are sensitive to outliers and assume constant variance
• Student-t regression with estimated degrees of freedom is robust to outliers
• Distributional regression models \(\sigma\) as a function of predictors
• Posterior predictive checks visualise model adequacy
• WAIC quantifies the predictive advantage of extended models