Bayesian Linear Regression

The Bayesian linear model

\[y_i \sim \mathrm{N}(\mu_i, \sigma^2)\] \[\mu_i = \beta_0 + \beta_1 x_i\]

Parameters \(\beta_0, \beta_1, \sigma^2\) receive prior distributions. MCMC samples from the posterior over all parameters jointly.

Fitting the model

M_5 <- brm(weight ~ height + gender,
  save_pars = save_pars(all = TRUE),
  data = weight_df
)
M_5

The summary output reports posterior mean, SD, and credible intervals for each parameter.

Reading the summary output

• Estimate: posterior mean
• Est.Error: posterior standard deviation
• l-95% CI / u-95% CI: 95% credible interval
• Rhat: convergence diagnostic (should be \(\approx 1.00\))
• Bulk_ESS / Tail_ESS: effective sample size

Trace plots

plot(M_5)

[Diagram: four coloured wavy lines running horizontally — the “caterpillar” trace plot]

A well-mixed chain shows no trends, no stuck periods. All four chains should overlap in the same region.

Posterior distributions

mcmc_plot(M_5, type = "hist")
mcmc_plot(M_5, type = "dens")
mcmc_plot(M_5, type = "areas")

Each parameter’s marginal posterior is a distribution, not a point estimate. The width of the distribution reflects remaining uncertainty.

Working with posterior samples directly

draws <- as_draws_df(M_5)

mean(draws$b_height)
quantile(draws$b_height, probs = c(0.025, 0.975))
mean(draws$b_height > 0)

Any posterior quantity can be computed from the samples.

Joint posterior

[Diagram: 2D contour plot of joint posterior over two parameters. Concentric oval contours showing correlation between parameters.]

The joint posterior captures correlations between parameters. Marginal summaries do not.

Effective sample size

• MCMC samples are autocorrelated; they are not independent
• Effective sample size (ESS) measures equivalent number of independent samples
• Target: Bulk_ESS and Tail_ESS both above 400 per chain
• Increase iterations if ESS is too low

Fixed effects summary

fixef(M_5)
prior_summary(M_5)
stancode(M_5)

Summary

• brm fits Bayesian regression using the same formula syntax as lm
• The output is a posterior distribution, not a point estimate
• Trace plots, Rhat, and ESS are the primary convergence diagnostics
• Posterior samples give direct access to any quantity of interest