Priors and Model Comparison

Default priors in brms

prior_summary(M_5)
get_prior(weight ~ height, data = weight_df)

• Weakly informative by default
• Student-t or normal centred at zero for regression coefficients
• Half-Student-t for sigma (keeps it positive)
• These regularise without strongly influencing the posterior

Setting custom priors

M_6 <- brm(weight ~ height + gender,
  prior = set_prior("normal(0, 10)"),
  save_pars = save_pars(all = TRUE),
  data = weight_df
)

A single set_prior call applies the same prior to all coefficients.

Detailed prior specification

new_priors <- c(
  set_prior("normal(0, 20)", class = "b", coef = "height"),
  set_prior("normal(50, 25)", class = "Intercept"),
  set_prior("student_t(1, 0, 50)", class = "sigma")
)

Different parameters can have different priors.

Prior shapes

[Diagram: left panel shows an irregular blob (arbitrary prior); right panel shows a normal curve with the density formula below it]

Priors can take any shape. Conjugate families (like beta for Bernoulli, normal for linear models) allow analytical posteriors. MCMC makes non-conjugate priors equally tractable.

Prior sensitivity analysis

mcmc_plot(M_5, type = "hist", variable = c("b_gendermale", "b_height"))
mcmc_plot(M_6, type = "hist", variable = c("b_gendermale", "b_height"))

If posteriors are similar across different reasonable priors: inference is robust. If they differ substantially: prior assumptions matter and should be examined.

Leave-one-out cross-validation

\[\text{elpd} = \sum_{i=1}^{N} \log p(x_i \mid x_{-i}, m)\]

\[\text{LOOIC} = -2 \times \text{elpd}\]

• Leave each observation out in turn
• Measure how well the model predicts the held-out observation
• Lower LOOIC = better estimated predictive accuracy

LOO and WAIC in practice

loo(M_10, M_11)
waic(M_10, M_11)

Comparison table shows ELPD difference and standard error. A difference greater than two standard errors is worth acting on.

Bayes factors

\[\text{BF} = \frac{p(D \mid M_0)}{p(D \mid M_1)}\]

• Ratio of marginal likelihoods
• \(\text{BF} > 1\): data favour \(M_0\)
• Naturally expressed on the log scale

bayes_factor(M_10, M_11, log = TRUE)

\[\log \text{BF} = 108 \implies \text{BF} = e^{108}\]

Overwhelming evidence.

LOO vs Bayes factors

Summary

• Default priors in brms are weakly informative; check them with prior_summary
• Custom priors are set with set_prior at any level of specificity
• Prior sensitivity analysis compares posteriors under different priors
• LOO and WAIC measure predictive accuracy; Bayes factors compare marginal likelihoods