Ordinal Logistic Regression

Ordinal outcome variables

• An ordinal variable has values that can be ranked but not measured on a common scale
• Examples: Likert responses, pain ratings, educational attainment
• Using a normal linear model implies equal spacing between levels — rarely defensible
• Treating as nominal ignores the ordering

Latent variable formulation of binary logistic regression

Binary logistic regression is equivalent to a latent variable model:

\[ \begin{aligned} z_i &\sim \mathrm{Logistic}\!\left(\beta_0 + \sum_k \beta_k x_{ki},\ 1\right)\\ y_i &= \begin{cases} 1 & \text{if } z_i \geq 0 \\ 0 & \text{if } z_i < 0 \end{cases} \end{aligned} \]

The logistic distribution is the key distribution here

Extension to ordinal outcomes

For \(L\) ordered values, introduce \(L-1\) cutpoints \(\zeta_1 < \zeta_2 < \cdots < \zeta_{L-1}\):

\[ y_i = l \quad \text{if} \quad \zeta_{l-1} \leq z_i < \zeta_l \]

where \(\zeta_0 = -\infty\) and \(\zeta_L = \infty\)

• The same regression coefficients shift the latent distribution
• The cutpoints divide the real line into \(L\) regions

The cumulative logit model

An equivalent formulation in terms of cumulative probabilities:

\[ \log\!\left(\frac{\Pr(y_i \leq l)}{\Pr(y_i > l)}\right) = \zeta_l - \sum_k \beta_k x_{ki} \]

• The same \(\beta\) coefficients for all thresholds (proportional odds assumption)
• A positive \(\beta_k\) shifts the latent distribution upward, raising the probability of higher values

Fitting with polr

library(MASS)
data("admit", package = "pscl")

M_8 <- polr(score ~ gre.quant, data = admit)
summary(M_8)

• Intercepts in the output are the cutpoints \(\zeta_l\)
• Coefficient for gre.quant is on the log-cumulative-odds scale

Predicted probabilities

admit_new <- tibble(gre.quant = seq(300, 800, by = 100))
add_predictions(admit_new, M_8, type = "prob")

The predictions give the probability for each ordered category

Fitting with clm (ordinal package)

library(ordinal)

M_8b <- clm(score ~ gre.quant, data = admit)
summary(M_8b)

• Equivalent model, more flexible extensions available
• clm supports partial proportional odds, flexible link functions, and scale models

The standard logistic distribution

• A Logistic(0, 1) distribution
• Symmetric, bell-shaped, heavier tails than the normal
• The CDF is the ilogit function: \(F(x) = \mathrm{ilogit}(x) = \dfrac{1}{1+e^{-x}}\)

Summary

• Ordinal logistic regression extends binary logistic regression via a latent variable argument
• Cutpoints \(\zeta_l\) divide the latent scale into \(L\) ordered regions
• Proportional odds: same \(\beta\) coefficients for all thresholds
• polr (MASS) and clm (ordinal) both fit the cumulative logit model