How do you model a yes/no outcome with several factors?

Start here

Why can't a straight line model a yes/no outcome?

Each patient's outcome is 0 or 1, and what you want to predict is the probability of 1. A straight line has no upper or lower limit, so at extreme values of the predictor it predicts probabilities below 0 or above 1.

Logistic regression changes the scale of the outcome in two steps. The probability p becomes the odds, p ÷ (1 − p), which runs from 0 to infinity. The odds become the log-odds, ln[p ÷ (1 − p)], which has no upper or lower limit. A straight line is allowed on that scale. The log-odds is called the logit.

ln[p ÷ (1 − p)] = b₀ + b₁x₁ + b₂x₂ + … + bₖxₖ

p = 1 ÷ [1 + e^{−(b₀ + b₁x₁ + … + bₖxₖ)}]

b₀

intercept: the log-odds when every predictor is 0

bᵢ

change in log-odds for a 1-unit rise in xᵢ, with the other predictors held constant

e^bᵢ

odds ratio for a 1-unit rise in xᵢ, adjusted for the other predictors

The coefficients are estimated by maximum likelihood: the software finds the values that make the observed pattern of 0s and 1s most probable. The S-curve and the odds ratio from a single predictor are introduced in the correlation and regression chapter.

Interactive

Move the intercept and the coefficient and watch the S-curve, the risks and the odds ratio

The odds ratio for a given change in age is the same at every age. The change in risk is not.

Reading the output

What does each coefficient mean when there are several predictors?

Each coefficient is the change in log-odds for a 1-unit rise in that predictor while the other predictors stay the same. Its exponential, e to the power b, is the adjusted odds ratio.

Per unit: choose a unit that means something

The age odds ratio of 1.049 is per year. For a 10-year difference, multiply the coefficient by 10 before taking the exponential: e^{10 × 0.0480} = 1.62 (95% CI 1.27 to 2.07). The albumin odds ratio of 0.940 is per g/L. For albumin 5 g/L lower: e^{−5 × −0.0621} = 1.36 (1.03 to 1.82). Scale the confidence limits the same way: e^{k × (b ± 1.96 × SE)} for a change of k units.

Per category: every comparison is against a reference

A predictor with k categories enters the model as k − 1 indicator (dummy) variables. Each coefficient compares one category with the reference category, not with the average patient.

Choose a reference category that has enough patients, because every comparison uses it. Changing the reference changes every odds ratio. Entering an ordered category as one score (I = 1, II = 2, III = 3, IV = 4) assumes the same odds ratio for every one-grade step.

Adjusted and unadjusted odds ratios

In the teaching data, albumin alone gives an odds ratio of 0.956 per g/L (95% CI 0.906 to 1.008). Adjusted for age, smoking and ASA grade it is 0.940 (0.888 to 0.995). An adjusted odds ratio compares patients who have the same values of the other predictors. STROBE item 16a asks for both. Without an interaction term the model also assumes that effects multiply: smoking multiplies the odds by 2.45 whatever the ASA grade. An interaction term (smoking × ASA) tests that assumption.

Measurements as predictors

How should age, albumin or BMI enter the model?

Entered as it is, a measurement is assumed to have a straight-line relation with the log-odds: each extra unit adds the same amount at every level. This assumption is called linearity in the logit, and it needs checking.

Ways to check linearity

1. Group the measurement into four or more categories, fit them as indicators, and plot the log-odds (or coefficients) against the category midpoints.
2. Add a squared term (x²) or ln x and compare the models with a likelihood ratio test.
3. Fit a flexible shape with restricted cubic splines or fractional polynomials. The BLISS study in this chapter considered fractional polynomials for each continuous predictor.

Cutting a measurement into “high” and “low” throws away the differences within each group and assumes that every patient on one side of the cut-off has the same risk. If you do categorise, report the category boundaries (STROBE item 16b).

Model fit

Is the model as a whole any use?

The likelihood is the probability of the observed outcomes under the model. Software reports −2 log-likelihood (−2LL), also called the deviance: the lower it is, the closer the model fits the data.

The likelihood ratio test

LR χ² = (−2LL of the smaller model) − (−2LL of the larger model), df = number of added parameters

The same test compares any two nested models. Removing albumin from the teaching model raises −2LL by 4.61 on 1 degree of freedom (p = 0.032). The Wald test on a single coefficient, (b ÷ SE)², usually agrees. For smoking it is (0.8954 ÷ 0.2803)² = 10.20.

Pseudo-R²

McFadden R² = 1 − ln L₁ ÷ ln L₀

Cox & Snell R² = 1 − (L₀ ÷ L₁)^2/n   Nagelkerke R² = Cox & Snell R² ÷ [1 − L₀^2/n]

L₀

likelihood of the model with no predictors

L₁

likelihood of the fitted model

n

number of patients

For the teaching model McFadden R² = 0.108, Cox & Snell R² = 0.096 and Nagelkerke R² = 0.158: three versions give three different numbers for one model. None of them is the proportion of variance explained, as R² is in linear regression. In the same data the C statistic is 0.720. Use pseudo-R² to compare models fitted to the same patients.

Akaike information criterion

AIC = −2LL + 2 × number of parameters. For the teaching model 333.77 + 2 × 5 = 343.77. Adding predictors one at a time gives AIC 364.80, 356.74, 346.38 and 343.77. A lower AIC means a better balance between fit and number of parameters, for models fitted to the same data.

Model fit

What does the Hosmer–Lemeshow test tell you, and what does it miss?

It sorts patients by predicted risk, splits them into groups (usually 10), and compares the observed and expected numbers of events in each group with a χ² statistic on g − 2 degrees of freedom.

Teaching model: χ² = 6.58 on 8 degrees of freedom, p = 0.58. The expected events in a group are the sum of the predicted risks of its patients, and the non-events are compared in the same way.

Its limitations

• A p above 0.05 does not show that the risks are right. It shows only that the test did not detect a difference. Van Calster and colleagues (2019) point out that it groups patients artificially, gives a p-value that says nothing about the type or size of miscalibration, and has low power. They recommend against using it to assess calibration.
• The result depends on the number of groups. The same teaching model gives p = 0.34 with 5 groups, 0.30 with 8, 0.58 with 10, 0.08 with 12 and 0.40 with 15.
• The result depends on the sample size. In the interactive element below, the same small miscalibration gives p = 0.57 with 250 patients and p < 0.001 with 5,000.

Report a calibration plot with the calibration slope and calibration-in-the-large instead.

Interactive

Change the sample size and the number of groups and watch the Hosmer–Lemeshow p-value

The model's miscalibration is identical at every setting. Only the sample size and the grouping change.

Model performance

Does the model separate patients, and are its risks correct?

These are two different questions. Discrimination asks whether patients who have the event get higher predicted risks than patients who do not. Calibration asks whether a predicted risk of 20% means that about 20 in 100 such patients have the event.

Discrimination: the C statistic

The C statistic equals the area under the ROC curve of the predicted risks. 0.5 means the model ranks patients no better than chance; 1 means every patient with the event has a higher predicted risk than every patient without. The teaching model has C = 0.72 (95% CI 0.65 to 0.79). Diagnostic test ROC curves are explained in the diagnostic tests chapter.

Calibration: plot, intercept and slope

• Calibration plot: observed risk against predicted risk, in groups or as a smoothed curve. Points on the diagonal mean agreement.
• Calibration-in-the-large: whether predicted risks are too high or too low on average. The ideal value is 0. A negative value means the model predicts too many events.
• Calibration slope: the spread of the predicted risks. The ideal value is 1. A slope below 1 means the predicted risks are too extreme: too low for low-risk patients and too high for high-risk patients.
• Expected ÷ observed events: the ideal value is 1.

Interactive

Shift and spread the predicted risks and watch the C statistic stay the same while the calibration plot moves

A model built where the outcome was more common predicts too high a risk for every patient. It still ranks the patients in the same order.

Sample size

How many events do you need, and what if one predictor predicts perfectly?

Rules of thumb for logistic regression count events, not patients: the number in the smaller outcome group. The teaching data have 400 patients but 71 events, which is 17.8 events for each of the 4 predictor coefficients.

Events per variable

Peduzzi and colleagues (1996) simulated logistic models and found that with fewer than 10 events per variable the coefficients were biased in both directions and the confidence limits did not have the proper coverage. With 10 or more no major problems occurred. van Smeden and colleagues (2016) concluded that the evidence behind such rules is weak: the problems also depend on other factors, such as the total sample size, and on a few data sets in which the outcome is perfectly predicted (separation). Count every coefficient you considered, including each indicator variable and any squared term. The BLISS authors, for example, planned for about 10 events per candidate predictor.

Separation

Separation occurs when a predictor, or a combination of predictors, perfectly separates patients with and without the outcome in the sample. Heinze and Schemper (2002) showed that Firth's penalised likelihood gives finite estimates in this situation, and that penalised likelihood ratio tests and profile penalised likelihood confidence intervals are often preferable to Wald tests. Warning signs are an odds ratio in the hundreds or thousands, a huge standard error, or a category with no events.

Matched case-control studies

What if each case was matched to a control?

Matching makes each case and its controls a set. Ordinary logistic regression ignores the sets. Conditional logistic regression compares each case only with its own controls.

Each matched set has its own baseline odds. Conditional logistic regression removes those set-specific intercepts from the likelihood, so it estimates only the coefficients of the exposures. For 1:1 pairs and a single binary exposure, its odds ratio is the number of pairs in which only the case was exposed divided by the number in which only the control was exposed.

Outcomes with more than two categories

What if the outcome has three or more categories?

Unordered categories, such as stroke type or discharge destination, need multinomial logistic regression. Ordered categories, such as pain grade or the modified Rankin scale, can use ordinal logistic regression, usually the proportional odds model.

The proportional odds model assumes that a predictor has the same odds ratio at every cut-point of the scale. SPSS's Ordinal Regression offers a test of parallel lines for this assumption. When the odds ratios clearly differ, use a multinomial model. A multinomial model estimates a separate set of coefficients for each non-reference category, so it needs more events than an ordinal model.

Odds ratio or risk ratio

When does the odds ratio overstate the risk ratio?

Logistic regression gives odds ratios. When the outcome is rare, the odds ratio is close to the risk ratio. When the outcome is common, the odds ratio is further from 1 than the risk ratio.

McNutt and colleagues (2003) wrote that the adjusted odds ratio approximates the adjusted relative risk when the incidence is below 10%, and that for more common outcomes the odds ratio always overstates the relative risk. They described the incorrect application of a proposed method for converting an adjusted odds ratio into an adjusted relative risk, and alternative methods. In cohort studies and trials with a common outcome, estimate the risk ratio directly. Zou (2004) proposed Poisson regression with robust standard errors for this, the “modified Poisson” approach, described in the count models chapter. In a case-control study the odds ratio is the measure available. The basic 2 × 2 comparison is in the risk and survival chapter.

Decide

Which logistic model fits your study?

Answer the questions in order. Each answer links to its section.

The formula

Worked example: reading a logistic regression table

Made-up teaching data: the 400 patients from the thesis at the top of this chapter, 71 with a postoperative pulmonary complication. Fitted in Python with statsmodels.

1. B is the change in log-odds. Smoking adds 0.895 to the log-odds of a complication, with age, ASA grade and albumin unchanged.
2. Odds ratio = e^B = e^0.895 = 2.45: smokers have 2.45 times the odds of a complication.
3. 95% CI = e^{B ± 1.96 × SE} = e^{0.8954 ± 1.96 × 0.2803} = e^0.346 to e^1.444 = 1.41 to 4.24.
4. Wald χ² = (B ÷ SE)² = (0.8954 ÷ 0.2803)² = 10.20 on 1 df, p = 0.001.
5. Units: 1.049 per year of age is 1.62 per 10 years. 0.940 per g/L of albumin is 1.36 for albumin 5 g/L lower.
6. Predicted risk for a 70-year-old smoker, ASA III–IV, albumin 30 g/L: log-odds = −2.848 + 0.0480 × 70 + 0.895 + 1.033 − 0.0621 × 30 = 0.577. Odds = e^0.577 = 1.78. Risk = 1.78 ÷ 2.78 = 64%.
7. Whole model: −2LL 374.07 → 333.77, likelihood ratio χ² 40.29 on 4 df, p < 0.001. Nagelkerke R² 0.158.
8. Performance: C statistic 0.72 (95% CI 0.65 to 0.79, Hanley and McNeil method). Hosmer–Lemeshow p = 0.58, which does not establish calibration.
9. Events per variable: 71 ÷ 4 = 17.8.

In a real paper

Logistic regression in two published prediction models

How to run it

Running logistic regression in SPSS, R and jamovi

Menu paths below are from the IBM SPSS Statistics 27 manuals. Function names and arguments are from the CRAN reference manuals.

SPSS

1. Analyze > Regression > Binary Logistic. Move the outcome to Dependent and the predictors to Covariates. Use Categorical to declare categorical predictors and choose the reference category.
2. In Options, tick CI for exp(B). Hosmer-Lemeshow goodness-of-fit is also offered there. The output gives the omnibus test of model coefficients, −2 log-likelihood, Cox & Snell and Nagelkerke R², and B, SE, Wald and Exp(B) for each predictor.
3. For the C statistic, save the predicted probabilities (Save), then use Analyze > Classify > ROC Curve.
4. Unordered outcome: Analyze > Regression > Multinomial Logistic Regression. Ordered outcome: Analyze > Regression > Ordinal, which offers a Test of parallel lines.
5. Matched sets: an IBM support note describes conditional logistic regression through Analyze > Survival > Cox Regression: code the time variable 1 for cases and 2 for controls, define the event as the case status, and put the matched-set identifier in Strata. The note advises this for 1:1 and 1:n matching.

R

fit <- glm(ppc ~ age + smoker + asa34 + albumin, family = binomial, data = d)
summary(fit)                         # B, SE, z (Wald) and p for each coefficient
exp(coef(fit))                       # odds ratios
exp(confint(fit))                    # 95% CIs by profile likelihood (confint.default() gives Wald CIs)
anova(fit, test = "Chisq")           # likelihood ratio tests
ResourceSelection::hoslem.test(fit$y, fitted(fit), g = 10)
pROC::auc(pROC::roc(d$ppc, fitted(fit)))
rms::val.prob(fitted(fit), d$ppc)    # calibration plot and measures

survival::clogit(case ~ antibiotic + strata(pair), data = m)   # matched sets
MASS::polr(pain ~ drug, data = p, Hess = TRUE)                # proportional odds
nnet::multinom(pain ~ drug, data = p)                          # unordered categories
logistf::logistf(ppc ~ marker, data = s)                       # Firth's penalised likelihood

jamovi

1. Analyses → Regression → 2 Outcomes (binomial). For more categories: N Outcomes (multinomial) or Ordinal Outcomes.
2. Move the outcome to Dependent Variable, measurements to Covariates and categorical predictors to Factors. Set reference levels under Reference levels.
3. Under Model Coefficients, tick Odds ratio and its confidence interval. Under Model Fit, tick Deviance, AIC, Overall Model Test and the pseudo-R² values. Prediction gives the classification table. The underlying jmv function, logRegBin(), also has auc and rocPlot options.

How to report it

What to write in the thesis and the paper

State how the model was built before giving its results, and give odds ratios with their units, reference categories and confidence intervals.

Methods

• The outcome, how many patients had it, and why logistic regression was used.
• How predictors were chosen (before the analysis, from clinical knowledge), how many were considered, and the events per candidate variable.
• How categorical predictors were coded (reference categories) and how continuous predictors were handled (TRIPOD+AI item 12b).
• Measures of model performance: discrimination, calibration, and how they were validated (TRIPOD+AI item 12e). Software and version.

Results

• Unadjusted and adjusted odds ratios with 95% confidence intervals (STROBE item 16a), with the unit for each measurement.
• For a prediction model: the full equation including the intercept (TRIPOD+AI item 22), and performance with confidence intervals (item 23a).

A worked sentence using this chapter's teaching numbers

Of 400 patients, 71 (17.8%) developed a postoperative pulmonary complication. In multivariable logistic regression, older age (adjusted odds ratio 1.62 per 10 years, 95% CI 1.27 to 2.07), current smoking (2.45, 1.41 to 4.24), ASA grade III–IV compared with I–II (2.81, 1.62 to 4.89) and lower serum albumin (1.36 per 5 g/L lower, 1.03 to 1.82) were associated with higher odds of complication (likelihood ratio χ² 40.29, 4 df, p < 0.001). The model had a C statistic of 0.72 (95% CI 0.65 to 0.79) in the development data. Its calibration has not been assessed in other patients.

By specialty

Logistic regression problems by specialty

Common myths

Common mistaken beliefs

• ✗Myth: An odds ratio of 1.05 for age is too small to matter.

  ✓It is per year. In the teaching model 1.049 per year is 1.62 per 10 years. Always read the unit.

• ✗Myth: A Hosmer–Lemeshow p above 0.05 proves the model is well calibrated.

  ✓It only fails to detect miscalibration. The teaching model gives p from 0.08 to 0.58 depending on the number of groups, and Van Calster and colleagues (2019) recommend against the test.

• ✗Myth: A high C statistic means the predicted risks are correct.

  ✓The C statistic measures ranking only. Adding the same amount to every predicted log-odds leaves it unchanged while every risk is wrong.

• ✗Myth: The odds ratio can be read as a relative risk.

  ✓Only when the outcome is rare. At a baseline risk of 30%, an odds ratio of 2 is a risk ratio of 1.54.

• ✗Myth: Nagelkerke R² of 0.16 means the model explains 16% of the variation.

  ✓Pseudo-R² values are not proportions of explained variance, and different versions give different values for the same model: 0.108, 0.096 and 0.158 here.

• ✗Myth: Matched case-control data can be analysed with ordinary logistic regression.

  ✓Use conditional logistic regression. In the teaching pairs the conditional odds ratio is 2.50 and the unconditional 2.80.

Why this matters to you

Five consequences of getting logistic regression wrong

1. Protocol sent back

   A plan to "enter all significant variables into logistic regression" gives no events per variable, no list of candidate predictors and no way of handling continuous predictors.

2. A wrong conclusion in the thesis

   Writing that age has "no effect" because its odds ratio per year is 1.05, or reporting an odds ratio of 2.45 for a common outcome as a 2.45-fold risk.

3. Questions you cannot answer at the viva

   "How do you know your predicted risks are right?" needs a calibration plot, slope and intercept, not a Hosmer–Lemeshow p-value.

4. Misreading papers

   Accepting an odds ratio in the thousands from sparse data, or a prediction model reported with only a C statistic in the patients used to build it.

5. Harm to patients

   A risk score that overestimates risk in a new hospital can lead to unnecessary treatment; one that underestimates risk can withhold it.

History

History of logistic regression

1. 1944

   Joseph Berkson

   Published "Application of the logistic function to bio-assay" in the Journal of the American Statistical Association.

2. 1958

   D. R. Cox

   Published "The regression analysis of binary sequences": tests and estimates for a sequence of 0s and 1s in which the chance of a 1 depends on one or more independent variables.

3. 1972

   John Nelder and Robert Wedderburn

   Published "Generalized linear models", showing that one fitting method, iterative weighted linear regression, gives maximum likelihood estimates for normal, binomial, Poisson and gamma outcomes.

4. 1979

   Ross Prentice and Ronald Pyke

   Published "Logistic disease incidence models and case-control studies" in Biometrika.

5. 1980

   David Hosmer and Stanley Lemeshow

   Published "Goodness of fit tests for the multiple logistic regression model".

6. 1980

   Peter McCullagh

   Published "Regression models for ordinal data", in which the proportional odds and proportional hazards models were described as likely to be the most useful in practice.

7. 1996

   Peter Peduzzi and colleagues

   Simulated logistic models and found biased coefficients and incorrect confidence limits with fewer than 10 events per variable.

8. 2002

   Georg Heinze and Michael Schemper

   Showed that Firth's penalised likelihood gives finite estimates when separation makes maximum likelihood estimates diverge.

9. 2016

   Maarten van Smeden and colleagues

   Concluded that the evidence supporting events-per-variable rules for binary logistic regression is weak.

10. 2019

    Ben Van Calster and colleagues

    Described calibration as "the Achilles heel of predictive analytics" and recommended against the Hosmer–Lemeshow test for assessing calibration.

11. 2024

    Gary Collins and colleagues

    Published TRIPOD+AI, a 27-item checklist for reporting prediction models built with regression or machine learning, superseding TRIPOD 2015.

Guidelines

Logistic regression in reporting guidelines

Reading a paper

Six checks on any logistic regression

• Are the number of events and the number of candidate predictors both given?
• Is the unit of every continuous predictor and the reference category of every categorical predictor stated?
• Was the shape of the relation checked for continuous predictors, or were they cut into categories without reason?
• Are unadjusted and adjusted odds ratios given with confidence intervals, and is a very large odds ratio explained?
• For a prediction model, are both discrimination and calibration reported, in patients not used to build the model?
• If the outcome is common, are odds ratios described as odds ratios? If cases were matched, was a conditional analysis used?

Why is it called that?

Word surgery

Exam answers

Model answer outlines

Past papers

TNMGRMU questions on logistic regression in depth

Questions from MD, MS, DM, MCh and Ph.D. papers that this chapter helps you answer. Tap a sitting to open the university's own question paper.

1. ×1Logistic regressionDM Neurology · Aug 2009

Further reading

Related Stateazy articles