Odds, Log-Odds and Interpretation

01 Learning objectives

02 Motivating example

A hospital has fitted a logistic regression to predict 30-day readmission using data from several thousand discharged patients. The fitted model is:

z = −2.1 + 0.03·age + 0.8·diabetes + 0.4·prior_admissions − 0.6·discharge_plan

where age is measured in years, diabetes is 1 if the patient has a diabetes diagnosis and 0 otherwise, prior_admissions counts the number of admissions in the previous year, and discharge_plan is 1 if a structured discharge plan was put in place and 0 otherwise.

The clinical team asks a simple question: how much does diabetes increase the risk of readmission?

The coefficient is 0.8. Someone looks at it and says: "diabetes increases the probability of readmission by 0.8." That would be an 80-percentage-point increase — which is obviously impossible, since probabilities cannot exceed 1.

Someone else, a bit more careful, says: "diabetes multiplies the probability by 0.8." That is also wrong, and in the opposite direction: a multiplier less than 1 would mean diabetes reduces readmission risk, which contradicts the positive sign.

Both attempts fail because they assume the coefficient lives on the probability scale. It does not. The coefficient lives on the log-odds scale, and moving to the probability scale requires a transformation that depends on where the patient started.

The correct statement turns out to be: diabetes multiplies the odds of readmission by about 2.23 — an increase of roughly 123% in odds. But even this is not the full story, because the corresponding change in probability ranges from about 4 percentage points to about 16 percentage points depending on the patient. Same coefficient, same odds ratio, very different clinical significance.

Understanding why is the subject of this tutorial.

03 Concept explanation

3.1 Why probability is not the natural scale

A probability is a number in [0, 1]. That range creates two problems for interpreting coefficients.

First, a probability cannot increase by a fixed amount indefinitely. If p = 0.9 and something increases the risk, there is only 0.1 of headroom. The same effect that raises a probability from 0.1 to 0.2 cannot raise one from 0.9 to 1.0.

Second, probabilities are not additive in the way coefficients are. A logistic regression says that each feature contributes a fixed amount to a score — but that fixed contribution does not translate into a fixed change in probability.

The odds scale solves the first problem. The log-odds scale solves both.

3.2 Odds

The odds of an event are the ratio of the probability that it happens to the probability that it does not:

odds = p / (1 − p)

If p = 0.75, the odds are 0.75 / 0.25 = 3. This is usually expressed as "3 to 1 in favour" — three ways to win for every one way to lose. If p = 0.20, the odds are 0.20 / 0.80 = 0.25, or "1 to 4 in favour."

The range of odds is [0, ∞). This is better than probability because there is now no upper ceiling — an event can become arbitrarily more likely as the odds grow. But odds still cannot be modelled by an unrestricted linear function, because they cannot go below zero.

Odds are the language of betting, and this is not a coincidence. A bookmaker quoting "3 to 1" is quoting odds, because odds are the natural way to express a multiplicative relationship between risk factors.

3.3 Odds ratios

When a predictor changes, the odds change. The odds ratio (OR) is the factor by which they change:

OR = odds(condition present) / odds(condition absent)

An odds ratio of 2 means the odds double. An odds ratio of 0.5 means the odds halve. An odds ratio of 1 means no change — the predictor has no effect on the odds.

Odds ratios are the standard way to report logistic regression results in medicine, epidemiology and the social sciences. They are convenient because they are dimensionless — an OR of 2.2 means the same thing regardless of the baseline risk — and because they are directly available from the coefficients.

3.4 Log-odds, and why the model uses them

The log-odds, also called the logit, is the natural logarithm of the odds:

log-odds = log( p / (1 − p) )

As T-02 established, this is the scale on which logistic regression is linear:

log( p / (1 − p) ) = β₀ + β₁x₁ + … + βₚxₚ

The log-odds range over (−∞, ∞), so an unrestricted linear function is perfectly suited to modelling them. This is why the model is expressed on this scale, and why the coefficients live here.

The relationship between the three scales is a chain of transformations:

The table below gives anchor values on all three scales. It is worth internalising a few of these — particularly that p = 0.5 corresponds to odds of 1 and log-odds of 0, and that p = 0.1 and p = 0.9 correspond to log-odds of roughly −2.2 and +2.2.

The symmetry is worth noting. Because σ(−z) = 1 − σ(z), the log-odds of complementary probabilities are exact negatives: logit(0.1) = −2.197 and logit(0.9) = +2.197.

3.5 Interpreting a coefficient: three scales

A single coefficient βⱼ can be read on all three scales. Each reading is correct; each answers a different question. Using the readmission model, with β = 0.8 for diabetes:

Notice that three of the four rows give a specific number, and only the probability-scale reading does not. That is the central difficulty: the probability-scale effect is not a property of the coefficient alone.

3.6 Continuous and binary predictors

The interpretation differs slightly depending on whether the predictor is binary or continuous.

Binary predictors. For a variable that is 0 or 1, the interpretation is direct: the odds ratio compares the two groups. For diabetes: OR = e^0.8 = 2.23 — patients with diabetes have 2.23 times the odds of readmission compared with patients without diabetes, holding all other variables constant.

Continuous predictors. For a variable measured on a continuous scale, the odds ratio refers to a one-unit change. For age with β = 0.03:

OR per year = e^0.03 = 1.0305

Each additional year of age multiplies the odds of readmission by 1.03 — a 3% increase per year. Per year this sounds small; the compounding is what matters. For a ten-year difference:

OR per decade = e^(10 × 0.03) = e^0.3 = 1.3499

A decade of age multiplies the odds by 1.35 — a 35% increase. This is a much more useful number to report, and it is the sort of re-scaling that makes continuous predictors interpretable in practice.

3.7 The same odds ratio, different probability effects

This is the point that causes the most confusion, and it deserves careful attention.

The odds ratio for diabetes is 2.23. That number does not depend on the patient. But the corresponding change in probability does — because the probability scale is non-linear in the log-odds.

Consider three patients with different baseline risks. For each, we compute the predicted probability before and after adding diabetes to the model (which simply adds 0.8 to the score).

All three rows have the same odds ratio of 2.23. The change in probability ranges from 16 percentage points for the low-risk patient to just over 4 percentage points for the high-risk patient.

Why? Because the high-risk patient is already near the ceiling. Their probability of 0.921 cannot rise much, however favourable the odds become. The odds can still double — from 11.7 to 26.0 — but the probability can only move from 0.921 to 0.963.

This is the saturation behaviour of the sigmoid made concrete. It means that reporting an odds ratio alone can be misleading, and reporting a probability change requires specifying for whom.

3.8 Marginal effects: communicating on the probability scale

The standard solution is the marginal effect — the change in predicted probability associated with a one-unit change in a predictor, holding the other predictors at specified values.

For a continuous predictor, the marginal effect at a particular point is:

∂p/∂xⱼ = βⱼ · p · (1 − p)

This is a genuinely useful formula, and it explains the phenomenon in Section 3.7 in a single line. The derivative of the sigmoid is p(1 − p), which is largest at p = 0.5 (where it equals 0.25) and shrinks towards zero as p approaches 0 or 1. So the same coefficient βⱼ has its greatest effect on probability when the baseline risk is near 0.5, and its smallest effect at the extremes.

For a binary predictor, the analogous quantity is the change in predicted probability when the indicator switches from 0 to 1, evaluated at specified values of the other predictors. This is sometimes called the average marginal effect when it is computed for every observation and then averaged.

Average marginal effects are increasingly the preferred way to communicate logistic regression results to non-technical audiences, because they express effects in the units people actually care about — percentage points of probability — rather than in odds ratios.

3.9 Confidence intervals and statistical significance

A coefficient is an estimate, not a known truth. The fitted value of 0.8 is the most likely value given the data, but other values are also plausible. This uncertainty is captured by a confidence interval.

Confidence intervals for coefficients are typically computed on the log-odds scale — where the sampling distribution is closer to normal — and then exponentiated to give a confidence interval for the odds ratio. A 95% interval for the diabetes coefficient might be:

β ∈ [0.538, 1.063]

OR ∈ [e^0.538, e^1.063] = [1.71, 2.89]

Three things can be read from this interval.

• The point estimate is OR = 2.23 — the single best guess.
• The uncertainty is substantial. The true effect could plausibly be anywhere from a 71% increase in odds to a 189% increase. Both are compatible with the data.
• Zero effect is excluded. The interval does not contain OR = 1, which corresponds to β = 0. This is what "statistically significant at the 5% level" means for a single coefficient.

3.10 The interpretation errors to avoid

The table below collects the mistakes that appear most often in practice. Each is followed by the correct statement.

3.11 Presenting results

Different audiences need different framings. The table below gives a rule of thumb.

A report that gives only odds ratios is not wrong, but it is incomplete. A report that gives odds ratios, confidence intervals, and average marginal effects gives readers everything they need to form their own judgement — which is the purpose of reporting results at all.

04 Visual illustration

Figure 1 plots odds against probability. The relationship is strongly non-linear, and the shape of the curve explains why the probability-scale effect of a coefficient depends on the baseline risk.

Read the curve at the three patients from Section 3.7. The low-risk patient sits at p = 0.206, where the curve is still relatively flat: doubling the odds moves them a long way up the probability scale. The high-risk patient sits at p = 0.921, in the steep region where the curve is nearly vertical: doubling the odds barely moves them at all.

Figure 2 shows the fitted odds ratios for the readmission model, with 95% confidence intervals. This is the standard way to present logistic regression results in a report, and it is worth learning to read.

Three features of Figure 2 are worth pointing out. First, the horizontal axis is on a logarithmic scale, so that OR = 0.5 and OR = 2 are equidistant from OR = 1. This is standard for odds-ratio plots, because the log-odds scale is where the uncertainty is symmetric. Second, the intervals vary enormously in width: the diabetes interval spans more than a factor of three, while the age-per-year interval is razor-thin. Third, the discharge-plan interval lies entirely below 1, indicating a protective effect — the only predictor in the model that reduces readmission odds.

05 Worked example

We work through the readmission model in detail, Computing every quantity that would appear in a report.

OR = e^0.8 = 2.2255

(2.2255 − 1) × 100 = 122.6%

Lower: e^0.538 = 1.712
Upper: e^1.063 = 2.895

OR ∈ [1.71, 2.89]

OR per year = e^0.03 = 1.0305

OR per decade = e^0.3 = 1.3499

OR = e^0.6 = 1.8221

∂p/∂age = 0.03 × 0.5 × 0.5 = 0.0075

∂p/∂age = 0.03 × 0.9 × 0.1 = 0.0027

OR = e^(−0.6) = 0.5488

(0.5488 − 1) × 100 = −45.1%

z = 0.8 − 0.6 = 0.2
p = σ(0.2) = 1 / (1 + e^(−0.2)) = 1 / 1.8187 = 0.5498

06 Interactive activity

import numpy as np
import matplotlib.pyplot as plt

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

def odds(p):
    return p / (1 - p)

def logit(p):
    return np.log(odds(p))

# Readmission model
def score(age, diabetes, prior_adm, discharge_plan):
    return -2.1 + 0.03*age + 0.8*diabetes + 0.4*prior_adm - 0.6*discharge_plan

# Patient: 60y, diabetes, 1 prior admission, discharge plan
z = score(60, 1, 1, 1)
p = sigmoid(z)
print(f"z = {z:.3f}")
print(f"p = {p:.4f}")
print(f"odds = {odds(p):.4f}")
print(f"logit(p) = {logit(p):.4f}  (should equal z)")

# Same patient without diabetes
z2 = score(60, 0, 1, 1)
p2 = sigmoid(z2)
print(f"\nWithout diabetes: z = {z2:.3f}, p = {p2:.4f}")
print(f"Odds ratio = {odds(p)/odds(p2):.4f}")
print(f"Change in p = {(p - p2)*100:.1f} percentage points")

# Probability change as a function of baseline risk
baseline_p = np.linspace(0.01, 0.99, 200)
baseline_z = logit(baseline_p)
with_diabetes_p = sigmoid(baseline_z + 0.8)

fig, ax = plt.subplots(figsize=(7.5, 5))
ax.plot(baseline_p, (with_diabetes_p - baseline_p) * 100, linewidth=2.6,
        color="#1f4e8c")
ax.set_xlabel("baseline probability (without diabetes)")
ax.set_ylabel("change in probability (percentage points)")
ax.set_title("The same odds ratio (2.23) produces different probability changes")
ax.grid(alpha=0.25)
plt.tight_layout()
plt.show()

The plot is the single most useful way to internalise Section 3.7. The curve peaks around p = 0.5 and falls away on both sides. At the peak, adding diabetes changes the probability by around 16 percentage points. Near p = 0.05 or p = 0.95, the same odds ratio changes the probability by only a few points.

This is the derivative β·p·(1−p) from Section 3.8 in graphical form. The quantity p(1−p) is the variance of a Bernoulli random variable, and it is largest at p = 0.5 — precisely where the sigmoid is steepest and where additional evidence has the most impact.

07 Self-check questions

Answer each question before revealing the solution.

08 Practice exercises

Work these by hand where possible, then verify computationally.

1. Convert each of the following probabilities to odds and then to log-odds: 0.05, 0.25, 0.5, 0.75, 0.95.
2. A coefficient is β = 1.5. What is the odds ratio? What percentage change in odds does it represent?
3. An odds ratio is reported as 0.75. What is the corresponding coefficient on the log-odds scale? What percentage reduction in odds does it represent?
4. A continuous predictor has β = 0.2 per unit. What is the odds ratio for a 5-unit increase? For a 10-unit increase?
5. Using the readmission model, compute the predicted probability for a 55-year-old with no diabetes, 3 prior admissions and no discharge plan.
6. For the same patient, compute the predicted probability if a discharge plan is put in place. What is the change in probability?
7. A logistic regression reports OR = 1.8 with 95% CI [1.1, 2.9]. Is the effect statistically significant at the 5% level? What is the range of plausible percentage increases in odds?
8. A marginal effect computation gives ∂p/∂x = 0.02 at a particular point. Interpret this in a sentence. Is this a property of the coefficient alone, or does it depend on where it is evaluated?

09 Problem sets

These problems require reasoning about interpretation rather than computation.

10 Discussion and projects

Discussion questions

1. Odds ratios are the standard reporting convention in medicine and epidemiology, despite being difficult to interpret on the probability scale. Is this a good convention, or should the field switch to marginal effects? What would be gained and lost?
2. The non-collapsibility of odds ratios means that a coefficient depends on the other variables in the model. Does this make logistic regression coefficients "unstable"? Or is it a legitimate feature of a conditional model?
3. A confidence interval can be narrow (precise) but centred on a trivial effect, or wide (imprecise) but centred on an important one. Which is more useful to a decision-maker? Does the answer depend on the decision?
4. The marginal effect of a predictor depends on where it is evaluated. Does this make the concept of "the effect of X" incoherent in logistic regression? How would you respond to someone who insists on a single number?

Mini-project · Interpreting a real logistic regression

In this project you will fit a logistic regression to a dataset of your choice, compute all the quantities discussed in this tutorial, and write a short report aimed at two different audiences.

import numpy as np
import pandas as pd
import statsmodels.api as sm

# Fit with statsmodels for full inference
X = sm.add_constant(df[predictors])
model = sm.Logit(df["outcome"], X).fit()

# Coefficients with confidence intervals
params = model.params
conf = model.conf_int()
conf.columns = ["lower", "upper"]

results = pd.DataFrame({
    "beta": params,
    "OR": np.exp(params),
    "OR_lower": np.exp(conf["lower"]),
    "OR_upper": np.exp(conf["upper"]),
    "p_value": model.pvalues,
})
print(results.round(3))

# Marginal effects for a continuous predictor
def marginal_effect(beta_j, p):
    return beta_j * p * (1 - p)

# Evaluate at several baseline probabilities
for p in [0.1, 0.3, 0.5, 0.7, 0.9]:
    me = marginal_effect(params["age"], p)
    print(f"p = {p:.1f}: marginal effect of age = {me:.4f}")

As you write the two report paragraphs, notice how differently the same model reads depending on the scale. The statistical audience wants the odds ratio and its interval; the general audience wants to know how much the probability changes for a typical case. Neither report is complete without the other, and a good analyst can produce both from the same fitted model.

11 Summary

This tutorial covered the three scales on which a logistic regression can be read, and the interpretation errors that arise from confusing them.

• Probability, odds and log-odds are three views of one quantity. Their ranges are [0, 1], [0, ∞) and (−∞, ∞). The model is linear on the log-odds scale.
• Coefficients are additive on the log-odds scale. A coefficient βⱼ is the change in log-odds per unit change in xⱼ.
• Coefficients are multiplicative on the odds scale. The odds ratio is e^β, and the percentage change in odds is (e^β − 1) × 100.
• There is no single probability-scale effect. The change in probability depends on the baseline risk. A coefficient has its largest probability effect near p = 0.5 and its smallest at the extremes.
• Marginal effects express results on the probability scale. For a continuous predictor, ∂p/∂xⱼ = βⱼ · p · (1 − p). For binary predictors, compute the change in predicted probability directly.
• Confidence intervals are essential. They express the range of plausible values for the coefficient or odds ratio. A significant result does not mean a large effect; a non-significant result does not mean no effect.
• Odds ratios are not risk ratios. They coincide only when the outcome is rare. For common outcomes, the difference is substantial.
• Odds ratios are non-collapsible. They change when other covariates are added or removed, even if those covariates are unrelated to the predictor of interest. A coefficient is always conditional on the model.
• The intercept is rarely interpretable. It is the log-odds when every predictor equals zero, which is often not a meaningful case.
• Different audiences need different scales. Odds ratios for statistical audiences; marginal effects and probabilities for clinical, policy and general audiences.

12 Further reading

Core

• Gelman, Hill & Vehtari — Regression and Other Stories, Chapters 13–14. An exceptionally clear treatment of logistic regression interpretation, with a strong emphasis on communicating results honestly and on the difference between the coefficient scale and the probability scale.
• Hosmer, Lemeshow & Sturdivant — Applied Logistic Regression, Chapter 3. The standard reference for interpretation, including detailed discussion of odds ratios, confidence intervals and the presentation of results.
• James, Witten, Hastie & Tibshirani — An Introduction to Statistical Learning, Section 4.3. Covers the basics of logistic regression interpretation, including the odds-ratio reading of coefficients.

Supplementary

• Norton, Dowd, Maciejewski & Wang — "Marginal Effects: Quantifying the Effect of Changes in Predictors" (2018). A widely cited paper on marginal effects in logistic regression, with worked examples and clear guidance on when to use them.
• Mood, C. — "Logistic Regression: Why We Cannot Do What We Think We Can Do, and What We Can Do About It" (2010). A clear treatment of the difficulties in interpreting and comparing coefficients across logistic regression models, including a discussion of unobserved heterogeneity.
• Pang, Kaufman & Platt — "Studying Noncollapsibility of the Odds Ratio with the Pearson Chi-Square Test" (2016). A short, accessible treatment of why odds ratios change when covariates are added.

On reporting and communication

• Schwartz & Woloshin — Know Your Chances. A book about communicating risk to the public, with extensive discussion of why ratios (including odds ratios) are so easily misread and why absolute risks are usually preferable.
• Gigerenzer, Gaissmaier, Kurz-Milcke, Schwartz & Woloshin — "Helping Doctors and Patients Make Sense of Health Statistics" (2007). The classic review of statistical literacy in medicine, with numerous examples of how ratio-based reporting misleads.
• Zikmund-Fisher et al. — "Percent Weight Change in Lifestyle Interventions" (2010). A study of how different framings of the same statistical result influence perception — a useful reminder that the choice of scale is not neutral.