Chapter 18 · Measurement, agreement and diagnosis

How does a test result change what you already suspected?

Bayes' rule, pre-test and post-test probability, serial and parallel testing, and decision analysis.

Read first: Diagnostic tests

The problem first

You are a DM Cardiology resident. A 50-year-old woman with atypical chest pain has, by your estimate, a 20% chance of coronary artery disease. Test A comes back positive, and so does test B (made-up teaching tests, with made-up accuracy).

At the case presentation you multiply the two likelihood ratios together and announce a 97% probability of disease. The consultant asks two questions.

“What probability did you start from, and do those two tests make their mistakes independently?”

Start here

Why does the same positive result mean different things in different patients?

A test result does not give a probability by itself. It changes the probability you already had before the test.

Picture 1,000 patients like yours, of whom 200 have the disease. Test A picks up 80% of those with the disease and wrongly flags 10% of those without it. Now count the positive results.

1,000 patients, pre-test probability 20% · one square = one patient160disease, test positive40disease, test negative80no disease, test positive720no disease, test negativepositive: 160 + 80 = 240160 ÷ 240 = 67%have the disease
Teaching example. The test finds 160 of the 200 patients with the disease, but it also flags 80 of the 800 without it. Of the 240 positive patients, 160 have the disease: 67%, not the 80% sensitivity. Counting whole patients like this is called using natural frequencies.

Doctors often find this hard when the numbers are given as percentages. In 1978, Casscells and colleagues asked 60 students, house officers and physicians at Harvard Medical School: if a test for a disease with a prevalence of 1 in 1,000 has a false-positive rate of 5%, what is the chance that a person with a positive result has the disease? Only 11 answered correctly, and the most common answer, 95%, was given by 27. In natural frequencies the answer is visible: of 1,000 people, 1 has the disease and about 50 of the other 999 test positive, so about 1 in 51, or 2%, if the test detects every case. Hoffrage and colleagues (2000) showed that presenting the same information as natural frequencies instead of probabilities considerably reduces these errors.

Sensitivity, specificity, predictive values and likelihood ratios are explained in the diagnostic tests chapter. This chapter uses them to update probabilities, combine tests and make decisions.

Bayes' rule

How do you turn the probability of a result into the probability of disease?

Studies tell you how often the test is positive in people with the disease. Your patient needs the reverse: how often people with a positive test have the disease. Bayes' rule converts one into the other.

disease 20%no disease 80%TP0.16FN 0.04FP 0.80 × 0.10 = 0.08TN 0.80 × 0.90 = 0.72width = prevalence · height = chance of each resultP(disease | positive)= TP ÷ (TP + FP)= 0.16 ÷ (0.16 + 0.08)= 0.667red height: sensitivity 80%blue height: 1 − specificity 10%
The same teaching numbers drawn as areas. Each patient group's width is its share of the population. Within each group, the height shows the chance of each test result. The probability of disease after a positive result is the red area divided by all the positive area.

P(D | +) = P(+ | D) × P(D) ÷ P(+)

P(+) = P(+ | D) × P(D) + P(+ | no D) × P(no D)

P(D | +) = Se × p ÷ [Se × p + (1 − Sp) × (1 − p)] = 0.80 × 0.20 ÷ (0.16 + 0.08) = 0.667

P(D | +)
probability of disease given a positive result (the post-test probability, equal to the PPV in this population)
p
pre-test probability of disease, 0.20
Se
sensitivity, P(+ | D) = 0.80
1 − Sp
false-positive rate, P(+ | no D) = 0.10

The same rule written with odds is shorter, and it is the form used at the bedside:

post-test odds = pre-test odds × likelihood ratio → 0.25 × 8 = 2 → 2 ÷ (1 + 2) = 0.667

LR+ = Se ÷ (1 − Sp) = 0.80 ÷ 0.10 = 8. For a negative result, LR− = (1 − Se) ÷ Sp = 0.20 ÷ 0.90 = 0.22, and the probability falls to 5.3%. Reading P(positive | disease) as if it were P(disease | positive) is the error in the Casscells question.

Pre-test probability

Where does the probability before the test come from?

Every post-test probability depends on the pre-test probability you put in. It is an estimate for this patient, not a fixed number for the disease.

  1. Prevalence in your setting: how common the disease is among patients like this one who present to your clinic, ward or programme.
  2. History and examination: each finding raises or lowers that starting probability. Findings are themselves tests with likelihood ratios.
  3. Validated clinical prediction scores: where a score has been tested in patients like yours, it gives a probability directly.
  4. Your own estimate: if nothing else is available, state a range (for example 10% to 30%) and check whether the decision changes across it.

How much does the test add?

The change from the pre-test to the post-test probability is the gain from the test. With test A (LR+ 8, LR− 0.22), the gain is largest when the pre-test probability is in the middle range, and small when you are already nearly sure.

Pre-test probabilityAfter a positive resultAfter a negative result
5%29.6% (+24.6 points)1.2% (−3.8 points)
20%66.7% (+46.7)5.3% (−14.7)
50%88.9% (+38.9)18.2% (−31.8)
80%97.0% (+17.0)47.1% (−32.9)

Teaching values. At 80%, a negative result still leaves almost a 1 in 2 chance of disease, so it cannot rule the disease out.

Combining results

Can the result of one test become the starting point for the next?

Yes. The post-test probability after test A becomes the pre-test probability for test B. With odds, you multiply by each likelihood ratio in turn.

A positive: odds × 8B positive: odds × 181%5%20%50%80%95%99%20%66.7%97.3%odds 0.25 → 2 → 36 · valid only if the two tests are conditionally independent
Teaching example. After a positive test A, the probability is 66.7%. Using that as the starting point for test B (LR+ 18) gives 97.3%. This is the calculation the resident did in the problem at the top of the chapter.

post-test odds = pre-test odds × LRA × LRB = 0.25 × 8 × 18 = 36 → 97.3%

This multiplication is valid only if the tests are conditionally independent: among patients with the disease, a positive A does not make a positive B more likely, and the same holds among patients without the disease. In practice, tests that detect the same abnormality, or react to the same interfering condition, tend to make the same mistakes. Then test B's likelihood ratio in patients who are already positive on A is smaller than its published value, and multiplying overstates the post-test probability.

What to do: use a likelihood ratio for the combination, measured in studies where both tests were done on the same patients, or report the chained result as an upper estimate and say that independence was assumed.

Interactive

Make the two tests share their errors and watch the post-test probability fall

Try this: move the dependence from 0% to 100%. At 0% the second positive test raises the probability to 97.3%. At 100% test B adds almost nothing once A is positive.

Teaching example: test A sensitivity 80%, specificity 90%; test B sensitivity 90%, specificity 95%. Dependence means that the patients who fool test A also tend to fool test B, for example because both tests measure the same thing.

20%
50%

0% = independent errors · 100% = the largest possible overlap

before any test20.0%after A positive66.7%after A and B positive, if independent97.3%after A and B positive, at this dependence87.4%0%25%50%75%100%
87.4%probability after two positive tests
9.9percentage points overstated by assuming independence

Among patients already positive on A, test B is positive in 95.0% of those with the disease and 27.5% of those without it. So B's likelihood ratio, once A is positive, is 3.5, not 18.

joint LR for two positives = 27.6 · product of the separate LRs = 8 × 18 = 144

Two tests in a pathway

Should you ask for both tests to be positive, or either?

Serial testing calls a patient positive only if both tests are positive. Parallel testing calls a patient positive if either test is positive. The choice changes the combined sensitivity and specificity.

SerialParallel1,000 patients: test AA positive: 240 → test BA negative: 760, called negativeboth positive: 148, called positiveB negative: 92, called negative1,000 patients: tests A and Beither positive: 312called positiveboth negative: 688called negativesensitivity 72% · specificity 99.5%PPV 97.3% · test B done 240 timessensitivity 98% · specificity 85.5%NPV 99.4% · test B done 1,000 times
Teaching example with 1,000 patients, pre-test probability 20%, test A (sensitivity 80%, specificity 90%) and test B (sensitivity 90%, specificity 95%), assuming independence.

Serial: Se = SeA × SeB = 0.72 · Sp = 1 − (1 − SpA)(1 − SpB) = 0.995

Parallel: Se = 1 − (1 − SeA)(1 − SeB) = 0.98 · Sp = SpA × SpB = 0.855

  • Serial testing raises specificity and PPV (97.3% here) and lowers sensitivity and NPV (93.4%). It suits confirming a diagnosis before a treatment that is risky, expensive or lifelong, and it saves tests: B is needed in only 240 patients.
  • Parallel testing raises sensitivity and NPV (99.4% here) and lowers specificity and PPV (62.8%). It suits urgent settings where missing the disease is dangerous and both tests can be done at once.
  • Order in a serial pathway does not change the combined sensitivity and specificity, but it changes how many second tests are needed. Starting with B would need test A in 220 patients instead of test B in 240. Put the cheaper or safer test first.
  • Both sets of formulas assume conditional independence. With related tests, serial testing gains less specificity and parallel testing gains less sensitivity than the formulas show.

Interactive

Send 1,000 patients through one test, two tests in series, or two in parallel, and watch PPV and NPV

Try this: start with "Serial" and read the PPV. Then press "Parallel" and watch the sensitivity rise while the specificity and PPV fall.

Teaching example with made-up test accuracy. The combined values assume the two tests make their errors independently.

How the two tests are combined
20%

From prevalence and the clinical findings

80%
90%
90%
95%
all patients1,000disease200no disease800A positive160A negative40A positive80A negative720B positive144B negative16B positive4B negative76red outline: called positive by this strategy · counts rounded to whole patients
97.3%PPV: of those called positive, how many have the disease
93.4%NPV

Test B is done only in the 240 patients positive on A. 148 are positive on both, and 144 of them have the disease. 56 patients with the disease are missed.

combined sensitivity 72.0% · combined specificity 99.5% · test B done 240 times

Comparison

Serial or parallel testing: which does what?

FeatureSerial testingParallel testing
Called positive whenBoth tests are positiveEither test is positive
Combined sensitivitySe_A × Se_B, lower than either test1 − (1 − Se_A)(1 − Se_B), higher than either test
Combined specificity1 − (1 − Sp_A)(1 − Sp_B), higher than either testSp_A × Sp_B, lower than either test
Predictive value that risesPPVNPV
Tests neededSecond test only for those positive on the firstBoth tests for everyone
Time to a decisionLonger, one test after the otherShorter, both at once
SuitsConfirming before risky or costly treatment, screening followed by confirmationEmergencies where a missed diagnosis is dangerous
Teaching exampleSe 72%, Sp 99.5%, PPV 97.3%Se 98%, Sp 85.5%, NPV 99.4%

Decisions

When is a test worth doing at all?

A test is useful only if its result could change what you do. If you would treat whatever the result, or would not treat whatever the result, the test adds cost and risk without changing the decision.

In 1975 Pauker and Kassirer described a treatment threshold: the probability of disease above which treating is better than not treating. It depends on the benefit of treating a patient who has the disease (B) and the harm of treating a patient who does not (H):

treatment threshold = H ÷ (H + B) = 1 ÷ (1 + 9) = 10%

In 1980 they added two more thresholds for when a test is available. Below the testing threshold, withhold treatment without testing. Above the test-treatment threshold, treat without testing. Between the two, test and let the result decide. Setting the expected values of the three strategies equal gives, for a test with no risk of its own:

testing threshold = (1 − Sp) × H ÷ [(1 − Sp) × H + Se × B] = 0.10 ÷ (0.10 + 7.2) = 1.4%

test-treatment threshold = Sp × H ÷ [Sp × H + (1 − Se) × B] = 0.90 ÷ (0.90 + 1.8) = 33.3%

test firsttreat without testingbelow 1.4%: no test, no treatmenttreatment threshold without a test: 10%test-treatment threshold 33.3%0%25%50%75%100%
Teaching example: benefit 9 and harm 1 (arbitrary units), test A with sensitivity 80% and specificity 90%. The patient in the problem, at 20%, lies in the zone where testing is best. If the test itself has risks or costs, the two thresholds move towards each other, and a test that is risky enough is never worth doing.

Benefit and harm must be in the same units: deaths prevented, quality-adjusted life years, or a value agreed with the patient. The thresholds are only as good as those estimates, so check whether the decision changes across a plausible range.

Interactive

Change the benefit, harm and test accuracy, and watch the testing and test-treatment thresholds move

Try this: raise the harm of treating a patient without the disease from 1 to 5, and watch all three thresholds move to the right. Then raise the test's sensitivity and watch the zone where testing is best become wider.

Teaching example in arbitrary benefit units. The test is assumed to carry no risk or cost of its own.

9
1
80%
90%
20%
90−10%25%50%75%100%probability of diseaseexpected valuetesting 1.4%treatment 10.0%test-treatment 33.3%treat everyonetest firstno treatment (0)the dot marks the best strategy at 20%
1.4%testing threshold
33.3%test-treatment threshold

At 20%: Test, and treat only if the test is positive: the probability lies between the two thresholds.

expected value: no treatment 0 · treat everyone 1.00 · test first 1.36 · treatment threshold without a test 10.0%

Decision analysis

How do you compare strategies whose outcomes have different values?

Draw a decision tree. Each strategy branches into its possible outcomes with their probabilities and values. The expected value of a strategy is the sum of probability × value over its outcomes.

strategyprobability · outcomevaluetreat everyoneEV 1.00test A firstEV 1.36no treatmentEV 00.20 · has the disease, treated+90.80 · no disease, treated−10.16 · disease, A positive, treated+90.04 · disease, A negative, not treated00.08 · no disease, A positive, treated−10.72 · no disease, A negative, not treated01.00 · nobody treated00.16 × 9 − 0.08 × 1 = 1.36square: decision · circle: chance · triangle: end point · values in benefit units (made-up)
Teaching decision tree for the patient at 20%. Testing first has the highest expected value (1.36), because it avoids most of the harm of treating patients without the disease while still treating 80% of those who have it.
  1. Decision node (square): the choice you control, such as treat, test first or wait.
  2. Chance nodes (circles): events you do not control, each with a probability. The probabilities leaving a node add up to 1.
  3. End points: each outcome gets a value, such as survival, quality-adjusted life years, cost, or a utility from 0 (death) to 1 (full health).
  4. Calculate from the right: at each chance node, multiply each outcome's value by its probability and add them. At the decision node, choose the branch with the highest expected value.
  5. Sensitivity analysis: change each uncertain input over its plausible range and see whether the best strategy changes. The thresholds above are the points where it does.

EV(test A first) = 0.16 × 9 + 0.04 × 0 + 0.08 × (−1) + 0.72 × 0 = 1.36

Expected value describes the average over many patients like this one. The values placed on outcomes are judgements, and they should reflect what matters to patients.

Bayesian statistics

What does a Bayesian analysis of a trial add to a p-value?

A p-value is the probability of data at least this extreme if the treatment had no effect. A Bayesian analysis answers the question clinicians usually ask: given these data, how probable is it that the treatment works?

It uses the same logic as the diagnostic test, with the treatment effect in place of the disease:

  • Prior: what was believed about the effect before the trial, written as a probability distribution. A sceptical prior is centred on no effect, an enthusiastic prior on a benefit, and a non-informative (flat) prior adds almost nothing to the data.
  • Likelihood: what the trial data say about each possible effect.
  • Posterior: the prior updated by the data. From it you read a point estimate, a 95% credible interval (the range with a 95% probability of containing the effect, given the prior and the model), and the posterior probability of benefit, such as P(risk ratio < 1) or P(risk ratio < 0.90).

ICH E9 defines Bayesian approaches as analyses that give a posterior probability distribution for a parameter such as the treatment effect, derived from the observed data and a prior distribution.

A simple calculation with the normal approximation

On the log risk-ratio scale, a normal prior and a normal likelihood give a normal posterior. Its precision (1 ÷ variance) is the sum of the two precisions, and its mean is their precision-weighted average.

1 ÷ SDpost² = 1 ÷ SDprior² + 1 ÷ SE²

meanpost = (meanprior ÷ SDprior² + log RR ÷ SE²) × SDpost²

Made-up trial: risk ratio for 30-day death 0.80, standard error of log RR 0.13 (95% confidence interval 0.62 to 1.03, p = 0.086). With a sceptical prior centred on RR 1 (SD 0.15 on the log scale):

precision 44.4 + 59.2 = 103.6 · mean = (0 × 44.4 + (−0.223) × 59.2) ÷ 103.6 = −0.127 · SD = 0.098

posterior RR 0.88, 95% credible interval 0.73 to 1.07 · P(RR < 1) = 90.3%

0.40.50.60.811.251.6risk ratio (log scale)posterior RR 0.88sceptical prior, RR 1trial, RR 0.80shaded: 90.3% of theposterior lies below RR 1
The posterior lies between the prior and the trial result and is narrower than either. With a flat prior, the probability that RR is below 1 would be 95.7%. With an enthusiastic prior centred on RR 0.80 (same SD), it would be 98.8%.

Uses in trials: reading a result with a non-significant p-value as a probability of benefit and of clinically important benefit, adding information from earlier studies through the prior, and writing decision rules for interim analyses in adaptive designs as posterior probabilities. In January 2026 the US FDA published a draft guidance on using Bayesian methods for the primary analysis of the main trials that support the effectiveness and safety of drugs and biological products.

Cautions: the prior must be chosen and justified before the data are seen, and results should be shown under a range of priors, from sceptical to enthusiastic. With little data, the prior dominates the result.

Interactive

Choose a prior and move the trial result, and watch the probability of benefit change

Try this: keep the trial at RR 0.80 and switch the prior from "No prior information" to "Sceptical". Watch the probability that the treatment reduces deaths fall from 95.7% as the posterior moves towards RR 1.

Teaching example: a made-up trial of a drug on 30-day mortality, analysed with the normal approximation on the log risk-ratio scale.

Prior belief before the trial
0.80

Below 1 means fewer deaths with the drug

0.13

A bigger trial has a smaller standard error

0.15

Smaller SD = a stronger prior belief

0.40.50.60.811.251.6RR 1: no effectrisk ratio (log scale)posteriorshaded: probability RR < 1
posteriortrial result (likelihood)prior
90.3%probability that the drug reduces deaths (RR < 1)
58.9%probability of at least a 10% reduction (RR < 0.90)

Posterior RR 0.88, 95% credible interval 0.73 to 1.07. The usual analysis of the same trial gives RR 0.80, 95% confidence interval 0.62 to 1.03, p = 0.086.

Comparison

Credible interval or confidence interval: what does each 95% mean?

Feature95% confidence interval (frequentist)95% credible interval (Bayesian)
The 95% refers toThe method: over many repeated studies, 95% of intervals calculated this way contain the true valueThis interval: given the prior, the model and the data, there is a 95% probability that the true value lies inside it
Needs a priorNoYes, stated in advance
Probability of benefitNot available. The p-value is P(data at least this extreme | no effect)Read directly, for example P(RR < 1 | data)
With a flat prior and a large trialNumerically close to the credible intervalNumerically close to the confidence interval, with a different meaning
Teaching trialRR 0.80, 95% CI 0.62 to 1.03, p = 0.086Sceptical prior: RR 0.88, 95% CrI 0.73 to 1.07, P(RR < 1) 90.3%

The confidence interval is explained in the hypothesis testing chapter.

Decide

Which approach answers your question?

Start from the decision you face.

What do you need to decide?

Which approach: all possible answers
  • Bayes' rule with one likelihood ratio: Convert the pre-test probability to odds, multiply by the likelihood ratio of the result, and convert back.
  • Chain the likelihood ratios: The post-test probability after the first result becomes the pre-test probability for the second. Say that you assumed the tests are independent.
  • Use accuracy measured for the combination: Multiplying the separate likelihood ratios will overstate the probability. Look for a study that reports the accuracy of both tests together in the same patients.
  • Parallel testing: Positive if either test is positive. Sensitivity and NPV rise, specificity and PPV fall.
  • Serial testing: Test B only after a positive test A, and positive only if both are positive. Specificity and PPV rise, sensitivity falls, and fewer B tests are needed.
  • Testing and test-treatment thresholds: Test only if the probability lies between the two thresholds, where a result can change what you do.
  • Decision tree with expected values: Lay out each strategy, its chance outcomes and their values. Choose the strategy with the highest expected value, then check how sensitive the choice is to the inputs.
  • Bayesian analysis of the trial: Combine a stated prior with the trial result to give a posterior distribution, a credible interval and the probability of benefit. Report more than one prior.

The formula

Worked example: the patient at 20%, step by step

All numbers are made-up teaching values, calculated in Python.

  1. Pre-test probability 20%, so pre-test odds = 0.20 ÷ 0.80 = 0.25.
  2. Test A: sensitivity 80%, specificity 90%. LR+ = 0.80 ÷ 0.10 = 8. LR− = 0.20 ÷ 0.90 = 0.222.
  3. Test A positive: odds 0.25 × 8 = 2, probability 2 ÷ 3 = 66.7%. Test A negative: odds 0.25 × 0.222 = 0.056, probability 5.3%.
  4. Test B: sensitivity 90%, specificity 95%, LR+ = 0.90 ÷ 0.05 = 18. Both positive, assuming independence: odds 2 × 18 = 36, probability 36 ÷ 37 = 97.3%.
  5. If the tests were strongly related (50% of the largest possible overlap in the interactive element), B's likelihood ratio after a positive A would be 3.5, and the probability after both would be 87.4%.
  6. Serial pathway in 1,000 patients: B done in 240, 148 called positive, combined sensitivity 72%, specificity 99.5%, PPV 97.3%, NPV 93.4%.
  7. Parallel pathway: 312 called positive, sensitivity 98%, specificity 85.5%, PPV 62.8%, NPV 99.4%.
  8. Thresholds with benefit 9 and harm 1: treatment threshold 10%, testing threshold 1.4%, test-treatment threshold 33.3%. At 20%, test first.
  9. Expected values at 20%: no treatment 0, treat everyone 0.20 × 9 − 0.80 × 1 = 1.00, test A first 1.36.

P(D | A+, B+) = 1 ÷ [1 + (1 − p) ÷ (p × LRA × LRB)] = 1 ÷ (1 + 0.8 ÷ 28.8) = 97.3%

In a real paper

Bayesian reanalysis and post-test probability in published studies

Open-access paper · CC BY 4.0

Lammers D, Richman J, Holcomb JB, Jansen JO. Use of Bayesian statistics to reanalyze data from the Pragmatic Randomized Optimal Platelet and Plasma Ratios trial. JAMA Netw Open. 2023;6(2):e230421.

The study: The PROPPR trial randomised 680 severely injured patients at 12 US level I trauma centres to a balanced 1:1:1 transfusion of plasma, platelets and red cells or a red cell-heavy 1:1:2 strategy. Neither primary outcome was statistically significant. This post hoc analysis refitted the data with Bayesian hierarchical models and non-informative priors, using the brms package in R 4.0.3, to report the probability that 1:1:1 was better.

What the authors wrote
“Using bayesian approaches, a 1:1:1 resuscitation was found to have a 93% (Bayes factor, 13.7; RR, 0.75 [95% credible interval, 0.45-1.11]) and 87% (Bayes factor, 6.56; RR, 0.82 [95% credible interval, 0.57-1.16]) probability of being superior to a 1:1:2 resuscitation with regards to 24-hour and 30-day mortality, respectively.”
In plain words

Given the trial data and a prior that adds almost no information, there is a 93% probability that balanced transfusion reduced 24-hour mortality (risk ratio below 1), and an 87% probability for 30-day mortality. The credible intervals still include 1, which is why the original analysis found no significant difference. The Bayes factor says how many times more probable the data are under one hypothesis than under the other: 13.7 for benefit against no benefit at 24 hours.

ReportedValueWhat it means
Frequentist result, 24 h12.7% vs 17.0%; RR 0.75 (95% CI 0.52-1.08); P = .12Not statistically significant at α 0.05
Posterior probability of benefit, 24 h93%P(RR < 1 | data) with a non-informative prior
Posterior probability of benefit, 30 d87%P(RR < 1 | data) for 30-day mortality
Probability of RR < 0.9 / < 0.8, 24 h85% / 68%The probability of at least a 10% or 20% relative reduction falls as the benefit you require grows
Bayes factor, 24 h13.7The data are 13.7 times more probable if 1:1:1 is better than if it is not
Check these when you read it
  • Which prior was used? A non-informative prior re-expresses the trial data as probabilities. It does not add outside evidence, so the result is not a stronger trial.
  • The analysis was post hoc, carried out years after the trial. Probabilities from an unplanned analysis are exploratory.
  • Check consistency: the results text and table give the 24-hour RR as 0.72 (95% CrI 0.45-1.11), while the abstract gives 0.75, and a table gives the 30-day Bayes factor as 6.2 against 6.56 in the abstract.
  • Look for the probability of a clinically important effect (RR < 0.9 or < 0.8), not only of any effect.
Open-access paper · CC BY 4.0

Brush JE Jr, Lee M, Sherbino J, Taylor-Fishwick JC, Norman G. Effect of teaching Bayesian methods using learning by concept vs learning by example on medical students' ability to estimate probability of a diagnosis: a randomized clinical trial. JAMA Netw Open. 2019;2(12):e1918023.

The study: A randomised trial in 61 medical students at McMaster University and Eastern Virginia Medical School. Students received explicit teaching on Bayesian revision (22), repeated worked cases with feedback (20), or neither (19). Each student then gave a pre-test probability for written cases, saw a test result, and gave a post-test probability, which was compared with the value calculated by Bayes' rule from the test's sensitivity and specificity.

What the authors wrote
“Mean (SE) probability estimates of students in the concept group were statistically significantly closer to calculated bayesian probability than the other 2 groups (concept, 0.4%; [0.7%]; experience, 3.5% [0.7%]; control, 4.3% [0.7%]; P < .001).”
In plain words

For each case the authors took the student's own pre-test probability, calculated with Bayes' rule how much a correct revision would change it, and compared that with the change the student actually made. On average, the concept group's revisions differed from the Bayes-rule revisions by 0.4 percentage points, against 3.5 and 4.3 points in the other groups. SE is the standard error of each mean.

ReportedValueWhat it means
Concept group0.4% (SE 0.7%)Average difference between the students’ change in probability and the change calculated by Bayes’ rule
Experience group3.5% (SE 0.7%)Repeated examples without explicit teaching
Control group4.3% (SE 0.7%)No teaching or examples
P value< .001The difference between groups is unlikely to be due to chance, but the authors describe it as relatively modest
Check these when you read it
  • Sensitivity, specificity and likelihood ratios were given with the questions only in the concept condition. Part of the concept group’s advantage may come from having those numbers.
  • The authors note that the study design may have alerted participants to the goals of the exercise, which could explain why all groups did better than earlier studies.
  • With 61 students, a difference of a few percentage points needs a confidence interval, not only a P value.

Software

How to run it

Most of the calculations in this chapter need only arithmetic. Software helps with confidence intervals for test accuracy and with Bayesian models.

R

# Post-test probability from a pre-test probability and a likelihood ratio
pre     <- 0.20
lr_pos  <- 0.80 / (1 - 0.90)            # LR+ = 8
odds    <- pre / (1 - pre) * lr_pos     # 2
odds / (1 + odds)                       # 0.667

# Sensitivity, specificity, PPV, NPV and likelihood ratios with 95% CIs
# epiR: dat = c(a, b, c, d) = c(TP, FP, FN, TN)
library(epiR)
epi.tests(c(160, 80, 40, 720), method = "exact", conf.level = 0.95)

# Normal-approximation posterior for a trial risk ratio
y   <- log(0.80); se <- 0.13            # trial estimate and SE of log RR
m0  <- 0;         s0 <- 0.15            # sceptical prior on log RR
w   <- c(1 / s0^2, 1 / se^2)
post_sd   <- sqrt(1 / sum(w))
post_mean <- (m0 * w[1] + y * w[2]) / sum(w)
exp(post_mean + c(-1, 1) * qnorm(0.975) * post_sd)   # 95% credible interval
pnorm(0, mean = post_mean, sd = post_sd)             # P(RR < 1) = 0.903

Functions checked against the documentation: epi.tests(dat, method = "exact", digits = 2, conf.level = 0.95) in the epiR package (version 2.0.96), where a, b, c and d are test-positive/outcome-positive, test-positive/outcome-negative, test-negative/outcome-positive and test-negative/outcome-negative counts. pnorm and qnorm are in base R (stats). Full Bayesian models, such as the one in the PROPPR reanalysis, can be fitted with the brms package.

SPSS

IBM SPSS Statistics (version 25 and later) has Bayesian procedures for continuous outcomes. For two independent groups: Analyze > Bayesian Statistics > Independent Samples Normal. Under Bayesian Analysis choose Characterize Posterior Distribution, Estimate Bayes Factor or Use Both Methods, and set the credible interval percentage and the priors in the dialog.

Reporting

How to report it

State where every input came from, what was assumed, and the result with its uncertainty.

Post-test probability and combined tests

“We estimated the pre-test probability of disease as 20% from the prevalence in our clinic. A positive result on test A (likelihood ratio 8.0) raised the probability to 67%. A second positive result on test B (likelihood ratio 18) raised it to 97%, assuming that the two tests are conditionally independent. In a serial pathway (test B only after a positive test A), the combined sensitivity was 72% and the combined specificity 99.5%.”

Bayesian analysis of a trial

“In a pre-specified Bayesian analysis with a sceptical prior centred on no effect (normal on the log risk-ratio scale, SD 0.15), the posterior risk ratio for 30-day mortality was 0.88 (95% credible interval 0.73 to 1.07). The posterior probability that the drug reduced mortality was 90.3%, and the probability of at least a 10% relative reduction was 58.9%. With a non-informative prior these probabilities were 95.7% and 81.7%.”

  • Report the prior (distribution, centre, spread), why it was chosen, and results under at least one other prior.
  • Report the model and software, and for full Bayesian models the sampling settings and convergence checks.
  • Keep the frequentist estimate with its 95% confidence interval alongside: CONSORT 2025 item 26 asks for the estimated effect size and its precision for each outcome, and for binary outcomes both absolute and relative effect sizes.

By specialty

Bayes' rule and combined tests by specialty

A patient with atypical chest pain has a positive stress test. You want to know whether coronary angiography is needed.

Estimate the pre-test probability first. The same positive result means a high probability in a patient with typical symptoms and a low one in a young patient without risk factors.

Common myths

Common mistaken beliefs

  • Myth: A positive result on a test with 90% sensitivity means a 90% chance of disease.

    Sensitivity is P(positive | disease). The chance of disease after a positive result depends on the pre-test probability and the false-positive rate.

  • Myth: Two positive tests always multiply their likelihood ratios.

    Only if the tests are conditionally independent. Related tests add less information, and multiplying overstates the probability.

  • Myth: Parallel testing is always better because it misses fewer cases.

    It also lowers specificity and PPV and needs more tests. In the teaching example PPV fell from 97.3% (serial) to 62.8%.

  • Myth: A serial pathway keeps the sensitivity of the first test.

    Combined sensitivity is Se_A × Se_B, lower than either test: 72% from tests of 80% and 90%.

  • Myth: Every available test should be done.

    Below the testing threshold or above the test-treatment threshold, no result would change management.

  • Myth: A 93% posterior probability of benefit proves the treatment works.

    It depends on the prior and the model, and a post hoc analysis is exploratory. Look at the probability of a clinically important benefit and at other priors.

Why this matters to you

Five consequences of misusing Bayes' rule and combined tests

  1. Overconfident diagnosis

    Chaining related tests gives a post-test probability that is too high, and further investigation stops too early.

  2. Unnecessary tests

    Tests ordered below the testing threshold or above the test-treatment threshold add cost and risk without changing management.

  3. A common viva error

    Answering "90%" when asked how likely disease is after a positive result on a test with 90% sensitivity.

  4. Misreading trials

    A non-significant trial is called negative, or a Bayesian probability from a flat prior is presented as independent confirmation.

  5. Harm to patients

    A parallel pathway used for confirmation labels many healthy patients as diseased. A serial pathway used for exclusion misses cases.

History

History of Bayesian reasoning in medicine

  1. 1763

    Thomas Bayes and Richard Price

    "An essay towards solving a problem in the doctrine of chances" by the late Rev. Thomas Bayes was communicated by Richard Price and published in the Philosophical Transactions of the Royal Society.

  2. 1959

    Robert Ledley and Lee Lusted

    Published "Reasoning foundations of medical diagnosis" in Science, describing how symbolic logic, probability and value theory can explain how physicians reason.

  3. 1975

    Stephen Pauker and Jerome Kassirer

    Described a therapeutic threshold: above it, treating is preferable, below it, withholding treatment is preferable.

  4. 1978

    Ward Casscells and colleagues

    Asked 60 students, house officers and physicians at Harvard Medical School to calculate the chance of disease after a positive result. 11 answered correctly, and the most common answer was 95%.

  5. 1980

    Pauker and Kassirer

    Published "The threshold approach to clinical decision making", with a testing threshold and a test-treatment threshold.

  6. 1995

    Gerd Gigerenzer and Ulrich Hoffrage

    Published "How to improve Bayesian reasoning without instruction: frequency formats" in Psychological Review.

  7. 1998

    ICH E9

    The statistical principles guideline for clinical trials stated that Bayesian and other approaches may be considered when the reasons for their use are clear and the conclusions do not depend on the particular assumptions and methods chosen.

  8. 2014

    Arjun Manrai and colleagues

    Repeated the Casscells question in Boston. 14 of 61 respondents (23%) answered correctly, and 95% was again the most common answer.

  9. 2026

    US Food and Drug Administration

    Published the draft guidance "Use of Bayesian Methodology in Clinical Trials of Drug and Biological Products" (notice of availability, 12 January 2026).

Guidelines

Bayesian methods and test results in guidelines

ICH E9 (1998), Introduction

“This should not be taken to imply that other approaches are not appropriate: the use of Bayesian (see Glossary) and other approaches may be considered when the reasons for their use are clear and when the resulting conclusions are sufficiently robust.”

ICH E9 (1998), Glossary

Bayesian approaches: “Approaches to data analysis that provide a posterior probability distribution for some parameter (e.g. treatment effect), derived from the observed data and a prior probability distribution for the parameter.”

US FDA draft guidance (January 2026)

“Use of Bayesian Methodology in Clinical Trials of Drug and Biological Products.” Its primary focus is “the use of Bayesian methods to support primary inference in pivotal clinical trials designed to support the effectiveness and safety of drug and biological products.”

CONSORT 2025, item 26

For each primary and secondary outcome, “the estimated effect size and its precision (such as 95% confidence interval)” and “for binary outcomes, presentation of both absolute and relative effect size”.

Reading a paper

Five checks when a paper uses Bayes' rule, combined tests or a Bayesian analysis

  • Is the pre-test probability stated, and does it match patients like yours?
  • When two tests are combined, were they studied together in the same patients, or were separate accuracies multiplied under an independence assumption?
  • For a serial or parallel pathway, are the combined sensitivity, specificity and number of tests reported, not only one of them?
  • In a Bayesian analysis, is the prior stated, justified before the data, and varied in a sensitivity analysis?
  • Is the probability of a clinically important effect reported, and not only the probability of any effect?

Why is it called that?

Word surgery

“Prior”

In everyday speech

Earlier, as in a prior appointment or a prior record.

In research

The probability distribution for an effect before the current data are seen. It must be stated and justified.

“Likelihood”

In everyday speech

Another word for probability or chance.

In research

How probable the observed data are for each possible value of the effect. A likelihood ratio compares two such probabilities.

“Independent”

In everyday speech

Separate, unconnected.

In research

For tests: within patients with the disease, and within patients without it, one result does not change the chance of the other.

Memory trick

Serial: both must be positive, so specificity rises. Parallel: either can be positive, so sensitivity rises.

Bayes' rule

named after the Rev. Thomas Bayes, whose essay was published after his death in 1763

Converts the probability of a result given a disease into the probability of the disease given the result.

Prior

Latin prior: "former, previous, first"

What was believed before the data.

Posterior

Latin posterior: "after, later, behind", from posterus "coming after"

What is believed after combining the prior with the data.

Credible

Latin credibilis: "worthy to be believed", from credere "to believe"

A credible interval is a range for the effect with a stated probability, given the prior and the data.

Exam answers

Model answer outlines

Bayesian methods in statistics, and their application in trials

MCh Surgical Oncology 2025 · DM Medical Oncology 2021
  1. Principle: probability expresses degree of belief. Bayes' theorem updates a prior distribution with the likelihood of the data to give a posterior distribution: posterior ∝ likelihood × prior.
  2. Contrast with frequentist methods: a p-value is P(data at least this extreme | no effect) and a confidence interval refers to repeated sampling. A Bayesian analysis gives P(effect | data), a credible interval and the probability of benefit.
  3. Priors: non-informative (flat), sceptical (centred on no effect), enthusiastic (centred on benefit), or informed by earlier trials. Pre-specify and justify, and show results under several priors.
  4. Worked example: trial RR 0.80 (95% CI 0.62 to 1.03, p = 0.086). With a sceptical prior (log RR mean 0, SD 0.15): posterior RR 0.88, 95% credible interval 0.73 to 1.07, P(RR < 1) = 90.3%. With a flat prior: 95.7%.
  5. Applications in trials: interpreting non-significant results as probabilities of benefit and of clinically important benefit, adding information from earlier evidence, interim decision rules in adaptive designs, and post hoc reanalysis (for example, the PROPPR transfusion trial: 93% probability that 1:1:1 reduced 24-hour mortality, JAMA Netw Open 2023).
  6. Regulatory view: ICH E9 (1998) allows Bayesian approaches when the reasons for their use are clear and the conclusions do not depend on the assumptions chosen. The US FDA published a draft guidance on Bayesian methodology in drug and biologic trials in January 2026.
  7. Limitations: the result depends on the prior, a strong prior can dominate a small trial, the models need expertise and software (R brms, SPSS Bayesian Statistics), and readers are less familiar with them.
  8. Other medical uses of Bayesian reasoning: diagnosis (pre-test to post-test probability), combining tests, and decision analysis.

Post-test probability

DM Neonatology 2024
  1. Definition: the probability that a patient has the disease after a test result is known. After a positive result it equals the PPV in a population with that pre-test probability.
  2. Pre-test probability: from prevalence in the setting, then adjusted by history, examination and validated scores.
  3. Calculation: pre-test odds = p ÷ (1 − p); post-test odds = pre-test odds × LR; post-test probability = odds ÷ (1 + odds). LR+ = Se ÷ (1 − Sp), LR− = (1 − Se) ÷ Sp.
  4. Example: pre-test 20%, sensitivity 80%, specificity 90%: LR+ 8, post-test after a positive result 66.7%, after a negative result 5.3%.
  5. Gain from a test: largest in the middle range of pre-test probability (+46.7 points at 20%, +17.0 at 80% in the example).
  6. Sequential tests: the post-test probability of one test becomes the pre-test probability of the next, valid only if the tests are conditionally independent.
  7. Tools: Fagan's nomogram (1975) and natural-frequency trees avoid the arithmetic.
  8. Use: compare the post-test probability with the treatment threshold to decide whether to treat, test further or stop.

Bayes' theorem and its clinical applications

DM Cardiology 2012
  1. Statement: P(D | T) = P(T | D) × P(D) ÷ P(T), where P(T) = P(T | D)P(D) + P(T | no D)P(no D). Named after Thomas Bayes (essay published in 1763).
  2. Meaning: converts what studies measure, P(result | disease), into what the clinician needs, P(disease | result). In odds form: post-test odds = pre-test odds × likelihood ratio.
  3. Diagnosis: interpreting a positive result depends on pre-test probability. For a disease with prevalence 1 in 1,000 and a 5% false-positive rate, a positive result means about a 2% chance of disease, yet in 1978 most Harvard respondents answered 95%.
  4. Combining tests: chaining likelihood ratios, and serial testing (both positive: Se = Se_A × Se_B, higher specificity) or parallel testing (either positive: higher sensitivity), assuming independence.
  5. Screening: low prevalence gives a low PPV, so screen-positive results need confirmation.
  6. Decision making: treatment threshold H ÷ (H + B), testing and test-treatment thresholds (Pauker and Kassirer, 1975 and 1980), and decision trees with expected values.
  7. Trials: Bayesian analysis gives the posterior probability of benefit from a prior and the trial data.
  8. Limits: depends on an accurate pre-test probability and on test accuracy measured in similar patients, and on the independence assumption when tests are combined.

The one thing to remember

A test result changes a probability. Say what the probability was before the test, and check that combined tests are independent.

Worked examples, test accuracies, benefit and harm values and the trial in the Bayesian example are made-up teaching numbers, calculated in Python (SciPy). The threshold formulas assume that the test itself has no risk or cost.

Sources: Bayes T. An essay towards solving a problem in the doctrine of chances. Phil Trans R Soc Lond 1763;53:370–418 (doi:10.1098/rstl.1763.0053). Ledley RS, Lusted LB. Science 1959;130:9–21 (doi:10.1126/science.130.3366.9; PMID 13668531). Pauker SG, Kassirer JP. N Engl J Med 1975;293:229–34 (doi:10.1056/NEJM197507312930505; PMID 1143303). Casscells W, Schoenberger A, Graboys TB. N Engl J Med 1978;299:999–1001 (doi:10.1056/NEJM197811022991808; PMID 692627). Pauker SG, Kassirer JP. N Engl J Med 1980;302:1109–17 (doi:10.1056/NEJM198005153022003; PMID 7366635). Gigerenzer G, Hoffrage U. Psychol Rev 1995;102:684–704 (doi:10.1037/0033-295X.102.4.684). Hoffrage U, Lindsey S, Hertwig R, Gigerenzer G. Science 2000;290:2261–2 (doi:10.1126/science.290.5500.2261; PMID 11188724). Manrai AK et al. JAMA Intern Med 2014;174:991–3 (doi:10.1001/jamainternmed.2014.1059; PMID 24756486). Nixon MP et al. BMC Med Inform Decis Mak 2024;24:210 (doi:10.1186/s12911-024-02610-3; PMID 39075421). Brush JE Jr et al. JAMA Netw Open 2019;2:e1918023 (doi:10.1001/jamanetworkopen.2019.18023; PMID 31860107). Lammers D et al. JAMA Netw Open 2023;6:e230421 (doi:10.1001/jamanetworkopen.2023.0421; PMID 36811858). ICH E9 Statistical Principles for Clinical Trials, 1998. US Food and Drug Administration. Use of Bayesian Methodology in Clinical Trials of Drug and Biological Products; draft guidance; notice of availability. Federal Register, 12 January 2026 (Docket FDA-2025-D-3217). Hopewell S et al. CONSORT 2025 statement. BMJ 2025;389:e081123 (doi:10.1136/bmj-2024-081123; PMID 40228833). Fagan TJ. N Engl J Med 1975;293:257 (doi:10.1056/NEJM197507312930513). epiR package reference manual, version 2.0.96 (CRAN). IBM SPSS Statistics 25 documentation, Bayesian Independent Sample Inference.

Further reading: Indrayan A. Medical Biostatistics. 4th ed. Chapman and Hall/CRC; 2017 (doi:10.4324/9781315100265), chapters 9 and 10. Jawlik AA. Statistics from A to Z: Confusing Concepts Clarified. Hoboken, NJ: John Wiley & Sons; 2016 (doi:10.1002/9781119272021).

Past papers

TNMGRMU questions on bayes' rule and combining tests

Questions from MD, MS, DM, MCh and Ph.D. papers that this chapter helps you answer. Tap a sitting to open the university's own question paper.

  1. ×1Bayesian methods and its application in trialsMCh Surgical Oncology · Feb 2025
  2. ×1Post-test probabilityDM Neonatology · Aug 2024
  3. ×1Bayesian methods in statisticsDM Medical Oncology · Aug 2021
  4. ×1Describe Bayes theorem and its clinical applicationsDM Cardiology · Aug 2012