Chapter 17 · Making sense of numbers

How good is a test?

Sensitivity, specificity, predictive values, likelihood ratios and the ROC curve.

28 TNMGRMU questions answered hereset 29 times, 1990–2025

The problem first

You are a DM Neonatology resident. In this made-up teaching example, a new bedside sepsis test has 90% sensitivity and 80% specificity. In your NICU, about 1 in 10 babies tested turns out to have culture-proven sepsis.

A baby's test comes back positive. The parents ask how likely it is that their baby has sepsis, and you tell them 90%. At your thesis presentation the examiner asks one question.

“Of the babies who test positive, how many actually have sepsis?”

Start here

How do you find out whether a test gives the right answer?

Do the new test and the best available test on the same patients, then count how often they agree. The counts go into a 2 × 2 table.

The test being studied is the index test. The best available method of deciding who has the disease is the reference standard, often called the gold standard. Every patient falls into one of four cells.

Reference standard: blood cultureSepsisNo sepsisTotalNew test +New test −Totaltrue positive (TP)90false positive (FP)180false negative (FN)10true negative (TN)7202707301009001,000PPV 33.3%NPV 98.6%Sensitivity 90%Specificity 80%
Teaching example: 1,000 babies, 100 with sepsis. Sensitivity and specificity are read down the columns (people grouped by disease). PPV and NPV are read across the rows (people grouped by test result).

Two properties of the test

Of the patients who have the disease, how many does the test pick up?

Test results in people with and without a disease usually overlap. Wherever you put the cut-off, some people land on the wrong side.

020406080100test result (teaching units)cut-off 35test negativetest positiveNo diseaseDiseasefalse negatives:disease missedfalse positives:healthy but positive
At a cut-off of 35, 77% of people with the disease test positive and 77% of people without it test negative. Moving the cut-off to the left detects more disease but labels more healthy people positive.

Sensitivity is the proportion of people with the disease whose test is positive. Specificity is the proportion of people without the disease whose test is negative. Both are calculated inside one disease group, so in principle they do not depend on how common the disease is.

Memory trick

SnNout: when a test with high Sensitivity is Negative, it rules the disease out. SpPin: when a test with high Specificity is Positive, it rules the disease in.

A very sensitive test misses few cases, so a negative result makes the disease unlikely. A very specific test gives few false positives, so a positive result makes the disease likely. Pewsner and colleagues (BMJ, 2004) added two cautions. The ability to rule out also falls when specificity is low, and the ability to rule in also depends on sensitivity. Studies quoted as showing SnNout or SpPin properties may be affected by spectrum bias, partial verification bias or incorporation bias, or may be too small.

The formula

Working through the 2 × 2 table

The numbers from the sepsis example (made-up teaching numbers).

TP
true positive: disease present, test positive (90)
FP
false positive: disease absent, test positive (180)
FN
false negative: disease present, test negative (10)
TN
true negative: disease absent, test negative (720)

Sensitivity = TP ÷ (TP + FN) = 90 ÷ 100 = 90%

Specificity = TN ÷ (TN + FP) = 720 ÷ 900 = 80%

PPV = TP ÷ (TP + FP) = 90 ÷ 270 = 33.3%

NPV = TN ÷ (TN + FN) = 720 ÷ 730 = 98.6%

Prevalence = (TP + FN) ÷ total = 100 ÷ 1,000 = 10%

Accuracy = (TP + TN) ÷ total = 810 ÷ 1,000 = 81%

Youden index J = sensitivity + specificity − 1 = 0.90 + 0.80 − 1 = 0.70

Diagnostic odds ratio = (TP × TN) ÷ (FP × FN) = 64,800 ÷ 1,800 = 36

Accuracy mixes the two kinds of error and changes with prevalence, so it is a poor single summary. The answer to the parents is 33%, not 90%.

The patient's question

My patient tested positive. How likely is the disease?

That is the positive predictive value. It depends on how common the disease is in the people you test.

Positive predictive value (PPV) is the proportion of people with a positive test who have the disease. Negative predictive value (NPV) is the proportion of people with a negative test who are free of it.

PPV = (sensitivity × prevalence) ÷ [sensitivity × prevalence + (1 − specificity) × (1 − prevalence)]

Prevalence 1%PPV 4.3%9 true positives + 198 false positivesPrevalence 10%PPV 33.3%90 true positives + 180 false positivesPrevalence 50%PPV 81.8%450 true positives + 100 false positives
true positivesfalse positives
Same test every time. When the disease is rare, the few true positives are outnumbered by false positives from the large healthy group. At 1% prevalence, 198 of 207 positive results are false. NPV moves the other way: 99.9% at 1% prevalence, 88.9% at 50%.

This is why a test that performs well in a referral hospital gives many more false positives when it is used to screen the general population.

When the disease is rare, PPV depends much more on specificity than on sensitivity. The number of false positives is set by the specificity and by the size of the group without the disease, and that group is large. For the sepsis test at 1% prevalence, raising sensitivity from 90% to 99% raises the PPV only from 4.3% to 4.8%. Raising specificity from 80% to 99% raises it to 47.6%. NPV depends more on sensitivity. At 50% prevalence, raising sensitivity to 99% raises the NPV from 88.9% to 98.8%, while raising specificity to 99% raises it only to 90.8%.

Interactive

Change the prevalence and watch PPV and NPV change

Try this: move the prevalence from 10% down to 1% and watch the PPV fall from 33.3% to 4.3% while the NPV rises.

Teaching example: 1,000 people tested. Change how common the disease is and watch what a positive or a negative result means.

10%

Rarer disease → lower PPV, higher NPV

90%

Higher sensitivity → fewer missed cases, higher NPV

80%

Higher specificity → fewer false positives, higher PPV

33.3%PPV: of those who test positive, have the disease
98.6%NPV: of those who test negative, are free of it

Per 1,000 tested (rounded): 90 true positive · 180 false positive · 10 false negative · 720 true negative

has disease, test positivehas disease, test negativeno disease, positiveno disease, negative0%50%100%0.5%1%2%5%10%20%50%prevalence (log scale)NPVPPV

Probability

Why is the chance of a positive test in disease different from the chance of disease after a positive test?

Sensitivity and PPV are both conditional probabilities. The condition is on opposite sides.

A conditional probability is a probability calculated within one group only. P(A | B) is read as “the probability of A given B”. Sensitivity is P(positive | disease), calculated among people who have the disease. PPV is P(disease | positive), calculated among people whose test is positive. In the sepsis example the first is 90% and the second is 33.3%.

Hospital records and published case series are usually organised by diagnosis, so they report how often patients with a disease had a symptom or a positive test. At the bedside you need the reverse. Bayes' rule converts one into the other.

P(disease | positive) = P(positive | disease) × P(disease) ÷ P(positive)

P(positive) = P(positive | disease) × P(disease) + P(positive | no disease) × P(no disease)

Sepsis example: 0.90 × 0.10 ÷ (0.90 × 0.10 + 0.20 × 0.90) = 0.09 ÷ 0.27 = 33.3%

P(disease)
the prior (pre-test) probability: prevalence in patients like yours, or your estimate from the history and examination (0.10)
P(positive | no disease)
1 − specificity, the false-positive rate (0.20)
P(positive)
the proportion of all patients tested who test positive (270 of 1,000 = 0.27)
P(disease | positive)
the posterior (post-test) probability, which is the PPV
FeatureSensitivity, P(positive | disease)PPV, P(disease | positive)
Calculated amongPeople with the disease (a column of the 2 × 2 table)People with a positive test (a row of the 2 × 2 table)
AnswersHow often the test detects the disease when it is presentHow likely the disease is in a patient with a positive result
Changes with prevalenceNo, in principleYes, strongly
Used byThe person choosing or evaluating a testThe clinician explaining a result to a patient
Sepsis example90%33.3%

The PPV formula in the section above is Bayes' rule written for a test result. The likelihood ratio method in the next section is the same rule written in odds. When several diagnoses compete for the same presentation, the rule extends to all of them at once, and when tests are combined their errors may not be independent. Both are covered in the Practical Statistics chapter on Bayes' rule and combined tests.

Likelihood ratios

How much does this result change the probability of disease?

A likelihood ratio tells you how many times more likely a result is in a person with the disease than in a person without it.

LR+ = sensitivity ÷ (1 − specificity) = 0.90 ÷ 0.20 = 4.5

LR− = (1 − sensitivity) ÷ specificity = 0.10 ÷ 0.80 = 0.125

Likelihood ratios work on odds, not probabilities. Start with the pre-test probability: your estimate before the test, from prevalence and clinical findings. The result gives the post-test probability.

Pre-test odds = 0.10 ÷ 0.90 = 0.111

Positive: 0.111 × 4.5 = 0.50 odds → 0.50 ÷ 1.50 = 33.3%

Negative: 0.111 × 0.125 = 0.0139 odds → 0.0139 ÷ 1.0139 = 1.4%

odds
probability ÷ (1 − probability)
probability
odds ÷ (1 + odds)
pre-test probability 10%1%2%5%10%20%50%80%90%95%99%positive: odds × 4.5negative: odds × 0.12533.3%1.4%
On a log-odds scale, multiplying the odds by a likelihood ratio moves the probability by a fixed distance. Fagan's nomogram (1975) does the same conversion with a ruler and no calculation.

LR above 1 raises the probability of disease, LR below 1 lowers it, and LR = 1 does not change it. McGee (2002) gave a bedside approximation that avoids odds. It is accurate to within 10 percentage points for pre-test probabilities between 10% and 90%.

Likelihood ratioApproximate change in probability
10+45 percentage points
5+30
2+15
1no change
0.5−15
0.2−30
0.1−45

Likelihood ratios come from sensitivity and specificity, so they do not change with prevalence in the way PPV does. You can apply them to your own patient's pre-test probability. Tests with several result levels can have a likelihood ratio for each level.

Interactive

Change the pre-test probability and the likelihood ratio, and watch the post-test probability

Try this: keep the pre-test probability at 10% and move the likelihood ratio from 4.5 (a positive result on the sepsis test) to 0.125 (a negative result). Watch the post-test probability fall from 33.3% to 1.4%.

10%

Your estimate before the test, from prevalence and clinical findings

4.5

Above 1 raises the probability, below 1 lowers it, 1 leaves it unchanged

pre-test 10%1%2%5%10%20%50%80%90%95%99%post-test 33.3%
33.3%post-test probability
+23.3percentage points of change

Pre-test odds = 0.10 ÷ 0.90 = 0.111

Post-test odds = 0.111 × 4.5 = 0.500

Post-test probability = 0.500 ÷ 1.50 = 33.3%

McGee's bedside table (2002) has no entry for a likelihood ratio of 4.5. Use the odds calculation.

ROC curve

The test gives a number. Which cut-off should you use?

Every cut-off gives a different pair of sensitivity and specificity. The receiver operating characteristic (ROC) curve plots all of them.

050100test resultno diseasediseaseABCABCchance (AUC 0.5)(1, 1): everyone positive(0, 0): everyone negative00.510.511 − specificitysensitivity
this test, AUC 0.86a better test, AUC 0.96
Drawn to scale from the teaching distributions. A (cut-off 15): sensitivity 96%, specificity 40%. B (35): 77% and 77%, the highest Youden index (0.55). C (55): 40% and 96%. Lowering the cut-off moves the point up and to the right.
  • Axes: sensitivity (true-positive rate) on the y-axis, 1 − specificity (false-positive rate) on the x-axis.
  • Corners: a cut-off above every result makes everyone negative, the point (0, 0). A cut-off below every result makes everyone positive, the point (1, 1). So every ROC curve runs from the lower left corner to the upper right corner. Tests differ in the path between them.
  • Diagonal: a test no better than chance lies on the line from (0, 0) to (1, 1). The closer the curve comes to the top left corner (0, 1), the better the test separates the two groups.
  • Area under the curve (AUC): 0.5 for chance, 1.0 for perfect separation. Hanley and McNeil (1982) showed that the AUC is the probability that a randomly chosen person with the disease is rated as more abnormal than a randomly chosen person without it, the same quantity estimated by the Wilcoxon statistic.
  • Choosing a cut-off: the Youden index (sensitivity + specificity − 1, the largest vertical distance from the diagonal) or the point closest to the top left corner give equal weight to both errors. If a missed case is worse, as in neonatal sepsis, choose a lower cut-off with higher sensitivity. If a false positive leads to harmful treatment, choose a higher cut-off.
  • Uses: choosing a cut-off for a laboratory test or score, comparing two tests over all cut-offs, judging how well a prediction model separates patients with and without an outcome, and reader studies in radiology.
  • Slope: the slope of the curve at a point is the likelihood ratio for a result at that cut-off.

Interactive

Move the cut-off and watch sensitivity, specificity and the ROC point change

Try this: move the cut-off from 35 down to 15. Watch sensitivity rise, specificity fall, and the point move up and to the right along the ROC curve.

Teaching example: the distributions from the figures above. Results average 20 in people without the disease and 50 in people with it, with SD 20 in both. A result at or above the cut-off counts as positive.

35

Lower cut-off → higher sensitivity, lower specificity

Prevalence, for PPV and NPV

In this model, sensitivity and specificity do not change with prevalence

050100test resultno diseasediseasecut-off 35negativepositiveAUC 0.8600.510.511 − specificitysensitivity
shaded: people with the disease who test negativeshaded: people without the disease who test positive
77.3%sensitivity: people with the disease who test positive
77.3%specificity: people without the disease who test negative

Youden index J = sensitivity + specificity − 1 = 0.55 · ROC point (0.23, 0.77)

This cut-off gives the highest Youden index for these two groups. Both groups have the same SD, and 35 is halfway between their means of 20 and 50.

Per 1,000 people tested at 10% prevalence (counts rounded to whole people):

ResultDiseaseNo diseasePredictive value
Test positive77 true positive204 false positivePPV 27.5%
Test negative23 false negative696 true negativeNPV 96.8%

The formula

How do you build a ROC curve and calculate its area from real counts?

Teaching example (made-up numbers): an ultrasound grade from 1 (normal) to 5 (clearly abnormal) in 100 children with appendicitis and 150 without.

Grade12345Total
Appendicitis46153045100
No appendicitis604525155150

Step 1. Take each grade in turn as the cut-off, and call that grade and every higher grade positive. Step 2. At each cut-off, calculate sensitivity and 1 − specificity from the counts. Step 3. Plot the points, add (0, 0) and (1, 1), and join them with straight lines.

Positive if grade ≥SensitivitySpecificity1 − specificityYouden JDistance to (0, 1)
545/100 = 0.450145/150 = 0.9670.0330.4170.551
475/100 = 0.750130/150 = 0.8670.1330.6170.283
390/100 = 0.900105/150 = 0.7000.3000.6000.316
296/100 = 0.96060/150 = 0.4000.6000.3600.601
1100/100 = 1.0000/150 = 0.0001.0000.0001.000

Distance to (0, 1) = √[(1 − sensitivity)² + (1 − specificity)²]

Grade 4 or more has both the highest Youden index (0.617) and the shortest distance to the top left corner (0.283). The two rules do not always choose the same cut-off.

AUC (trapezoidal rule) = sum over the strips of width × average height

AUC = 0.0075 + 0.0600 + 0.1375 + 0.2790 + 0.3920 = 0.876

The same number comes from comparing children in pairs. Of the 15,000 possible pairs of one child with and one child without appendicitis, the child with appendicitis has the higher grade in 12,360 pairs and the same grade in 1,560. Counting each tie as half, (12,360 + 780) ÷ 15,000 = 0.876. This is the Mann–Whitney (Wilcoxon) statistic divided by the number of pairs, which is why Hanley and McNeil could describe the AUC as a probability.

  • Empirical or smooth curve: the trapezoidal method uses only the observed points and makes no assumption about the shape of the distributions. A binormal (parametric) method fits a smooth curve, like the one in the ROC figure above, by assuming that the results in each group follow normal distributions after a suitable transformation.
  • Precision: report the AUC with its 95% confidence interval. When two tests are done on the same patients, their AUCs are correlated. DeLong, DeLong and Clarke-Pearson (Biometrics, 1988) gave a nonparametric method that estimates this correlation so that the two areas can be compared.
  • Curves that cross: two tests can have the same AUC and still differ. One is better where high sensitivity is needed, the other where high specificity is needed. The partial AUC, the area over a chosen range of false-positive rates, compares them only over the range that matters clinically.
  • AUC below 0.5: the test ranks people the wrong way round, with lower values in people with the disease. Reverse the direction of positivity, and the AUC becomes 1 minus the original value.

Interactive

Change the counts at each grade and watch the ROC curve and its area change

Try this: press "Grades reversed" and watch the curve fall below the diagonal and the AUC drop from 0.876 to 0.124.

Teaching example, made-up numbers: an ultrasound grade from 1 (normal) to 5 (clearly abnormal). The worked example above.

Number of children at each grade, editable
Grade12345Total
Appendicitis100
No appendicitis150
0.876area under the ROC curve (trapezoidal rule)
grade ≥ 4cut-off with the highest Youden index (0.617): sensitivity 75.0%, specificity 86.7%
Sensitivity and specificity at each cut-off
Positive if grade ≥TPFPSensitivity1 − specificityYouden J
545 of 1005 of 1500.4500.0330.417
475 of 10020 of 1500.7500.1330.617
390 of 10045 of 1500.9000.3000.600
296 of 10090 of 1500.9600.6000.360
1100 of 100150 of 1501.0001.0000.000
≥5≥4≥3≥200.510.511 − specificitysensitivityAUC 0.876shaded area, trapezoidal ruleHighest Youden index: grade ≥ 4sensitivity 75.0%, specificity 86.7%J = 0.617 (red point)dashed line: a test no better than chance

pairs of one child with and one without appendicitis: 15000 · child with appendicitis has the higher grade: 12360 · same grade: 1560

(12360 + 1560 ÷ 2) ÷ 15000 = 0.876. This is the same as the trapezoidal AUC, because the area under an empirical ROC curve equals the Mann–Whitney statistic divided by the number of pairs.

Pre-test and post-test probability

When will the test result change what you do for the patient?

A test changes management only if its result can move the probability of disease across the level at which you would treat.

11%59%+30 points0%50%100%0%30%50%100%pre-test probabilitypost-test probabilityCut-off Bsensitivity and specificity 77%red line: treat if 30% or morePre-test below 11%no test, no treatmentPre-test 11% to 59%do the test, act on the resultPre-test above 59%treat without the test
after a positive resultafter a negative resultno test: post-test equals pre-test
Teaching example: cut-off B of the test in the ROC figure (sensitivity and specificity 77%), with treatment given if the probability of disease is 30% or more. For each pre-test probability, the upper curve gives the probability after a positive result (the PPV at that prevalence) and the lower curve the probability after a negative result (1 − NPV).

The vertical distance between a curve and the diagonal is the change the result makes. For cut-off B, a positive result adds the most, 30 percentage points, when the pre-test probability is about 35%. At a pre-test probability of 90% it adds about 7 points, to 96.9%. When the pre-test probability is very high or very low, the result changes it little.

The cut-off changes the curves. Cut-off C (specificity 96%) can raise the probability by up to 52 percentage points after a positive result. Cut-off A (sensitivity 96%) can lower it by up to 52 points after a negative result. A highly specific test is most useful for ruling in, and a highly sensitive test for ruling out, as SpPin and SnNout state.

The threshold approach

Pauker and Kassirer (N Engl J Med, 1980) defined two threshold probabilities. If the probability of disease is below the testing threshold, withhold treatment and do not test. If it is above the test-treatment threshold, treat without testing. Between the two, do the test and let the result decide. In their method the thresholds also depend on the risks of the test and on the benefits and risks of the treatment.

Teaching example, ignoring any risk from the test itself. You would treat at a probability of 30% or more, and the test is cut-off B (LR+ 3.413, LR− 0.293). The testing threshold is the pre-test probability at which a positive result just reaches 30%. The test-treatment threshold is the pre-test probability at which a negative result just falls to 30%.

Odds at the treatment threshold = 0.30 ÷ 0.70 = 0.4286

Testing threshold: odds 0.4286 ÷ 3.413 = 0.126, probability 0.126 ÷ 1.126 = 11.2%

Test-treatment threshold: odds 0.4286 ÷ 0.293 = 1.463, probability 1.463 ÷ 2.463 = 59.4%

The test is worth doing when the pre-test probability lies between about 11% and 59%. Below 11%, even a positive result leaves the probability under 30%. Above 59%, even a negative result leaves it at 30% or more. The two vertical red lines in the figure mark the same points.

The ROC curve shows sensitivity and specificity but not predictive values. Shiu and Gatsonis (2008) described the predictive ROC (PROC) curve, which plots every pair of PPV and NPV as the cut-off for a positive result is varied.

Decide

Which measure answers your question?

Start from the clinical question, not from the formula.

What do you want to know?

Which measure: all possible answers
  • ROC curve and AUC: Plot sensitivity against 1 − specificity at every cut-off. The AUC summarises all cut-offs. Choose the working cut-off from the curve.
  • Sensitivity and specificity: Read down the columns of the 2 × 2 table. Report both with 95% confidence intervals.
  • PPV and NPV: Read across the rows of the 2 × 2 table. They apply only where the prevalence is similar to the study.
  • Likelihood ratios: Convert the pre-test probability to odds, multiply by the likelihood ratio, and convert back to a probability.
  • A highly sensitive test with a negative result: SnNout. Check that the negative likelihood ratio is small and its confidence interval is narrow.
  • A highly specific test with a positive result: SpPin. Check that the positive likelihood ratio is large and its confidence interval is narrow.
  • Reliability, not accuracy: Use kappa for categories and the intraclass correlation coefficient for measurements. Two observers can agree and both be wrong.
  • Screening criteria and a randomised trial: Check the Wilson and Jungner criteria, then look for a randomised trial showing fewer deaths from the disease in the screened group.
  • Pre-test probability and the threshold approach: Estimate the pre-test probability. Do the test only if a positive or a negative result could move the probability across the level at which you would treat.

Comparison

Why does a screening test need high sensitivity, and a diagnostic test high specificity?

A screening test only decides who needs further tests. A diagnostic test decides who is treated.

FeatureScreening testDiagnostic test
Who is testedPeople without symptoms, or a defined high-risk groupPeople with symptoms or a positive screening test
PurposeFind people who need further testingConfirm or exclude the disease before treatment
PriorityHigh sensitivity: a missed case loses the chance of early treatmentHigh specificity: a false positive can lead to surgery, chemotherapy or a lifelong label
MnemonicSnNout: a negative result rules outSpPin: a positive result rules in
False positivesSorted out by the next, diagnostic testNot corrected by any later test
Prevalence and PPVLow prevalence, so PPV is lowHigher prevalence, so PPV is higher
RequirementsCheap, quick, safe and acceptable to large numbersCan be more invasive and expensive
ExampleClinical breast examination at 30–65 yearsBreast ultrasound and biopsy in a screen-positive woman

Using two tests together

Parallel testing (positive if either test is positive) raises sensitivity. Serial testing (positive only if both are positive, the usual screening-then-confirmation pathway) raises specificity. The formulas assume the two tests make errors independently.

Parallel: sensitivity = 1 − (1 − SeA)(1 − SeB), specificity = SpA × SpB

Serial: sensitivity = SeA × SeB, specificity = 1 − (1 − SpA)(1 − SpB)

Teaching example: test A has sensitivity 90% and specificity 80%, test B 80% and 95%. In parallel: sensitivity 98%, specificity 76%. In series: sensitivity 72%, specificity 99%.

These formulas use the product rule of probability. For independent events, the probability that both occur is the product of their probabilities. If the two tests tend to make the same mistakes, for example two imaging tests that both miss small lesions, the formulas overstate the gain in sensitivity from parallel testing and the gain in specificity from serial testing. Combining tests is treated in more depth in the Practical Statistics chapter on Bayes' rule and combined tests.

At 10% prevalence, test A alone has PPV 33.3% and NPV 98.6%. In series, the PPV rises to 88.9% and the NPV falls to 97.0%. In parallel, the NPV rises to 99.7% and the PPV falls to 31.2%. Serial testing is used to confirm a diagnosis and parallel testing to exclude one.

Serial testing is usually done in sequence, with test B given only to people who are positive on test A. The classification is the same as giving both tests to everyone and calling positive only those positive on both, but far fewer B tests are needed. Among 1,000 people tested at 10% prevalence, 270 are positive on test A (90 true and 180 false positives), so 270 people receive test B instead of 1,000.

Screening programmes

When is a disease worth screening for?

In 1968 WHO published Wilson and Jungner's report, Principles and practice of screening for disease. Its ten criteria are still the starting point.

  1. The condition sought should be an important health problem.
  2. There should be an accepted treatment for patients with recognized disease.
  3. Facilities for diagnosis and treatment should be available.
  4. There should be a recognizable latent or early symptomatic stage.
  5. There should be a suitable test or examination.
  6. The test should be acceptable to the population.
  7. The natural history of the condition, including development from latent to declared disease, should be adequately understood.
  8. There should be an agreed policy on whom to treat as patients.
  9. The cost of case-finding (including diagnosis and treatment of patients diagnosed) should be economically balanced in relation to possible expenditure on medical care as a whole.
  10. Case-finding should be a continuing process and not a “once and for all” project.

Why screen-detected patients seem to do better

Without screeningsymptoms, diagnoseddies (same date)2 years after diagnosisWith screeningfound by screeningdies (same date)5 years after diagnosis0246810years from the start of the disease
Lead-time bias: screening moves the date of diagnosis earlier without moving the date of death, so survival measured from diagnosis looks longer. Teaching timeline.
  • Length-time bias: slow-growing disease stays longer in the stage that screening can detect, so screening finds a larger share of the slow, less dangerous cases.
  • Overdiagnosis: screening finds disease that would never have caused symptoms or death in that person's lifetime. These patients are counted as cures.
  • Volunteer (healthy screenee) bias: people who come for screening are often healthier and more health-conscious than people who do not.

Because of these biases, longer survival after diagnosis does not prove that screening works. The evidence needed is a randomised trial showing fewer deaths from the disease in the group invited for screening.

Characteristics of a test

Apart from accuracy, what makes a test good?

A test must give the right answer, give the same answer when repeated, and be usable in the setting where it is needed.

CharacteristicQuestionMeasured by
Validity (accuracy)Does it classify people correctly?Sensitivity, specificity, PPV, NPV, likelihood ratios, ROC curve and AUC
Reliability (precision)Does it give the same result when repeated, by the same or another observer or laboratory?Kappa for categories. Intraclass correlation coefficient and coefficient of variation for measurements
YieldHow many previously unknown cases does it find?New cases detected per number screened
Acceptability and safetyWill people agree to it, and can it harm them?Uptake, complications, discomfort
Cost and feasibilityCan it be done at this level of care?Cost per case detected, equipment and training needed
Clinical usefulnessDoes the result change management and outcome?Trials comparing strategies with and without the test

A reliable test is not always valid: two observers can agree and both be wrong. A valid test must be reasonably reliable.

Analytical and diagnostic sensitivity are different things

FeatureAnalytical (laboratory)Diagnostic (clinical)
Studied inSamples, in the laboratoryPatients, against a reference standard
SensitivityThe smallest amount of substance in a sample that can accurately be measuredThe percentage of people with the disorder who test positive
SpecificityThe ability to measure one particular organism or substance rather than othersThe percentage of people without the condition who test negative
LinkHigh analytical sensitivity does not guarantee acceptable diagnostic sensitivitySample contamination causes false positives and lowers diagnostic specificity

Definitions from Saah and Hoover (1997), who advise always adding the word “analytical” or “diagnostic”.

Biases that make a test look better than it is

  • Spectrum bias: comparing clear-cut severe cases with healthy volunteers. Ransohoff and Feinstein (1978) showed that without a broad spectrum of patients, sensitivity and specificity can be falsely high.
  • Review bias: reading the index test while knowing the reference result, or the reverse. The two must be interpreted independently.
  • Partial verification (work-up) bias: only patients with a positive index test receive the reference standard.
  • Incorporation bias: the index test is part of the reference standard.
  • Differential verification: patients receive different reference standards depending on the index test result, for example surgery for positives and follow-up for negatives. Lijmer and colleagues (JAMA, 1999) found that this overestimated diagnostic performance (relative diagnostic odds ratio 2.2, 95% CI 1.5 to 3.3).
  • Imperfect reference standard: the reference standard itself makes some errors. Every disagreement is then counted as an error of the index test, so the accuracy estimates are biased. If the index test and the reference standard tend to make the same errors, accuracy is overestimated (Reitsma and colleagues, 2009).

Comparison

Which study design gives which measures of accuracy?

Sensitivity and specificity can come from either design below. PPV and NPV come directly only from a study that enrols patients as they present.

FeatureSingle-gate (cross-sectional) studyTwo-gate (diagnostic case-control) study
Who is enrolledA consecutive or random series of patients in whom the disease is suspected, before anyone knows who has itOne group known to have the disease and a separate group known not to have it
Proportion with the diseaseSet by the setting, so it estimates the pre-test probability thereSet by the investigator, for example 100 cases and 100 controls
Sensitivity and specificityYesYes, but often inflated because the cases are clear-cut and the controls are healthy
PPV and NPVYes, read across the rows of the 2 × 2 tableNot directly. Calculate them with Bayes' rule, using a prevalence from another source
Evidence of biasThe reference design in the comparison by Lijmer and colleagues (studies in a clinical population)Lijmer and colleagues (JAMA, 1999): diagnostic performance overestimated, relative diagnostic odds ratio 3.0 (95% CI 2.0 to 4.5)
Place in test evaluationShows how the test performs in the patients who will receive itAn early, cheaper check that a new test can separate clear cases from people without the disease

Lijmer and colleagues examined 184 original studies of 218 diagnostic tests, taken from published meta-analyses, and compared the diagnostic odds ratios of studies with and without each design fault. A study that enrols patients according to their test result, for example all test-positives and a sample of test-negatives, gives PPV and NPV but not sensitivity and specificity, unless the sampling fractions are known and used to reweight the counts.

By specialty

Diagnostic test problems by specialty

A sepsis screen decides whether to start antibiotics while the blood culture is pending.

Missing sepsis is dangerous, so the screen needs high sensitivity and a high NPV. Accept more false positives.

Common myths

Common mistaken beliefs

  • Myth: A test with 90% sensitivity means a positive patient has a 90% chance of disease.

    That chance is the PPV, which depends on prevalence. In the sepsis example it is 33%.

  • Myth: Sensitivity and specificity are fixed properties of a test.

    STARD 2015: "sensitivity and specificity are not fixed test properties." They vary with the patients studied and the setting.

  • Myth: A PPV from a hospital study applies to screening in the community.

    Prevalence is lower in the community, so PPV is lower and false positives are more common.

  • Myth: A negative result on any sensitive test rules the disease out.

    The ability to rule out also depends on specificity and on the pre-test probability. Check the negative likelihood ratio and its confidence interval.

  • Myth: Screen-detected patients live longer after diagnosis, so screening works.

    Lead-time bias, length-time bias and overdiagnosis produce longer survival from diagnosis without fewer deaths.

  • Myth: A high AUC means the test performs well at the cut-off I use.

    AUC summarises all cut-offs. In practice you use one cut-off, so look at sensitivity and specificity at that point.

Why this matters to you

Five consequences of misreading a diagnostic test

  1. Protocol sent back

    A diagnostic accuracy study with no named reference standard, or with the reference test only for positives, is returned by the committee.

  2. Inflated results in your thesis

    Comparing confirmed severe cases with healthy volunteers gives sensitivity and specificity that fall when the test is used on real patients.

  3. A common viva error

    Quoting sensitivity when the examiner asks how likely a patient with a positive test is to have the disease.

  4. Misreading papers

    Applying a PPV from a tertiary referral centre to screening in a district population.

  5. Harm to patients

    When prevalence is low, most positive results are false. Each can mean anxiety, further invasive tests or treatment that was not needed.

History

History of diagnostic test evaluation

  1. 1947

    J. Yerushalmy

    Published "Statistical problems in assessing methods of medical diagnosis, with special reference to X-ray techniques". It introduced the terms sensitivity and specificity for the proportions of diseased and non-diseased people correctly classified.

  2. 1950

    W. J. Youden

    Published "Index for rating diagnostic tests" in Cancer. The Youden index, sensitivity + specificity − 1, is still used to choose cut-offs.

  3. 1950s

    Signal detection theory

    ROC analysis originated in electronic signal detection theory in the early 1950s. One of its first applications was radar.

  4. 1960

    Lee B. Lusted

    Calculated the first ROC curve in diagnostic radiology, re-analysing published data on reading chest films for pulmonary tuberculosis. He introduced ROC analysis to medical imaging.

  5. 1968

    J. M. G. Wilson and G. Jungner, WHO

    Published Principles and practice of screening for disease, with ten criteria for deciding whether a condition is suitable for screening.

  6. 1975

    T. J. Fagan

    Published a nomogram for Bayes’s theorem in a letter to the New England Journal of Medicine, converting pre-test to post-test probability with a likelihood ratio.

  7. 1978

    David Ransohoff and Alvan Feinstein

    Showed that a narrow spectrum of patients and non-independent interpretation of tests can falsely raise sensitivity and specificity, using the carcinoembryonic antigen and nitro-blue tetrazolium tests as examples.

  8. 1980

    Stephen Pauker and Jerome Kassirer

    Described the threshold approach to clinical decision making, with a testing threshold and a test-treatment threshold for the probability of disease.

  9. 1982

    James Hanley and Barbara McNeil

    Showed that the area under the ROC curve is the probability that a randomly chosen diseased subject is ranked above a randomly chosen non-diseased subject.

  10. 1988

    Elizabeth DeLong, David DeLong and Daniel Clarke-Pearson

    Published a nonparametric method for comparing the areas under correlated ROC curves from tests done on the same patients.

  11. 1999

    Jeroen Lijmer and colleagues

    Showed in 184 studies of 218 tests that design faults inflate diagnostic accuracy. Studies of cases and separate controls overestimated the diagnostic odds ratio about threefold.

  12. 2003

    STARD group

    Published the Standards for Reporting of Diagnostic Accuracy: a 25-item checklist and a flow diagram.

  13. 2011

    Whiting and colleagues

    Published QUADAS-2 for judging risk of bias in diagnostic accuracy studies, with four domains.

  14. 2015

    STARD group

    Published STARD 2015 with 30 items. A flow diagram of participants is now expected in every report.

Guidelines

Diagnostic tests in guidelines

STARD 2015, items 1, 12a and 13a–b

Title or abstract: “Identification as a study of diagnostic accuracy using at least one measure of accuracy (such as sensitivity, specificity, predictive values, or AUC)”. Methods: “Definition of and rationale for test positivity cut-offs or result categories of the index test, distinguishing pre-specified from exploratory”, and whether the readers of each test knew the results of the other.

STARD 2015, items 19, 23 and 24

Results: “Flow of participants, using a diagram”; “Cross tabulation of the index test results (or their distribution) by the results of the reference standard”; “Estimates of diagnostic accuracy and their precision (such as 95% confidence intervals)”.

QUADAS-2 (2011)

Risk of bias in a diagnostic accuracy study is judged in four domains: patient selection, index test, reference standard, and flow and timing. The first three are also judged for concerns about applicability.

Wilson and Jungner, WHO (1968)

Ten criteria for screening, beginning with “The condition sought should be an important health problem” and ending with “Case-finding should be a continuing process and not a ‘once and for all’ project”.

MoHFW Operational Framework: Management of Common Cancers (2016)

Breast: women 30–65 years, clinical breast examination once in 5 years; positives referred to a surgeon at the CHC or district hospital for confirmation using breast ultrasound, followed by biopsy as appropriate. Cervix: VIA, and oral cavity: oral visual examination, at the same ages and interval.

Steps

Designing a diagnostic accuracy study, step by step

  1. State the target condition, the intended use of the index test (screening, diagnosis, triage) and its role beside existing tests.
  2. Choose the reference standard, and write why it was chosen.
  3. Enrol a consecutive or random series of patients in whom the test would be used in practice.
  4. Fix the cut-off for a positive result before the data are collected.
  5. Do both tests on every patient, with a short interval and no treatment in between.
  6. Read each test without knowing the result of the other.
  7. Calculate the sample size for the precision you need around sensitivity and specificity (see the sample size chapter).
  8. Report the flow diagram, the 2 × 2 table, how indeterminate results were handled, and estimates with 95% confidence intervals.

Reading a paper

Five checks on any diagnostic accuracy study

  • Were the patients a consecutive or random series in whom the disease was suspected, rather than severe cases and healthy volunteers?
  • Did every patient receive the same reference standard, whatever the index test result?
  • Were the index test and the reference standard read without knowledge of each other’s result?
  • Was the cut-off fixed before the data were seen?
  • Is the 2 × 2 table given, with 95% confidence intervals for sensitivity and specificity?

Why is it called that?

Word surgery

“Sensitive”

In everyday speech

Easily upset, or delicate.

In research

Sensitivity is the proportion of people with the disease whose test is positive.

“Positive”

In everyday speech

Good news, hopeful.

In research

The result is on the abnormal side of the cut-off. For the patient it is usually unwelcome news.

“Predictive value”

In everyday speech

Something that tells the future.

In research

The proportion of positive (or negative) results that are correct in a given population at the time of testing.

Memory trick

Sensitivity and specificity: read down the columns of the 2 × 2 table. Predictive values: read across the rows.

Sensitivity

from sensitive: Medieval Latin sensitivus "capable of sensation", from Latin sentire "feel, perceive"

How well the test detects the disease when it is present.

Specificity

from specific: Late Latin specificus "constituting a kind or sort", from species "kind, sort"

How well the test responds only to this disease, and stays negative in people without it.

Prevalence

Latin praevalere "to be more able", from prae "before" + valere "be strong"

The proportion of a population that has the disease at a given time. It sets the pre-test probability.

Predictive

from predict: Latin praedicere "foretell", from prae "before" + dicere "to say"

PPV and NPV say how often a positive or negative result is correct.

Diagnosis

Greek diagnosis "a discerning, distinguishing", from dia "between" + gignoskein "to come to know"

Telling one condition apart from others. A diagnostic test separates people with the disease from people without it.

Screening

screen (verb): "sift by passing through a screen" by the 1660s; "examine systematically for suitability" from 1943

Sorting apparently healthy people into those who need further tests and those who do not.

ROC

Receiver · Operating · Characteristic

A name from electronic signal detection, where one of the first uses of the method was radar.

Exam answers

Model answer outlines

Draw a 2 × 2 table and calculate sensitivity, specificity, PPV and NPV

DM Neonatology 2017, 2019 · DM Cardiology 2016 · Ph.D. RM 2023
Disease presentDisease absentTotal
Test positivea (TP)b (FP)a + b
Test negativec (FN)d (TN)c + d
Totala + cb + dN
  1. Sensitivity = a/(a + c): proportion of diseased who test positive. High sensitivity → few false negatives (SnNout).
  2. Specificity = d/(b + d): proportion of non-diseased who test negative. High specificity → few false positives (SpPin).
  3. PPV = a/(a + b): probability of disease given a positive test.
  4. NPV = d/(c + d): probability of no disease given a negative test.
  5. Worked example (1,000 babies, prevalence 10%, a = 90, b = 180, c = 10, d = 720): sensitivity 90%, specificity 80%, PPV 33.3%, NPV 98.6%.
  6. Prevalence effect: with the same test, PPV is 4.3% at 1% prevalence and 81.8% at 50%. NPV falls as prevalence rises.
  7. Sensitivity and specificity vary with the patient spectrum. Report all four with 95% confidence intervals (STARD 2015, item 24).
  8. Direction of the probability: sensitivity is P(positive | disease). PPV is P(disease | positive). Bayes' rule links them: PPV = (sensitivity × prevalence) ÷ [sensitivity × prevalence + (1 − specificity) × (1 − prevalence)].
  9. Which design gives what: a consecutive series of suspected patients gives all four measures. A case-control (two-gate) design gives sensitivity and specificity only, and tends to overestimate them.

Receiver operating characteristic (ROC) curve: construction, significance and applications

MD Biochemistry 2011, 2015, 2016, 2018 · MD Community Medicine 2013, 2020 · DM Neonatology 2014, 2025
  1. Definition: a plot of sensitivity (y-axis) against 1 − specificity (x-axis) for every possible cut-off of a test with a continuous or graded result.
  2. Construction: list the cut-offs. At each, calculate sensitivity and specificity from a 2 × 2 table against the reference standard. Plot the points and join them.
  3. Area from a graded result: join the observed points with straight lines and add up the trapezoids. This equals the Mann–Whitney statistic divided by the number of pairs of one diseased and one non-diseased person, with ties counted as half. A binormal model gives a smooth curve instead. Report the AUC with a 95% confidence interval.
  4. Reading it: the curve runs from (0, 0) to (1, 1). The diagonal is a useless test. The nearer the curve to the top left corner, the better the test.
  5. AUC: 0.5 = chance, 1.0 = perfect. It equals the probability that a randomly chosen diseased person is ranked above a randomly chosen non-diseased person (Hanley and McNeil, 1982).
  6. Optimal cut-off: Youden index (sensitivity + specificity − 1) or the point nearest (0, 1). Shift towards sensitivity when missing disease is dangerous, towards specificity when false positives lead to harm.
  7. Significance: shows the trade-off between sensitivity and specificity at a glance, does not depend on a single arbitrary cut-off, and lets two tests be compared over all cut-offs (compare AUCs, using a paired method when both tests were done on the same patients).
  8. Applications: cut-offs for biochemical tests and scores, comparing imaging techniques and readers, evaluating prediction models, choosing the positivity threshold for screening tests.
  9. Limitations: AUC ignores the clinical cost of each error. Curves may cross. Accuracy depends on the patient spectrum.
  10. Comparing tests: AUCs of two tests done on the same patients are correlated, so use a paired method such as DeLong's (1988). If the curves cross, compare the partial AUC over the clinically relevant range of false-positive rates. An AUC below 0.5 means the direction of positivity is reversed.
  11. History: originated in electronic signal detection theory (radar) in the early 1950s, and was introduced to radiology by Lusted (1960).

A ROC curve must pass through the lower left corner and the upper right corner. Explain

MD Biochemistry 2021
  • Both statements are true for every ROC curve.
  • Lower left (0, 0): set the cut-off above the highest result. Nobody tests positive, so sensitivity = 0 and 1 − specificity = 0 (specificity 100%).
  • Upper right (1, 1): set the cut-off below the lowest result. Everybody tests positive, so sensitivity = 100% and 1 − specificity = 1 (specificity 0).
  • These two extreme cut-offs exist for any test, good or useless, so all ROC curves share the two end points. Tests differ only in the path between them: a useless test follows the diagonal, a good test bows towards (0, 1).

Likelihood ratios: definition, interpretation and use (with the ROC curve)

DM Neonatology 2017 · DM Endocrinology 2022 · Ph.D. RM 2025
  1. Definition: the probability of a given test result in people with the disease divided by the probability of the same result in people without it.
  2. LR+ = sensitivity/(1 − specificity). LR− = (1 − sensitivity)/specificity.
  3. Use: pre-test odds × LR = post-test odds (Bayes’ theorem). Odds = p/(1 − p) and p = odds/(1 + odds). Fagan's nomogram (1975) avoids the arithmetic.
  4. Worked example: sensitivity 90%, specificity 80% → LR+ 4.5, LR− 0.125. Pre-test probability 10% → 33.3% after a positive, 1.4% after a negative result.
  5. Interpretation: LR = 1 means no change. LR > 1 raises and LR < 1 lowers the probability. Approximate change (McGee, 2002): LR 2, 5, 10 → +15, +30, +45 percentage points. LR 0.5, 0.2, 0.1 → −15, −30, −45 (for pre-test probability 10–90%).
  6. Advantages: not changed by prevalence in the way PPV is. Applied to the individual patient’s pre-test probability. Can be given for several result levels. Tests can be applied in sequence, each post-test probability becoming the next pre-test probability (if the tests are independent).
  7. Link with ROC: the slope of the ROC curve at a cut-off is the likelihood ratio at that point. LR+ at a cut-off is the slope of the line from (0, 0) to that point.
  8. Will the result change management? Pauker and Kassirer (1980): test only when the pre-test probability lies between the testing threshold and the test-treatment threshold. Below the first, do not treat. Above the second, treat without testing.

Characteristics of a diagnostic test and statistical methods of validation (utility of ROC curves and likelihood ratios in clinical practice)

DM Neonatology 2011, 2018, 2022 · MD Biochemistry 2025
  1. Validity (accuracy) against a reference standard: sensitivity, specificity, PPV, NPV, likelihood ratios, diagnostic odds ratio, overall accuracy, Youden index, ROC curve and AUC, each with a 95% confidence interval.
  2. Reliability: repeatability and inter-observer agreement, measured by kappa for categorical results and by the intraclass correlation coefficient or coefficient of variation for measurements.
  3. Practical characteristics: yield, safety, acceptability, cost, speed, availability.
  4. Clinical usefulness: whether the result changes management and improves outcome.
  5. Validation study design: consecutive patients in whom the test would be used, the same reference standard for all, blinded interpretation, a pre-specified cut-off, and a sample size for the precision required.
  6. Biases to avoid: spectrum bias, review bias, partial verification bias, differential verification, incorporation bias, and an imperfect reference standard.
  7. Design evidence: case-control (two-gate) designs overestimated the diagnostic odds ratio about threefold compared with studies in a clinical population (Lijmer et al., JAMA 1999).
  8. Validation of the cut-off: a cut-off chosen from the ROC curve in one data set should be confirmed in a separate group of patients.
  9. ROC in practice: choose the cut-off to match the clinical purpose, compare tests; LR in practice: convert each patient's pre-test probability to a post-test probability.
  10. Reporting and appraisal: STARD 2015 (30 items) for reports, and QUADAS-2 (four domains) for risk of bias.

A screening test requires high sensitivity but a diagnostic test requires high specificity. Role of sensitivity and specificity of screening in clinical medicine, with a note on ROC curves

MD Community Medicine 2012, 2023
  • Screening tests apparently healthy people to find those who need further tests. Missing a case loses the chance of early treatment, so sensitivity is the priority. False positives are sorted out by the diagnostic test that follows.
  • Diagnostic tests decide treatment. A false positive leads to unnecessary surgery, drugs or labelling, so specificity is the priority.
  • SnNout and SpPin: a negative result on a highly sensitive test rules out, and a positive result on a highly specific test rules in.
  • Prevalence: in screening, prevalence is low, so PPV is low even with good specificity (4.3% at 1% prevalence with sensitivity 90% and specificity 80%).
  • Serial testing (screening then confirmation) raises overall specificity; parallel testing raises sensitivity.
  • Predictive values of combined tests: for tests with sensitivity and specificity of 90% and 80%, and 80% and 95%, at 10% prevalence, serial testing gives PPV 88.9% and parallel testing gives NPV 99.7%. The formulas assume that the tests make errors independently.
  • Examples: clinical breast examination, VIA and oral visual examination at 30–65 years (MoHFW, 2016), with confirmation by biopsy.
  • ROC note: for a screening test choose a cut-off on the upper right part of the curve (high sensitivity). For a confirmatory test, choose the lower left part (high specificity).

How is a cancer screening test evaluated? Sensitivity, specificity and predictive value, the breast cancer screening programme, and biases of screening

MCh Surgical Oncology 2004 · MD Radio Therapy 2019
  1. Definitions in screening terms. Sensitivity = screen-detected cancers ÷ all cancers in the screened group (screen-detected plus those appearing between screens, interval cancers). Specificity = screen-negative ÷ all without cancer. PPV = cancers ÷ screen-positives referred for further tests.
  2. Suitability: Wilson and Jungner criteria (WHO, 1968): important problem, detectable early stage, suitable and acceptable test, accepted treatment, facilities, agreed policy, balanced cost, continuing process.
  3. Programme measures: coverage, recall and biopsy rates, PPV, cancer detection rate, interval cancer rate, stage at diagnosis.
  4. Outcome measure: reduction in cancer-specific mortality in a randomised trial analysed by invitation to screening.
  5. Breast cancer programme in India: women 30–65 years, clinical breast examination once in 5 years by trained health workers. Positives are referred to a surgeon at the CHC or district hospital for breast ultrasound and biopsy as appropriate (MoHFW Operational Framework, 2016).
  6. Lead-time bias: earlier diagnosis without later death makes survival from diagnosis longer.
  7. Length-time bias: screening preferentially detects slow-growing tumours.
  8. Overdiagnosis: cancers that would never have caused harm are detected and treated.
  9. Volunteer bias: attenders are healthier than non-attenders.
  10. Harms: false positives, anxiety, unnecessary biopsies, and false reassurance from false negatives.

Analytical sensitivity and specificity

MD Biochemistry 2018
  • Analytical sensitivity: the smallest amount of a substance in a sample that an assay can accurately measure (limit of detection).
  • Analytical specificity: the ability of an assay to measure one particular organism or substance rather than others. Cross-reactivity and interference reduce it.
  • Diagnostic sensitivity: the percentage of people with the disorder whom the assay identifies as positive.
  • Diagnostic specificity: the percentage of people without the condition whom the assay identifies as negative.
  • High analytical sensitivity does not guarantee acceptable diagnostic sensitivity. False positives from sample contamination reduce diagnostic specificity.
  • Always use the adjective, analytical or diagnostic, because the meanings differ (Saah and Hoover, Ann Intern Med 1997).

Sensitivity and specificity of imaging for suspected common bile duct calculi, and for pancreatitis versus pancreatic carcinoma

MD Radio Diagnosis 1990
  1. Frame every modality the same way: the target condition, the reference standard, the patients studied, sensitivity and specificity with 95% confidence intervals, and what a positive or negative result changes.
  2. CBD stones, reference standard (Cochrane review, Giljaca 2015): stones extracted at endoscopy or surgery for a positive result, and negative duct exploration or at least six months without symptoms for a negative result.
  3. MRCP: summary sensitivity 0.93 (95% CI 0.87–0.96), specificity 0.96 (0.90–0.98), 7 studies, 996 participants.
  4. EUS: summary sensitivity 0.95 (0.91–0.97), specificity 0.97 (0.94–0.99), 13 studies, 1,537 participants. No evidence of a difference from MRCP.
  5. Post-test probability at a pre-test probability of 41%: 0.96 after a positive and 0.03 after a negative EUS; 0.94 and 0.05 for MRCP. All included studies were of low methodological quality.
  6. Ultrasound is usually the first examination. ERCP and intraoperative cholangiography are invasive and allow treatment in the same sitting.
  7. Pancreatitis versus carcinoma: compare CT, MRI with MRCP and EUS against histology or follow-up. Report accuracy separately for the difficult group (a focal mass in chronic pancreatitis), because a study of obvious cases gives inflated figures (spectrum bias). Quote current figures from a systematic review.

Randomised clinical trials: external validity, allocation concealment, stratification, sensitivity analysis, Fragility Index

DM Critical Care Medicine 2022
  1. External validity: how far the trial results apply to patients outside the trial. It depends on eligibility criteria, setting, co-interventions and the comparator. See bias and validity.
  2. Allocation concealment: the person enrolling a patient cannot know the next assignment until the patient is enrolled, for example by central telephone or web randomisation, or sequentially numbered, sealed, opaque envelopes. It prevents selection bias and is different from blinding. See clinical trials.
  3. Stratification: randomising separately within groups of a strong prognostic factor (centre, disease severity), usually with blocks, so the arms are balanced for that factor.
  4. Sensitivity analysis: repeating the analysis under different assumptions (methods for missing data, per-protocol instead of intention-to-treat, excluding doubtful data) to see whether the conclusion changes.
  5. Fragility Index (Walsh et al., J Clin Epidemiol 2014): the number of patients in the group with fewer events whose status would have to change from non-event to event for the P-value to exceed 0.05. In 399 trials the median was 8, and 25% had an index of 3 or less. In 53% of trials it was smaller than the number lost to follow-up.

The one thing to remember

Sensitivity and specificity describe the test, but the chance that a patient with a positive result has the disease also depends on how common the disease is.

Worked examples and the sepsis test use made-up numbers for teaching. The ROC figure uses normal distributions with SD 20 (means 20 and 50, and 70 for the better test). The graded ultrasound example and the 30% treatment threshold are also made up.

Sources: Yerushalmy J. Public Health Rep 1947;62:1432–49 (doi:10.2307/4586294; PMID 20340527). Youden WJ. Cancer 1950;3:32–5 (PMID 15405679). Lusted LB. Radiology 1960;74:178–93 (doi:10.1148/74.2.178). Lusted LB. Science 1971;171:1217–9 (doi:10.1126/science.171.3977.1217). Wilson JMG, Jungner G. Principles and practice of screening for disease. Geneva: WHO; 1968. Andermann A et al. Bull World Health Organ 2008;86:317–9 (doi:10.2471/BLT.07.050112). Fagan TJ. N Engl J Med 1975;293:257 (doi:10.1056/NEJM197507312930513). Ransohoff DF, Feinstein AR. N Engl J Med 1978;299:926–30 (doi:10.1056/NEJM197810262991705). Pauker SG, Kassirer JP. N Engl J Med 1980;302:1109–17 (doi:10.1056/NEJM198005153022003; PMID 7366635). Hanley JA, McNeil BJ. Radiology 1982;143:29–36 (doi:10.1148/radiology.143.1.7063747). DeLong ER, DeLong DM, Clarke-Pearson DL. Biometrics 1988;44:837–45 (doi:10.2307/2531595; PMID 3203132). Saah AJ, Hoover DR. Ann Intern Med 1997;126:91–4 (doi:10.7326/0003-4819-126-1-199701010-00026). Lijmer JG et al. JAMA 1999;282:1061–6 (doi:10.1001/jama.282.11.1061; PMID 10493205). McGee S. J Gen Intern Med 2002;17:646–9 (doi:10.1046/j.1525-1497.2002.10750.x). Bossuyt PM et al. BMJ 2003;326:41–4 (doi:10.1136/bmj.326.7379.41). Pewsner D et al. BMJ 2004;329:209–13 (doi:10.1136/bmj.329.7459.209). Shiu SY, Gatsonis C. Philos Trans A Math Phys Eng Sci 2008;366:2313–33 (doi:10.1098/rsta.2008.0043; PMID 18407893). Reitsma JB et al. J Clin Epidemiol 2009;62:797–806 (doi:10.1016/j.jclinepi.2009.02.005; PMID 19447581). Whiting PF et al. QUADAS-2. Ann Intern Med 2011;155:529–36 (doi:10.7326/0003-4819-155-8-201110180-00009). Hajian-Tilaki K. Caspian J Intern Med 2013;4:627–35 (PMID 24009950). Walsh M et al. J Clin Epidemiol 2014;67:622–8 (doi:10.1016/j.jclinepi.2013.10.019). Giljaca V et al. Cochrane Database Syst Rev 2015;(2):CD011549 (doi:10.1002/14651858.CD011549). Bossuyt PM et al. STARD 2015. BMJ 2015;351:h5527 (doi:10.1136/bmj.h5527). Ministry of Health and Family Welfare, Government of India. Operational Framework: Management of Common Cancers. 2016.

Past papers

Every TNMGRMU question on diagnostic tests

Each one is answered in this chapter. Tap a sitting to open the university's own question paper.

  1. ×2Receiver operating characteristic curve (ROC)DM Neonatology · Aug 2025DM Neonatology · Feb 2014
  2. ×1Statistical methods of validation of diagnostic testsMD Biochemistry · Oct 2025
  3. ×1Likelihood ratios & ROC curvePh.D. RM · Jan 2025
  4. ×1A screening test requires high sensitivity but a diagnostic test requires high specificity. ExplainMD Community Medicine · May 2023
  5. ×1Predictive valuesPh.D. RM · Jan 2023
  6. ×1Regarding randomised clinical trials, explain the following terms: a. External validity. b. Allocation concealment. c. Stratification. d. Sensitivity analysis. e. Fragility IndexDM Critical Care Medicine · Aug 2022
  7. ×1Discuss on validation of diagnostic tests. Write briefly on the Receiver operating characteristic curves and likelihood in clinical practiceDM Neonatology · Aug 2022
  8. ×1Interpretation of likelihood ratiosDM Endocrinology · Aug 2022
  9. ×1A receiver operating characteristic curve must inevitably pass through the upper right cornerMD Biochemistry · Jul 2021
  10. ×1An Receiver Operating Charecteristic curve must inevitably pass through the lower left cornerMD Biochemistry · Nov 2021
  11. ×1Receiver Operating Characteristic (ROC) CurveMD Community Medicine · Nov 2020
  12. ×1Define sensitivity, specificity and predictive value of a cancer screening test. Discuss screening programme for cancer breastMD Radio Therapy · May 2019
  13. ×1Draw a 2 X 2 table and describe how to calculate sensitivity, specificity, positive predictive value and negative predictive value for a diagnostic testDM Neonatology · Aug 2019
  14. ×1Analytical sensitivity and specificityMD Biochemistry · Oct 2018
  15. ×1Receiver operating characteristic curve and its applicationsMD Biochemistry · Oct 2018
  16. ×1Describe the various characteristics of a Diagnostic test. Discuss the utility of Receiver Operating curves and Likelihood ratios in clinical practiceDM Neonatology · Aug 2018
  17. ×1Positive predictive value, Negative predictive valueDM Neonatology · Feb 2017
  18. ×1Likelihood RatioDM Neonatology · Aug 2017
  19. ×1Receiver operating curveMD Biochemistry · Oct 2016
  20. ×1Sensitivity and specificity of a diagnostic testDM Cardiology · Aug 2016
  21. ×1ROC Curve and its significanceMD Biochemistry · Apr 2015
  22. ×1Significance of ROC CurveMD Community Medicine · Apr 2013
  23. ×1Describe the role of sensitivity and specificity of screening in clinical medicine with examples. Add a note on receiver operative charecterestic curvesMD Community Medicine · Oct 2012
  24. ×1Receiver Operating Characteristics CurveMD Biochemistry · May 2011
  25. ×1Discuss various characteristics of a diagnostic test in research methodologyDM Neonatology · Aug 2011
  26. ×1How is a screening test for cancer evaluated? Discuss the biases caused as a result of a screening programmeMCh Surgical Oncology · Apr 2004
  27. ×1Describe the imaging techniques and procedures avallable for the diagnosis of suspected calculi In the common bile duct. Highlight the sensitivity and specificity of the various techniques. Describe the roentgen features and write short notesMD Radio Diagnosis · Mar 1990
  28. ×1Describe the imaging procedures available with their specific features, for the diagnosis of a suspected case of pancreatitis versus pancreatic carcinoma High- light the sensitivity and specificity of the procedures usedMD Radio Diagnosis · Oct 1990