Chapter 15 · Making sense of numbers

Which statistical test do you use?

Parametric and non-parametric tests: t-test, ANOVA, chi-square, Mann-Whitney, Kruskal-Wallis.

36 TNMGRMU questions answered hereset 44 times, 1994–2025

The problem first

You are a second-year Anaesthesiology resident. Your thesis records VAS pain scores in 40 patients before and 30 minutes after a nerve block. You enter the scores in two columns and click the first t-test in the software menu, the independent-samples test. The software reports no significant difference, and you write that the block did not reduce pain.

At the pre-submission seminar, the head of department reads one line of your methods section.

“These are the same 40 patients. Why did you use an unpaired test?”

Start here

What decides which test you use?

Every test asks the same question: could chance alone explain the difference you see? Which test answers it depends on four facts about your data.

  1. Type of outcome: a measurement, ordered grades, or categories.
  2. Number of groups: two, or three or more.
  3. Independent or paired: different patients in each group, or the same patients measured twice (or matched pairs).
  4. Distribution, for measurements: roughly normal, or skewed with outliers.
A measurementHb in g/dLSummary: mean and SDt-test, ANOVACategoriesinfected: yes or noSummary: counts and %chi-square,Fisher's exactOrdered gradespain grade 0 to IIISummary: median and ranksMann-Whitney,Kruskal-Wallis910111213140IIIIII
The type of outcome decides the family of tests. Nominal data are named categories with no order (blood group). Ordinal data have an order but unequal steps (pain grade). Measurements on an interval or ratio scale allow a mean and SD. p-values and the null hypothesis are explained in the hypothesis testing chapter, and summaries in the descriptive statistics chapter.

Two types of tests

Parametric or non-parametric: what is the difference?

A parametric test assumes the data come from a known distribution, usually the normal distribution, described by parameters such as the mean and SD. A non-parametric test does not assume a distribution. It works on ranks or signs instead of the raw values.

SymmetricSkewed to the rightmean = medianHaemoglobinmedianmeanDays in hospitala few long stays pull the mean to the right
When data are symmetric, the mean describes the centre and parametric tests work well. When a few patients stay much longer, the mean moves towards them and no longer describes a typical patient. The median stays with the majority.

Assumptions of parametric tests

  1. Normality: the outcome is roughly normal in each group (for paired tests, the differences). Look at a histogram, and use the Shapiro-Wilk test if needed.
  2. Equal variances: the groups have similar spread. Check with the Levene test.
  3. Independence: one patient's value gives no information about another's, except where pairing is part of the design.
  4. Scale: the outcome is a measurement on an interval or ratio scale, so that a mean has a meaning.

Non-parametric tests

How can a test work without the actual values?

It replaces each value with its rank. The smallest value gets rank 1, the next rank 2, and tied values share the average of their ranks.

LaparoscopicOpen0 d5 d10 d15 d20 done 21-day stay pulls the mean upAll 16 stays ranked together, shortest = 12225557.57.510101012.512.5141516t-test p = 0.75 · Mann-Whitney p = 0.015
Made-up teaching data: length of stay after laparoscopic and open surgery, eight patients in each group. One 21-day stay raises the laparoscopic mean to 5.0 days, close to the open group's 5.75, and the t-test finds no difference (p = 0.75). As a rank, the 21-day stay counts only as 16. The laparoscopic ranks add up to 44.5 against 91.5 for open surgery, and the Mann-Whitney U test gives p = 0.015.

Interactive

Move one long stay and watch the t-test and the Mann-Whitney test

The t-test uses the actual values, so one long stay raises the mean and the SD of its group. The Mann-Whitney test uses ranks, so a stay longer than all the others counts as rank 16 whatever its length.

Try this: move the eighth laparoscopic stay from 4 days up to 21 days. The t-test p-value rises above 0.05. The Mann-Whitney p-value stops changing once this stay is longer than all the other stays.

Made-up teaching data from this chapter: length of stay after laparoscopic and open surgery, 8 patients in each group.

4 days

The other 15 stays do not change

< 0.001p, unpaired t-test
0.001p, Mann-Whitney U test

Both tests are significant at α 0.05. The eighth laparoscopic stay has rank 8 of 16.

means 2.88 and 5.75 days · medians 3.0 and 5.5 days · t = -5.32, df 14 · rank sums 37 and 99, U = 1

LaparoscopicOpenmean 2.88mean 5.75051015202530daysAll 16 stays ranked together, shortest = rank 122255588811111113.513.51516
laparoscopic (red: the stay you move)open

Decide

Which test fits your study?

Answer the questions in order. Each answer links to its worked example.

What kind of outcome are you comparing?

Which test: all possible answers
  • Paired t-test: Works on the difference within each pair. Degrees of freedom = number of pairs − 1.
  • Wilcoxon signed-rank test: Ranks the differences within pairs by size and keeps their signs.
  • Unpaired (independent samples) t-test: Compares two means. If the variances clearly differ, use the version for unequal variances (Welch test).
  • Mann-Whitney U test: Ranks all values from both groups together and compares the ranks. Also called the Wilcoxon rank-sum test.
  • One-way ANOVA: Tests whether at least one group mean differs. If it is significant, a post hoc test (Tukey, Bonferroni) shows which pairs differ.
  • Kruskal-Wallis test: The rank version of one-way ANOVA. If it is significant, compare pairs with a correction for multiple testing.
  • Repeated measures ANOVA: Compares means across time points in the same patients. Check sphericity with the Mauchly test.
  • Friedman test: The rank version of repeated measures ANOVA.
  • One-sample t-test: Compares one sample mean with a reference value, such as Hb 12 g/dL.
  • Wilcoxon signed-rank test (one sample): Compares the median of one sample with a reference value.
  • Chi-square test: Compares observed counts with the counts expected if there were no association. Degrees of freedom = (rows − 1) × (columns − 1).
  • Fisher's exact test: Calculates the exact probability for a table with small expected counts. Most packages give it for 2 × 2 tables.
  • McNemar test: For paired yes/no results, such as test positive or negative in the same patients before and after treatment. Only the pairs that changed enter the test.
  • Binomial test (one proportion): Compares the proportion in one sample with a reference value. The z-test is an approximation that needs large expected counts.
  • Chi-square goodness-of-fit test: Compares observed counts with the counts expected from stated proportions.
  • Correlation or regression: Pearson r for two normal measurements, Spearman rho for ranks. Regression predicts one from the other.
  • Survival analysis: Kaplan-Meier curves compared with the log-rank test, or Cox regression.

Comparison

Parametric or non-parametric: which one should you use?

Use a parametric test when its assumptions hold, because it uses the actual values. Use a non-parametric test when they do not hold.

FeatureParametricNon-parametric
AssumesA distribution, usually normal, with similar variancesNo particular distribution. Observations must still be independent
Works onThe actual valuesRanks or signs
Summary reportedMean and SDMedian and IQR
SuitsMeasurements that are roughly normalSkewed measurements, outliers, ordinal scores
OutliersPull the mean and inflate the SDCount only as the highest or lowest rank
EfficiencyBest when the assumptions holdAsymptotic relative efficiency 0.955 against the t-test when data are normal, and higher than the t-test when data are heavily skewed
Confidence intervalStraightforward for a difference in meansHarder to calculate
Examplest-test, ANOVA, Pearson correlationMann-Whitney, Wilcoxon, Kruskal-Wallis, Friedman, Spearman correlation

The chi-square test is usually grouped with the non-parametric tests because it works on counts and assumes no distribution for a measurement.

Comparison

Paired or unpaired: are the two sets of values linked?

Values are paired when each value in one set belongs with one value in the other: the same patient before and after, two eyes of one patient, or a case matched to a control.

Analysed as two separate groupsAnalysed as pairs05100510beforeafterbeforeafterunpaired t-test p = 0.20paired t-test p < 0.001
Made-up teaching data: pain scores in 8 patients before and 1 hour after an analgesic. The patients differ from each other (3 to 9 before treatment), but every patient falls by 1 or 2 points. The unpaired test compares the 1.4-point mean fall with all the variation between patients: t = 1.35, p = 0.20. The paired test compares each patient with themselves, removes the variation between patients, and gives t = 7.51, p < 0.001.
FeaturePaired t-testUnpaired t-test
DataTwo values from the same patient, or from a matched pairOne value from each patient in two different groups
ExampleHb before and after 3 months of iron in the same womenHb in women given oral iron and in other women given IV iron
Works onThe difference within each pairThe difference between the two group means
Standard errorSD of the differences ÷ √nPooled SD × √(1/n₁ + 1/n₂)
Degrees of freedomNumber of pairs − 1n₁ + n₂ − 2
Variation between patientsRemoved, because each patient is compared with themselvesStays in the standard error
Equal numbers neededYes, every value needs its pairNo
Non-parametric versionWilcoxon signed-rank testMann-Whitney U test

Interactive

Change how much the women differ and watch the paired and unpaired p-values

The paired test works on each patient's own change, so the variation between patients does not enter its standard error. The unpaired test keeps that variation.

Try this: move the SD of Hb before treatment from 1.2 up to 2.0 g/dL. The unpaired p-value rises. The paired p-value stays the same, because each woman is compared with herself.

Made-up teaching data: Hb in the same 10 women before and after 3 months of oral iron. Both tests use the same 20 numbers.

1.0 g/dL

Larger rise: smaller p in both tests

0.8 g/dL

How much the rise differs from woman to woman

1.2 g/dL

How much the women differ from each other. The paired test does not use it

0.003p, paired t-test
0.069p, unpaired t-test

paired t = 3.95, df 9 · unpaired t = 1.93, df 18

The unpaired test puts the variation between women (SD 1.2 g/dL) into its standard error. The paired test works on each woman's own rise, so only the SD of the rises (0.8 g/dL) enters its standard error.

Analysed as two separate groupsAnalysed as pairs481216481216beforeafterbeforeafterHb, g/dLHb, g/dLunpaired p = 0.069paired p = 0.003
group meanbeforeafter

The formula

Student's t-test: is the difference between two means larger than chance would produce?

t is the difference between the means divided by its standard error. The larger t is, the less likely it is that chance alone produced the difference.

Drug ADrug Bdifference 4.046810121416mmHg
Made-up teaching data: fall in systolic blood pressure after 4 weeks, 8 patients on each drug. The means differ by 4.0 mmHg. Whether that is more than chance depends on how much patients vary within each group.

t = (mean₁ − mean₂) ÷ SE of the difference

Three types of t-test

  • One-sample t-test: one sample mean against a reference value. t = (mean − reference) ÷ (SD ÷ √n), df = n − 1.
  • Unpaired (independent samples) t-test: two independent groups. df = n₁ + n₂ − 2.
  • Paired t-test: two measurements on the same patients. t = mean difference ÷ (SD of the differences ÷ √n), df = number of pairs − 1.

Worked example 1: unpaired t-test in two small groups

Drug A: 12, 15, 9, 14, 11, 16, 10, 13 mmHg. Drug B: 8, 10, 6, 11, 7, 9, 12, 5 mmHg.

  1. Means: A = 100 ÷ 8 = 12.5 and B = 68 ÷ 8 = 8.5. Difference = 4.0 mmHg.
  2. Squared deviations from the mean in group A: 0.25, 6.25, 12.25, 2.25, 2.25, 12.25, 6.25, 0.25. They add up to 42. Group B also adds up to 42.
  3. SD = √(42 ÷ 7) = √6 = 2.45 mmHg in each group.
  4. Pooled SD = √[(7 × 6 + 7 × 6) ÷ 14] = 2.45 mmHg.
  5. SE of the difference = 2.45 × √(1/8 + 1/8) = 1.225.
  6. t = 4.0 ÷ 1.225 = 3.27, with 8 + 8 − 2 = 14 degrees of freedom.
  7. The critical t for 14 df at 5% (two-sided) is 2.145. 3.27 is larger, so p < 0.05. The exact p = 0.006.
pooled SD
√{[(n₁ − 1)SD₁² + (n₂ − 1)SD₂²] ÷ (n₁ + n₂ − 2)}
SE
pooled SD × √(1/n₁ + 1/n₂)
df
degrees of freedom, used to read the critical t from the table

t = 4.0 ÷ 1.225 = 3.27, df 14, p = 0.006 → 95% CI for the difference 1.37 to 6.63 mmHg

Worked example 2: paired t-test

Pain scores before: 4, 9, 6, 8, 5, 7, 3, 8. After 1 hour: 3, 7, 5, 6, 3, 6, 2, 7.

  1. Differences: 1, 2, 1, 2, 2, 1, 1, 1. Mean difference = 11 ÷ 8 = 1.375.
  2. SD of the differences = 0.518. SE = 0.518 ÷ √8 = 0.183.
  3. t = 1.375 ÷ 0.183 = 7.51, df = 7, p = 0.0001. 95% CI for the mean fall 0.94 to 1.81 points.
  4. The same 16 numbers put through an unpaired t-test give t = 1.35, p = 0.20.

Worked example 3: one-sample t-test

Made-up teaching data. Hb in 10 adolescent girls: 11.2, 10.8, 12.1, 11.5, 10.9, 11.8, 11.0, 12.3, 10.6, 11.4 g/dL. Does the mean differ from 12 g/dL?

t = (11.36 − 12) ÷ (0.568 ÷ √10) = −0.64 ÷ 0.180 = −3.56, df 9 → p = 0.006

Why the t-test matters

  • It gives correct p-values for small samples. Methods for small samples were the problem W. S. Gosset worked on as a brewer at Guinness.
  • It does not need the population SD, so it can be used at any sample size.
  • It is the basis of the confidence interval for a mean and for a difference between two means.
  • It is one of the most used statistical tests in medicine: comparing a sample with a standard, two treatments, or values before and after treatment.

Assumptions: a measurement outcome, roughly normal (for the paired test, the differences), independent patients, and for the unpaired test similar variances. If the variances clearly differ, use the t-test for unequal variances (Welch test).

The formula

Three or more groups: why not run a t-test for each pair?

Each t-test carries a 5% chance of a false positive. The more tests you run, the higher the chance that at least one of them is falsely significant.

5%: one test3 groups · 3 t-tests14.3%4 groups · 6 t-tests26.5%5 groups · 10 t-tests40.1%
Chance of at least one false positive = 1 − 0.95^k, for k tests at 5% if the tests were independent. With 3 groups: 1 − 0.95³ = 14.3%. One-way ANOVA tests all the groups in a single test at 5%.

The principle of ANOVA

Analysis of variance (ANOVA) compares means by comparing two kinds of variation. If all groups have the same true mean, the group means should differ from each other only about as much as individual patients differ within a group. F is the ratio of the two.

4 h6 h8 h10 hgrand meanDrug alone+ adjuvant A+ adjuvant Bbetween groupswithin a group
Made-up teaching data: duration of analgesia with a drug alone or with one of two adjuvants, six patients per group. The group means (red lines) lie far from the grand mean (dashed line) compared with the spread inside each group, so F is large.

F = variance between groups (mean square between) ÷ variance within groups (mean square within)

MS between
Σ nᵢ (group mean − grand mean)² ÷ (k − 1)
MS within
Σ (each value − its group mean)² ÷ (N − k)
k, N
number of groups, total number of patients

If H₀ is true, both mean squares estimate the same patient-to-patient variance, so F is expected to be close to 1. Group means that truly differ make the mean square between groups larger. Only large values of F count against H₀, so the p-value is the area in the upper tail of the F distribution, although the alternative hypothesis (at least one mean differs) has no direction. F cannot be negative. With only two groups, F = t²: in worked example 1 below, t = 3.266 and F = t² = 10.67, with the same p = 0.006.

Worked example 4: one-way ANOVA

Duration of analgesia (hours). Drug alone: 5.2, 6.1, 5.8, 4.9, 6.4, 5.6 (mean 5.67). With adjuvant A: 7.9, 8.4, 7.2, 8.8, 7.6, 8.1 (mean 8.00). With adjuvant B: 8.6, 9.4, 8.1, 9.9, 8.8, 9.2 (mean 9.00). Grand mean 7.56.

SourceSum of squaresdfMean squareFp
Between groups35.11217.5650.7< 0.001
Within groups5.19150.346
Total40.3017

F = 17.56 ÷ 0.346 = 50.7. The critical F for (2, 15) df at 5% is 3.68, so at least one mean differs. The Levene test gives p = 0.91, so the variances are similar. Tukey's post hoc test shows that all three pairs differ, including adjuvant A against adjuvant B (difference 1.0 hour, p = 0.026).

Types of ANOVA

  • One-way ANOVA: one grouping factor with three or more independent groups. Example: HbA1c with three drugs.
  • Two-way (factorial) ANOVA: two factors at once, such as drug and sex. It tests the effect of each factor and their interaction, that is, whether the effect of the drug differs between men and women. In a factorial design every level of one factor is combined with every level of the other (2 drugs × 2 sexes = 4 groups). Main effects, interaction plots and worked two-factor designs are in the Practical Statistics chapter on two-way ANOVA.
  • Repeated measures ANOVA: the same patients at three or more time points. It extends the paired t-test and needs sphericity (Mauchly test). If sphericity fails, apply the Greenhouse-Geisser or Huynh-Feldt correction. The analysis of repeated measurements is covered further in the Practical Statistics chapter on repeated measures.
  • Mixed (two-way repeated measures) ANOVA: one factor between patients (treatment group) and one within patients (time).
  • ANCOVA: ANOVA adjusted for a continuous covariate, such as the baseline value.

Assumptions of ANOVA

  1. Observations are independent (except the planned repetition in repeated measures designs).
  2. The outcome is roughly normal in each group.
  3. Variances are similar across groups (Levene test). If not, use a post hoc test for unequal variances, such as Games-Howell.
  4. For repeated measures, sphericity holds (Mauchly test).
  5. The outcome is a measurement. If normality fails, use the Kruskal-Wallis or Friedman test.

How to check normality and equal variances, and what to do when they fail, is in the Practical Statistics chapter on checking assumptions.

After a significant F: post hoc tests

A significant F shows that at least one mean differs, not which one. Post hoc tests compare pairs while controlling the false-positive rate: Tukey, Scheffé, Holm, and Bonferroni, which divides α by the number of comparisons (0.05 ÷ 3 = 0.017). Dunnett's test compares each treatment group with one control group, instead of comparing every pair. How to choose among these tests is covered in the Practical Statistics chapter on multiple comparisons.

Interactive

Move the group means apart or spread the patients out, and watch F

F compares how far the group means lie from the grand mean with how far patients lie from their own group mean. The same distance between the means can give a large or a small F, depending on the spread within the groups.

Try this: leave the group means 1 hour apart and raise the SD within groups from 1.0 to 2.0 hours. F falls from 6.00 to 1.50 and p rises above 0.05. Then move the means 2 hours apart. F is back at 6.00, because the distance between the means has doubled along with the spread.

Made-up teaching data: duration of analgesia in three groups of six patients. The grand mean stays at 7 hours.

1.0 h

Group means 6.0, 7.0 and 8.0 hours

1.0 h

How much patients differ inside a group

6.00F = MS between ÷ MS within
0.012p, at least one mean differs
SourceSum of squaresdfMean squareFp
Between groups12.0026.006.000.012
Within groups15.00151.00

F = 6.00 ÷ 1.00 = 6.00 · critical F (2, 15) at α 0.05 = 3.68 · F ≥ critical F

02468101214hoursgrand meanDrug alone+ adjuvant A+ adjuvant B
group meangrand mean

Non-parametric tests

Mann-Whitney, Wilcoxon, Kruskal-Wallis and Friedman: what does each one do?

Each rank test takes the place of a parametric test when the data are skewed, have outliers, or are ordinal.

TestUsed instead ofDesignHow it works
Mann-Whitney U (Wilcoxon rank-sum)Unpaired t-testTwo independent groupsRanks all values together and compares the rank sums of the two groups
Wilcoxon signed-rankPaired t-test, one-sample t-testPaired values, or one sample against a referenceRanks the absolute differences, then adds the ranks of the positive differences
Sign testPaired t-testPaired valuesCounts only how many differences are + and how many are −
Kruskal-Wallis HOne-way ANOVAThree or more independent groupsRanks all values together and compares the rank sums of the groups
FriedmanRepeated measures ANOVASame patients on three or more occasionsRanks each patient's values across the occasions
Spearman rank correlationPearson correlationTwo variables in the same patientsCorrelates the ranks instead of the values

The same teaching data, analysed with rank tests

  • Mann-Whitney U on length of stay: medians 3 and 5.5 days, U = 8.5, p = 0.015. The t-test on the same data gave p = 0.75.
  • Wilcoxon signed-rank on the pain scores: all 8 differences are positive, p = 0.008.
  • Kruskal-Wallis on the analgesia data: H = 13.69, df = 2, p = 0.001. One-way ANOVA on the same data gave p < 0.001.

Why non-parametric tests are useful

  • They do not need the normality assumption, so they are less likely to give a wrong conclusion when that assumption fails.
  • Outliers have little effect, because an extreme value is only the highest or lowest rank.
  • They are the correct choice for ordinal data: pain grades, Likert items, Apgar scores.
  • They can be used in small samples, where normality cannot be checked.
  • When data are normal they lose little: the asymptotic relative efficiency of the Mann-Whitney test against the t-test is 0.955.

Limitations: ranks use less of the information in the data, confidence intervals are harder to calculate, and there are fewer methods for complex designs. The result describes ranks or medians, not a difference in means.

A significant Mann-Whitney test does not by itself show that the medians differ. The test compares the ranks of the two groups, so groups with similar medians but different spread can also give a small p-value. Hart made this point in a 2001 BMJ statistics note, whose title says that differences in spread can be important for this test. Report the median and IQR of each group, so that readers can see which one differs.

The formula

Comparing counts: the chi-square test

Chi-square compares the counts you observed with the counts you would expect if there were no difference between the groups.

813Skin preparation A(8 − 13)² ÷ 13 = 1.921813Skin preparation B(18 − 13)² ÷ 13 = 1.92observedexpected ifno difference
Made-up teaching data, example 1. If both preparations had the same infection rate, the 26 infections would be shared 13 and 13. The observed counts are 8 and 18.

χ² = Σ (O − E)² ÷ E

O
observed count in a cell
E
expected count = row total × column total ÷ grand total
df
(rows − 1) × (columns − 1)

Applications of the chi-square test

  1. Test of association (independence): are two categorical variables related? Smoking and wound infection.
  2. Comparison of proportions (homogeneity): cure rates in two or more treatment groups.
  3. Goodness of fit: do observed counts match a stated distribution? Admissions per weekday, or a genetic ratio.
  4. Trend: chi-square for trend across ordered groups, such as dose levels.
  5. Paired proportions: the related McNemar test, for the same patients classified twice.

Worked example 5: test of association in a 2 × 2 table

InfectedNot infectedTotal
Skin preparation A892100
Skin preparation B1882100
Total26174200
  1. Expected infections in each row = 100 × 26 ÷ 200 = 13. Expected non-infected = 100 × 174 ÷ 200 = 87.
  2. Infected cells: (8 − 13)² ÷ 13 = 1.923 and (18 − 13)² ÷ 13 = 1.923. Non-infected cells: (92 − 87)² ÷ 87 = 0.287 each.
  3. χ² = 1.923 + 1.923 + 0.287 + 0.287 = 4.42, with (2 − 1) × (2 − 1) = 1 df.
  4. The critical χ² for 1 df at 5% is 3.84. 4.42 is larger: p = 0.036. Infection rates were 8% and 18%.

Yates' continuity correction

Counts are whole numbers, but the chi-square distribution is continuous. For 2 × 2 tables, Yates proposed subtracting 0.5 from each |O − E| before squaring.

χ²(Yates) = Σ (|O − E| − 0.5)² ÷ E

In example 5: (5 − 0.5)² ÷ 13 = 1.558 for each infected cell and 0.233 for each non-infected cell, so χ² = 3.58 and p = 0.058. The correction moved p above 0.05. The Yates-corrected test is overly conservative, and many authors advise against it. State in the protocol which version you will use, before you see the data.

When to use Fisher's exact test

The chi-square p-value is an approximation. It needs adequate expected counts: no more than 20% of cells with an expected count below 5, and no cell below 1. When a table fails this rule, use Fisher's exact test, which calculates the exact probability and is valid at any sample size.

ComplicationNoneTotal
New technique1910
Standard technique5510
Total61420

Made-up teaching data. Expected complications = 10 × 6 ÷ 20 = 3 in each row, so 2 of the 4 cells (50%) are below 5. Fisher's exact test gives p = 0.14. The uncorrected chi-square would have given χ² = 3.81, p = 0.051.

Worked example 6: goodness of fit

Made-up teaching data: 140 admissions for acute myocardial infarction by weekday. Monday 32, Tuesday 21, Wednesday 18, Thursday 17, Friday 19, Saturday 16, Sunday 17. If admissions were spread evenly, each day would expect 140 ÷ 7 = 20.

  1. (O − E)² ÷ E: Monday (32 − 20)² ÷ 20 = 7.2. Tuesday 0.05, Wednesday 0.2, Thursday 0.45, Friday 0.05, Saturday 0.8, Sunday 0.45.
  2. χ² = 9.2, with 7 − 1 = 6 df. The critical χ² for 6 df at 5% is 12.59.
  3. p = 0.16. These 140 admissions give no evidence that admissions differ by weekday, despite the high Monday count.

Worked example 7: an R × C table

Made-up teaching data: anaemia in adolescent girls by family diet, 80 girls in each group.

DietAnaemicNot anaemicTotal
Vegetarian305080
Mixed225880
Non-vegetarian146680
Total66174240
  1. Expected anaemic in each row = 80 × 66 ÷ 240 = 22. Expected not anaemic = 80 × 174 ÷ 240 = 58. Every expected count is well above 5.
  2. (O − E)² ÷ E: vegetarian (30 − 22)² ÷ 22 = 2.9091 and (50 − 58)² ÷ 58 = 1.1034. Mixed 0 and 0. Non-vegetarian 2.9091 and 1.1034.
  3. χ² = 2 × (2.9091 + 1.1034) = 8.03, with (3 − 1) × (2 − 1) = 2 df. The critical χ² for 2 df at 5% is 5.99, and p = 0.018. Anaemia was 37.5%, 27.5% and 17.5%.
  4. A significant R × C test shows only that the proportions are not all equal. To find which groups differ, compare pairs with a correction for multiple testing. With Bonferroni (0.05 ÷ 3 = 0.017), only vegetarian against non-vegetarian differs (p = 0.005). Vegetarian against mixed gives p = 0.18, and mixed against non-vegetarian p = 0.13.

Interactive

Edit the 2 × 2 table and check the expected counts before choosing the test

The chi-square p-value needs adequate expected counts. In a 2 × 2 table, one expected count below 5 is already more than 20% of the cells, so Fisher's exact test is used instead.

Try this: press "Small table" and compare the chi-square p-value with Fisher's exact p-value. Then type your own counts and see which test the expected counts allow.

Teaching examples from this chapter, made-up numbers. Worked example 5: wound infection with skin preparation A (group 1) and B (group 2).

Observed counts, editable
ObservedEventNo eventTotal
Group 1100
Group 2100
Total26174200
Expected counts if the groups did not differ
ExpectedEventNo event
Group 113.087.0
Group 213.087.0

expected = row total × column total ÷ 200 · event rate: group 1 8.0% · group 2 18.0%

0.036chi-square, χ² = 4.42, df 1
0.058with Yates' correction, χ² = 3.58
0.057Fisher's exact test

Every expected count is 5 or more, so the chi-square test can be used: χ² = 4.42, df 1, p = 0.036. The Yates-corrected test is overly conservative. State in the protocol which version you will use.

Comparing proportions

Which test compares proportions: z-test, McNemar test or binomial test?

Yes-or-no outcomes are compared in three designs: two independent groups, the same patients classified twice, or one group against a reference value. Each design has its own test.

Two independent groups: the z-test for two proportions

In worked example 5, infection was 8% with preparation A and 18% with preparation B. Under H₀ both groups share one infection rate, estimated from all 200 patients as 26 ÷ 200 = 0.13.

z = (p₂ − p₁) ÷ √[p(1 − p)(1/n₁ + 1/n₂)]

z = 0.10 ÷ √(0.13 × 0.87 × 0.02) = 0.10 ÷ 0.0476 = 2.103, p = 0.036

z² = 2.103² = 4.42, the same value as the chi-square statistic for this 2 × 2 table, with the same p-value, so the two are one test written in two ways. The z form can be used one-tailed, and it leads directly to a confidence interval for the difference: 10 percentage points, 95% CI 0.8 to 19.2 (calculated with the unpooled SE, 0.047).

The same patients classified twice: the McNemar test

After: anaemicAfter: not anaemicBefore: anaemicBefore: not anaemica = 30same both times: not usedb = 22status changed: usedc = 6status changed: usedd = 22same both times: not used(b − c)² ÷ (b + c) = (22 − 6)² ÷ 28 = 9.14
Made-up teaching data: anaemia in 80 pregnant women before and after 12 weeks of supplementation. 30 women were anaemic on both occasions and 22 on neither. These 52 concordant pairs give no information about change, so the test does not use them. Of the 28 women whose status changed, 22 recovered and 6 became anaemic.

McNemar χ² = (b − c)² ÷ (b + c)

χ² = (22 − 6)² ÷ 28 = 256 ÷ 28 = 9.14, df 1, p = 0.002

b
pairs positive on the first occasion only (anaemic before, not after): 22
c
pairs positive on the second occasion only (anaemic after, not before): 6
a, d
pairs with the same result both times: not used

With the continuity correction, χ² = (|b − c| − 1)² ÷ (b + c) = 8.04, p = 0.005. When b + c is small, use the exact binomial version, which here gives p = 0.004. Analysing the two occasions as independent groups (65% against 45% anaemic, chi-square p = 0.011) is wrong, because the same 80 women appear in both groups.

One group against a reference value: the binomial test

Made-up teaching data: 7 of 20 patients (35%) had a surgical site infection after a new procedure, against an expected rate of 15%. The expected number is only 20 × 0.15 = 3, too small for the normal approximation. The binomial test calculates the probability directly. If the true rate were 15%, the chance of 7 or more infections in 20 patients is 0.022, and the two-tailed p-value is 0.022. The z approximation gives z = 2.50 and a smaller p-value, 0.012, because it is inaccurate with an expected count of 3. The sign test for paired measurements is a binomial test with a reference proportion of 0.5.

DesignTestTeaching example in this chapter
Two independent groupsChi-square, or the z-test for two proportions (z² = χ²). Fisher's exact test when expected counts are small8% against 18%: p = 0.036
Same patients twice, or matched pairsMcNemar test, exact version when b + c is small65% against 45% anaemic: p = 0.002
One group against a reference valueBinomial test, or the z-test when expected counts are large35% against 15%: p = 0.022
Three or more groupsChi-square for an R × C tableWorked example 7: p = 0.018

Decide

The complete test map

Find your outcome in the left column and your design across the top.

OutcomeTwo independent groupsTwo paired setsThree or more independent groupsThree or more repeated measurementsRelation between two variables
Measurement, roughly normalUnpaired t-testPaired t-testOne-way ANOVARepeated measures ANOVAPearson correlation, linear regression
Measurement skewed, or ordinal scoresMann-Whitney UWilcoxon signed-rankKruskal-WallisFriedmanSpearman correlation
Categories (counts)Chi-square, or Fisher's exact if expected counts are smallMcNemarChi-square (r × c table)Cochran QChi-square, logistic regression
Time to an eventLog-rank test—Log-rank test—Cox regression

One group against a reference value: one-sample t-test (normal measurement), Wilcoxon signed-rank test (skewed measurement), chi-square goodness of fit (counts in several categories), or the binomial test (one yes-or-no proportion). Correlation and regression are covered in the correlation and regression chapter, and survival methods in the risk and survival chapter.

Software

Which software can you use for the analysis?

Any of these packages will run the tests in this chapter, but none of them chooses the right test for you. Name the software and its version in your methods section.

SoftwareMade or maintained byCostNotes
IBM SPSS StatisticsIBMPaid subscriptionMenu-driven. Fisher's exact test is given automatically for 2 × 2 tables
RA GNU project, free softwareFree (GNU General Public License)Code-based, with add-on packages from CRAN
jamoviThe jamovi project (community-developed)Free and open sourceMenus, runs on R and shows the R code behind each analysis
JASPSupported by the Psychological Methods Group, University of AmsterdamFree and open sourceMenus
StataStataCorp LLCPaid licence (6-month, annual or perpetual)Stata 1.0 was released in 1985
SASSASSAS OnDemand for Academics is free for learningPoint-and-click or programming
Epi InfoUS Centers for Disease Control and Prevention (CDC)Free, public domainData entry forms and epidemiological analysis. CDC stopped development and technical assistance in September 2025. The desktop version still runs on Windows 11
OpenEpiDean, Sullivan and SoeFree and open sourceRuns in a web browser: 2 × 2 and R × C tables, sample size, chi-square for dose-response
PSPPGNU projectFree (GNU GPL version 3 or later)A free replacement for SPSS that reads SPSS syntax and data files
GraphPad PrismGraphPadPaid subscription, with academic pricingWindows and Mac
MedCalcMedCalc Software LtdPaid licenceMore than 220 tests and graphs, including ROC curves and Bland-Altman plots
Microsoft Excel (Analysis ToolPak)MicrosoftAdd-in included with Excelt-tests (paired, equal and unequal variances), ANOVA, correlation
Python with SciPyOpen-source communityFree (BSD licence)Code-based

Qualitative data (interview and focus group transcripts) needs different software: NVivo (Lumivero), ATLAS.ti (paid licences, free trial), MAXQDA (VERBI Software, Berlin), and Taguette (free and open source). Qualitative methods are covered in the qualitative research chapter.

Types of analysis

What types of data analysis are there?

Analyses are classified by purpose, by the number of variables, by the type of data, and by whether they were planned in advance.

BasisTypesWhat it means
PurposeDescriptiveSummarises the data: counts, percentages, mean and SD, median and IQR
PurposeInferentialDraws conclusions about the population: confidence intervals and tests of significance
Number of variablesUnivariate, bivariate, multivariableOne variable at a time. Two variables (t-test, chi-square, correlation). Several at once (multiple or logistic regression, ANCOVA)
Type of dataQuantitative, qualitativeNumbers analysed with statistics. Text from interviews and focus groups analysed for codes and themes
AssumptionsParametric, non-parametricWhether a distribution is assumed (this chapter)
PlanningPrespecified (confirmatory), exploratory (post hoc)Only analyses planned in the protocol count as confirmatory
Patients included, in trialsIntention-to-treat (full analysis set), per-protocolAll randomised patients in their allocated groups, or only those who followed the protocol
Additional analysesInterim, subgroup, sensitivityAnalyses before the planned end, within subgroups, or under different assumptions

By specialty

Choosing a test by specialty

VAS pain scores in the same 40 patients before and 30 minutes after a nerve block.

The same patients measured twice are paired. Use the paired t-test, or the Wilcoxon signed-rank test if the differences are skewed.

Common myths

Common mistaken beliefs

  • Myth: Three t-tests are fine for three groups.

    Each test has a 5% false-positive rate. Three t-tests give a 14.3% chance of at least one false positive if the tests were independent. Use one-way ANOVA, then a post hoc test.

  • Myth: Non-parametric tests are weak, so avoid them.

    When data are normal, the Mann-Whitney and Kruskal-Wallis tests have an asymptotic relative efficiency of 0.955 against the t-test and ANOVA. With heavily skewed data they have more power than the parametric tests.

  • Myth: The t-test is only for samples smaller than 30.

    When the population SD is unknown, which is almost always, the t-test can be used at any sample size.

  • Myth: A significant ANOVA shows which groups differ.

    It shows only that at least one mean differs. A post hoc test (Tukey, Bonferroni) shows which pairs differ.

  • Myth: Always apply Yates' correction to a 2 × 2 table.

    The Yates-corrected test is overly conservative. Decide in the protocol, and use Fisher's exact test when expected counts are small.

  • Myth: Analysing paired data with an unpaired test loses only a little.

    In worked example 2, the same 16 numbers gave p = 0.20 unpaired and p = 0.0001 paired.

Why this matters to you

Five consequences of choosing the wrong test

  1. Protocol sent back

    ICH E9 expects the principal features of the statistical analysis in the protocol. "Appropriate statistical tests will be applied" does not meet that.

  2. A wrong conclusion in the thesis

    Analysing paired data with an unpaired test, or skewed data with a t-test, can make a real effect appear as "no significant difference".

  3. Questions you cannot answer at the viva

    "Why this test?" needs an answer in terms of outcome type, number of groups, pairing and distribution. "The software gave it" is not an answer.

  4. Misreading papers

    Applying parametric tests to non-normal data is one of the most common statistical errors found in journals. Check also for repeated t-tests across several groups.

  5. Harm to patients

    A false-positive result can bring an ineffective treatment into practice. A missed real effect can keep a useful one out.

History

History of the common statistical tests

  1. 1900

    Karl Pearson

    Published the chi-square criterion for judging whether observed frequencies could have arisen by random sampling from an expected distribution: the chi-square goodness-of-fit test.

  2. 1908

    "Student" (William Sealy Gosset)

    A brewer at Arthur Guinness in Dublin from 1899, he published "The probable error of a mean" in Biometrika, the paper behind the t-test for small samples. Guinness allowed him to publish only under a pseudonym, so he signed it "Student".

  3. 1925

    Ronald A. Fisher

    Developed analysis of variance in the 1920s at Rothamsted Experimental Station. His first ANOVA paper, with W. A. Mackenzie, appeared in 1923. His 1925 book Statistical Methods for Research Workers included chapters on the analysis of variance.

  4. 1934

    Frank Yates

    Published "Contingency tables involving small numbers and the χ² test", which introduced the continuity correction for 2 × 2 tables.

  5. 1942

    Jacob Wolfowitz

    Used the term "non-parametric" for methods that do not assume a distribution.

  6. 1945

    Frank Wilcoxon

    Introduced tests based on ranks in "Individual comparisons by ranking methods".

  7. 1947

    Henry Mann and Donald Whitney

    Extended Wilcoxon's rank method to two groups of different sizes.

  8. 1952

    William Kruskal and W. Allen Wallis

    Published "Use of ranks in one-criterion variance analysis", the rank test for three or more groups.

Guidelines

Statistical analysis in guidelines

ICH E9 (1998), section 5.1

“When designing a clinical trial the principal features of the eventual statistical analysis of the data should be described in the statistical section of the protocol.” “Only results from analyses envisaged in the protocol (including amendments) can be regarded as confirmatory.”

ICH E9 (1998), section 5.4

“Transformations (e.g. square root, logarithm) should be specified in the protocol and a rationale provided, especially for the primary variable(s).”

CONSORT 2025, items 21a and 21d

Trial reports must give the “statistical methods used to compare groups for primary and secondary outcomes, including harms” and “methods for any additional analyses (eg, subgroup and sensitivity analyses), distinguishing prespecified from post hoc”.

ICMJE Recommendations, Statistics

“Describe statistical methods with enough detail to enable a knowledgeable reader with access to the original data to judge its appropriateness for the study and to verify the reported results.” Authors should also “specify the statistical software package(s) and versions used”.

Steps

Choosing and running a test, step by step

  1. Write the primary outcome and name its type: measurement, ordinal or categorical.
  2. Count the groups, and decide whether they are independent or paired.
  3. For a measurement, plan how you will check the distribution (histogram, Shapiro-Wilk test) and what you will do if it is skewed.
  4. Choose the test from the map, with a post hoc test or correction if there are three or more groups.
  5. Write the analysis plan in the protocol: tests, α, software and version.
  6. After data entry, check the assumptions, run the planned test, and report the effect size with its 95% confidence interval, not only the p-value.
  7. Label every analysis you did not plan as exploratory.

Reading a paper

Five checks on the statistical tests in any paper

  • Does the test match the outcome: means for normal measurements, ranks for skewed or ordinal data, counts for categories?
  • Were paired measurements (before and after, matched pairs) analysed with a paired test?
  • With three or more groups, did the authors use ANOVA or Kruskal-Wallis with a named post hoc test, instead of repeated t-tests?
  • For a chi-square test, were the expected counts large enough, or was Fisher's exact test used?
  • Are the tests, the software and its version named, and are unplanned analyses labelled as exploratory?

Why is it called that?

Word surgery

“Normal”

In everyday speech

Healthy, usual, nothing wrong.

In research

A bell-shaped, symmetric distribution. "Normally distributed" says nothing about whether the patients are healthy.

“Independent”

In everyday speech

Self-reliant, not controlled by anyone.

In research

One value gives no information about another. Two groups are independent when they contain different, unmatched patients.

Memory trick

Same patients twice: paired test. Two groups: t-test or Mann-Whitney. Three or more groups: ANOVA or Kruskal-Wallis. Counts: chi-square, or Fisher's exact test when expected counts are below 5.

Parametric

Greek para- "beside" + metron "measure", through "parameter"

A parametric test assumes a distribution defined by parameters such as the mean and SD.

Variance

Latin variantia, from variare "to change"

The average squared deviation from the mean. ANOVA compares means by analysing variance.

Analysis

Greek analysis: "a breaking up, a loosening"

ANOVA breaks the total variation into a part between groups and a part within groups.

Normal

Latin norma: "carpenter's square", a rule or pattern

In statistics, the bell-shaped distribution that parametric tests assume.

Nominal

Latin nomen: "name"

Nominal data are categories that have names but no order, such as blood group.

Chi-square

the Greek letter χ (chi), squared

The test statistic is written χ². It adds up (O − E)² ÷ E over all the cells of the table.

Exam answers

Model answer outlines

Parametric and non-parametric tests: differences, assumptions, importance and uses

Ph.D. RM 2013, 2014, 2016, 2018 · MD Community Medicine 2002, 2025 · MD Pharmacology 2011, 2024 · MD Nuclear Medicine 2018

Definitions: parametric tests assume the data follow a known distribution, usually normal, described by parameters such as the mean and SD. Non-parametric tests work on ranks or signs and assume no particular distribution.

FeatureParametricNon-parametric
DistributionNormal, similar variancesNone assumed
DataMeasurements (interval, ratio)Ordinal, skewed measurements, outliers
UsesActual valuesRanks or signs
SummaryMean and SDMedian and IQR
PowerHigher when assumptions holdEfficiency 0.955 against the t-test with normal data, higher with heavy skew
Examplest-test, ANOVA, Pearson rMann-Whitney, Wilcoxon, Kruskal-Wallis, Friedman, Spearman rho

Assumptions of parametric tests: normal distribution (of the differences, for paired tests), equal variances (Levene test), independent observations, a measurement on an interval or ratio scale, and random sampling.

Importance and uses of non-parametric tests: ordinal data (pain grade, Likert items, Apgar score), skewed measurements (length of stay), outliers, small samples, and any situation where normality fails. They remain valid without a distributional assumption and are little affected by extreme values.

Limitations: they use less of the information in the data, confidence intervals are harder to calculate, there are fewer methods for complex designs, and they have less power when parametric assumptions hold. A significant Mann-Whitney test is not proof that the medians differ, because differences in spread also affect it (Hart, BMJ 2001).

Matching pairs: unpaired t-test and Mann-Whitney U. Paired t-test and Wilcoxon signed-rank. One-way ANOVA and Kruskal-Wallis. Repeated measures ANOVA and Friedman. Pearson and Spearman correlation.

Student's t-test: principle, types, importance and applications

MD Pharmacology 2018, 2025, and undated · Ph.D. RM 2014, 2017 · MD Biochemistry 2014
  1. Origin: W. S. Gosset, a brewer at Guinness, published it under the name “Student” (Biometrika, 1908), to handle small samples.
  2. Principle: t = difference between means ÷ standard error of the difference, compared with the t distribution at the correct degrees of freedom.
  3. Assumptions: a measurement outcome, roughly normal, independent observations, and similar variances for the unpaired test (otherwise the Welch test).
  4. Types: one-sample (df = n − 1), unpaired (df = n₁ + n₂ − 2), paired (df = number of pairs − 1).
  5. Steps: state H₀, calculate means, SDs and SE, calculate t and df, compare with the critical t or obtain p, and give the 95% CI.
  6. Applications: a sample mean against a standard (mean Hb against 12 g/dL), two treatments (fall in BP with two drugs), before and after treatment (pain scores), and each coefficient in linear regression.
  7. Importance: correct p-values in small samples, no need for the population SD, the basis of confidence intervals for means, and one of the most used tests in medicine.
  8. Limitations: only two groups (use ANOVA for more). In small samples outliers and skew distort the mean (use the Mann-Whitney or Wilcoxon test).

Paired versus unpaired t-test

MD Pharmacology 2024 · MD Nuclear Medicine 2021
FeaturePaired t-testUnpaired t-test
DataSame patients twice, or matched pairsTwo independent groups
ExamplePain before and after a blockPain with drug A and with drug B in different patients
Formulat = mean difference ÷ (SD of differences ÷ √n)t = (mean₁ − mean₂) ÷ [pooled SD × √(1/n₁ + 1/n₂)]
dfPairs − 1n₁ + n₂ − 2
Between-patient variationRemovedIncluded in the SE
Non-parametric versionWilcoxon signed-rankMann-Whitney U

Worked point: 8 patients whose pain fell by 1 or 2 points each. Paired t = 7.51, p = 0.0001. The same numbers analysed as unpaired: t = 1.35, p = 0.20. An unpaired test ignores the pairing and can miss a real effect.

Calculation of standard deviation and t-value in small experimental groups

MD Biochemistry 1994
  1. SD = √[Σ(x − mean)² ÷ (n − 1)]. Divide by n − 1, not n, in a sample.
  2. Example: group A 12, 15, 9, 14, 11, 16, 10, 13 (mean 12.5). Group B 8, 10, 6, 11, 7, 9, 12, 5 (mean 8.5).
  3. Σ(x − mean)² = 42 in each group, so SD = √(42 ÷ 7) = 2.45.
  4. Pooled SD = 2.45. SE of the difference = 2.45 × √(1/8 + 1/8) = 1.225.
  5. t = (12.5 − 8.5) ÷ 1.225 = 3.27, df = 14. Critical t at 5% = 2.145, so p < 0.05 (exact p = 0.006).

ANOVA: principle, types (including factorial ANOVA), assumptions and importance

MD Biochemistry 2013, 2025 · Ph.D. RM 2025 · MD Pharmacology 2023 · MD Community Medicine 2013
  1. Principle: H₀: all group means are equal. Total variation is divided into variation between groups and within groups. F = mean square between ÷ mean square within. A large F means the group means differ more than patients within a group. Under H₀ both mean squares estimate the same variance, so F is close to 1, and only large values of F (the upper tail) count against H₀.
  2. Types: one-way (one factor, three or more groups), two-way or factorial (two or more factors and their interaction), repeated measures (same patients over time), mixed (between- and within-patient factors), ANCOVA (adjusted for a covariate).
  3. Factorial ANOVA: every level of one factor is combined with every level of another (drug A or B × male or female = 4 groups). It gives the main effect of each factor and the interaction, which shows whether the drug effect differs by sex.
  4. Assumptions: independent observations, normal distribution within groups, equal variances (Levene test), sphericity for repeated measures (Mauchly test), measurement outcome.
  5. Post hoc tests: after a significant F, Tukey, Bonferroni, Scheffé, or Games-Howell when variances are unequal. Dunnett's test when each group is compared only with one control group.
  6. Importance: one test keeps the false-positive rate at 5% (three separate t-tests would allow up to 14.3%), several factors and interactions in one analysis, repeated measurements, adjustment for baseline values, and the standard analysis for designed experiments since Fisher developed it in the 1920s.
  7. Example: analgesia duration 5.67, 8.00 and 9.00 hours in three groups of six. F(2, 15) = 50.7, p < 0.001. Tukey test: all pairs differ.

How Student's t-test and ANOVA are used, with examples

Ph.D. RM 2019
  • Unpaired t-test, two groups: fall in systolic BP 12.5 mmHg with drug A and 8.5 mmHg with drug B, 8 patients each. t = 3.27, df = 14, p = 0.006.
  • Paired t-test, same patients: pain before and after an analgesic in 8 patients. t = 7.51, df = 7, p = 0.0001.
  • One-sample t-test: mean Hb 11.36 g/dL in 10 girls against 12 g/dL. t = −3.56, p = 0.006.
  • One-way ANOVA, three groups: analgesia 5.67, 8.00 and 9.00 hours. F(2, 15) = 50.7, p < 0.001, followed by Tukey's test.
  • Link between them: with only two groups, one-way ANOVA gives the same p-value as the unpaired t-test (F = t²).

ANOVA and the Kruskal-Wallis test

Ph.D. RM 2025
FeatureOne-way ANOVAKruskal-Wallis
TypeParametricNon-parametric
DataNormal measurements, similar variancesSkewed measurements or ordinal scores
Works onMeans and variancesRanks of all values pooled
StatisticF, df (k − 1, N − k)H, compared with χ² on k − 1 df
After a significant resultTukey, BonferroniPairwise rank tests with a correction for multiple testing
Same teaching dataF = 50.7, p < 0.001H = 13.69, p = 0.001

Kruskal and Wallis published the test in 1952 as the rank-based version of one-way analysis of variance.

Chi-square test: principle, applications with two examples, Yates' correction and Fisher's exact test

MD Pharmacology 2019, 2024, 2025 · Ph.D. RM 2019, 2020 · MD Biochemistry 2025 · MD Nuclear Medicine 2024
  1. Principle: χ² = Σ (O − E)² ÷ E, where E = row total × column total ÷ grand total. df = (r − 1)(c − 1). Introduced by Karl Pearson in 1900.
  2. Conditions: counts (not percentages or means), independent observations, no more than 20% of cells with expected count below 5 and none below 1.
  3. Applications: association between two categorical variables, comparison of proportions, goodness of fit, trend across ordered groups. McNemar test for paired proportions.
  4. Example 1 (association): wound infection 8/100 with preparation A and 18/100 with B. Expected 13 each. χ² = 4.42, df = 1, p = 0.036.
  5. Example 2 (goodness of fit): 140 MI admissions over 7 weekdays, expected 20 per day. χ² = 9.2, df = 6, p = 0.16. No evidence of a weekday difference.
  6. Yates' correction (2 × 2 tables): Σ (|O − E| − 0.5)² ÷ E. In example 1, χ² = 3.58, p = 0.058. It is overly conservative.
  7. Fisher's exact test: when expected counts are small. Complications 1/10 against 5/10: expected 3 per cell, Fisher p = 0.14.
  8. R × C tables: df = (r − 1)(c − 1). Anaemia 37.5%, 27.5% and 17.5% in three diet groups of 80: χ² = 8.03, df 2, p = 0.018. A significant result does not show which groups differ, so compare pairs with a correction (Bonferroni).
  9. z-test for two proportions: the same test as a 2 × 2 chi-square, since z² = χ². In example 1, z = 2.103 and z² = 4.42.
  10. Paired proportions (McNemar): χ² = (b − c)² ÷ (b + c), using only the pairs that changed. Anaemia before and after supplementation in 80 women, b = 22 and c = 6: χ² = 9.14, p = 0.002.
  11. Strength of association: odds ratio or phi for 2 × 2 tables, Cramér's V for larger tables.

Tests of significance: definition and the tests for different types of data

Ph.D. RM 2021

Definition: a test of significance calculates the probability (p) of a difference at least as large as the one observed, if the null hypothesis were true. p below the chosen α (usually 0.05) is called statistically significant.

DataTwo independent groupsPairedThree or more groupsRelationship
Measurement, normalUnpaired t-test (z-test for large samples with known SD)Paired t-testANOVAPearson r, regression
Skewed or ordinalMann-Whitney UWilcoxon signed-rankKruskal-Wallis, FriedmanSpearman rho
Nominal (counts)Chi-square, Fisher's exact, z-test for two proportionsMcNemarChi-squareChi-square, logistic regression
Time to eventLog-rankLog-rankCox regression

One group against a reference value: one-sample t-test (normal measurement), Wilcoxon signed-rank test (skewed measurement), binomial test or z-test (one proportion), chi-square goodness of fit (counts in several categories).

The choice rests on four facts: outcome type, number of groups, paired or independent, and distribution.

Common software for data analysis

Ph.D. RM 2015, 2018
  • Paid: IBM SPSS Statistics (IBM, menu-driven), Stata (StataCorp LLC), GraphPad Prism, MedCalc (MedCalc Software Ltd). SAS is free for learning through SAS OnDemand for Academics.
  • Free: R (GNU General Public License, code-based), jamovi and JASP (open source, menu-driven), PSPP (free replacement for SPSS), OpenEpi (browser-based), Python with SciPy.
  • Public health: Epi Info from the CDC, public domain. Development and technical assistance stopped in September 2025.
  • Spreadsheet: Microsoft Excel with the Analysis ToolPak add-in (t-tests, ANOVA, correlation).
  • Qualitative data: NVivo, ATLAS.ti, MAXQDA, Taguette (free).
  • Choosing: licence cost, menus or code, what your institution supports, and whether the analysis can be saved as syntax or script and repeated.
  • Reporting: name the software package and version in the methods section (ICMJE).

Opportunities and threats from easy access to statistical software

MD Community Medicine 2019

Opportunities

  • Analyses that took days by hand take seconds.
  • Complex methods (multivariable regression, repeated measures, survival analysis) are available to every researcher.
  • Free packages (R, jamovi, JASP, PSPP, OpenEpi) cost nothing.
  • Saved syntax and scripts make an analysis repeatable, and code can be shared with the paper (CONSORT 2025, item 4).

Threats

  • Clicking “OK” to get a p-value without understanding the method increases the risk of the wrong test or a misread result.
  • Parametric tests applied to non-normal data, one of the most common errors in journals.
  • Running many tests until one is significant, and reporting only that one.
  • Relying on p-values without effect sizes and confidence intervals.
  • Assumptions (normality, expected counts, sphericity) left unchecked.
  • Software that stops being developed: CDC ended Epi Info development in September 2025.

Safeguards: a statistical analysis plan in the protocol (ICH E9), training, a statistician on the team, prespecified and exploratory analyses kept apart, and the software and version reported.

Qualitative research: study designs, tools, and software for analysing qualitative data

MD Community Medicine 2023
  1. Designs: phenomenology (lived experience of a condition), grounded theory (building a theory from the data), ethnography (the culture and practices of a group), case study, and narrative research.
  2. Tools: in-depth interviews, focus group discussions, key informant interviews, participant observation, and document review, each with a topic guide.
  3. Analysis: transcription, coding, then thematic or content analysis until no new themes appear.
  4. Software: NVivo (Lumivero), ATLAS.ti, MAXQDA (VERBI Software), and Taguette (free and open source). The software organises and codes text. The researcher still decides the codes and themes.

Designs and sampling are covered in the qualitative research chapter.

Types of data analysis

Ph.D. RM 2019
  • By purpose: descriptive (summaries) and inferential (confidence intervals, tests of significance).
  • By number of variables: univariate, bivariate, multivariable.
  • By type of data: quantitative (statistics) and qualitative (thematic or content analysis).
  • By assumptions: parametric and non-parametric.
  • By planning: prespecified (confirmatory) and exploratory (post hoc). ICH E9: only analyses envisaged in the protocol can be regarded as confirmatory.
  • In trials: intention-to-treat (full analysis set) and per-protocol.
  • Additional analyses: interim, subgroup and sensitivity analyses.

Statistical analysis: planning and conducting the analysis of a study

DM Neurology 2015
  1. Write the principal features of the analysis in the protocol before the study starts (ICH E9, section 5.1).
  2. Enter and code the data, check for errors, missing values and outliers.
  3. Describe the sample: n, mean and SD or median and IQR, counts and percentages.
  4. Check distributions and assumptions.
  5. Choose the test from outcome type, number of groups, pairing and distribution.
  6. Report the effect size with its 95% CI, and the exact p-value, at a two-sided α of 0.05.
  7. Adjust for confounders with multivariable models, and correct for multiple comparisons.
  8. In trials, state the analysis population (intention-to-treat or per-protocol) and how missing data were handled (CONSORT 2025, items 21b and 21c).
  9. Name the software and version, and label exploratory analyses.

Role of pilot testing in health research

Ph.D. RM 2024
  1. Definition: a small-scale run of the planned study to examine whether its methods are feasible.
  2. Roles: test the feasibility of recruitment, randomisation, retention, assessment procedures, new methods and delivery of the intervention.
  3. Pretest questionnaires for clarity, wording and time taken, and train data collectors.
  4. Try out data entry and the analysis plan.
  5. Identify changes needed before the main study.
  6. Limits: a pilot study is not a hypothesis-testing study. It does not evaluate safety or efficacy, and its effect size is too imprecise to base the main study's sample size on. Feasibility results apply only within the pilot's own eligibility criteria.
  7. Reporting: the CONSORT 2010 extension to randomised pilot and feasibility trials (26-item checklist, 2016) asks for the prespecified criteria used to decide whether to proceed to the main trial.

Project proposal: reasons for and factors in non-compliance among patients with type 2 diabetes, including the plan for statistical analysis

Ph.D. RM 2023
  1. Title, background and rationale: non-compliance with drugs, diet and follow-up worsens glycaemic control.
  2. Objectives: primary, the proportion of patients who are non-compliant. Secondary, the factors associated with non-compliance and the reasons patients give.
  3. Design: cross-sectional analytical study with a qualitative component (mixed methods). See the study designs chapter.
  4. Setting, population and criteria: adults with type 2 diabetes on treatment for at least 6 months at a diabetes clinic, excluding those unable to consent.
  5. Sample size: single-proportion formula with the expected proportion from earlier studies, the precision required and an allowance for non-response (sample size chapter).
  6. Sampling: systematic random sampling from the clinic register. Purposive sampling for the interviews.
  7. Variables: outcome, compliant or non-compliant. Factors: age, sex, education, income, duration of diabetes, number of tablets, insulin use, side effects, cost, knowledge, family support, distance to clinic.
  8. Tools: pretested structured questionnaire, pill count or a self-report adherence scale, and an interview guide for in-depth interviews.
  9. Plan for statistical analysis: descriptive statistics. Proportion non-compliant with 95% CI. Chi-square test (Fisher's exact if expected counts are small) for categorical factors. Unpaired t-test or Mann-Whitney U for continuous factors. Multivariable logistic regression for adjusted odds ratios with 95% CI. Two-sided α 0.05. Software and version named. Interview transcripts coded and analysed thematically.
  10. Ethics: ethics committee approval, written informed consent, confidentiality (research ethics chapter).
  11. Timeline, budget, and plan for dissemination.

Role of correlation and regression coefficients in statistical analysis

MD Community Medicine 2009
  • Correlation coefficient (r): measures the strength and direction of a linear relationship, from −1 to +1, with no units. Pearson r for normal measurements, Spearman rho for ranks.
  • r² gives the proportion of variation in one variable that is explained by the other.
  • Correlation does not show cause, and it does not show agreement between two methods of measurement.
  • Regression coefficient (b): the change in the outcome for a one-unit change in the predictor, in the outcome's units (y = a + bx). It allows prediction.
  • In multiple regression each coefficient gives that factor's effect adjusted for the others. In logistic regression the coefficients convert to odds ratios.
  • Each coefficient is reported with its 95% CI and a test of whether it differs from zero.

Details are in the correlation and regression chapter.

The one thing to remember

Choose the test from four facts about your data, and write it in the protocol before you see the results: outcome type, number of groups, paired or independent, and distribution.

All worked examples use made-up numbers for teaching. Every statistic was computed in Python (SciPy 1.15) with two-sided tests at α 0.05.

Sources: Pearson K. Philos Mag 1900;50(302):157–75 (doi:10.1080/14786440009463897). Plackett RL. Int Stat Rev 1983 (doi:10.2307/1402731). Student. Biometrika 1908;6(1):1 (doi:10.1093/biomet/6.1.1). Cals J, Winkens B. Ned Tijdschr Geneeskd 2018;162:D3204 (PMID 30212007). Burgin C. J Child Neurol 1995;10(4):293 (doi:10.1177/088307389501000407). Breathnach CS. Ir J Psychol Med 1993;10(3):164–5 (doi:10.1017/S0790966700012659). Fisher RA, Mackenzie WA. J Agric Sci 1923;13(3):311–20 (doi:10.1017/S0021859600003592). Fisher RA. Statistical Methods for Research Workers. Edinburgh: Oliver and Boyd; 1925. Cochran WG. Fisher and the analysis of variance. 1980 (doi:10.1007/978-1-4612-6079-0_4). Parolini G. J Hist Biol 2015;48(2):301–35 (doi:10.1007/s10739-014-9394-z). Yates F. Suppl J R Stat Soc 1934;1(2):217–35 (doi:10.2307/2983604). Serra N et al. Epidemiol Biostat Public Health (doi:10.2427/13059). Haviland MG. Stat Med 1990;9(4):363–7 (doi:10.1002/sim.4780090403). Wolfowitz J. Ann Math Stat 1942;13(3):247–79 (doi:10.1214/aoms/1177731566). Wilcoxon F. Biometrics Bull 1945;1(6):80 (doi:10.2307/3001968). Mann HB, Whitney DR. Ann Math Stat 1947;18(1):50–60 (doi:10.1214/aoms/1177730491). Kruskal WH, Wallis WA. J Am Stat Assoc 1952;47(260):583–621 (doi:10.1080/01621459.1952.10483441). Nahm FS. Korean J Anesthesiol 2016;69(1):8–14 (doi:10.4097/kjae.2016.69.1.8). Kitchen CMR. Am J Ophthalmol 2009;147(4):571–2 (doi:10.1016/j.ajo.2008.06.031). Mishra P et al. Ann Card Anaesth 2019;22(4):407–11 (doi:10.4103/aca.ACA_94_19). Kim TK. Korean J Anesthesiol 2017;70(1):22–6 (doi:10.4097/kjae.2017.70.1.22). Kim HY. Restor Dent Endod 2017;42(2):152–5 (doi:10.5395/rde.2017.42.2.152). Hart A. BMJ 2001;323(7309):391–3 (doi:10.1136/bmj.323.7309.391, PMID 11509435). McHugh ML. Biochem Med 2013;23(2):143–9 (doi:10.11613/BM.2013.018). Leon AC, Davis LL, Kraemer HC. J Psychiatr Res 2011;45(5):626–9 (doi:10.1016/j.jpsychires.2010.10.008). Eldridge SM et al. BMJ 2016;355:i5239 (doi:10.1136/bmj.i5239). ICH E9 Statistical Principles for Clinical Trials, 5 February 1998. Hopewell S et al. CONSORT 2025 statement. BMJ 2025;389:e081123. ICMJE Recommendations, section on statistics. Software details from the developers' websites, accessed 14 September 2026.

Past papers

Every TNMGRMU question on choosing a test

Each one is answered in this chapter. Tap a sitting to open the university's own question paper.

  1. ×7Chi-square testMD Pharmacology · May 2025MD Biochemistry · May 2025MD Pharmacology · Oct 2024MD Nuclear Medicine · Dec 2024MD Pharmacology · Oct 2019Ph.D. RM · Jan 2019Ph.D. RM · Jul 2019
  2. ×3Student t testMD Pharmacology · Oct 2018Ph.D. RM · Jan 2017Ph.D. RM · Jan 2014
  3. ×1Student T test and its applicationsMD Pharmacology · Oct 2025
  4. ×1Principle and types of ANOVA (analysis of variance)MD Biochemistry · Oct 2025
  5. ×1Discuss the importance and usage of non-parametric test in medical statisticsMD Community Medicine · Oct 2025
  6. ×1ANOVA and Kruskal-Wallis testPh.D. RM · Jan 2025
  7. ×1ANOVA and its typesPh.D. RM · Jul 2025
  8. ×1Compare Parametric and Non parametric testsMD Pharmacology · May 2024
  9. ×1Paired versus Unpaired t-testMD Pharmacology · May 2024
  10. ×1Role of Pilot Testing in Health ResearchPh.D. RM · Jul 2024
  11. ×1Factorial AnovaMD Pharmacology · Oct 2023
  12. ×1Describe the study designs for Qualitative research? What are the tools that can be used for qualitative studies? List the various software available for Analysing Qualitative DataMD Community Medicine · Oct 2023
  13. ×1As a researcher you are asked to identify the reasons for and factors influencing non-compliance among Diabetic (type 2). Write a detailed project proposal highlighting the research design including the plan for statistical analysisPh.D. RM · Jul 2023
  14. ×1Paired t testMD Nuclear Medicine · Jul 2021
  15. ×1What are the Test of Significance? Describe the different tests for different types of dataPh.D. RM · Aug 2021
  16. ×1Explain the applications of Chi-Square test with two examplesPh.D. RM · Dec 2020
  17. ×1Discuss the opportunities and threats subsequent to the easy availability of software for statistical analysis on research outputMD Community Medicine · Oct 2019
  18. ×1Explain how student’s T-test and ANOVA are used with appropriate examplesPh.D. RM · Jan 2019
  19. ×1Types of Data analysisPh.D. RM · Jul 2019
  20. ×1Non-parametric testsMD Nuclear Medicine · May 2018
  21. ×1Parametric and nonparametric testPh.D. RM · Jan 2018
  22. ×1Describe the common software used for analysis of dataPh.D. RM · Jul 2018
  23. ×1What is student t-Test and explain its importancePh.D. RM · Jul 2017
  24. ×1Non parametric tests / analysisPh.D. RM · Jul 2016
  25. ×1Statistical analysisDM Neurology · Aug 2015
  26. ×1Software for data analysisPh.D. RM · Jul 2015
  27. ×1Students’ t-testMD Biochemistry · Oct 2014
  28. ×1Parametric and non-parametric testsPh.D. RM · Jul 2014
  29. ×1Importance of ANOVAMD Biochemistry · Apr 2013
  30. ×1Assumptions in ANOVAMD Community Medicine · Oct 2013
  31. ×1Distinguish between Parametric and Non Parametric Tests. What are the assumptions of the Parametric Tests?Ph.D. RM · Jul 2013
  32. ×1Value of non-parametric statistical testsMD Pharmacology · May 2011
  33. ×1What is the role of correlation and regression coefficient in statistical analysis?MD Community Medicine · Sep 2009
  34. ×1Non parametric tests. (bh) Use of pictograma in health education. ic} Communication barriers. fe) Multidrug resistant tuberculosisMD Community Medicine · Mar 2002
  35. ×1Calculation of standard deviation and t-value of small experimental groups. ]MD Biochemistry · Apr 1994
  36. ×1Student t test (@) Pebrile convulsions in children, (Sxto=59}MD Pharmacology

Further reading

Related Stateazy articles