Chapter 15 · Making sense of numbers
Which statistical test do you use?
Parametric and non-parametric tests: t-test, ANOVA, chi-square, Mann-Whitney, Kruskal-Wallis.
36 TNMGRMU questions answered hereset 44 times, 1994–2025
The problem first
You are a second-year Anaesthesiology resident. Your thesis records VAS pain scores in 40 patients before and 30 minutes after a nerve block. You enter the scores in two columns and click the first t-test in the software menu, the independent-samples test. The software reports no significant difference, and you write that the block did not reduce pain.
At the pre-submission seminar, the head of department reads one line of your methods section.
“These are the same 40 patients. Why did you use an unpaired test?”
Start here
What decides which test you use?
Every test asks the same question: could chance alone explain the difference you see? Which test answers it depends on four facts about your data.
- Type of outcome: a measurement, ordered grades, or categories.
- Number of groups: two, or three or more.
- Independent or paired: different patients in each group, or the same patients measured twice (or matched pairs).
- Distribution, for measurements: roughly normal, or skewed with outliers.
Two types of tests
Parametric or non-parametric: what is the difference?
A parametric test assumes the data come from a known distribution, usually the normal distribution, described by parameters such as the mean and SD. A non-parametric test does not assume a distribution. It works on ranks or signs instead of the raw values.
Assumptions of parametric tests
- Normality: the outcome is roughly normal in each group (for paired tests, the differences). Look at a histogram, and use the Shapiro-Wilk test if needed.
- Equal variances: the groups have similar spread. Check with the Levene test.
- Independence: one patient's value gives no information about another's, except where pairing is part of the design.
- Scale: the outcome is a measurement on an interval or ratio scale, so that a mean has a meaning.
Non-parametric tests
How can a test work without the actual values?
It replaces each value with its rank. The smallest value gets rank 1, the next rank 2, and tied values share the average of their ranks.
Interactive
Move one long stay and watch the t-test and the Mann-Whitney test
The t-test uses the actual values, so one long stay raises the mean and the SD of its group. The Mann-Whitney test uses ranks, so a stay longer than all the others counts as rank 16 whatever its length.
Try this: move the eighth laparoscopic stay from 4 days up to 21 days. The t-test p-value rises above 0.05. The Mann-Whitney p-value stops changing once this stay is longer than all the other stays.
Made-up teaching data from this chapter: length of stay after laparoscopic and open surgery, 8 patients in each group.
The other 15 stays do not change
Both tests are significant at α 0.05. The eighth laparoscopic stay has rank 8 of 16.
means 2.88 and 5.75 days · medians 3.0 and 5.5 days · t = -5.32, df 14 · rank sums 37 and 99, U = 1
Decide
Which test fits your study?
Answer the questions in order. Each answer links to its worked example.
What kind of outcome are you comparing?
Which test: all possible answers
- Paired t-test: Works on the difference within each pair. Degrees of freedom = number of pairs − 1.
- Wilcoxon signed-rank test: Ranks the differences within pairs by size and keeps their signs.
- Unpaired (independent samples) t-test: Compares two means. If the variances clearly differ, use the version for unequal variances (Welch test).
- Mann-Whitney U test: Ranks all values from both groups together and compares the ranks. Also called the Wilcoxon rank-sum test.
- One-way ANOVA: Tests whether at least one group mean differs. If it is significant, a post hoc test (Tukey, Bonferroni) shows which pairs differ.
- Kruskal-Wallis test: The rank version of one-way ANOVA. If it is significant, compare pairs with a correction for multiple testing.
- Repeated measures ANOVA: Compares means across time points in the same patients. Check sphericity with the Mauchly test.
- Friedman test: The rank version of repeated measures ANOVA.
- One-sample t-test: Compares one sample mean with a reference value, such as Hb 12 g/dL.
- Wilcoxon signed-rank test (one sample): Compares the median of one sample with a reference value.
- Chi-square test: Compares observed counts with the counts expected if there were no association. Degrees of freedom = (rows − 1) × (columns − 1).
- Fisher's exact test: Calculates the exact probability for a table with small expected counts. Most packages give it for 2 × 2 tables.
- McNemar test: For paired yes/no results, such as test positive or negative in the same patients before and after treatment. Only the pairs that changed enter the test.
- Binomial test (one proportion): Compares the proportion in one sample with a reference value. The z-test is an approximation that needs large expected counts.
- Chi-square goodness-of-fit test: Compares observed counts with the counts expected from stated proportions.
- Correlation or regression: Pearson r for two normal measurements, Spearman rho for ranks. Regression predicts one from the other.
- Survival analysis: Kaplan-Meier curves compared with the log-rank test, or Cox regression.
Comparison
Parametric or non-parametric: which one should you use?
Use a parametric test when its assumptions hold, because it uses the actual values. Use a non-parametric test when they do not hold.
| Feature | Parametric | Non-parametric |
|---|---|---|
| Assumes | A distribution, usually normal, with similar variances | No particular distribution. Observations must still be independent |
| Works on | The actual values | Ranks or signs |
| Summary reported | Mean and SD | Median and IQR |
| Suits | Measurements that are roughly normal | Skewed measurements, outliers, ordinal scores |
| Outliers | Pull the mean and inflate the SD | Count only as the highest or lowest rank |
| Efficiency | Best when the assumptions hold | Asymptotic relative efficiency 0.955 against the t-test when data are normal, and higher than the t-test when data are heavily skewed |
| Confidence interval | Straightforward for a difference in means | Harder to calculate |
| Examples | t-test, ANOVA, Pearson correlation | Mann-Whitney, Wilcoxon, Kruskal-Wallis, Friedman, Spearman correlation |
The chi-square test is usually grouped with the non-parametric tests because it works on counts and assumes no distribution for a measurement.
Comparison
Paired or unpaired: are the two sets of values linked?
Values are paired when each value in one set belongs with one value in the other: the same patient before and after, two eyes of one patient, or a case matched to a control.
| Feature | Paired t-test | Unpaired t-test |
|---|---|---|
| Data | Two values from the same patient, or from a matched pair | One value from each patient in two different groups |
| Example | Hb before and after 3 months of iron in the same women | Hb in women given oral iron and in other women given IV iron |
| Works on | The difference within each pair | The difference between the two group means |
| Standard error | SD of the differences ÷ √n | Pooled SD × √(1/n₁ + 1/n₂) |
| Degrees of freedom | Number of pairs − 1 | n₁ + n₂ − 2 |
| Variation between patients | Removed, because each patient is compared with themselves | Stays in the standard error |
| Equal numbers needed | Yes, every value needs its pair | No |
| Non-parametric version | Wilcoxon signed-rank test | Mann-Whitney U test |
Interactive
Change how much the women differ and watch the paired and unpaired p-values
The paired test works on each patient's own change, so the variation between patients does not enter its standard error. The unpaired test keeps that variation.
Try this: move the SD of Hb before treatment from 1.2 up to 2.0 g/dL. The unpaired p-value rises. The paired p-value stays the same, because each woman is compared with herself.
Made-up teaching data: Hb in the same 10 women before and after 3 months of oral iron. Both tests use the same 20 numbers.
Larger rise: smaller p in both tests
How much the rise differs from woman to woman
How much the women differ from each other. The paired test does not use it
paired t = 3.95, df 9 · unpaired t = 1.93, df 18
The unpaired test puts the variation between women (SD 1.2 g/dL) into its standard error. The paired test works on each woman's own rise, so only the SD of the rises (0.8 g/dL) enters its standard error.
The formula
Student's t-test: is the difference between two means larger than chance would produce?
t is the difference between the means divided by its standard error. The larger t is, the less likely it is that chance alone produced the difference.
t = (mean₁ − mean₂) ÷ SE of the difference
Three types of t-test
- One-sample t-test: one sample mean against a reference value. t = (mean − reference) ÷ (SD ÷ √n), df = n − 1.
- Unpaired (independent samples) t-test: two independent groups. df = n₁ + n₂ − 2.
- Paired t-test: two measurements on the same patients. t = mean difference ÷ (SD of the differences ÷ √n), df = number of pairs − 1.
Worked example 1: unpaired t-test in two small groups
Drug A: 12, 15, 9, 14, 11, 16, 10, 13 mmHg. Drug B: 8, 10, 6, 11, 7, 9, 12, 5 mmHg.
- Means: A = 100 ÷ 8 = 12.5 and B = 68 ÷ 8 = 8.5. Difference = 4.0 mmHg.
- Squared deviations from the mean in group A: 0.25, 6.25, 12.25, 2.25, 2.25, 12.25, 6.25, 0.25. They add up to 42. Group B also adds up to 42.
- SD = √(42 ÷ 7) = √6 = 2.45 mmHg in each group.
- Pooled SD = √[(7 × 6 + 7 × 6) ÷ 14] = 2.45 mmHg.
- SE of the difference = 2.45 × √(1/8 + 1/8) = 1.225.
- t = 4.0 ÷ 1.225 = 3.27, with 8 + 8 − 2 = 14 degrees of freedom.
- The critical t for 14 df at 5% (two-sided) is 2.145. 3.27 is larger, so p < 0.05. The exact p = 0.006.
- pooled SD
- √{[(n₁ − 1)SD₁² + (n₂ − 1)SD₂²] ÷ (n₁ + n₂ − 2)}
- SE
- pooled SD × √(1/n₁ + 1/n₂)
- df
- degrees of freedom, used to read the critical t from the table
t = 4.0 ÷ 1.225 = 3.27, df 14, p = 0.006 → 95% CI for the difference 1.37 to 6.63 mmHg
Worked example 2: paired t-test
Pain scores before: 4, 9, 6, 8, 5, 7, 3, 8. After 1 hour: 3, 7, 5, 6, 3, 6, 2, 7.
- Differences: 1, 2, 1, 2, 2, 1, 1, 1. Mean difference = 11 ÷ 8 = 1.375.
- SD of the differences = 0.518. SE = 0.518 ÷ √8 = 0.183.
- t = 1.375 ÷ 0.183 = 7.51, df = 7, p = 0.0001. 95% CI for the mean fall 0.94 to 1.81 points.
- The same 16 numbers put through an unpaired t-test give t = 1.35, p = 0.20.
Worked example 3: one-sample t-test
Made-up teaching data. Hb in 10 adolescent girls: 11.2, 10.8, 12.1, 11.5, 10.9, 11.8, 11.0, 12.3, 10.6, 11.4 g/dL. Does the mean differ from 12 g/dL?
t = (11.36 − 12) ÷ (0.568 ÷ √10) = −0.64 ÷ 0.180 = −3.56, df 9 → p = 0.006
Why the t-test matters
- It gives correct p-values for small samples. Methods for small samples were the problem W. S. Gosset worked on as a brewer at Guinness.
- It does not need the population SD, so it can be used at any sample size.
- It is the basis of the confidence interval for a mean and for a difference between two means.
- It is one of the most used statistical tests in medicine: comparing a sample with a standard, two treatments, or values before and after treatment.
Assumptions: a measurement outcome, roughly normal (for the paired test, the differences), independent patients, and for the unpaired test similar variances. If the variances clearly differ, use the t-test for unequal variances (Welch test).
The formula
Three or more groups: why not run a t-test for each pair?
Each t-test carries a 5% chance of a false positive. The more tests you run, the higher the chance that at least one of them is falsely significant.
The principle of ANOVA
Analysis of variance (ANOVA) compares means by comparing two kinds of variation. If all groups have the same true mean, the group means should differ from each other only about as much as individual patients differ within a group. F is the ratio of the two.
F = variance between groups (mean square between) ÷ variance within groups (mean square within)
- MS between
- Σ nᵢ (group mean − grand mean)² ÷ (k − 1)
- MS within
- Σ (each value − its group mean)² ÷ (N − k)
- k, N
- number of groups, total number of patients
If H₀ is true, both mean squares estimate the same patient-to-patient variance, so F is expected to be close to 1. Group means that truly differ make the mean square between groups larger. Only large values of F count against H₀, so the p-value is the area in the upper tail of the F distribution, although the alternative hypothesis (at least one mean differs) has no direction. F cannot be negative. With only two groups, F = t²: in worked example 1 below, t = 3.266 and F = t² = 10.67, with the same p = 0.006.
Worked example 4: one-way ANOVA
Duration of analgesia (hours). Drug alone: 5.2, 6.1, 5.8, 4.9, 6.4, 5.6 (mean 5.67). With adjuvant A: 7.9, 8.4, 7.2, 8.8, 7.6, 8.1 (mean 8.00). With adjuvant B: 8.6, 9.4, 8.1, 9.9, 8.8, 9.2 (mean 9.00). Grand mean 7.56.
| Source | Sum of squares | df | Mean square | F | p |
|---|---|---|---|---|---|
| Between groups | 35.11 | 2 | 17.56 | 50.7 | < 0.001 |
| Within groups | 5.19 | 15 | 0.346 | ||
| Total | 40.30 | 17 |
F = 17.56 ÷ 0.346 = 50.7. The critical F for (2, 15) df at 5% is 3.68, so at least one mean differs. The Levene test gives p = 0.91, so the variances are similar. Tukey's post hoc test shows that all three pairs differ, including adjuvant A against adjuvant B (difference 1.0 hour, p = 0.026).
Types of ANOVA
- One-way ANOVA: one grouping factor with three or more independent groups. Example: HbA1c with three drugs.
- Two-way (factorial) ANOVA: two factors at once, such as drug and sex. It tests the effect of each factor and their interaction, that is, whether the effect of the drug differs between men and women. In a factorial design every level of one factor is combined with every level of the other (2 drugs × 2 sexes = 4 groups). Main effects, interaction plots and worked two-factor designs are in the Practical Statistics chapter on two-way ANOVA.
- Repeated measures ANOVA: the same patients at three or more time points. It extends the paired t-test and needs sphericity (Mauchly test). If sphericity fails, apply the Greenhouse-Geisser or Huynh-Feldt correction. The analysis of repeated measurements is covered further in the Practical Statistics chapter on repeated measures.
- Mixed (two-way repeated measures) ANOVA: one factor between patients (treatment group) and one within patients (time).
- ANCOVA: ANOVA adjusted for a continuous covariate, such as the baseline value.
Assumptions of ANOVA
- Observations are independent (except the planned repetition in repeated measures designs).
- The outcome is roughly normal in each group.
- Variances are similar across groups (Levene test). If not, use a post hoc test for unequal variances, such as Games-Howell.
- For repeated measures, sphericity holds (Mauchly test).
- The outcome is a measurement. If normality fails, use the Kruskal-Wallis or Friedman test.
How to check normality and equal variances, and what to do when they fail, is in the Practical Statistics chapter on checking assumptions.
After a significant F: post hoc tests
A significant F shows that at least one mean differs, not which one. Post hoc tests compare pairs while controlling the false-positive rate: Tukey, Scheffé, Holm, and Bonferroni, which divides α by the number of comparisons (0.05 ÷ 3 = 0.017). Dunnett's test compares each treatment group with one control group, instead of comparing every pair. How to choose among these tests is covered in the Practical Statistics chapter on multiple comparisons.
Interactive
Move the group means apart or spread the patients out, and watch F
F compares how far the group means lie from the grand mean with how far patients lie from their own group mean. The same distance between the means can give a large or a small F, depending on the spread within the groups.
Try this: leave the group means 1 hour apart and raise the SD within groups from 1.0 to 2.0 hours. F falls from 6.00 to 1.50 and p rises above 0.05. Then move the means 2 hours apart. F is back at 6.00, because the distance between the means has doubled along with the spread.
Made-up teaching data: duration of analgesia in three groups of six patients. The grand mean stays at 7 hours.
Group means 6.0, 7.0 and 8.0 hours
How much patients differ inside a group
| Source | Sum of squares | df | Mean square | F | p |
|---|---|---|---|---|---|
| Between groups | 12.00 | 2 | 6.00 | 6.00 | 0.012 |
| Within groups | 15.00 | 15 | 1.00 |
F = 6.00 ÷ 1.00 = 6.00 · critical F (2, 15) at α 0.05 = 3.68 · F ≥ critical F
Non-parametric tests
Mann-Whitney, Wilcoxon, Kruskal-Wallis and Friedman: what does each one do?
Each rank test takes the place of a parametric test when the data are skewed, have outliers, or are ordinal.
| Test | Used instead of | Design | How it works |
|---|---|---|---|
| Mann-Whitney U (Wilcoxon rank-sum) | Unpaired t-test | Two independent groups | Ranks all values together and compares the rank sums of the two groups |
| Wilcoxon signed-rank | Paired t-test, one-sample t-test | Paired values, or one sample against a reference | Ranks the absolute differences, then adds the ranks of the positive differences |
| Sign test | Paired t-test | Paired values | Counts only how many differences are + and how many are − |
| Kruskal-Wallis H | One-way ANOVA | Three or more independent groups | Ranks all values together and compares the rank sums of the groups |
| Friedman | Repeated measures ANOVA | Same patients on three or more occasions | Ranks each patient's values across the occasions |
| Spearman rank correlation | Pearson correlation | Two variables in the same patients | Correlates the ranks instead of the values |
The same teaching data, analysed with rank tests
- Mann-Whitney U on length of stay: medians 3 and 5.5 days, U = 8.5, p = 0.015. The t-test on the same data gave p = 0.75.
- Wilcoxon signed-rank on the pain scores: all 8 differences are positive, p = 0.008.
- Kruskal-Wallis on the analgesia data: H = 13.69, df = 2, p = 0.001. One-way ANOVA on the same data gave p < 0.001.
Why non-parametric tests are useful
- They do not need the normality assumption, so they are less likely to give a wrong conclusion when that assumption fails.
- Outliers have little effect, because an extreme value is only the highest or lowest rank.
- They are the correct choice for ordinal data: pain grades, Likert items, Apgar scores.
- They can be used in small samples, where normality cannot be checked.
- When data are normal they lose little: the asymptotic relative efficiency of the Mann-Whitney test against the t-test is 0.955.
Limitations: ranks use less of the information in the data, confidence intervals are harder to calculate, and there are fewer methods for complex designs. The result describes ranks or medians, not a difference in means.
A significant Mann-Whitney test does not by itself show that the medians differ. The test compares the ranks of the two groups, so groups with similar medians but different spread can also give a small p-value. Hart made this point in a 2001 BMJ statistics note, whose title says that differences in spread can be important for this test. Report the median and IQR of each group, so that readers can see which one differs.
The formula
Comparing counts: the chi-square test
Chi-square compares the counts you observed with the counts you would expect if there were no difference between the groups.
χ² = Σ (O − E)² ÷ E
- O
- observed count in a cell
- E
- expected count = row total × column total ÷ grand total
- df
- (rows − 1) × (columns − 1)
Applications of the chi-square test
- Test of association (independence): are two categorical variables related? Smoking and wound infection.
- Comparison of proportions (homogeneity): cure rates in two or more treatment groups.
- Goodness of fit: do observed counts match a stated distribution? Admissions per weekday, or a genetic ratio.
- Trend: chi-square for trend across ordered groups, such as dose levels.
- Paired proportions: the related McNemar test, for the same patients classified twice.
Worked example 5: test of association in a 2 × 2 table
| Infected | Not infected | Total | |
|---|---|---|---|
| Skin preparation A | 8 | 92 | 100 |
| Skin preparation B | 18 | 82 | 100 |
| Total | 26 | 174 | 200 |
- Expected infections in each row = 100 × 26 ÷ 200 = 13. Expected non-infected = 100 × 174 ÷ 200 = 87.
- Infected cells: (8 − 13)² ÷ 13 = 1.923 and (18 − 13)² ÷ 13 = 1.923. Non-infected cells: (92 − 87)² ÷ 87 = 0.287 each.
- χ² = 1.923 + 1.923 + 0.287 + 0.287 = 4.42, with (2 − 1) × (2 − 1) = 1 df.
- The critical χ² for 1 df at 5% is 3.84. 4.42 is larger: p = 0.036. Infection rates were 8% and 18%.
Yates' continuity correction
Counts are whole numbers, but the chi-square distribution is continuous. For 2 × 2 tables, Yates proposed subtracting 0.5 from each |O − E| before squaring.
χ²(Yates) = Σ (|O − E| − 0.5)² ÷ E
In example 5: (5 − 0.5)² ÷ 13 = 1.558 for each infected cell and 0.233 for each non-infected cell, so χ² = 3.58 and p = 0.058. The correction moved p above 0.05. The Yates-corrected test is overly conservative, and many authors advise against it. State in the protocol which version you will use, before you see the data.
When to use Fisher's exact test
The chi-square p-value is an approximation. It needs adequate expected counts: no more than 20% of cells with an expected count below 5, and no cell below 1. When a table fails this rule, use Fisher's exact test, which calculates the exact probability and is valid at any sample size.
| Complication | None | Total | |
|---|---|---|---|
| New technique | 1 | 9 | 10 |
| Standard technique | 5 | 5 | 10 |
| Total | 6 | 14 | 20 |
Made-up teaching data. Expected complications = 10 × 6 ÷ 20 = 3 in each row, so 2 of the 4 cells (50%) are below 5. Fisher's exact test gives p = 0.14. The uncorrected chi-square would have given χ² = 3.81, p = 0.051.
Worked example 6: goodness of fit
Made-up teaching data: 140 admissions for acute myocardial infarction by weekday. Monday 32, Tuesday 21, Wednesday 18, Thursday 17, Friday 19, Saturday 16, Sunday 17. If admissions were spread evenly, each day would expect 140 ÷ 7 = 20.
- (O − E)² ÷ E: Monday (32 − 20)² ÷ 20 = 7.2. Tuesday 0.05, Wednesday 0.2, Thursday 0.45, Friday 0.05, Saturday 0.8, Sunday 0.45.
- χ² = 9.2, with 7 − 1 = 6 df. The critical χ² for 6 df at 5% is 12.59.
- p = 0.16. These 140 admissions give no evidence that admissions differ by weekday, despite the high Monday count.
Worked example 7: an R × C table
Made-up teaching data: anaemia in adolescent girls by family diet, 80 girls in each group.
| Diet | Anaemic | Not anaemic | Total |
|---|---|---|---|
| Vegetarian | 30 | 50 | 80 |
| Mixed | 22 | 58 | 80 |
| Non-vegetarian | 14 | 66 | 80 |
| Total | 66 | 174 | 240 |
- Expected anaemic in each row = 80 × 66 ÷ 240 = 22. Expected not anaemic = 80 × 174 ÷ 240 = 58. Every expected count is well above 5.
- (O − E)² ÷ E: vegetarian (30 − 22)² ÷ 22 = 2.9091 and (50 − 58)² ÷ 58 = 1.1034. Mixed 0 and 0. Non-vegetarian 2.9091 and 1.1034.
- χ² = 2 × (2.9091 + 1.1034) = 8.03, with (3 − 1) × (2 − 1) = 2 df. The critical χ² for 2 df at 5% is 5.99, and p = 0.018. Anaemia was 37.5%, 27.5% and 17.5%.
- A significant R × C test shows only that the proportions are not all equal. To find which groups differ, compare pairs with a correction for multiple testing. With Bonferroni (0.05 ÷ 3 = 0.017), only vegetarian against non-vegetarian differs (p = 0.005). Vegetarian against mixed gives p = 0.18, and mixed against non-vegetarian p = 0.13.
Interactive
Edit the 2 × 2 table and check the expected counts before choosing the test
The chi-square p-value needs adequate expected counts. In a 2 × 2 table, one expected count below 5 is already more than 20% of the cells, so Fisher's exact test is used instead.
Try this: press "Small table" and compare the chi-square p-value with Fisher's exact p-value. Then type your own counts and see which test the expected counts allow.
Teaching examples from this chapter, made-up numbers. Worked example 5: wound infection with skin preparation A (group 1) and B (group 2).
| Observed | Event | No event | Total |
|---|---|---|---|
| Group 1 | 100 | ||
| Group 2 | 100 | ||
| Total | 26 | 174 | 200 |
| Expected | Event | No event |
|---|---|---|
| Group 1 | 13.0 | 87.0 |
| Group 2 | 13.0 | 87.0 |
expected = row total × column total ÷ 200 · event rate: group 1 8.0% · group 2 18.0%
Every expected count is 5 or more, so the chi-square test can be used: χ² = 4.42, df 1, p = 0.036. The Yates-corrected test is overly conservative. State in the protocol which version you will use.
Comparing proportions
Which test compares proportions: z-test, McNemar test or binomial test?
Yes-or-no outcomes are compared in three designs: two independent groups, the same patients classified twice, or one group against a reference value. Each design has its own test.
Two independent groups: the z-test for two proportions
In worked example 5, infection was 8% with preparation A and 18% with preparation B. Under H₀ both groups share one infection rate, estimated from all 200 patients as 26 ÷ 200 = 0.13.
z = (p₂ − p₁) ÷ √[p(1 − p)(1/n₁ + 1/n₂)]
z = 0.10 ÷ √(0.13 × 0.87 × 0.02) = 0.10 ÷ 0.0476 = 2.103, p = 0.036
z² = 2.103² = 4.42, the same value as the chi-square statistic for this 2 × 2 table, with the same p-value, so the two are one test written in two ways. The z form can be used one-tailed, and it leads directly to a confidence interval for the difference: 10 percentage points, 95% CI 0.8 to 19.2 (calculated with the unpooled SE, 0.047).
The same patients classified twice: the McNemar test
McNemar χ² = (b − c)² ÷ (b + c)
χ² = (22 − 6)² ÷ 28 = 256 ÷ 28 = 9.14, df 1, p = 0.002
- b
- pairs positive on the first occasion only (anaemic before, not after): 22
- c
- pairs positive on the second occasion only (anaemic after, not before): 6
- a, d
- pairs with the same result both times: not used
With the continuity correction, χ² = (|b − c| − 1)² ÷ (b + c) = 8.04, p = 0.005. When b + c is small, use the exact binomial version, which here gives p = 0.004. Analysing the two occasions as independent groups (65% against 45% anaemic, chi-square p = 0.011) is wrong, because the same 80 women appear in both groups.
One group against a reference value: the binomial test
Made-up teaching data: 7 of 20 patients (35%) had a surgical site infection after a new procedure, against an expected rate of 15%. The expected number is only 20 × 0.15 = 3, too small for the normal approximation. The binomial test calculates the probability directly. If the true rate were 15%, the chance of 7 or more infections in 20 patients is 0.022, and the two-tailed p-value is 0.022. The z approximation gives z = 2.50 and a smaller p-value, 0.012, because it is inaccurate with an expected count of 3. The sign test for paired measurements is a binomial test with a reference proportion of 0.5.
| Design | Test | Teaching example in this chapter |
|---|---|---|
| Two independent groups | Chi-square, or the z-test for two proportions (z² = χ²). Fisher's exact test when expected counts are small | 8% against 18%: p = 0.036 |
| Same patients twice, or matched pairs | McNemar test, exact version when b + c is small | 65% against 45% anaemic: p = 0.002 |
| One group against a reference value | Binomial test, or the z-test when expected counts are large | 35% against 15%: p = 0.022 |
| Three or more groups | Chi-square for an R × C table | Worked example 7: p = 0.018 |
Decide
The complete test map
Find your outcome in the left column and your design across the top.
| Outcome | Two independent groups | Two paired sets | Three or more independent groups | Three or more repeated measurements | Relation between two variables |
|---|---|---|---|---|---|
| Measurement, roughly normal | Unpaired t-test | Paired t-test | One-way ANOVA | Repeated measures ANOVA | Pearson correlation, linear regression |
| Measurement skewed, or ordinal scores | Mann-Whitney U | Wilcoxon signed-rank | Kruskal-Wallis | Friedman | Spearman correlation |
| Categories (counts) | Chi-square, or Fisher's exact if expected counts are small | McNemar | Chi-square (r × c table) | Cochran Q | Chi-square, logistic regression |
| Time to an event | Log-rank test | — | Log-rank test | — | Cox regression |
One group against a reference value: one-sample t-test (normal measurement), Wilcoxon signed-rank test (skewed measurement), chi-square goodness of fit (counts in several categories), or the binomial test (one yes-or-no proportion). Correlation and regression are covered in the correlation and regression chapter, and survival methods in the risk and survival chapter.
Software
Which software can you use for the analysis?
Any of these packages will run the tests in this chapter, but none of them chooses the right test for you. Name the software and its version in your methods section.
| Software | Made or maintained by | Cost | Notes |
|---|---|---|---|
| IBM SPSS Statistics | IBM | Paid subscription | Menu-driven. Fisher's exact test is given automatically for 2 × 2 tables |
| R | A GNU project, free software | Free (GNU General Public License) | Code-based, with add-on packages from CRAN |
| jamovi | The jamovi project (community-developed) | Free and open source | Menus, runs on R and shows the R code behind each analysis |
| JASP | Supported by the Psychological Methods Group, University of Amsterdam | Free and open source | Menus |
| Stata | StataCorp LLC | Paid licence (6-month, annual or perpetual) | Stata 1.0 was released in 1985 |
| SAS | SAS | SAS OnDemand for Academics is free for learning | Point-and-click or programming |
| Epi Info | US Centers for Disease Control and Prevention (CDC) | Free, public domain | Data entry forms and epidemiological analysis. CDC stopped development and technical assistance in September 2025. The desktop version still runs on Windows 11 |
| OpenEpi | Dean, Sullivan and Soe | Free and open source | Runs in a web browser: 2 × 2 and R × C tables, sample size, chi-square for dose-response |
| PSPP | GNU project | Free (GNU GPL version 3 or later) | A free replacement for SPSS that reads SPSS syntax and data files |
| GraphPad Prism | GraphPad | Paid subscription, with academic pricing | Windows and Mac |
| MedCalc | MedCalc Software Ltd | Paid licence | More than 220 tests and graphs, including ROC curves and Bland-Altman plots |
| Microsoft Excel (Analysis ToolPak) | Microsoft | Add-in included with Excel | t-tests (paired, equal and unequal variances), ANOVA, correlation |
| Python with SciPy | Open-source community | Free (BSD licence) | Code-based |
Qualitative data (interview and focus group transcripts) needs different software: NVivo (Lumivero), ATLAS.ti (paid licences, free trial), MAXQDA (VERBI Software, Berlin), and Taguette (free and open source). Qualitative methods are covered in the qualitative research chapter.
Types of analysis
What types of data analysis are there?
Analyses are classified by purpose, by the number of variables, by the type of data, and by whether they were planned in advance.
| Basis | Types | What it means |
|---|---|---|
| Purpose | Descriptive | Summarises the data: counts, percentages, mean and SD, median and IQR |
| Purpose | Inferential | Draws conclusions about the population: confidence intervals and tests of significance |
| Number of variables | Univariate, bivariate, multivariable | One variable at a time. Two variables (t-test, chi-square, correlation). Several at once (multiple or logistic regression, ANCOVA) |
| Type of data | Quantitative, qualitative | Numbers analysed with statistics. Text from interviews and focus groups analysed for codes and themes |
| Assumptions | Parametric, non-parametric | Whether a distribution is assumed (this chapter) |
| Planning | Prespecified (confirmatory), exploratory (post hoc) | Only analyses planned in the protocol count as confirmatory |
| Patients included, in trials | Intention-to-treat (full analysis set), per-protocol | All randomised patients in their allocated groups, or only those who followed the protocol |
| Additional analyses | Interim, subgroup, sensitivity | Analyses before the planned end, within subgroups, or under different assumptions |
By specialty
Choosing a test by specialty
VAS pain scores in the same 40 patients before and 30 minutes after a nerve block.
The same patients measured twice are paired. Use the paired t-test, or the Wilcoxon signed-rank test if the differences are skewed.
HbA1c after 6 months in three groups, each on a different oral drug.
Three independent groups: one-way ANOVA, then a post hoc test. Three separate t-tests raise the false-positive rate.
Surgical site infection (yes or no) with two skin preparations, 60 patients in each group, with few infections expected.
Counts need the chi-square test. If expected counts fall below 5, use Fisher's exact test.
Length of hospital stay in bronchiolitis. Most children go home in 2 to 3 days and a few stay for weeks.
Skewed data: report median and IQR, and compare groups with the Mann-Whitney U test.
Apgar scores at 5 minutes in babies born after two methods of labour induction.
Apgar is an ordered score, not a measurement. Report the median and use the Mann-Whitney U test.
HAM-D scores in the same patients at baseline, 4 weeks and 8 weeks of treatment.
Repeated measurements in the same patients: repeated measures ANOVA, or the Friedman test if the scores are skewed.
Anaemia (present or absent) in adolescent girls from vegetarian, mixed-diet and non-vegetarian families.
A 2 × 3 table of counts: chi-square test with (2 − 1) × (3 − 1) = 2 degrees of freedom.
Serum enzyme levels in rats given saline or one of three doses of a test drug, six rats per group.
Four independent groups with a normal outcome: one-way ANOVA, then a post hoc test. With six rats per group, check for outliers first.
Common myths
Common mistaken beliefs
Myth: Three t-tests are fine for three groups.
Each test has a 5% false-positive rate. Three t-tests give a 14.3% chance of at least one false positive if the tests were independent. Use one-way ANOVA, then a post hoc test.
Myth: Non-parametric tests are weak, so avoid them.
When data are normal, the Mann-Whitney and Kruskal-Wallis tests have an asymptotic relative efficiency of 0.955 against the t-test and ANOVA. With heavily skewed data they have more power than the parametric tests.
Myth: The t-test is only for samples smaller than 30.
When the population SD is unknown, which is almost always, the t-test can be used at any sample size.
Myth: A significant ANOVA shows which groups differ.
It shows only that at least one mean differs. A post hoc test (Tukey, Bonferroni) shows which pairs differ.
Myth: Always apply Yates' correction to a 2 × 2 table.
The Yates-corrected test is overly conservative. Decide in the protocol, and use Fisher's exact test when expected counts are small.
Myth: Analysing paired data with an unpaired test loses only a little.
In worked example 2, the same 16 numbers gave p = 0.20 unpaired and p = 0.0001 paired.
Why this matters to you
Five consequences of choosing the wrong test
Protocol sent back
ICH E9 expects the principal features of the statistical analysis in the protocol. "Appropriate statistical tests will be applied" does not meet that.
A wrong conclusion in the thesis
Analysing paired data with an unpaired test, or skewed data with a t-test, can make a real effect appear as "no significant difference".
Questions you cannot answer at the viva
"Why this test?" needs an answer in terms of outcome type, number of groups, pairing and distribution. "The software gave it" is not an answer.
Misreading papers
Applying parametric tests to non-normal data is one of the most common statistical errors found in journals. Check also for repeated t-tests across several groups.
Harm to patients
A false-positive result can bring an ineffective treatment into practice. A missed real effect can keep a useful one out.
History
History of the common statistical tests
- 1900
Karl Pearson
Published the chi-square criterion for judging whether observed frequencies could have arisen by random sampling from an expected distribution: the chi-square goodness-of-fit test.
- 1908
"Student" (William Sealy Gosset)
A brewer at Arthur Guinness in Dublin from 1899, he published "The probable error of a mean" in Biometrika, the paper behind the t-test for small samples. Guinness allowed him to publish only under a pseudonym, so he signed it "Student".
- 1925
Ronald A. Fisher
Developed analysis of variance in the 1920s at Rothamsted Experimental Station. His first ANOVA paper, with W. A. Mackenzie, appeared in 1923. His 1925 book Statistical Methods for Research Workers included chapters on the analysis of variance.
- 1934
Frank Yates
Published "Contingency tables involving small numbers and the χ² test", which introduced the continuity correction for 2 × 2 tables.
- 1942
Jacob Wolfowitz
Used the term "non-parametric" for methods that do not assume a distribution.
- 1945
Frank Wilcoxon
Introduced tests based on ranks in "Individual comparisons by ranking methods".
- 1947
Henry Mann and Donald Whitney
Extended Wilcoxon's rank method to two groups of different sizes.
- 1952
William Kruskal and W. Allen Wallis
Published "Use of ranks in one-criterion variance analysis", the rank test for three or more groups.
Guidelines
Statistical analysis in guidelines
“When designing a clinical trial the principal features of the eventual statistical analysis of the data should be described in the statistical section of the protocol.” “Only results from analyses envisaged in the protocol (including amendments) can be regarded as confirmatory.”
“Transformations (e.g. square root, logarithm) should be specified in the protocol and a rationale provided, especially for the primary variable(s).”
Trial reports must give the “statistical methods used to compare groups for primary and secondary outcomes, including harms” and “methods for any additional analyses (eg, subgroup and sensitivity analyses), distinguishing prespecified from post hoc”.
“Describe statistical methods with enough detail to enable a knowledgeable reader with access to the original data to judge its appropriateness for the study and to verify the reported results.” Authors should also “specify the statistical software package(s) and versions used”.
Steps
Choosing and running a test, step by step
- Write the primary outcome and name its type: measurement, ordinal or categorical.
- Count the groups, and decide whether they are independent or paired.
- For a measurement, plan how you will check the distribution (histogram, Shapiro-Wilk test) and what you will do if it is skewed.
- Choose the test from the map, with a post hoc test or correction if there are three or more groups.
- Write the analysis plan in the protocol: tests, α, software and version.
- After data entry, check the assumptions, run the planned test, and report the effect size with its 95% confidence interval, not only the p-value.
- Label every analysis you did not plan as exploratory.
Reading a paper
Five checks on the statistical tests in any paper
- Does the test match the outcome: means for normal measurements, ranks for skewed or ordinal data, counts for categories?
- Were paired measurements (before and after, matched pairs) analysed with a paired test?
- With three or more groups, did the authors use ANOVA or Kruskal-Wallis with a named post hoc test, instead of repeated t-tests?
- For a chi-square test, were the expected counts large enough, or was Fisher's exact test used?
- Are the tests, the software and its version named, and are unplanned analyses labelled as exploratory?
Why is it called that?
Word surgery
“Normal”
Healthy, usual, nothing wrong.
A bell-shaped, symmetric distribution. "Normally distributed" says nothing about whether the patients are healthy.
“Independent”
Self-reliant, not controlled by anyone.
One value gives no information about another. Two groups are independent when they contain different, unmatched patients.
Same patients twice: paired test. Two groups: t-test or Mann-Whitney. Three or more groups: ANOVA or Kruskal-Wallis. Counts: chi-square, or Fisher's exact test when expected counts are below 5.
Parametric
Greek para- "beside" + metron "measure", through "parameter"
A parametric test assumes a distribution defined by parameters such as the mean and SD.
Variance
Latin variantia, from variare "to change"
The average squared deviation from the mean. ANOVA compares means by analysing variance.
Analysis
Greek analysis: "a breaking up, a loosening"
ANOVA breaks the total variation into a part between groups and a part within groups.
Normal
Latin norma: "carpenter's square", a rule or pattern
In statistics, the bell-shaped distribution that parametric tests assume.
Nominal
Latin nomen: "name"
Nominal data are categories that have names but no order, such as blood group.
Chi-square
the Greek letter χ (chi), squared
The test statistic is written χ². It adds up (O − E)² ÷ E over all the cells of the table.
Exam answers
Model answer outlines
Parametric and non-parametric tests: differences, assumptions, importance and uses
Ph.D. RM 2013, 2014, 2016, 2018 · MD Community Medicine 2002, 2025 · MD Pharmacology 2011, 2024 · MD Nuclear Medicine 2018Definitions: parametric tests assume the data follow a known distribution, usually normal, described by parameters such as the mean and SD. Non-parametric tests work on ranks or signs and assume no particular distribution.
| Feature | Parametric | Non-parametric |
|---|---|---|
| Distribution | Normal, similar variances | None assumed |
| Data | Measurements (interval, ratio) | Ordinal, skewed measurements, outliers |
| Uses | Actual values | Ranks or signs |
| Summary | Mean and SD | Median and IQR |
| Power | Higher when assumptions hold | Efficiency 0.955 against the t-test with normal data, higher with heavy skew |
| Examples | t-test, ANOVA, Pearson r | Mann-Whitney, Wilcoxon, Kruskal-Wallis, Friedman, Spearman rho |
Assumptions of parametric tests: normal distribution (of the differences, for paired tests), equal variances (Levene test), independent observations, a measurement on an interval or ratio scale, and random sampling.
Importance and uses of non-parametric tests: ordinal data (pain grade, Likert items, Apgar score), skewed measurements (length of stay), outliers, small samples, and any situation where normality fails. They remain valid without a distributional assumption and are little affected by extreme values.
Limitations: they use less of the information in the data, confidence intervals are harder to calculate, there are fewer methods for complex designs, and they have less power when parametric assumptions hold. A significant Mann-Whitney test is not proof that the medians differ, because differences in spread also affect it (Hart, BMJ 2001).
Matching pairs: unpaired t-test and Mann-Whitney U. Paired t-test and Wilcoxon signed-rank. One-way ANOVA and Kruskal-Wallis. Repeated measures ANOVA and Friedman. Pearson and Spearman correlation.
Student's t-test: principle, types, importance and applications
MD Pharmacology 2018, 2025, and undated · Ph.D. RM 2014, 2017 · MD Biochemistry 2014- Origin: W. S. Gosset, a brewer at Guinness, published it under the name “Student” (Biometrika, 1908), to handle small samples.
- Principle: t = difference between means ÷ standard error of the difference, compared with the t distribution at the correct degrees of freedom.
- Assumptions: a measurement outcome, roughly normal, independent observations, and similar variances for the unpaired test (otherwise the Welch test).
- Types: one-sample (df = n − 1), unpaired (df = n₁ + n₂ − 2), paired (df = number of pairs − 1).
- Steps: state H₀, calculate means, SDs and SE, calculate t and df, compare with the critical t or obtain p, and give the 95% CI.
- Applications: a sample mean against a standard (mean Hb against 12 g/dL), two treatments (fall in BP with two drugs), before and after treatment (pain scores), and each coefficient in linear regression.
- Importance: correct p-values in small samples, no need for the population SD, the basis of confidence intervals for means, and one of the most used tests in medicine.
- Limitations: only two groups (use ANOVA for more). In small samples outliers and skew distort the mean (use the Mann-Whitney or Wilcoxon test).
Paired versus unpaired t-test
MD Pharmacology 2024 · MD Nuclear Medicine 2021| Feature | Paired t-test | Unpaired t-test |
|---|---|---|
| Data | Same patients twice, or matched pairs | Two independent groups |
| Example | Pain before and after a block | Pain with drug A and with drug B in different patients |
| Formula | t = mean difference ÷ (SD of differences ÷ √n) | t = (mean₁ − mean₂) ÷ [pooled SD × √(1/n₁ + 1/n₂)] |
| df | Pairs − 1 | n₁ + n₂ − 2 |
| Between-patient variation | Removed | Included in the SE |
| Non-parametric version | Wilcoxon signed-rank | Mann-Whitney U |
Worked point: 8 patients whose pain fell by 1 or 2 points each. Paired t = 7.51, p = 0.0001. The same numbers analysed as unpaired: t = 1.35, p = 0.20. An unpaired test ignores the pairing and can miss a real effect.
Calculation of standard deviation and t-value in small experimental groups
MD Biochemistry 1994- SD = √[Σ(x − mean)² ÷ (n − 1)]. Divide by n − 1, not n, in a sample.
- Example: group A 12, 15, 9, 14, 11, 16, 10, 13 (mean 12.5). Group B 8, 10, 6, 11, 7, 9, 12, 5 (mean 8.5).
- Σ(x − mean)² = 42 in each group, so SD = √(42 ÷ 7) = 2.45.
- Pooled SD = 2.45. SE of the difference = 2.45 × √(1/8 + 1/8) = 1.225.
- t = (12.5 − 8.5) ÷ 1.225 = 3.27, df = 14. Critical t at 5% = 2.145, so p < 0.05 (exact p = 0.006).
ANOVA: principle, types (including factorial ANOVA), assumptions and importance
MD Biochemistry 2013, 2025 · Ph.D. RM 2025 · MD Pharmacology 2023 · MD Community Medicine 2013- Principle: H₀: all group means are equal. Total variation is divided into variation between groups and within groups. F = mean square between ÷ mean square within. A large F means the group means differ more than patients within a group. Under H₀ both mean squares estimate the same variance, so F is close to 1, and only large values of F (the upper tail) count against H₀.
- Types: one-way (one factor, three or more groups), two-way or factorial (two or more factors and their interaction), repeated measures (same patients over time), mixed (between- and within-patient factors), ANCOVA (adjusted for a covariate).
- Factorial ANOVA: every level of one factor is combined with every level of another (drug A or B × male or female = 4 groups). It gives the main effect of each factor and the interaction, which shows whether the drug effect differs by sex.
- Assumptions: independent observations, normal distribution within groups, equal variances (Levene test), sphericity for repeated measures (Mauchly test), measurement outcome.
- Post hoc tests: after a significant F, Tukey, Bonferroni, Scheffé, or Games-Howell when variances are unequal. Dunnett's test when each group is compared only with one control group.
- Importance: one test keeps the false-positive rate at 5% (three separate t-tests would allow up to 14.3%), several factors and interactions in one analysis, repeated measurements, adjustment for baseline values, and the standard analysis for designed experiments since Fisher developed it in the 1920s.
- Example: analgesia duration 5.67, 8.00 and 9.00 hours in three groups of six. F(2, 15) = 50.7, p < 0.001. Tukey test: all pairs differ.
How Student's t-test and ANOVA are used, with examples
Ph.D. RM 2019- Unpaired t-test, two groups: fall in systolic BP 12.5 mmHg with drug A and 8.5 mmHg with drug B, 8 patients each. t = 3.27, df = 14, p = 0.006.
- Paired t-test, same patients: pain before and after an analgesic in 8 patients. t = 7.51, df = 7, p = 0.0001.
- One-sample t-test: mean Hb 11.36 g/dL in 10 girls against 12 g/dL. t = −3.56, p = 0.006.
- One-way ANOVA, three groups: analgesia 5.67, 8.00 and 9.00 hours. F(2, 15) = 50.7, p < 0.001, followed by Tukey's test.
- Link between them: with only two groups, one-way ANOVA gives the same p-value as the unpaired t-test (F = t²).
ANOVA and the Kruskal-Wallis test
Ph.D. RM 2025| Feature | One-way ANOVA | Kruskal-Wallis |
|---|---|---|
| Type | Parametric | Non-parametric |
| Data | Normal measurements, similar variances | Skewed measurements or ordinal scores |
| Works on | Means and variances | Ranks of all values pooled |
| Statistic | F, df (k − 1, N − k) | H, compared with χ² on k − 1 df |
| After a significant result | Tukey, Bonferroni | Pairwise rank tests with a correction for multiple testing |
| Same teaching data | F = 50.7, p < 0.001 | H = 13.69, p = 0.001 |
Kruskal and Wallis published the test in 1952 as the rank-based version of one-way analysis of variance.
Chi-square test: principle, applications with two examples, Yates' correction and Fisher's exact test
MD Pharmacology 2019, 2024, 2025 · Ph.D. RM 2019, 2020 · MD Biochemistry 2025 · MD Nuclear Medicine 2024- Principle: χ² = Σ (O − E)² ÷ E, where E = row total × column total ÷ grand total. df = (r − 1)(c − 1). Introduced by Karl Pearson in 1900.
- Conditions: counts (not percentages or means), independent observations, no more than 20% of cells with expected count below 5 and none below 1.
- Applications: association between two categorical variables, comparison of proportions, goodness of fit, trend across ordered groups. McNemar test for paired proportions.
- Example 1 (association): wound infection 8/100 with preparation A and 18/100 with B. Expected 13 each. χ² = 4.42, df = 1, p = 0.036.
- Example 2 (goodness of fit): 140 MI admissions over 7 weekdays, expected 20 per day. χ² = 9.2, df = 6, p = 0.16. No evidence of a weekday difference.
- Yates' correction (2 × 2 tables): Σ (|O − E| − 0.5)² ÷ E. In example 1, χ² = 3.58, p = 0.058. It is overly conservative.
- Fisher's exact test: when expected counts are small. Complications 1/10 against 5/10: expected 3 per cell, Fisher p = 0.14.
- R × C tables: df = (r − 1)(c − 1). Anaemia 37.5%, 27.5% and 17.5% in three diet groups of 80: χ² = 8.03, df 2, p = 0.018. A significant result does not show which groups differ, so compare pairs with a correction (Bonferroni).
- z-test for two proportions: the same test as a 2 × 2 chi-square, since z² = χ². In example 1, z = 2.103 and z² = 4.42.
- Paired proportions (McNemar): χ² = (b − c)² ÷ (b + c), using only the pairs that changed. Anaemia before and after supplementation in 80 women, b = 22 and c = 6: χ² = 9.14, p = 0.002.
- Strength of association: odds ratio or phi for 2 × 2 tables, Cramér's V for larger tables.
Tests of significance: definition and the tests for different types of data
Ph.D. RM 2021Definition: a test of significance calculates the probability (p) of a difference at least as large as the one observed, if the null hypothesis were true. p below the chosen α (usually 0.05) is called statistically significant.
| Data | Two independent groups | Paired | Three or more groups | Relationship |
|---|---|---|---|---|
| Measurement, normal | Unpaired t-test (z-test for large samples with known SD) | Paired t-test | ANOVA | Pearson r, regression |
| Skewed or ordinal | Mann-Whitney U | Wilcoxon signed-rank | Kruskal-Wallis, Friedman | Spearman rho |
| Nominal (counts) | Chi-square, Fisher's exact, z-test for two proportions | McNemar | Chi-square | Chi-square, logistic regression |
| Time to event | Log-rank | Log-rank | Cox regression |
One group against a reference value: one-sample t-test (normal measurement), Wilcoxon signed-rank test (skewed measurement), binomial test or z-test (one proportion), chi-square goodness of fit (counts in several categories).
The choice rests on four facts: outcome type, number of groups, paired or independent, and distribution.
Common software for data analysis
Ph.D. RM 2015, 2018- Paid: IBM SPSS Statistics (IBM, menu-driven), Stata (StataCorp LLC), GraphPad Prism, MedCalc (MedCalc Software Ltd). SAS is free for learning through SAS OnDemand for Academics.
- Free: R (GNU General Public License, code-based), jamovi and JASP (open source, menu-driven), PSPP (free replacement for SPSS), OpenEpi (browser-based), Python with SciPy.
- Public health: Epi Info from the CDC, public domain. Development and technical assistance stopped in September 2025.
- Spreadsheet: Microsoft Excel with the Analysis ToolPak add-in (t-tests, ANOVA, correlation).
- Qualitative data: NVivo, ATLAS.ti, MAXQDA, Taguette (free).
- Choosing: licence cost, menus or code, what your institution supports, and whether the analysis can be saved as syntax or script and repeated.
- Reporting: name the software package and version in the methods section (ICMJE).
Opportunities and threats from easy access to statistical software
MD Community Medicine 2019Opportunities
- Analyses that took days by hand take seconds.
- Complex methods (multivariable regression, repeated measures, survival analysis) are available to every researcher.
- Free packages (R, jamovi, JASP, PSPP, OpenEpi) cost nothing.
- Saved syntax and scripts make an analysis repeatable, and code can be shared with the paper (CONSORT 2025, item 4).
Threats
- Clicking “OK” to get a p-value without understanding the method increases the risk of the wrong test or a misread result.
- Parametric tests applied to non-normal data, one of the most common errors in journals.
- Running many tests until one is significant, and reporting only that one.
- Relying on p-values without effect sizes and confidence intervals.
- Assumptions (normality, expected counts, sphericity) left unchecked.
- Software that stops being developed: CDC ended Epi Info development in September 2025.
Safeguards: a statistical analysis plan in the protocol (ICH E9), training, a statistician on the team, prespecified and exploratory analyses kept apart, and the software and version reported.
Qualitative research: study designs, tools, and software for analysing qualitative data
MD Community Medicine 2023- Designs: phenomenology (lived experience of a condition), grounded theory (building a theory from the data), ethnography (the culture and practices of a group), case study, and narrative research.
- Tools: in-depth interviews, focus group discussions, key informant interviews, participant observation, and document review, each with a topic guide.
- Analysis: transcription, coding, then thematic or content analysis until no new themes appear.
- Software: NVivo (Lumivero), ATLAS.ti, MAXQDA (VERBI Software), and Taguette (free and open source). The software organises and codes text. The researcher still decides the codes and themes.
Designs and sampling are covered in the qualitative research chapter.
Types of data analysis
Ph.D. RM 2019- By purpose: descriptive (summaries) and inferential (confidence intervals, tests of significance).
- By number of variables: univariate, bivariate, multivariable.
- By type of data: quantitative (statistics) and qualitative (thematic or content analysis).
- By assumptions: parametric and non-parametric.
- By planning: prespecified (confirmatory) and exploratory (post hoc). ICH E9: only analyses envisaged in the protocol can be regarded as confirmatory.
- In trials: intention-to-treat (full analysis set) and per-protocol.
- Additional analyses: interim, subgroup and sensitivity analyses.
Statistical analysis: planning and conducting the analysis of a study
DM Neurology 2015- Write the principal features of the analysis in the protocol before the study starts (ICH E9, section 5.1).
- Enter and code the data, check for errors, missing values and outliers.
- Describe the sample: n, mean and SD or median and IQR, counts and percentages.
- Check distributions and assumptions.
- Choose the test from outcome type, number of groups, pairing and distribution.
- Report the effect size with its 95% CI, and the exact p-value, at a two-sided α of 0.05.
- Adjust for confounders with multivariable models, and correct for multiple comparisons.
- In trials, state the analysis population (intention-to-treat or per-protocol) and how missing data were handled (CONSORT 2025, items 21b and 21c).
- Name the software and version, and label exploratory analyses.
Role of pilot testing in health research
Ph.D. RM 2024- Definition: a small-scale run of the planned study to examine whether its methods are feasible.
- Roles: test the feasibility of recruitment, randomisation, retention, assessment procedures, new methods and delivery of the intervention.
- Pretest questionnaires for clarity, wording and time taken, and train data collectors.
- Try out data entry and the analysis plan.
- Identify changes needed before the main study.
- Limits: a pilot study is not a hypothesis-testing study. It does not evaluate safety or efficacy, and its effect size is too imprecise to base the main study's sample size on. Feasibility results apply only within the pilot's own eligibility criteria.
- Reporting: the CONSORT 2010 extension to randomised pilot and feasibility trials (26-item checklist, 2016) asks for the prespecified criteria used to decide whether to proceed to the main trial.
Project proposal: reasons for and factors in non-compliance among patients with type 2 diabetes, including the plan for statistical analysis
Ph.D. RM 2023- Title, background and rationale: non-compliance with drugs, diet and follow-up worsens glycaemic control.
- Objectives: primary, the proportion of patients who are non-compliant. Secondary, the factors associated with non-compliance and the reasons patients give.
- Design: cross-sectional analytical study with a qualitative component (mixed methods). See the study designs chapter.
- Setting, population and criteria: adults with type 2 diabetes on treatment for at least 6 months at a diabetes clinic, excluding those unable to consent.
- Sample size: single-proportion formula with the expected proportion from earlier studies, the precision required and an allowance for non-response (sample size chapter).
- Sampling: systematic random sampling from the clinic register. Purposive sampling for the interviews.
- Variables: outcome, compliant or non-compliant. Factors: age, sex, education, income, duration of diabetes, number of tablets, insulin use, side effects, cost, knowledge, family support, distance to clinic.
- Tools: pretested structured questionnaire, pill count or a self-report adherence scale, and an interview guide for in-depth interviews.
- Plan for statistical analysis: descriptive statistics. Proportion non-compliant with 95% CI. Chi-square test (Fisher's exact if expected counts are small) for categorical factors. Unpaired t-test or Mann-Whitney U for continuous factors. Multivariable logistic regression for adjusted odds ratios with 95% CI. Two-sided α 0.05. Software and version named. Interview transcripts coded and analysed thematically.
- Ethics: ethics committee approval, written informed consent, confidentiality (research ethics chapter).
- Timeline, budget, and plan for dissemination.
Role of correlation and regression coefficients in statistical analysis
MD Community Medicine 2009- Correlation coefficient (r): measures the strength and direction of a linear relationship, from −1 to +1, with no units. Pearson r for normal measurements, Spearman rho for ranks.
- r² gives the proportion of variation in one variable that is explained by the other.
- Correlation does not show cause, and it does not show agreement between two methods of measurement.
- Regression coefficient (b): the change in the outcome for a one-unit change in the predictor, in the outcome's units (y = a + bx). It allows prediction.
- In multiple regression each coefficient gives that factor's effect adjusted for the others. In logistic regression the coefficients convert to odds ratios.
- Each coefficient is reported with its 95% CI and a test of whether it differs from zero.
Details are in the correlation and regression chapter.
The one thing to remember
Choose the test from four facts about your data, and write it in the protocol before you see the results: outcome type, number of groups, paired or independent, and distribution.
All worked examples use made-up numbers for teaching. Every statistic was computed in Python (SciPy 1.15) with two-sided tests at α 0.05.
Sources: Pearson K. Philos Mag 1900;50(302):157–75 (doi:10.1080/14786440009463897). Plackett RL. Int Stat Rev 1983 (doi:10.2307/1402731). Student. Biometrika 1908;6(1):1 (doi:10.1093/biomet/6.1.1). Cals J, Winkens B. Ned Tijdschr Geneeskd 2018;162:D3204 (PMID 30212007). Burgin C. J Child Neurol 1995;10(4):293 (doi:10.1177/088307389501000407). Breathnach CS. Ir J Psychol Med 1993;10(3):164–5 (doi:10.1017/S0790966700012659). Fisher RA, Mackenzie WA. J Agric Sci 1923;13(3):311–20 (doi:10.1017/S0021859600003592). Fisher RA. Statistical Methods for Research Workers. Edinburgh: Oliver and Boyd; 1925. Cochran WG. Fisher and the analysis of variance. 1980 (doi:10.1007/978-1-4612-6079-0_4). Parolini G. J Hist Biol 2015;48(2):301–35 (doi:10.1007/s10739-014-9394-z). Yates F. Suppl J R Stat Soc 1934;1(2):217–35 (doi:10.2307/2983604). Serra N et al. Epidemiol Biostat Public Health (doi:10.2427/13059). Haviland MG. Stat Med 1990;9(4):363–7 (doi:10.1002/sim.4780090403). Wolfowitz J. Ann Math Stat 1942;13(3):247–79 (doi:10.1214/aoms/1177731566). Wilcoxon F. Biometrics Bull 1945;1(6):80 (doi:10.2307/3001968). Mann HB, Whitney DR. Ann Math Stat 1947;18(1):50–60 (doi:10.1214/aoms/1177730491). Kruskal WH, Wallis WA. J Am Stat Assoc 1952;47(260):583–621 (doi:10.1080/01621459.1952.10483441). Nahm FS. Korean J Anesthesiol 2016;69(1):8–14 (doi:10.4097/kjae.2016.69.1.8). Kitchen CMR. Am J Ophthalmol 2009;147(4):571–2 (doi:10.1016/j.ajo.2008.06.031). Mishra P et al. Ann Card Anaesth 2019;22(4):407–11 (doi:10.4103/aca.ACA_94_19). Kim TK. Korean J Anesthesiol 2017;70(1):22–6 (doi:10.4097/kjae.2017.70.1.22). Kim HY. Restor Dent Endod 2017;42(2):152–5 (doi:10.5395/rde.2017.42.2.152). Hart A. BMJ 2001;323(7309):391–3 (doi:10.1136/bmj.323.7309.391, PMID 11509435). McHugh ML. Biochem Med 2013;23(2):143–9 (doi:10.11613/BM.2013.018). Leon AC, Davis LL, Kraemer HC. J Psychiatr Res 2011;45(5):626–9 (doi:10.1016/j.jpsychires.2010.10.008). Eldridge SM et al. BMJ 2016;355:i5239 (doi:10.1136/bmj.i5239). ICH E9 Statistical Principles for Clinical Trials, 5 February 1998. Hopewell S et al. CONSORT 2025 statement. BMJ 2025;389:e081123. ICMJE Recommendations, section on statistics. Software details from the developers' websites, accessed 14 September 2026.
Past papers
Every TNMGRMU question on choosing a test
Each one is answered in this chapter. Tap a sitting to open the university's own question paper.
- ×7Chi-square test
- ×3Student t test
- ×1Student T test and its applications
- ×1Principle and types of ANOVA (analysis of variance)
- ×1Discuss the importance and usage of non-parametric test in medical statistics
- ×1ANOVA and Kruskal-Wallis test
- ×1ANOVA and its types
- ×1Compare Parametric and Non parametric tests
- ×1Paired versus Unpaired t-test
- ×1Role of Pilot Testing in Health Research
- ×1Factorial Anova
- ×1Describe the study designs for Qualitative research? What are the tools that can be used for qualitative studies? List the various software available for Analysing Qualitative Data
- ×1As a researcher you are asked to identify the reasons for and factors influencing non-compliance among Diabetic (type 2). Write a detailed project proposal highlighting the research design including the plan for statistical analysis
- ×1Paired t test
- ×1What are the Test of Significance? Describe the different tests for different types of data
- ×1Explain the applications of Chi-Square test with two examples
- ×1Discuss the opportunities and threats subsequent to the easy availability of software for statistical analysis on research output
- ×1Explain how student’s T-test and ANOVA are used with appropriate examples
- ×1Types of Data analysis
- ×1Non-parametric tests
- ×1Parametric and nonparametric test
- ×1Describe the common software used for analysis of data
- ×1What is student t-Test and explain its importance
- ×1Non parametric tests / analysis
- ×1Statistical analysis
- ×1Software for data analysis
- ×1Students’ t-test
- ×1Parametric and non-parametric tests
- ×1Importance of ANOVA
- ×1Assumptions in ANOVA
- ×1Distinguish between Parametric and Non Parametric Tests. What are the assumptions of the Parametric Tests?
- ×1Value of non-parametric statistical tests
- ×1What is the role of correlation and regression coefficient in statistical analysis?
- ×1Non parametric tests. (bh) Use of pictograma in health education. ic} Communication barriers. fe) Multidrug resistant tuberculosis
- ×1Calculation of standard deviation and t-value of small experimental groups. ]
- ×1Student t test (@) Pebrile convulsions in children, (Sxto=59}
Further reading