Game 15 · Testing and estimation
What do all the trials say together?
Meta-analysis, forest and funnel plots, heterogeneity, and levels of evidence.
Read with: Systematic review and meta-analysis126 DNB questions
Play
Place the diamond, judge the spread, find the missing studies
Each round is a made-up meta-analysis. Guess where the pooled diamond lands and whether it crosses the line of no effect, guess I² from the spread of the intervals, read a funnel plot, order study designs by level of evidence and grade the certainty of a body of evidence. You have three lives.
Loading the first round…
The formula
Each study is weighted by its precision
weight w = 1 ÷ SE²
pooled log RR = Σ(w × log RR) ÷ Σw, with SE = 1 ÷ √Σw
- RR
- risk ratio in each study. Ratios are pooled on the log scale, then converted back
- SE
- standard error of the study's log RR
- Σ
- sum over all studies
In a forest plot each row is a study: the square is its estimate, with area proportional to its weight, and the line is its 95% confidence interval. The vertical line at RR 1 means no effect. The diamond is the pooled estimate, and its width is the pooled confidence interval. The pooled effect is statistically significant when the diamond does not cross the line of no effect. Read the outcome and the direction first, then the diamond, then I², then the largest squares.
Heterogeneity
Do the studies agree?
I² = 100% × (Q − df) ÷ Q
- Q
- Cochran's Q: weighted sum of squared differences between each study and the pooled result
- df
- number of studies − 1
- I²
- share of the variation between studies due to heterogeneity rather than chance. Negative values are set to 0%
Higgins and colleagues (2003) tentatively called I² values of 25%, 50% and 75% low, moderate and high. The Q test has low power with few studies, so a non-significant Q does not show agreement. When heterogeneity is present, look for its cause with subgroup analysis or meta-regression planned in the protocol, use a random-effects model, or do not pool.
| Feature | Fixed effect | Random effects (DerSimonian-Laird) |
|---|---|---|
| Assumes | Every study estimates the same true effect | True effects differ and are spread around an average |
| Weights | 1 ÷ SE² | 1 ÷ (SE² + τ²), so more equal |
| Confidence interval | Narrower | Wider when heterogeneity is present |
| Caution | Overconfident if the studies really differ | Gives small studies more weight, which matters if small studies are biased |
Publication bias
The funnel plot
A funnel plot shows each study's effect against its standard error. Large studies sit near the top close to the true value, and small studies scatter widely at the bottom, equally on both sides. If small studies that found no benefit stay unpublished, the bottom corner on the side of no effect is empty, and the pooled result of the published studies overstates the benefit.
Asymmetry does not prove publication bias: heterogeneity, poorer quality of small studies and chance can produce it. Egger's regression test measures asymmetry, but tests should not be used with fewer than 10 studies (Sterne et al., 2011).
Levels of evidence
The pyramid and GRADE
| Oxford CEBM 2011 level | Does this intervention help? |
|---|---|
| 1 | Systematic review of randomised trials or n-of-1 trials |
| 2 | Randomised trial or observational study with dramatic effect |
| 3 | Non-randomised controlled cohort or follow-up study |
| 4 | Case series, case-control studies or historically controlled studies |
| 5 | Mechanism-based reasoning |
GRADE rates the certainty of evidence for each outcome as high, moderate, low or very low. Randomised trials start high and observational studies start low. Certainty is rated down by one or two levels for each of risk of bias, inconsistency, indirectness, imprecision and publication bias. It is rated up for a large magnitude of effect (one or two levels), a dose-response gradient, or when all plausible confounding would reduce the demonstrated effect (one level each). Recommendations are graded strong or weak.
The pyramid, PRISMA 2020 and worked pooling are in the systematic review chapter.
Writing the answer
How to write this in a 10-mark DNB answer
- Define systematic review and meta-analysis in one line each, and say a meta-analysis is the optional pooling step of a systematic review.
- Draw a labelled forest plot of 5 made-up studies: squares sized by weight, CI lines, the line of no effect, the diamond, and I² below.
- Write the inverse-variance formula and explain fixed against random effects in a small table.
- Define Q and I², with the 25%, 50% and 75% benchmarks, and say what to do when I² is high.
- Draw a funnel plot, symmetric and with an empty corner, and list the causes of asymmetry other than publication bias.
- End with levels of evidence (OCEBM 2011 table) and GRADE factors, and name PRISMA 2020 for reporting.
The one thing to remember
The diamond is a weighted average, so it is only as good as the studies under it and only as meaningful as their agreement.
Past papers
DNB questions this game helps you answer
Questions from NBE theory papers for DNB, DrNB and Diploma courses, most often asked first. The tag after each question is the specialty and the year it was set.
- ×4Differences between a systematic review and meta-analysis.
- ×4Meta-analysis.
- ×4Meta analysis.
- ×3Forest Plot.
- ×2Forest plot.
- ×2Level of evidence.
- ×2What is meta-analysis?
- ×1Briefly discuss forest plot and its use in pediatric practice. [4]
- ×1Differences between review and systematic review. [4]
- ×1Interpretation of Forest plot. [2.5]
- ×1Confidence interval and Forest Plot.
- ×1Critically comment and compare research article, systematic review and meta-analysis.
- ×1Enlist the steps for conducting a meta-analysis. [5]
- ×1Forest plot in metanalysis.
- ×1Interpretation of a forest plot.
- ×1What is a forest plot and how to interpret it?
- ×1Level of evidence in research.
- ×1Levels of evidence in research studies.
- ×1What are the Levels of Evidence in biomedical research?
- ×1Meta analysis vs systematic review.
- ×1Network meta-analysis.
- ×1Systematic reviews are an important component of scientific writing and deliver a meticulous summary of all the available primary research in response to the research question : a) What are the steps of a systematic review? [3] b) Comment on: Are systematic review and meta-analysis different? [3] c) Draw a Forest Plot, with a brief description of its components.
- ×1Take any two examples of Meta analysis and describe how the results of a meta analysis serve to change clinical practice.
- ×1What is a Forest plot? Describe the characteristics of a Forest plot with a diagram.
- ×1a) Health care related quality of life assessment in gastrointestinal cancer patients. b) Level of evidence and grade of recommendations. c) Describe a ‘Forest plot’.
- ×1Define evidence based medicine and hierarchy of evidence. Discuss in brief the various types of research studies.
- ×1GRADE of recommendation.
- ×1What is evidence based medicine? What is personalized medicine in psychiatry? Discuss pharmacogenetics.
- ×1Describe principle of metaanalysis.
- ×1Discuss metaanalysis and systematic reviews.
- ×1Funnel plot.
- ×1Metaanalysis.
- ×1What is PRISMA for systematic review and metaanalysis?
- ×1a) Interpretation of Forest Plot. [3] b) Interpretation of Funnel plot. [3] c) Box-whisker Plot. [2] d) What is I2 in Forest Plot?
- ×1Define evidence based medicine (EBM). What are the principles of EBM? Briefly discuss the five steps of EBM model.
- ×1Describe the levels of evidences in research methodology.
- ×1Discuss evidence based medicine.
- ×1Evidence based medicine.
- ×1Level of Evidences (LOEs) in Medical Research.
- ×1Describe in brief meta-analysis.
- ×1Discuss differences between meta-analysis and systematic review. Briefly discuss main finding of MACH-NC analysis published recently (last 2-3 years).
- ×1Forest Plot and its interpretation in Research.
- ×1Forest plot and its interpretation.
- ×1How do you conduct a systematic review and meta-analysis?
- ×1Network meta analysis.
- ×1Classify various levels of evidence.
- ×1Define Evidence based medicine. Describe strengths and weakness of evidence based medicine.
- ×1Describe levels of evidence for clinical practice.
- ×1Describe various levels of evidences based on types of studies in research methodology.
- ×1GRADE process in evidence based medicine.
- ×1Levels of evidence in research.
- ×1What are the principles of evidence based medicine that will help you to identify good practices?
- ×1What do you understand by evidence-based medicine?
- ×1What is Evidence based medicine? Explain with examples relevance of Evidence based medicine in the planning of Disease Control Programs.
- ×1Compare and contrast systematic review and narrative review.
- ×1Compare systematic review and meta analysis.
- ×1Describe the steps of conducting a systematic review and meta-analysis.
- ×1Designing a meta-analysis study.
- ×1Difference between systematic review and review article.
- ×1Merits and demerits of Network meta-analysis.
- ×1Meta Analysis.
- ×1Meta-analysis of observational studies.
- ×1Systematic reveiw and meta-analysis.
- ×1Define levels of evidence in relation to evidence based medicine.
- ×1Levels of Evidence.
- ×1Levels of evidence in clinical research.
- ×1Pyramid of evidence-based medicine.
- ×1What are the various level of evidence for therapeutic studies from the Oxford Centre for Evidence Based Medicine (OCEBM)?
- ×1Define meta-analysis. Explain in brief the steps involved in carrying out a meta-analysis.
- ×1Difference between systemic review and meta-analysis.
- ×1Systematic Review.
- ×1What is a forest plot? Where is it used? How will you read and assess a study/studies with the help of it?
- ×1What is meta-analysis and what are the steps in doing it? Write short note on Forest plot.
- ×1Write short notes on: Forest plot.
- ×1About research methodology: b) Levels of evidence.
- ×1Levels of evidence.
- ×1What is evidence based medicine? What are the levels of evidence?
- ×1What is evidence-based medicine? How would you apply the 10 process to your clinical practice?
- ×1Define meta-analysis. List its advantages and disadvantages. Write differences between meta-analysis and systematic review.
- ×1Describe a Forest Plot.
- ×1Describe the Steps involved in conducting systematic review.
- ×1Describe the steps involved in conduct of meta-analysis.
- ×1What are study designs and discuss about meta-analysis?
- ×1a) Meta-analysis in medical research. b) Cross sectional study. c) Controlled trials- discuss its types and merits.
- ×1a) Systematic review. b) Confidence interval.
- ×1Describe the levels of evidences in medical research.
- ×1Give your comments on importance and limitations of evidence based medicine in clinical practice. Enumerate by giving suitable examples with appropriate reasoning the rationality of various approved Fixed Dose Combinations.
- ×1Give your comments on importance and limitations of evidence based medicine in clinical practice. Enumerate by giving suitable examples with appropriate reasoning. The rationality of various approved Fixed Dose Combinations.
- ×1Levels of evidences of various study (like case report, cases series, case control study, etc.).
- ×1What are the levels of evidence?
- ×1Enumerate the steps involved in conducting a meta-analysis.
- ×1Forest Plot in Meta-Analysis.
- ×1Meta-Analysis
- ×1What is a Forest plot? Mention the key features in its interpretation.
- ×1What is a “Forest Plot”? Describe with a diagram.
- ×1What is meant by a ‘systematic review’ and ‘meta-analysis’?
- ×1Define evidence-based orthopaedics and how it can be applied in clinical practice. List the hierarchy of evidence (various types of studies).
- ×1Draw the Pyramid of levels of Evidence and explain the salient points of each level of evidence in Medicine.
- ×1What are the ‘levels of evidence’?
- ×1What is Evidence Based Medicine?
- ×1List the components of a Forest plot.
- ×1What is hierarchy of evidence in medical literature?
- ×1Write short notes on: a) What are the components of ethical decision making and principles of medical ethics? b) What is informed consent? c) What are levels of evidence in research?
- ×1Forest plot
- ×1Forest plot in meta-analysis.
- ×1Meta-analysis: a) Definition. b) Conduct. c) Limitations.
- ×1What is a systematic review? Outline its role in clinical medicine.
- ×1What is the difference between a narrative review and systematic review? Which is better and why?
- ×1Define five levels of evidence with one example for each level that is being used in clinical practice.
- ×1Different levels of evidence.
- ×1Grade of recommendation.
- ×1Levels of evidence
- ×1Various levels of evidence, with one example from medical literature for each level of evidence.
- ×1What is Evidence-based Medicine?
- ×1Meta analysis in Research Methodology.
- ×1Steps involved in conducting systematic review.
- ×1Level of Evidence.
- ×1Necessity of evidence based medicine.
- ×1Define systemic meta analysis. Describe briefly its role and place in evidence based medicine
- ×1Enumerate and describe steps involved in conduct of meta-analysis.
- ×1Write a note on meta-analysis.
- ×1Mention the various levels of evidences and their significance.
- ×1What is evidence based medicine? Describe the hierarchy of evidence. Discuss the differences in various types of studies.
- ×1Define hierarchy of evidence. Define broad principles of each level of evidence.
- ×1What are the various steps for practicing evidence based medicine. What are the NCCN categories of consensus for treatment guidelines?
- ×1What is evidence based medicine (EBM)? Describe strength of evidence and quality of evidence. Discuss importance of EBM in relation to neonatal resuscitation.
Read next