You are a first-year Community Medicine resident studying tobacco use among college students. You write a 40-item questionnaire in two evenings and hand it out in six classrooms the following week.
When the forms come back, many students have ticked two boxes for “How often do you smoke?”, the income question is mostly blank, and “regularly” meant daily to some students and weekly to others. At your progress review, your guide asks one question.
“Did you pilot the questionnaire before you used it?”
Two types of data
Is it a category or a number?
Data are the facts you record about each participant. Each item you record is a variable. The first question about any variable is whether it places people in groups or measures an amount.
Qualitative (categorical) data place each person in a group. If the groups have no order, the data are nominal. If the groups have an order, the data are ordinal. Quantitative (numerical) data are counts or measurements.
Counts and measurements behave differently.
Discrete data are counts and take only whole numbers: seizures, children, admissions. Continuous data are measurements and can take any value in a range. The number of decimal places depends only on the instrument. Made-up values for teaching.
You can turn continuous data into categories, for example haemoglobin recorded as “anaemic” or “not anaemic”, but you lose information and cannot turn the categories back into values. Record the actual value and group it at analysis if you need to.
Scales of measurement
What can you do with the numbers you record?
The scale of a variable decides which summaries and tests have meaning. Each scale has the properties of the one before it, plus one more.
In 1946 the psychologist S. S. Stevens described four scales of measurement in Science: nominal (labels), ordinal (order), interval (equal gaps, arbitrary zero) and ratio (equal gaps and a true zero). He listed the statistics that suit each scale. He used the Centigrade and Fahrenheit temperature scales as examples of interval scales, where saying that one value is twice another has no meaning.
Scale
Clinical examples
Summary
Common tests
Nominal
Sex, blood group, diagnosis, cured yes/no
Counts, percentages, mode
Chi-square, Fisher's exact
Ordinal
Pain grade, NYHA class, cancer stage, Likert items
Median, percentiles
Mann–Whitney U, Wilcoxon, Spearman
Interval
Temperature in °C, calendar dates
Mean and SD (if roughly normal)
t test, ANOVA, Pearson
Ratio
Weight, height, haemoglobin, blood pressure, age
Mean and SD, ratios, coefficient of variation
t test, ANOVA, Pearson
Coding does not change the scale. If you enter blood groups as 1, 2, 3 and 4, they are still nominal, and their average means nothing. Stevens wrote that on a nominal scale the numerals are used only as labels. Which test to use in detail is covered in choosing a test.
Interactive
Classify each variable by its scale of measurement
Try this: pick the scale for each variable and read why. If your answer is wrong, try another before you move on.
Variable 1 of 14 · Variables from the examples in this chapter.
Body temperature in °C
Comparison
Collect it yourself, or use what someone else recorded?
Primary data are collected by you, for your study. Secondary data were collected by someone else, for another purpose, and you use them again.
The purpose for which data were recorded decides what they contain. A case file records what the treating doctor needed, not what your protocol needs. NFHS-5 (2019–21) collected its data with four questionnaires (household, woman's, man's and biomarker) using computer-assisted personal interviewing. For the survey team these were primary data. For a resident who analyses the NFHS data set, they are secondary data.
Feature
Primary data
Secondary data
Collected by
You, for your research question
Someone else, for another purpose
Examples
Your questionnaire, interview, observation, examination, tests
Case files, registers, census, NFHS, published tables
Fit to your question
Exact: you choose the variables and definitions
Partial: you use what was recorded
Time and cost
High
Low
Quality control
In your hands: training, pilot, daily checks
Out of your hands: check completeness and definitions
Main problems
Slow, expensive, needs field staff
Missing values, changed definitions, variables not recorded
Ethics
Informed consent from participants
Permission from the data custodian, confidentiality of records
Before you use records, check five things
Completeness: for each variable you need, count how many records contain it.
Who is in the records: hospital records contain only the patients who reached that hospital, who may be more severely ill than other people with the disease.
Definitions: check whether a definition, test or form changed during the period, and whether different hospitals recorded the same item in the same way.
Legibility: handwritten doses, dates and results can be misread when they are copied.
Linkage: check that one patient's visits, test results and outcome can be joined by a unique identifier, without duplicates.
Methods
Which methods can you use to collect data?
There are five main methods. Most theses use more than one, for example an interview for history, an examination for signs and records for test results.
Method
What it collects
Example
Strength
Limitation
Questionnaire (self-administered)
Knowledge, attitudes, practices, history
Tobacco use among college students
Cheap, large numbers, can be anonymous
Needs literate respondents, no chance to clarify, non-response
Works with low literacy, can clarify, fewer blanks
Costly, interviewer bias, needs training
Structured observation
What people do
Hand hygiene on a ward
Records behaviour directly
People change when watched, time-consuming
Records and documents
Past events and results
Case files, laboratory register, death certificates
Quick, cheap, covers the past
Incomplete, not recorded for research
Examination and investigations
Signs, measurements, test results
Blood pressure, BMI, haemoglobin, spirometry
Objective
Observer and instrument error, cost
The form on which you record data is the data collection tool or instrument: a questionnaire, an interview schedule, an observation checklist, or a case record form (proforma). Focus group discussions and in-depth interviews are covered in the qualitative research chapter. Checking that a tool is reliable and valid is covered in bias and validity.
Kinds of information
Are you asking for a fact, what people know, what they think, or what they do?
One questionnaire often asks for all of these. Each kind has its own best source and its own usual error, so plan the method for each item.
Kind of information
Example
Best source
Usual error
Facts and measurements
Date of birth, weight, blood pressure, HbA1c
Documents, examination, laboratory
Instrument and observer error, values given from memory or rounded
Past events
Vaccinations, admissions, episodes of diarrhoea
Cards and records, or recall over a short, stated period
Events forgotten or placed in the wrong period
Knowledge
"Which foods are rich in iron?"
Questions with a correct answer
Guessing, and answers copied from others when forms are filled in a group
Attitudes, beliefs and opinions
"Vaccines are safe for my child"
Rating scales such as Likert scales
Answers the respondent thinks are expected, agreeing with every statement
Practices
Hand washing, taking tablets
Observation, pill counts, pharmacy records
Self-report that differs from what people actually do
Symptoms and perceptions
Pain, fatigue, breathlessness
Rating scales, symptom diaries
The answer shows how the person perceives the symptom, which can change with mood and circumstances
A knowledge, attitude and practice (KAP) survey combines three of these kinds in one questionnaire. Analyse and report them separately. Good knowledge does not prove good practice, and only observation or records show what people actually do.
Interview
How much freedom does the interviewer have?
Interviews differ in how fixed the questions are. Fixed questions give answers that are easy to count. Open questions tell you more about reasons.
A structured interview uses an interview schedule. The interviewer reads each question exactly as written and records the answer. A semi-structured interview uses a topic guide with set questions and follow-up questions (probes). An unstructured interview starts from a few broad topics and follows what the respondent says.
Feature
Self-administered questionnaire
Interview schedule
Who reads the questions
The respondent
The interviewer
Who records the answers
The respondent
The interviewer
Literacy needed
Yes
No
Explaining a question
Not possible
Possible, with standard wording
Cost per respondent
Low
High: interviewer time and travel
Blank or incomplete forms
Common
Fewer
Main errors
Non-response, misread questions
Interviewer bias, answers the respondent thinks are expected
Sensitive topics
Can be anonymous
Needs privacy and trained interviewers
Structured observation
How do you record what people do, not what they say they do?
In structured observation a trained observer watches for defined events and records them on a checklist, in the same way for every person observed.
Made-up teaching example. Each row is one nurse and each circle is one opportunity for hand hygiene. What people report and what they do can differ. Boynton and Greenhalgh (2004) note that general practitioners' questionnaire answers about how they manage clinical conditions differ significantly from their actual practice.
How to carry it out
Define each behaviour so that two observers would record it the same way (an operational definition).
Decide what is recorded: every event (event sampling), or what happens in set time periods (time sampling).
Prepare the checklist, and pilot it on the ward or in the field.
Train observers together, and check that two observers watching the same events agree.
Decide whether the people observed know about the observation. Observation without their knowledge needs specific justification to the ethics committee.
Record at the time, not from memory afterwards.
In participant observation the observer takes part in the activity. In non-participant observation the observer only watches. People can change their behaviour when they know they are being studied. This is called the Hawthorne effect, after studies at the Western Electric factory at Hawthorne, near Chicago, between 1924 and 1933. A systematic review by McCambridge and colleagues (2014) concluded that research participation does change behaviour, but that its size and the conditions under which it happens are uncertain. Report how long observers were present and whether the people observed knew about the study.
Questionnaire
What makes a good questionnaire?
A questionnaire is a written set of questions that respondents answer themselves. A good one measures what it claims to measure, gives consistent answers, and is clear and short enough that people finish it.
Parts of a questionnaire
Title, and the name of the institution.
Introduction: purpose, time needed, confidentiality, voluntary participation and consent.
Instructions: how to mark answers, and which questions to skip if they do not apply.
A code number instead of a name, then sociodemographic questions.
Main sections in a logical order, general before specific, with sensitive questions after easier ones.
Space for comments, and a thank-you.
Types of question
Closed questions offer fixed options: yes or no, multiple choice, or a rating scale. They are quick to answer and analyse, but the researcher sets the range of answers. Open questions let respondents answer in their own words. Boynton and Greenhalgh advise that if you ask open questions, you plan in advance how you will analyse the answers.
Recording and coding the answers
State the unit and precision on the form: for example age in completed years, weight in kg to one decimal place, haemoglobin in g/dL to one decimal place.
Make the options complete: add "Other (specify)". If many pilot answers fall under "Other", read them and add the common ones as options.
A blank is not a zero: use separate codes for "not applicable", "test not done" and "no response", and enter 0 only for a real zero. A blank cell could mean any of these. The Practical Statistics chapter on missing data covers how missing values are handled in the analysis.
In 1932 Rensis Likert published a method in which respondents react to statements with one of five responses: strongly approve, approve, undecided, disapprove, strongly disapprove. Agreement scales of this kind are called Likert scales. Some respondents tend to agree with any statement, so word some items positively and some negatively.
Qualities of a good questionnaire
Linked to the objectives: every question maps to a variable in your analysis plan.
Short: ask only what you will analyse. Shorter postal questionnaires are more likely to be returned (Edwards et al., 2002).
Clear language in the respondent's own language, translated and back-translated where needed.
One idea per question, without leading, vague or technical words.
Response options that cover every possible answer and do not overlap, with “not applicable” or “other” where needed.
Readable layout. Boynton and Greenhalgh describe a 10-page questionnaire squeezed into four sides with smaller print. Colleagues found it hard to read and refused to complete it. Restored to 10 pages, it had a much higher response rate.
Reliable and valid, and tested in a pilot before use.
Ethical: consent, confidentiality, and no sensitive questions that the study does not need.
Pre-coded for data entry.
Use an existing validated questionnaire where one exists. A questionnaire is not shown to be valid or reliable because colleagues piloted it, earlier studies used it or a journal published it (Boynton and Greenhalgh, 2004).
Rating scales
How do you score a Likert scale?
Each Likert item is ordinal. Several items on one topic are added to give a scale score, after the negatively worded items have been reversed.
Teaching example: five made-up statements about an antenatal clinic, coded from 1 (strongly disagree) to 5 (strongly agree). Statements marked R are worded negatively, so agreeing with them means less satisfaction. Their codes are reversed before adding: on a 1 to 5 scale the new code is 6 minus the old code.
Scale score = 4 + (6 − 2) + 5 + (6 − 1) + 4 = 22, possible range 5 to 25
Code each response, for example from 1 to 5.
Reverse the negatively worded items: new code = (lowest code + highest code) − old code.
Add the items that measure the same topic to get the scale score.
Report a single item as percentages or a median, because each item is ordinal.
The scale score has many possible values and is often summarised with the mean and SD. Look at its distribution first.
Check that the items measure one topic before you add them, for example with Cronbach's alpha. Validity and reliability of tools are introduced in bias and validity, and the Practical Statistics chapter on reliability and validity of scales covers how a scale is checked in more depth.
The numbers printed on a scale can change the answers. Schwarz and Hippler (1995) asked people to rate six German politicians on a scale with the same end labels, numbered either from −5 to +5 or from 0 to 10. Ratings were more favourable on the −5 to +5 version: 29.3% of respondents gave a mean rating below the midpoint, compared with 40.2% on the 0 to 10 version. Keep the numbering of a validated scale unchanged.
Interactive
Choose answers to six statements and compare the scale score with the unreversed total
Try this: press "Agree with every statement". The total of the unreversed codes rises to 24, but the correct scale score falls to 18, the midpoint, because agreeing with the positive and the negative statements cancels out.
Teaching example: six made-up statements for adults with type 2 diabetes who have been advised to start insulin. Codes run from 1 (strongly disagree) to 5 (strongly agree). Statements marked R are worded negatively.
Statement
Answer (code)
Scored as
4
6 − 2 = 4
5
6 − 1 = 5
4
6 − 2 = 4
26scale score, after reversing statements 2, 4 and 6 (possible range 6 to 30)
The scale score of 26 is above the midpoint (a favourable attitude). Adding the codes without reversing gives 18, which is at the midpoint. That total mixes agreement with positive statements and agreement with negative ones, so it does not measure the attitude.
correct scale scoremidpoint
Cumulative scales
What if a yes to a harder item means yes to every easier item?
Some yes-or-no items can be put in order of difficulty, so that a person who can do a hard task can also do every easier one. Items that behave like this form a Guttman scale.
Teaching example with made-up answers. Items are ordered from easiest to hardest and people from the highest score to the lowest. In a perfect Guttman scale the yes answers form a staircase, so the score alone shows which items a person answered yes. Person 4 has 3 yes answers but not on the first 3 items, so 2 answers differ from the perfect pattern. Louis Guttman published this approach to scaling in 1944.
Coefficient of reproducibility = 1 − errors ÷ total answers
1 − 2 ÷ 40 = 0.95
Errors
answers that differ from the perfect staircase pattern for the same number of yes answers (the counting used in this example)
Total answers
people × items = 8 × 5 = 40
Feature
Likert scale
Guttman scale
Items
Statements rated for agreement, each counted equally
Yes-or-no items in order of difficulty
Score
Sum of the item codes, after reversing negatively worded items
Number of yes answers
What the score tells you
The overall level. Different answer patterns can give the same score
In a perfect scale, exactly which items were answered yes
Checked with
Internal consistency, for example Cronbach's alpha
Coefficient of reproducibility
Health example
Satisfaction with a clinic, attitude towards insulin
Mobility tasks, or vision tasks of increasing difficulty
Gothwal and colleagues (2009) tested the Distance Vision Scale, a self-assessment of vision whose questions become harder in order, in 108 people with cataract. It met the criteria for a Guttman scale, with a coefficient of reproducibility of 0.99.
Writing questions
Which wording errors spoil the answers?
Most questionnaire errors are in the wording. Each error below produces answers that you cannot interpret.
Boynton and Greenhalgh list words to avoid in questions: frequently, regularly, commonly, usually, many, some and hardly ever. Respondents read them differently. Ask about a defined period and a number instead.
Assumed behaviour: “How many cigarettes do you smoke a day?” asked of non-smokers. Add a filter question first.
No time frame: “Have you had diarrhoea?” Ask “in the last 2 weeks”.
Long recall: questions about events years ago give inaccurate answers.
Hypothetical questions: “Would you use a new clinic?” predicts actual use poorly.
Sensitive questions too early: income or sexual behaviour on the first page increases refusals.
Interactive
Name the wording error in each question, then compare the rewrite
Try this: read the question, name its wording error, then compare it with the rewritten version.
Question 1 of 12 · Made-up questionnaire items, each with one of the six wording errors shown above.
"How satisfied are you with the doctors and the nurses in this hospital?"
Pilot testing
Why test the questionnaire on a few people first?
A pilot shows whether people understand the questions and can complete the form, before you use it on your whole sample.
Thabane and colleagues (2010) list what a pilot checks about data collection tools: whether participants give no answer, several answers, qualified answers or unexpected answers, and how long the forms take to fill in. Pilot participants should resemble the study population and meet the same eligibility criteria.
Pretesting checks the instrument. A pilot study is a small version of the whole study and checks recruitment, consent, data collection, time, cost and data entry. Thabane and colleagues group the reasons for a pilot under process, resources, management and scientific questions.
A pilot study cannot give a trustworthy effect size for your sample size calculation. Leon, Davis and Kraemer (2011) explain that estimates from small samples are too imprecise for this. See sample size. Pilot data can be combined with the main study only if the methods did not change and the combination was planned in the protocol (Thabane et al., 2010).
Measurement and recording errors
What errors occur while values are measured, recorded and entered?
A larger sample reduces chance error. It does not reduce errors in how each value is measured, written down or entered in the computer.
Made-up teaching example: the last digit of 200 systolic pressures recorded with a manual device. With no preference each digit would appear about 20 times (10%, dashed line). Here 56 readings (28.0%) end in 0, and even digits are more common than odd ones. Readings rounded to 130 or 140 change who is classified above a cut-off.
This is digit preference, and it occurs in real clinics. Kottke and colleagues (2022) studied 11 primary care clinics where staff were trained to record blood pressure to the nearest even digit, so 20% of readings would be expected to end in 0. With manual measurement, 32.8% of systolic readings ended in 0. After automated monitors were introduced, 12.4% did, against an expected 10%, and the mean systolic pressure estimated with a statistical model was 5.09 mm Hg higher.
Error
What happens
How to prevent or detect it
Instrument error
An uncalibrated scale or analyser reads consistently high or low
Calibrate against a reference before and during the study, and record which device was used
Observer error
Two observers measure or grade the same thing differently
A written procedure, joint training, and a check of agreement between observers
Digit preference
Values are rounded to 0 or 5
Automated devices, recording the exact reading, and a count of last digits during the study
Unclear instructions
Data collectors interpret a question or definition in different ways
A manual of procedures with operational definitions
Transcription error
A value is copied wrongly, for example haemoglobin 11.8 entered as 18.1
Forms checked the same day, range and consistency checks
Data entry and coding error
A wrong code, or a missing value entered as 0
Double entry by two people with comparison, separate codes for missing values
Repeated testing
Participants remember a test, or observers become more skilled, between measurements
A control group measured at the same times, with the same observers and instruments throughout
What are the strengths and weaknesses of a survey?
A survey collects the same information, in the same way, from a defined sample, usually by questionnaire or interview.
Made-up teaching example. A low response rate matters when the people who do not respond differ from those who do. Handing out more forms does not correct this.
Response rate = completed forms ÷ eligible people approached × 100
240 ÷ 400 × 100 = 60%
Completed forms
forms returned with the main questions answered
Eligible people approached
everyone eligible who was given the form or asked for an interview
A systematic review of 292 randomised trials of postal questionnaires (Edwards et al., 2002) found that the odds of response more than doubled with a monetary incentive (odds ratio 2.02) and were higher with short questionnaires (1.86). Questionnaires containing sensitive questions were less likely to be returned (0.92).
What can you do about non-response?
Unit non-response means a person does not take part at all. Item non-response means a person takes part but leaves some questions unanswered. Both have two effects. Fewer answers give wider confidence intervals, which you can allow for by adding the expected non-response to the sample size. If the people who do not answer differ from those who do, the estimate is also biased, and a larger sample does not remove that bias.
Hansen and Hurwitz (1946) proposed taking a random subsample of the non-responders and collecting their answers with more effort, for example by personal interview after an unreturned postal questionnaire. The subsample then stands for all non-responders.
Estimate = (240 ÷ 400) × 15% + (160 ÷ 400) × 30%
= 9.0 + 12.0 = 21.0%
Teaching example
400 students approached, 240 returned the form (15% use tobacco), 160 did not
30%
tobacco use in a random subsample of 40 of the 160 non-responders, interviewed at home (12 of 40)
15%
the estimate from returned forms alone, 6 percentage points too low in this example
Advantages
Disadvantages
Large numbers, quickly and at low cost
Non-response bias
Standard questions allow comparison between groups, places and years
Self-report can differ from actual behaviour
With random sampling, results apply to the population
Recall errors
Can cover wide areas (postal, telephone, online)
Fixed questions give little depth
Self-completed forms can be anonymous
Misread questions cannot be corrected (self-administered)
Can study many variables at once
Interviewer bias (interview surveys)
Repeated surveys show trends
A single survey cannot show whether exposure came before disease
Interactive
Change the response rate and watch the survey estimate move away from the true value
Try this: set tobacco use among students who did not return the form to 15%, the same as those who did, and watch the error disappear. Then put it back to 30%, choose 4,000 forms, and watch the interval become narrower while the error stays.
Teaching example (made-up numbers): a questionnaire on tobacco use handed to college students. You can see the answers only from students who return the form.
Lower response → the non-responders count for more
This is what the survey reports
Unknown in a real survey
15%survey estimate, from 240 returned forms
true value: 60% × 15% + 40% × 30% = 21% · 95% CI 10.5% to 19.5%
The survey is 6 percentage points too low. The 95% confidence interval reflects chance only. It runs from 10.5% to 19.5% and does not include the true 21%. Handing out more forms makes the interval narrower, but the error from non-response stays the same.
survey estimate with its 95% confidence intervaltrue value
In the top panel, the width of each block is the share of students, and the red part is the share using tobacco. The dashed block is the students whose answers you never see.
Decide
Which data collection method fits your question?
Answer a few questions. Each answer links to the section that explains it.
What do you need to find out?
Which collection method: all possible answers
Self-administered questionnaire: Closed questions with fixed options, tested in a pilot before use. Quick and cheap for large literate groups.
Structured interview (interview schedule): The interviewer reads the same questions in the same words and order and records the answers.
Semi-structured interview: A topic guide with set questions and follow-up questions. Gives numbers and the reasons behind them.
Unstructured or in-depth interview: A few broad topics, and the respondent leads. This is qualitative research.
Structured observation: A trained observer records defined behaviours on a checklist. Plan for people changing behaviour because they are watched.
Clinical examination and investigations: Use calibrated instruments, a written procedure and trained examiners, the same for every participant.
Secondary data from records: Quick and cheap. Write a data extraction form and report how many records had missing values.
Collect primary data: Records made for another purpose cannot answer your question. Design your own form and collect the data yourself.
By specialty
Data collection problems by specialty
A retrospective study of HbA1c control uses OPD files. One patient in three has no HbA1c recorded in the study year.
Secondary data contain only what someone chose to record. Report how many records were incomplete.
Surgical site infection is judged from the operating surgeon's notes, written for clinical care, not for research.
Define infection in advance and have a trained assessor examine every wound the same way.
Intraoperative blood pressure is copied by hand from the monitor every 15 minutes, and short falls in pressure between readings are not recorded.
Decide the recording interval before the study, or use data stored by the monitor.
Immunisation status is taken from the mother's memory when the vaccination card is not available.
State the source for each child (card or recall) and report the results separately.
A survey asks antenatal women about domestic violence in the clinic corridor, with relatives nearby.
Sensitive questions need privacy, trained interviewers and a plan for referral.
A depression rating scale developed in English is used in Tamil after one person translated it.
Translate, back-translate and pilot a scale before use. Check its validity in your population.
A household survey of tobacco use in a block, with interviewers entering answers on tablets.
Train interviewers on one written manual, pilot the questionnaire, and check a sample of forms every day.
Culture results are taken from the laboratory register, where some specimens have no ward or date written.
Registers are secondary data. Write inclusion rules for incomplete entries before extraction.
Common myths
Common mistaken beliefs
✗Myth: Data means numbers.
✓Categories are data too. Blood group, sex and NYHA class are qualitative data.
✗Myth: Entering blood groups as 1, 2, 3 and 4 makes them quantitative.
✓Numbers used as labels are still nominal. Their average means nothing.
✗Myth: 40 °C is twice as hot as 20 °C.
✓Celsius is an interval scale with an arbitrary zero. Ratios have meaning only on a ratio scale such as weight or age.
✗Myth: A questionnaire published in a journal is valid for my study.
✓An instrument developed in another country, time or culture may not be valid in your population (Boynton and Greenhalgh, 2004). Check it, and pilot it.
✗Myth: Hospital records are ready-made thesis data.
✓Records are written for patient care. Expect missing values and changing definitions, and check completeness before you plan the analysis.
✗Myth: The pilot study will give me the effect size for my sample size.
✓Pilot samples are too small to estimate an effect precisely. Use a pilot to test feasibility and procedures (Leon et al., 2011).
Why this matters to you
Five consequences of poor data collection
Answers you cannot use
Vague or double-barrelled questions give answers you cannot code, and you find out only after the data are collected.
Protocol sent back
Committees ask for the questionnaire, where it came from, whether it was validated and how it will be piloted.
The wrong statistics
Averaging ordinal scores or treating codes as numbers leads to summaries and tests that do not fit the data.
Viva questions you cannot answer
"How did you validate the questionnaire?" and "What was your response rate?" need an answer with numbers.
Misreading papers
A survey with a low response rate may describe only the people who chose to answer.
History
History of data collection methods
1662
John Graunt, London
Published Natural and Political Observations Made upon the Bills of Mortality, an analysis of London's Bills of Mortality. It is an early use of data recorded by others to answer questions about population and disease.
1924–1933
Western Electric, Hawthorne factory
Studies of workers at the Hawthorne factory near Chicago. The term "Hawthorne effect" was first used in a methodology textbook in 1953, and is now used for a change in behaviour caused by being studied.
1932
Rensis Likert
Published "A technique for the measurement of attitudes" (Archives of Psychology, no. 140), with statements answered on five responses from strongly approve to strongly disapprove.
1944
Louis Guttman
Published "A basis for scaling qualitative data" in the American Sociological Review. Scales of yes-or-no items in order of difficulty are named after him.
1946
S. S. Stevens
Described nominal, ordinal, interval and ratio scales in Science, with the statistics permissible on each.
1946
Morris Hansen and William Hurwitz
Proposed following up a random subsample of non-responders in sample surveys, in the Journal of the American Statistical Association.
2002
Edwards and colleagues
Systematic review of 292 randomised trials, with 258,315 participants, of methods to increase response to postal questionnaires.
2004
Petra Boynton and Trisha Greenhalgh
Published a practical BMJ guide to selecting, designing and developing a questionnaire.
2016
Eldridge and colleagues
Published the CONSORT extension to randomised pilot and feasibility trials, a 26-item checklist.
2021
Sharma and colleagues
Published CROSS, a consensus checklist for reporting survey studies, with 40 items in 19 sections.
Guidelines
Data collection in guidelines
STROBE (2007), item 8
Data sources/measurement: “For each variable of interest give sources of data and details of methods of assessment (measurement). Describe comparability of assessment methods if there is more than one group.”
CROSS (2021), items 5c and 11c
Survey reports should “provide information on pretesting of the questionnaire, if performed” (5c) and “report response rate, present the definition of response rate or the formula used to calculate response rate” (11c).
ICH E6(R2) (9 November 2016), glossary 1.11 and 1.51
A case report form is “a printed, optical, or electronic document designed to record all of the protocol required information to be reported to the sponsor on each trial subject.” Source data are “all information in original records and certified copies of original records of clinical findings, observations, or other activities in a clinical trial necessary for the reconstruction and evaluation of the trial.” ICH E6(R3), adopted on 6 January 2025, calls CRFs data acquisition tools.
Steps
Data collection procedure, step by step
List every variable from your objectives, with its definition, scale and unit.
Choose the method and source for each variable: questionnaire, interview, observation, examination or records.
Select the instrument. Use a validated tool where one exists, and translate and back-translate it if needed.
Prepare the case record form (proforma) with a code for each response.
Obtain ethics committee approval, and prepare the information sheet and consent form.
Write a manual of procedures, train the data collectors, and calibrate instruments.
Pilot the instrument and the procedure, then revise.
Collect the data: take consent, then administer, interview, observe, examine or extract, recording at the time.
Check forms the same day for missing or impossible values, and keep a log of refusals and dropouts.
Enter the data with checks (for example double entry), clean and code it, and store it securely with names kept separately from codes.
Reading a paper
Five checks on how a study collected its data
For each main variable, do they say what was measured, how and by whom?
Was the questionnaire or scale validated, and is it provided or referenced?
Was it pretested or piloted, and in whom?
What was the response rate, and how was it calculated?
Were the same methods used in every group, and were assessors unaware of group where that mattered?
Why is it called that?
Word surgery
“Data”
In everyday speech
Computer files, a mobile data pack, information in general.
In research
Recorded observations or measurements on each participant, collected by a stated method.
“Survey”
In everyday speech
Looking over something, or measuring land.
In research
Collecting the same information in the same way from a defined sample, usually by questionnaire or interview.
“Pilot”
In everyday speech
The person who flies an aircraft.
In research
A small trial run of the instrument or the study procedures before the main study.
Memory trick
NOIR: Nominal, Ordinal, Interval, Ratio. Each scale adds one property to the one before it: order, then equal gaps, then a true zero.
Data
Latin datum: "(thing) given", from dare, "to give"
Plural of datum. The facts recorded about each participant.
Questionnaire
French questionnaire: "list of formal questions", in English from 1901
A written list of questions that respondents answer themselves.
Survey
Medieval Latin supervidere: "oversee, inspect"
Systematic collection of information from a defined group. The sense "poll" is recorded in English from 1927.
Nominal
Latin nomen: "name"
A scale whose values are names or labels only.
Ordinal
Late Latin ordinalis: "showing order", from ordo, "row, series"
A scale whose values have an order, with unequal gaps between them.
Interval
Latin intervallum: "space between palisades or ramparts"
A scale with equal gaps between values but no true zero.
Ratio
Latin ratio: "a reckoning, calculation"
A scale with equal gaps and a true zero, so "twice as heavy" has meaning.
Pilot
French pillote: "one who steers a ship". As an adjective, "serving as a prototype" from 1928
A small trial run of a study or an instrument before the main study.
Exam answers
Model answer outlines
Data collection methods, with suitable examples (and construction of a research question)
Ph.D. RM 2014, 2015, 2018, 2023, 2025
Definition: data collection is the systematic gathering of information on the study variables, by a stated method, from the study participants or from existing sources.
Questionnaire (self-administered): knowledge, attitudes and practices. Example: tobacco use among college students.
Interview: structured (interview schedule), semi-structured (topic guide) or unstructured. Example: infant feeding practices from mothers with low literacy.
Observation: structured or unstructured, participant or non-participant. Example: hand-hygiene adherence on a ward.
Clinical examination and investigations: example: blood pressure, BMI, haemoglobin in an anaemia survey.
Records and documents (secondary data): case files, registers, census, NFHS. Example: audit of caesarean rates from labour ward registers.
Qualitative methods: focus group discussions, in-depth interviews.
Choice depends on: the objective and variable, respondents' literacy, sensitivity of the topic, time, cost, trained staff and ethics.
Kind of information: facts and measurements (documents, examination), past events (records, short recall periods), knowledge (questions with a correct answer), attitudes (rating scales), practices (observation, records). A KAP survey combines knowledge, attitudes and practices.
Quality measures: validated tools, pilot testing, trained data collectors, a manual of procedures, daily checks, and reporting of response rate.
(a) Construction of a research question: start from a problem seen in practice, review the literature, narrow the question, then write it in PICO form (Population, Intervention or exposure, Comparison, Outcome) and check it against FINER (Feasible, Interesting, Novel, Ethical, Relevant). See research question.
List of methods of data collection. Describe the structured observation method
Ph.D. RM 2023
Methods: questionnaire, interview, observation, examination and investigations, records and documents, focus groups.
Structured observation: a trained observer records predefined behaviours or events on a standard checklist, in the same way for every person or setting.
Define each behaviour operationally.
Choose event sampling (every event) or time sampling (set intervals).
Prepare and pilot the checklist or coding sheet.
Train observers, and check agreement between two observers on the same events.
Decide overt or covert observation, participant or non-participant, with ethics approval.
Record at the time of observation.
Example: hand hygiene before patient contact, recorded per opportunity.
Merits: records actual behaviour rather than reported behaviour, and does not need literate respondents.
Demerits: the Hawthorne effect (people change behaviour when studied), observer bias, time and cost, and it cannot record attitudes or past events.
Classification of data with suitable examples (how data are collected and classified; concept of sampling; sample size)
Ph.D. RM 2019, 2023, 2025
Basis
Classes
Examples
Nature
Qualitative: nominal, ordinal
Blood group; NYHA class
Nature
Quantitative: discrete, continuous
Number of seizures; haemoglobin
Scale (Stevens, 1946)
Nominal, ordinal, interval, ratio
Sex; pain grade; °C; weight
Source
Primary, secondary
Your interview; hospital records
Kind of information
Fact, knowledge, attitude or opinion, practice
Weight, knowing which foods contain iron, belief that vaccines are safe, hand washing
Time
Cross-sectional, time series
One survey; monthly admissions
Place
Geographical
District-wise prevalence
How data are collected: questionnaire, interview, observation, examination and investigations, records (see the frame above).
Sampling: studying a part (sample) chosen from the population through a sampling frame. Probability methods (simple random, systematic, stratified, cluster, multistage) give each unit a known chance. Non-probability methods (convenience, consecutive, purposive, quota, snowball) do not. See sampling.
Sample size: depends on the significance level, power, the smallest difference that matters, the SD or expected proportion, the precision needed, and expected dropouts. For a prevalence: n = Z² × p(1 − p) ÷ d². With p 30% and d ±5 points: 1.96² × 0.30 × 0.70 ÷ 0.05² = 322.7, so 323. See sample size.
Primary data and secondary data: distinguish between them
Ph.D. RM 2023, 2025
Feature
Primary data
Secondary data
Definition
Collected first-hand by the investigator for the present study
Before using secondary data, check: who collected them and why, definitions used and whether they changed, completeness, who is included (hospital records hold only patients who reached that hospital), legibility, linkage of records by a unique identifier, period covered and permission to use them.
Designing a questionnaire: components, qualities, and testing its reliability and validity (data collection tools in quantitative designs)
MD Community Medicine 2025 · Ph.D. RM 2016, 2023
Tools in quantitative designs: questionnaire, interview schedule, observation checklist, rating scales (Likert), case record form, clinical and laboratory measurements.
Main sections in logical order, sensitive items later.
Response formats: closed (dichotomous, multiple choice, Likert) and open.
Closing and thank-you. Pre-coding for data entry.
Coding: units and precision stated, "Other (specify)", separate codes for not applicable, not done and no response.
Qualities: linked to objectives, short, simple language, one idea per question, no leading or vague words, options that cover all answers without overlap, clear layout, translated and back-translated, ethical, piloted.
Scales: a Likert scale adds the item codes after reversing negatively worded items. A Guttman scale orders yes-or-no items by difficulty, and its fit is checked with the coefficient of reproducibility (1 − errors ÷ total answers).
Reliability (consistency):
Test–retest: same respondents twice, agreement by correlation, ICC or kappa.
Internal consistency: Cronbach's alpha across items of a scale.
Inter-rater: agreement between interviewers or observers (kappa).
Validity (measures what it claims):
Face and content validity: review by experts and by respondents.
Criterion validity: agreement with a gold standard, at the same time (concurrent) or later (predictive).
Construct validity: relationships with other measures as expected, factor analysis.
Definition: a small-scale trial run of the instrument (pretest) or of the whole study procedure (pilot study) before the main study, in people like the study population.
Roles, grouped by Thabane et al. (2010):
Process: recruitment, refusal and retention rates, clarity of eligibility criteria, whether participants understand the questionnaire.
Resources: time to complete forms, equipment, capacity of centres.
Management: data entry problems, space on forms, staff difficulties.
Scientific: safety, and preliminary estimates of variability.
Outcome: revise wording, order, response options, layout and procedures, or decide the main study is not feasible.
Limits: it does not test a hypothesis, and it does not give a precise effect size for sample size (Leon et al., 2011). Randomised pilot trials are reported with the CONSORT extension (Eldridge et al., 2016).
Pros and cons of the survey method
Ph.D. RM 2023
Survey: collection of standard information from a defined sample by questionnaire or interview (face to face, telephone, postal, online).
Pros:
Large samples, quickly and cheaply.
Standard questions allow comparison across groups, places and time.
Results can be generalised when sampling is random.
Many variables at once. Anonymous self-completion possible.
Cons:
Non-response bias. Report the response rate and its formula (CROSS 11c).
Self-report differs from behaviour. Recall error.
Little depth. Misread questions cannot be corrected.
Interviewer bias in interview surveys.
Cross-sectional surveys cannot show temporal sequence.
Improving response: in a review of 292 trials, monetary incentives (OR 2.02) and short questionnaires (OR 1.86) increased postal response (Edwards et al., 2002).
Dealing with non-response: report unit and item non-response. Add the expected non-response to the sample size for precision. To reduce bias, follow up a random subsample of non-responders with extra effort (Hansen and Hurwitz, 1946).
Data collection procedure
Ph.D. RM 2015
Identify variables from the objectives, with operational definitions and scales.
Choose methods and sources for each variable.
Select or develop tools. Translate, back-translate, validate.
Prepare the case record form and a manual of procedures.
Ethics approval and consent forms.
Train and standardise data collectors, calibrate instruments.
Pilot and revise.
Collect data: consent, administer, record at the time.
Supervise: check forms daily, re-interview a sample, log refusals, count the last digits of measurements to detect digit preference.
Data entry with checks (double entry, range checks, separate codes for missing values), cleaning, coding, secure storage and confidentiality.
Types of data analysis, and tests of significance for different types of data
Ph.D. RM 2019, 2021
Types of analysis:
Descriptive: frequencies, percentages, mean and SD or median and IQR, tables and graphs.
Inferential: estimation (confidence intervals) and hypothesis testing (tests of significance).
By number of variables: univariate, bivariate, multivariable (regression).
Qualitative analysis: coding and themes from text.
Test of significance: a procedure that gives the probability (p value) of a difference at least as large as the one observed if the null hypothesis were true.
Define every variable and how it will be measured, and test the method on a few people, before you collect data from your first participant.
Observation sheet, response-rate, seizure-count, Likert, Guttman, last-digit and non-response subsample examples use made-up numbers for teaching.
Sources: Graunt J. Natural and Political Observations Made upon the Bills of Mortality. London; 1662. Connor H. John Graunt F.R.S. (1620–74). J Med Biogr 32(1):57–69 (online 2022; doi:10.1177/09677720221079826). Likert R. A technique for the measurement of attitudes. Arch Psychol 1932;no. 140. Stevens SS. On the theory of scales of measurement. Science 1946;103:677–80 (doi:10.1126/science.103.2684.677). McCambridge J, Witton J, Elbourne DR. J Clin Epidemiol 2014;67:267–77 (doi:10.1016/j.jclinepi.2013.08.015). Edwards P et al. BMJ 2002;324:1183 (doi:10.1136/bmj.324.7347.1183). Boynton PM, Greenhalgh T. BMJ 2004;328:1312–5 (doi:10.1136/bmj.328.7451.1312). Thabane L et al. BMC Med Res Methodol 2010;10:1 (doi:10.1186/1471-2288-10-1; correction 2023, doi:10.1186/s12874-023-01880-1). Leon AC, Davis LL, Kraemer HC. J Psychiatr Res 2011;45:626–9 (doi:10.1016/j.jpsychires.2010.10.008). Eldridge SM et al. BMJ 2016;355:i5239 (doi:10.1136/bmj.i5239). Sharma A et al. CROSS. J Gen Intern Med 2021;36:3179–87 (doi:10.1007/s11606-021-06737-1). von Elm E et al. STROBE statement. PLoS Med 2007;4:e296 (doi:10.1371/journal.pmed.0040296). Vandenbroucke JP et al. STROBE explanation and elaboration. PLoS Med 2007;4:e297 (doi:10.1371/journal.pmed.0040297). ICH E6(R2) Guideline for Good Clinical Practice, 9 November 2016. ICH E6(R3), adopted 6 January 2025. International Institute for Population Sciences. National Family Health Survey (NFHS-5), 2019–21. Guttman L. A basis for scaling qualitative data. Am Sociol Rev 1944;9(2):139 (doi:10.2307/2086306). Hansen MH, Hurwitz WN. The problem of non-response in sample surveys. J Am Stat Assoc 1946;41:517–29 (doi:10.1080/01621459.1946.10501894, PMID 20279350). Schwarz N, Hippler HJ. The numeric values of rating scales: a comparison of their impact in mail surveys and telephone interviews. Int J Public Opin Res 1995;7:72–4 (doi:10.1093/ijpor/7.1.72). Gothwal VK et al. Guttman scale analysis of the distance vision scale. Invest Ophthalmol Vis Sci 2009;50:4496–501 (doi:10.1167/iovs.08-3330, PMID 19369236). Kottke TE et al. Association of an automated blood pressure measurement quality improvement program with terminal digit preference and recorded mean blood pressure in 11 clinics. JAMA Netw Open 2022;5:e2229098 (doi:10.1001/jamanetworkopen.2022.29098, PMID 36044216).
Past papers
Every TNMGRMU question on data collection
Each one is answered in this chapter. Tap a sitting to open the university's own question paper.
×1As a health researcher, you are planning to do health research among college students using questionnaire as the only data collecting tool. Discuss various components that you will consider while designing the questionnaire and the methods you will apply to test reliability and validity of a questionnaireMD Community Medicine · Oct 2025
×1Explain the classification of data with suitable examples. Explain the basic concept of sampling. How do you determine the sample size of a research study?Ph.D. RM · Jan 2025