We need to know whether drugs and other healthcare treatments work. The field of evidence-based medicine developed important tools, like randomised controlled trials, to do this. While they have helped drive huge advances in medical science, we should be cautious about relying solely on trials to evaluate new healthcare interventions. Although valuable, they can be misleading if badly designed, and they aren’t intrinsically superior to all other evidence. Evidence-based medicine should use real-life observations alongside trial data to get richer and more accurate answers for patients.

If you had enough of a potentially life-saving drug for half the patients with a deadly disease but not enough for all of them, how would you decide whom to treat? Would you flip a coin? Allocate sequentially (patient 1 gets the treatment, patient 2 nothing, and so on)? Write ‘drug’ or ‘no drug’ on hundreds of cards, put them in identical sealed envelopes and shuffle thoroughly before picking? And what if the drug was possibly lifesaving but its benefits had not yet been proven? Might you use one of these allocation methods and a placebo group (who would get an inactive dummy pill) to test exactly how effective the drug was?

SUGGESTED VIEWING The medicine myth With Denis Noble, David Malone, Guy Brown, Nessa Carey

This was the dilemma faced in 1947 by a British statistician, Dr (later Sir)  Austin Bradford Hill, who was a stickler for methodology.  He had been concerned for some time that new drugs tended to be compared with placebos using either historical controls or sequential allocation. The former approach meant that one group of patients had received yesterday’s standard of care while those taking the active drug were getting today’s (hopefully better) standard in addition to the drug, making comparisons tricky. Sequential allocation was an improvement on this, but it meant that a doctor would know whether a patient was to get the active drug before deciding whether to put them in the trial. Misplaced compassion might nudge the doctor to tweak the allocation process—for example by re-ordering the patients or leaving sicker patients out of the trial altogether if they were in line for placebo. The results of such trials would suffer from bias—defined as anything that systematically distorts comparisons between groups. If an active treatment group contains sicker patients than the placebo group, the drug will appear less effective than it actually is.

Bradford Hill had written a series of articles in the medical journal The Lancet in 1937 arguing to replace historical controls and sequential allocation with some kind of concealed allocation (in his day, this was done by putting cards in sealed envelopes; these days we use computer-generated random numbers). But despite the soundness of Bradford Hill’s arguments, practice was slow to change.

SUGGESTED VIEWING How medicine ignores our bodies With David Healy