Photo: soldiersmediacenterHow do we know that parachutes are really a good treatment for preventing serious injury in someone falling from an airplane?
That’s the subject of this tongue-in-cheek paper on the limits of evidence-based medicine, written by two physicians and published in the British Medical Journal.
After applying to parachutes the guidelines usually used to test new drugs, the authors find that, since nobody has ever conducted a placebo-controlled trial on parachutes, the only evidence we have of their effectiveness is “anecdotal” and worthy of healthy skepticism:
It might be argued that the pressure exerted on individuals to use parachutes is yet another example of a natural, life-enhancing experience being turned into a situation of fear and dependency. The widespread use of the parachute may just be another example of doctors’ obsession with disease prevention and their misplaced belief in unproved technology to provide effective protection against occasional adverse events.
Ian Ayres tackled evidence-based medicine on this blog back in March. What he found made him feel a little better about going to the dentist.
and 
. You can also watch it on 
Is this sort of question amenable to Bayesian analysis? I know I’ve seen Bayesian analysis of other medical issues, and it tends to be straightforward and convincing. A simple example being that false positives for AIDS screenings are more common than real positives, so retesting a positive is generally worthwhile.
…of course, there’s well over a century’s worth of ‘placebo-controlled’ parachute experience around the world.
Humans have been falling out of the sky with and without operational parachutes for a long, long time.
Parachute ‘Cause & Effect’ on human survival is obvious and easily demonstrated even without a placebo — because there is only a single, easily measured variable to test.
Not so with medical testing… which involves a large number of variables — many of which are difficult/impossible to measure objectively. Subjective (anecdotal) data often results… an is often proffered as “science”.
Everyone nowadays knows that high-cholesterol causes heart-disease deaths — except that there is no objective scientific testing demonstrating that ‘fact’. And everyone knows cancer-screening significantly reduces overall cancer-deaths– except there’s no hard proof of that either.
Perhaps the value of controlled medical testing should not be lightly dismissed ?
Funny and all, but properly controlled trials with human analogues *have* been done: even the Mythbusters dropped a crash-test dummy from an airplane with various parachute substitutes.
Evidence-based medicine looks at the risk vs. benefit of a given intervention: and *every* intervention has risk. Properly controlled studies are required so that one can be sure that the benefit is real before subjecting a patient to additional risk.
What we need to do is find and exploit a natural experiment. Do we have any data on survival rates for people who have fallen out of airplanes without parachutes?
Evidence-based medicine is what everyone assumes we’re getting, but surprisingly, this is not always the case.
A couple of doctors at UBC’s Therapeutics Initiative (Canada) have a podcast on evidence-based medicine, where they discuss drug therapies in terms of what really works versus what doesn’t really do much. They’re always talking emphasizing patient outcomes, instead of statistics or measurements (e.g., cholesterol numbers, PSA counts, or how a tumor looks on an X-ray). A lot of medical treatment is given despite the fact that for most patients, all it does is make their numbers look better — it doesn’t alleviate symptoms they can feel, make them live longer, or otherwise have a real effect on their lives/health.
#2 and #3 missed the point:
No prior trials involving the efficacy of parachutes were done with blind controls. Therefore, concerns over a bias such as the “healthy cohort” effect have not been put to rest.
No … wait … the real point was that it’s hilarious.
It’s a poorly written article that shows the authors don’t actually understand evidence-based medicine. A placebo doesn’t also have to be nothing. Much research is done with two or more situations where there is a supposed effect. For example, one wouldn’t do a cancer study by comparing a novel treatment to doing absolutely nothing. One would compare a standard treatment to either a novel treatment or the same standard plus a novel treatment.
Whilst technically correct, I do suspect that you have walked past the main point stressed by the authors. They are clearly not against well-conducted trials. They are asking for people to apply reasonable balance in what level of evidence is required when.
Well conducted trials are great; my family and I have benefitted from them on many occasions. However, where a clinician looks for the right treatment for the patient, the researcher looks for the right patient for the treatment. We can end up with evidence from trials with carefully selected groups, then we have to apply the results to the real world – where patients that do not meet the description of the trial sample have to be treated.
So, please keep on with the well-conducted trials, and use the information from them and other sources with care. Which you would do, I’m sure!
Since when are parachutes medicine?
Medicine requires stronger evidence than anecdotal evidence because history has shown anecdotes to be extremely unreliable. Anecdotal evidence is fine to determine if a normal invention, like a parachute work, but the effects of medicine is a lot more subtle.
Europeans used blood letting since classical times because it ‘seemed to work’. Patients would report, and doctors would notice, a lessening of the symptoms. But it wasn’t until the procedure was subjected to clinical trials, with a control group, did it become apparent that it was killing patients, famously including George Washington.
Scientific medicine is supposed to be flawed because it can’t be applied in certain cases, and it’s usually because of ethics (we can’t throw people out of planes)? There are very few situations like this. If a treatment can be ethically tested, it should be! There’s no reason to not demand the highest of standards of evidence from medical treatments, especially considering the history of failed medical treatments and outright quackery.
People who promote alternative/complimentary medicine adore the above study, they think it frees them from having to provide evidence for their often dubious treatments.
Unfortunately for them, we should still demand scientific evidence for their treatments, because there’s no reason not to. Because of the fallibility of human perception (confirmation bias), the placebo effect, and the complexity of the human body, it is nearly impossible to tell if a medical intervention helps, or even hurts without proper clinical trials.
For more on the importance of evidence based medicine, and the evidence for or against popular alternative treatments, I highly recommend Simon Singh’s book ‘Trick or Treatment’.