Photo: soldiersmediacenterHow do we know that parachutes are really a good treatment for preventing serious injury in someone falling from an airplane?
That’s the subject of this tongue-in-cheek paper on the limits of evidence-based medicine, written by two physicians and published in the British Medical Journal.
After applying to parachutes the guidelines usually used to test new drugs, the authors find that, since nobody has ever conducted a placebo-controlled trial on parachutes, the only evidence we have of their effectiveness is “anecdotal” and worthy of healthy skepticism:
It might be argued that the pressure exerted on individuals to use parachutes is yet another example of a natural, life-enhancing experience being turned into a situation of fear and dependency. The widespread use of the parachute may just be another example of doctors’ obsession with disease prevention and their misplaced belief in unproved technology to provide effective protection against occasional adverse events.
Ian Ayres tackled evidence-based medicine on this blog back in March. What he found made him feel a little better about going to the dentist.
and 
. You can also watch it on 
I’m pretty sure that NOBODY voluntarily jumping out of a plane could be placed in a psychiatric “healthy cohort.” We’ll have to throw some people out of planes against their will.
Best sentence in the article is the last sentence of the abstract: “We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.”
—————-
Note the use of “most radical”. The article is not arguing that “evidence-based” evaluation is unnecessary. It’s simply pointing out that those who reject any data in any context unless it is backed by randomized controlled studies are going a little too far.
Anecdotal evidence *is* useful: after all, it’s how most of our medical practice came about. Do you think someone did a randomized controlled experiment to discover aspirin? No, they just observed that many people who chewed on willow tree bark reported pain relief. Controlled experiments came centuries later.
@2: You don’t understand the meaning of the word “placebo”. Unless those falling out of planes sans chutes had every reason to believe they had a chute, then they didn’t have a placebo. The tongue-in-cheek reasoning here is that it’s not the chutes that save lives but the belief that they will be safe. And those without any sort of chute (or even a clearly malfunctioning chute) believe they will die. Thus, you cannot [grin] discount the psychological influence on the outcome.
When I was in the US Army, I was detailed for a short time to the Airborne and Special Operations Test Board. Testing experimental parachutes. They had person-shaped dummies for the really dangerous stuff. Lots of fun otherwise.
First define serious injury.
A fractured ankle, certainly not serious in the life-threatening sense, will seriously affect a ballerina or a football player or a military person trying to get out of the line of fire, but perhaps not an accountant. A fall-related head injury is another matter.
A better metric [IMHO} would be to use the "number needed to treat [NNT]” to find out if parachutes are beneficial.
According to the Canadian Centre for Evidence Based Medicine the definition is: The NNT is the number of patients who need to be treated in order to prevent one additional bad outcome [one patient suffering a serious injury].
An example: Some people with untreated high blood pressure live to be 100 and others die young. How many patients would you have to put on a medicine to prolong or save one life?” For example, if 100 people are treated with a test medication, and 25 have their life prolonged, then the NNT is 4.
In the case of parachutes, assuming that the chute either works or doesn’t, the NNT should be unity (or darn close.)
If you really want a control (no parachute) group perhaps we could use death row inmates, or maybe economists . . .
The Bayesian approach simply considers what is the probability of a given fact or condition (or in medicine, specifically the presence of disease) prior to performance of a test. Thus, from the earlier comment, screening for an uncommon disease such as HIV in the general population will yield relatively high numbers of false positives, even if the screening test is quite “accurate”, simply because the so-called “base-rate” of HIV was low to begin with and no test is 100%. Here, the pre-test probability of death after skydiving unassisted by parachute is essentially 100% (if high enough), based on empiric data (failed parachute cases, or simulations with dummies etc). Although conceptually quite elegant, Bayesian reasoning is not always straightforward, since estimation of the pre-test probability itself (the critical starting point) can be debated in various circumstances, and itself depends on prior data-sets and experience and thus is not a simple “fact”. actually, every single medical trial ever published operates on the exact opposite principle, that of the “frequentists”, and uses P-value methods that assume no prior knowledge (the opposite of Bayes). The main circumstance in which bayesian reasoning enters medical decision making is interpretation of diagnostic test results in routine care (like a screening test mentioned above), but even then, it is rarely actually done. correctly.
**Anecdotal evidence *is* useful: after all, it’s how most of our medical practice came about. Do you think someone did a randomized controlled experiment to discover aspirin? No, they just observed that many people who chewed on willow tree bark reported pain relief. Controlled experiments came centuries later.**
And how many of those who found relief died of stomach bleeding? The problem with the “old” medications is precisely that they *haven’t” been tested in a double-blind, randomized way. Thus the ongoing inappropriate use of digoxin, lidocaine, sugar infusions for a heart attack and a few others.
Unfortunately such trials cost money and most are currently funded by drug companies who want to sell their new “me-too” drug.
regarding many of the comments…
is it just me, but surely the “point” of this article is a bit of light-hearted amusement.
Definitions of placebo? Explanation of medical variables??
Aren’t these people simply demonstrating that they missed the point?