We all know that information is valuable, and that more information is generally better than less.
But in the realm of pharmaceutical research (as in others, to be sure), there’s a troubling paradox: while successes are widely publicized, and while the results of clinical trials are usually published, the research from projects that fail before that stage is usually kept hidden. “A result,” writes Natasha Singer in The Times, “is that companies waste many millions going down experimental paths that their competitors have already found to be dead ends.”
You should read Singer’s article in full, but here are a few highlights:
Now big-picture thinkers, within the industry and outside it, are re-examining every stage of drug development — from molecule to market — in an effort to foster faster innovation. It’s a holistic approach, called “systems thinking,” that originated in methods that engineers used to streamline projects in the aeronautics and automotive industries.
The Massachusetts Institute of Technology, for one, started a pharmaceutical innovation program this year to help drug companies adapt some successful approaches now used in aeronautics, like lean management and information-sharing among rivals.
And:
The M.I.T. project, called New Drug Development Paradigms, has gathered a powerful consortium of interested parties — including major drug makers and federal health authorities. One short-term goal is to identify, and rectify, the root causes of bottlenecks in the existing system. Longer term, the ambition is to create new prediction models, new ways to share information about the biology of diseases, and a new inclusiveness involving earlier participation of regulators, health insurers, health care providers, and patients.
And:
In the absence of a public database on failed drug compounds, a small group from the Sloan School of Management at M.I.T. and the Harvard Business School has created Pharmer’s Market (not this one), an online prediction market that uses crowd-sourcing to forecast the likelihood of a drug’s success. Introduced last month, the market has invited biomedical researchers and other drug industry experts to place anonymous bets, using virtual money, on the likelihood that certain breast cancer drugs, currently in clinical trials, will succeed or fail.
After the clinical trials conclude, researchers can assess whether this kind of collective intelligence may be a useful predictive tool for drug companies, said Ragu Bharadwaj, who helped devise the project as an M.I.T. graduate student.
There is an obvious conflict between open-sourcing and a competitive marketplace, but it is exciting to see smart people pushing hard to change the shape of that marketplace for the public good.
and 
. You can also watch it on 
Did someone think of the moral hazard? Researchers sabotaging their own or others projects to collect a huge windfall from their betting?
As a general proposition in research it might be useful to have a way for researchers to document and publish approaches that failed so that others do not go down the same blind alley.
@ puneet – the betting is anonymous and with fake money, so I doubt it.
Of course, the sharing of past experiments will definitely benefit society in that, as explained, companies will not have to go through the same path as the other ones did, spending millions of dollars in the process. However, it is easy to understand why the companies DON’T do so. Why help the other company out if doing so would just give them an advantage? It doesn’t make sense if you spend tons of money on something that didn’t work and then give a piece of advice to the rival company that will only help them not make the same mistakes.
So what can be done so it’s all fair and competitive? Bah, who knows. The only thing that comes to my mind is just one giant pharmaceutical company, but that’s not fair. Or is it?
Water is a public good. Protection is a public good. Even a representative in court is public to anyone and everyone. So why can’t life-saving cures be made public? It doesn’t even have to be a government institution, just one big company with regulated prices so as to not rip the customers off. I’m not saying this is the answer…but could it be a possibility?
“There is an obvious conflict between open-sourcing and a competitive marketplace…”
Is there an obvious conflict? Similar things have happened and are happening in the “open source” world, in which large amounts of source code are shared, developed, maintained, and improved via market competitors. For example, the Linux kernel is co-developed between the market competitors IBM, Red Hat, Novell, and Ubuntu, and the result is an improved code base which is more widely used than if it were tied up with a single company.
The patent system is another example — the information is intended to be shared (which is why patents expire), thus increasing the amount of shared knowledge usable by…market competitors.
I would instead argue that this “open sourcing” — sharing of information — is REQUIRED for a competitive marketplace. (Think of the opposite — if nothing were shared, then there would larger costs in entering new markets, resulting in fewer competitors, and a less efficient market. This is why we even have a patent system in the first place, to encourage sharing, improving the market!)
Please say more about the obvious conflict between open-sourcing and a competitive market. How is open-sourcing anticompetitive? Doesn’t open-sourcing certain information simply change the goal of the competitive game? Open-source companies are still competing in some arena.
Why not start a voluntary, but subscription-based database managed by a neutral third party whereby results of failed experiments are documented robustly but the contributors remain anonymous? (This is sort of the same as they way benchmarking works…everyone wants to know where they stand amongst their peers, but no one wants to see their name at the bottom of a category)
This way, Pharmas can see what’s being done and make their decisions about whether to persue certain lines of research or not.
This is a problem in all of science. The push is to publish positive results, not negative results. The Journal of Negative Results in BioMedicine is one resource to combat this though.