Why Is Failure a Sign of a Healthy Economy? A Guest Post by Tim Harford

I’ve known Tim Harford for a while; to me, he’s one of the best writers who also happens to be an economist (although in recent years he’s spent more time as a writer than a practicing economist, which may explain everything). Disclosure: Harford profiled Steve Levitt back when Freakonomics came out, and he’s had the two of us on his BBC radio show More or Less.

He writes a Financial Times column (on Saturdays) called “Undercover Economist,” and that was the title of his first book, published in 2005 (and just updated). His latest book, out this week, is called Adapt: Why Success Always Starts with Failure. It examines the incremental, adaptive ways by which success is achieved across a number of fields. Here’s a taste, in the form of a guest post. It’s very good, and to my mind, here’s the best passage:

[W]here’s the churn in education policy or healthcare policy or policing? These are difficult problems. Why would we expect them to be solved the first time? They are surely no simpler than the business problems which seem so prone to experiment and error.

Failure: It’s Everywhere

by Tim Harford

In 1982, the management consultants Tom Peters and Robert Waterman published In Search of Excellence, a colossally popular business title. The book aimed to learn lessons from the world’s best companies, and Peters and Waterman produced a list of 43. But just a couple of years after In Search of Excellence had been published, BusinessWeek ran a cover story with the simple title: “Oops! Who’s Excellent Now?” Almost a third of the companies singled out for praise by Peters and Waterman were in financial trouble.

My aim isn’t to mock Peters and Waterman, but to point out that the rise and fall of business models is an unavoidable part of economic growth. In a complex world, things fail – a lot. According to the economist Paul Ormerod, 10 percent of U.S. firms go bankrupt every year. Ormerod – an iconoclastic figure who enjoys beating fellow economists at their own game, mathematics – has studied the statistical patterns that emerge from these bankruptcies. He thinks they suggest that failure and success in business are far more random than our culture of CEO-worship would have us believe.

The curious thing is that these individual failures need not be a problem for the economy as a whole. Far from it. There are good reasons to believe that more successful economies play host to more failures. To get an instinctive grasp of this, just compare the astonishing failure rate of Silicon Valley firms with the situation of the Big 3 in Detroit, who seem to be in a perpetual state of slow-motion failure without ever quite leaving the economic stage.

A more rigorous attempt to look at this question, a study by Kathy Fogel, Randall Morck, and Bernard Yeung, found statistical evidence that economies with more churn in the corporate sector also had faster economic growth. The relationship even seems causal: churn today is correlated with fast economic growth tomorrow. The real benefit of this creative destruction, say Fogel and her colleagues, is not the appearance of “rising stars” but the disappearance of old, inefficient companies. Failure is not only common and unpredictable, it’s healthy.

But if this is true, it makes nonsense out of much of the way we approach complex problems in the world – anything from fighting wars to fighting poverty.

For one thing, where’s the churn in education policy or healthcare policy or policing? These are difficult problems. Why would we expect them to be solved the first time? They are surely no simpler than the business problems which seem so prone to experiment and error.

When the pizza chain Sbarro’s filed for bankruptcy in April, one journalist called it “America’s least essential restaurant.” The likely result is that one way or another, the chefs and locations and ingredients that used to be turned into bland pizzas will now be used to make something that people actually want to pay for. Good.

But in politics, where are the bad ideas that have been tested, found wanting, and replaced with something better? It’s rare – but not unheard of – for politicians to seriously test out their policies, perhaps because they realize that we voters pay more attention to soundbites. And so there’s rarely a really good evidence base to shut down failing policies and replace them with something else.

I’m not sure I’d blame the politicians. Not many people want to vote for a candidate who says, “I really am not sure how to improve schools in our area, so I plan to pilot half a dozen ideas, and we’ll keep the one or two that we can prove to have worked.” But the other guy – the one who claims he does know – isn’t telling us the truth about uncertainty and failure. He’s just telling us what we want to hear.

Ideally, I’d like to see many more complex problems approached with a willingness to experiment. The process has three components: first, try lots of different things; second, make sure the experiments are at a small scale so that when things go wrong, it’s not a catastrophe; and third, make sure there’s a reliable way to tell the difference between success and failure.

Governments often fall down on all three: they have a particular ideology and so push a single-minded policy; they bet big; and they don’t bother to evaluate the results too carefully, perhaps through overconfidence.

But markets can fail badly too, and for much the same reason. Just think about the subprime crisis. It failed the same three tests. First, many big banks and insurance companies were taking similar bets at similar times, so that when subprime loans started to go bad, much of Wall Street started struggling simultaneously. Second, the bets were gigantic. Fancy derivatives such as credit default swaps and complex mortgage-backed securities were new, rapidly growing, and largely untested. And third, many investment bankers were being paid large bonuses on the assumption that their performance could be measured properly – and it couldn’t, because profitable-seeming bets concealed large risks.

I’ve been studying how these processes of experimenting and adapting have worked, or failed to work, in dealing with some of the important problems we face today, from promoting innovation to preventing future financial meltdowns. And I’ve also been exploring our psychological responses to failure – why we fear even small setbacks, and how we can over-react when things are going badly. I’ve concluded that adapting in the face of failure is absolutely vital – and a more painful challenge than I at first expected.

Nowhere is that more true than on the battlefield, and in my next post I’ll be exploring how the military learns from failure – from the Vietnam War to the hunt for Bin Laden.

Tyler Sibley

Great article. It makes sense that bankruptcy would cause more firms to improve their product/service and that this adaptation to inprovement increases value of firms in the long-run.


Interesting article, and if you are interested in more stuff by Tim Harford, his radio show is available as a podcast at www.bbc.co.uk/podcasts/series/moreorless

Lohring Miller

I am just finishing up a dental public health grant that intended to do exactly what you are talking about. The goal was to fund fifteen different clinics for five years to provide low cost dental care to HIV positive individuals in several different settings. The models ranged from large central city clinics to mobile vans, established dental schools, and, in my case, a dental hygiene program in a community college.

Dental care is a microcosm of health care in general, but is mostly privately funded. People have an easy time deciding on treatment where they need to pay all the costs. Public health dental programs run into serious funding problems, so care gets rationed by the small number of paid practitioners available to meet the demand. Programs usually can only provide minimal treatment to relieve pain.

The problem can be attacked from both sides. Conventional treatments can replaced by less expensive approaches. Expensive practitioners can be replaced by less expensive ones. Schools have the advantage that the students actually pay to provide patient care. There are serious limitations, though.

Will any of the various models be the single new solution? I doubt it. However, the people involved will move on into new programs with what they learned. New ideas will be added and new solutions will be developed. In the mean time a lot of care was provided. In those ways "failure" is a long term success.

Lohring Miller, DMD