Or at least that is the impression you might get if you read this article in today’s Wall Street Journal.
I will post a longer blog entry once I have had time to fully digest the working paper by Foote and Goetz which is the basis for the article.
For now, I will say just a few things:
1) It is not at all clear from the WSJ article is that Foote and Goetz are talking about only one of the five different pieces of evidence we put forth in our paper. They have no criticisms of the other four approaches, all of which point to the same conclusion.
2) There was a coding error that led the final table of my paper with John Donohue on legalized abortion to have specifications that did not match what we said we did in the text. (We’re still trying to figure out where we went wrong on this.) This is personally quite embarrassing because I pride myself on being careful with data. Still, that embarrassment aside, when you run the specifications we meant to run, you still find big, negative effects of abortion on arrests (although smaller in magnitude than what we report). The good news is that the story we put forth in the paper is not materially changed by the coding error.
3) Only when you make other changes to the specification that Foote and Goetz think are appropriate, do the results weaken further and in some cases disappear. The part of the paper that Foote and Goetz focus on is one that is incredibly demanding of the data. For those of you who are technically minded, our results survive if you include state*age interactions, year*age interactions, and state*year interactions. (We can include all these interactions because we have arrest data by state and single year of age.) Given how imperfect the abortion data are, I think most economists would be shocked that our results stand up to removing all of this variation, not that when you go even further in terms of demands on the data things get very weak.
Again, as I said, I will post again on this subject once I have had a chance to carefully study the details of what they have done, and after I have been able to go back to the raw data and understand why the results change when one does what Foote and Goetz do.
From 1 to 25 of 40 Comments
“Mr. Levitt”? I thought it was “Dr. Levitt.”
— econopeteHaving read the article in its entirety, I felt it was rather balanced except for the title (”faulted” should be replaced by “questioned”).
The Royal Society of England, of which Issac Newton was a part of, was formed largely to help encourage debate and discussion of complex issues. This is the tradition of science.
Since I got a D in applied econometrics (the second time I took it) I don’t dare comment on the methodology.
— econopeteThe Wall St. J. summarizes your theory as follows:
Unwanted children are more likely to become troubled adolescents, prone to crime and drug use, than are wanted children.
My question: is this really your theory? Because this summary seems unsubstantiated. Your results show that there is a causal connection between people and crime, that is, between the people who would have existed, but now do not, because they have been aborted, and the amount and frequency of crime. And it does NOT show a causal connection between the environmental influences on such people and the likelihood they would commit crime. Whether or not a child is ‘wanted’ is certainly an environmental factor. So the environmental and genetic causes are confounded. I don’t remember your work controlling for this. Pehaps I should look again. But I look forward to your response to Foote.
— MPGibsonOn a lighter note…now you have one of those cool WSJ “dot” drawings of yourself…
Looking forward to your response to Foote–this is how good ideas become great ideas–through testing, questioning, and rethinking.
Good stuff. Thanks, as always, for keeping us in the loop and letting us learn along side you guys.
— JdTEver since I debated Dr. Levitt in Slate.com in 1999, people have been telling me that my simpleminded little graphs and ratios of national crime trends showing that Dr. Levitt hadn’t met the burden of proof couldn’t possibly be right because Dr. Levitt’s state-level evidence was so much more gloriously, glamorously, incomprehensibly complicated than mine, and Occam’s Butterknife says that the guy with the most convoluted argument wins.
Well, now we now why Dr. Levitt’s abstruse state-level analysis didn’t match up with my straight-forward national-level analysis: because he made two big mistakes in his work.
— SteveSailerAbortion and Crime - Freakonomics Disputed
I have not yet read Freakonomics, but I have heard good things about the book from friends. Apparently one chapter within the book discusses the effect of the legalization of abortion in the 1970s on the amount of crime in the 1990s. An article in the …
— Consumerism CommentaryWould it be appropriate to calculate powers for the various specifications for which Foote and Goetz find insignificant coefficients on abortion to determine exactly how demanding of the data the specifications are?
Also, by saying that the specifications are too demanding what assumptions are you making in your model by not including the state-age-year interactions?
— GamblingEconomist“For those of you who are technically minded, our results survive if you include state*age interactions, year*age interactions, and state*year interactions.”
Including interaction terms is standard in data analysis BY UNDERGRADUATE PSYCHOLOGY MAJORS but you think it’s high tech. What you consider “technically advanced” and “incredibly demanding of the data” is something psychologists teach undergraduates to do in introductory research methods classes.
Thanks! In just one sentence you do more to support the hypothesis “Whoever called economics the dismal science got it half right” than anyone in recent history.
— south(west)pawTechnically advanced clearly refers to being technically advanced for the blog audience not for economists. Incredibly demanding of the data is not difficulty in performing the computations, it is having enough variation to have a reasonable chance at finding significant results.
— GamblingEconomistEconomist and blogger Mahalanobis writes:
“Levitt’s response is on his website (see here) where he notes:
>
“3 interaction variables are necessary to get the right sign and significance? I think that is _incredibly demanding._ In my experience, interaction variables are kitchen sink type regressors that induce severe multicollinearity and give spurious results. It’s like an economist saying his results only appear after doing 3-stage least squares. I have to think something’s not really there if you can’t normalize the data somehow and show in a simple graph that the pattern is there (in this case, say, by showing the change in arrest rates for abortion and non-abortion states for the relevant age cohort).
http://mahalanobis.twoday.net/stories/1203397/
— SteveSailerI would consider any college level statistics course to be considered technically advanced. I don’t think this blog’s audience is limited to psychology majors.
However, I am still familiar with the general idea of statistics, and I think I can respond to SteveSailor, who has done a lot of research, it would seem. His biggest flaw is that he compares homicide rates per age of the criminals, rather than the number of criminals per age. There is nothing in Levitt’s argument that would suggest the homicide rates per criminals would be any lower. The argument is that the size of the population in the groups typically commiting homicide were reduced.
I would be somewhat interested to see some research supporting the idea that legalizing abortions reduced unwanted children, but that really does seem fairly straightforward.
— e6matteomatt says: “There is nothing in Levitt’s argument that would suggest the homicide rates per criminals would be any lower. The argument is that the size of the population in the groups typically commiting homicide were reduced.”
Sailer: No, Dr. Levitt is also saying something much more interesting than just, to take it to a reductio ad absurdum, “If all unborn children were aborted and the human race died out, then the crime rate would go down.” As Stalin used to say, “No man, no problem.”
That’s part of his theory but not the interesting part. Of course, as Dr. Levitt himself admits in “Freakonomics,” the impact of legalized abortion on the birthrate was fairly minimal. He wrote: “Conceptions rose by nearly 30 percent, but births actually fell by 6 percent …” That’s because legalizing abortion drove up the amount of unprotected sex Jonathan Klick discovered that sexually transmitted disease rates rose by 30 percent as well. Apparently, Roe v. Wade made condoms unfashionable.
No, the interesting aspect is that Dr. Levitt has argued repeatedly that legalized abortion has a selection effect, that there is, in effect, a eugenic and/or eucultural impact on the composition of the next generation.
Levitt and Donohue’s 1998 draft paper and 2001 published papers contained an explicitly racial element (that blacks have more abortions and commit more murders, thus, presumably, legalizing abortion lowered the number of murderers on the streets 20 years later) of the kind that Bill Bennett was recently excoriated for. In 2001, Levitt and Donohue wrote:
“Fertility declines for black women are three times greater than for whites (12 percent compared to 4 percent). Given that homicide rates of black youths are roughly nine times higher than those of white youths, racial differences in the fertility effects of abortion are likely to translate into greater homicide reductions. Under the assumption that those black and white births eliminated by legalized abortion would have experienced the average criminal propensities of their respective races, then the predicted reduction in homicide is 8.9 percent. In other words, taking into account differential abortion rates by race raises the predicted impact of abortion legalization on homicide from 5.4 percent to 8.9 percent.”
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=174508
In “Freakonomics,” Levitt prudently cut out the racial argument (and, when, Bill Bennett got in trouble for referring to Levitt’s race-based subtheory, Levitt would not step up and admit to what he wrote).
Now, Dr. Levitt makes the anodyne claim that “wanted babies” are more likely to get better mothering than “unwanted babies.” To support this, he cites European studies.
The reason that in Dr. Levitt’s theory of _American_ crime trends, he cites _European_ studies claiming that women who have abortions would make worse mothers than the ones who went ahead and had their children is because the American studies of the impact of abortion came to the opposite conclusion.
Trent and Griner’s research, along with other studies undermining Levitt’s central argument, was pointed out to Levitt by CCNY economist Ted Joyce in his response to Levitt & Donohue in the Journal of Human Resources, which was entitled “Did Legalized Abortion Lower Crime?” Joyce summed up two reason why Levitt’s theory didn’t work. The second was:
“Second, analysts, I being one, have tended to overestimate the selection effects associated with abortion. A careful examination of studies of pregnancy resolution reveals that women who abort are at lower risk of having children with criminal propensities than women of similar age, race and marital status who instead carried to term. For instance, in an early study of teens in Ventura County, California between 1972 and 1974, researchers demonstrated that pregnant teens with better grades, more completed schooling, and not on public assistance were much more likely to abort than their poorer, less academically oriented counterparts (Leibowitz, Eisen, and Chow 1986).
“Studies based on data from the National Health and Social Life Survey (NHSLS) and the National Longitudinal Survey of Youth (NLSY) make the same point (Michael 2000; Hotz, McElroy, and Sanders 1999). Indeed, Hotz, McElroy, and Sanders (1999) found that teens who abort are similar along observed characteristics to teens that were never pregnant, both of whom differ significantly from pregnant teens that spontaneously abort or carry to term.
“Nor is favorable selection limited to teens. Unmarried women that abort have more completed schooling and higher AFQT [the military's IQ test for applicants for enlistment] scores than their counterparts that carry the pregnancy to term (Powell-Griner and Trent 1987; Currie, Nixon, and Cole 1995).
“In sum, legalized abortion has improved the lives of many women by allowing them to avoid an unwanted birth. I found little evidence to suggest, however, that the legalization of abortion had an appreciable effect on the criminality of subsequent cohorts.”
http://islandia.law.yale.edu/donohue/Joyce%20%282004%29.pdf
— SteveSailer[...] Oh, no: Everything in Freakonomics is wrong! according to the books’ authors, Levitt and Dunbar, the subject of a WSJ story on the criticisms levied against the book’s claims about the correlation between legalized abortion and falling crime rates in a paperfrom the Boston Federal Reseve Bank. [...]
— The New Market Machines » Blog Archive » Dismal DebateThe good news is . . .
Shouldn’t you at least maintain an appearance of impartiality, not to say open-mindedness, on this empirical issue?
— lukeleaEconopete: at the University of Chicago, all of the faculty are addressed as Mr./Ms.
JDT: those are not drawings, they are photographs that have been processed to remove intermediate tones.
— Robert SchwartzA few points to clear up the technical nonsense:
Sailor said, “3 interaction variables are necessary to get the right sign and significance?”
No, what Levitt said is that his results SURVIVE despite adding all the interaction controls.
To the person who asked why not use state*year*age interaction. The observations are at the state year age level so obviously you can’t interact at that level. That’d be one dummy variable uniquely assigned to each observation, what’s the point?
If people were to read the actual working paper by Foote and Goetz the main issue is not the programming error but the respecification of the model which involves adding in a variable which is known to have measurement error.
Much of applied econometrics is an art, not a science, so this will not be easily sorted out but it is certainly refreshing to hear constructive criticism for once as opposed to certain individuals jumping in whenever the issue comes up in a pathetic attempt at publicity for themselves.
— sophistrysophistry - you are right about the state-year-age interaction. I got confused because I thought Levitt was saying that even though he doesn’t include all the interactions that FG wanted, his results stand up to some interactions. Now I realize that he is saying that his results DO stand up to the interactions that FG wanted - just with a smaller magnitude once the error is corrected. And that that is impressive because it is demanding of the data.
— GamblingEconomistFrom reading the working draft of the commentary paper I have a couple thoughts. First, the authors believe all alternative explinations of the decline in the crime rate can be eliminated from the data set by removing state-year, year-age, and state-age interactions. They do this by normalizing the three dimensional data (state by age by year) so that the every state-year combination has a mean of zero, age-year, and state-age likewise. The thought is your left with pure state-age-year interactions.
However, I believe that throws away too much data, and doesn’t accomplish what the authers want. The authers want to adjust for trends not caused by abotion, such as a crack wave slowing. Crack, however also moves along all three dimensions. It spreads and wanes through states slowly, atacking one age group at a time. Any reasonable alternative hypothises would have to be more local than the Levitt and Donohue thesis, and would not be eliminated just by normalizing.
— AlpharigelAs I explained to Dr. Levitt in Slate in 1999
http://www.slate.com/id/33569/entry/33571/
what actually happened, simplifying greatly, was that the crack crime wave took off first in socially liberal states where legal abortion also had taken off first about 16-17 years earlier, most notably New York and California, which legalized abortion in 1970, three years before Roe v. Wade.
In other words, there was a _positive_ correlation, when weighted by population of state, between the legal abortion rate in the early 1970s and the homicide offending rate among youths born after legalization in the late 1980s and early 1990s. Unfortunately, in his original specification of his analysis, Dr. Levitt only looked at crime rates in 1985 and 1997 (and only looked at the overly crude age groups of over and under 25), so he completely missed how his theory had failed its most obvious historical test.
Second, this vast youth murder wave took off first specifically in the demographic group that had the highest legal abortion rate, urban blacks. As Donohue and Levitt wrote in 2001, under their theory, the opposite was supposed to happen: ““Fertility declines for black women are three times greater than for whites (12 percent compared to 4 percent). Given that homicide rates of black youths are roughly nine times higher than those of white youths, racial differences in the fertility effects of abortion are likely to translate into greater homicide reductions.” Instead, the black to white youth homicide offending rate almost doubled in the first cohort born after legalization. So, Levitt-Donohue failed the first two historical tests in massive fashion.
Then, two things happened historically that helped create the (presumably) negative correlation between 1970s abortion rates and mid 1990s crime rates that Levitt and Donohue have emphasized so repeatedly, while trying to cover up the earlier negative correlation.
1. From NY and CA, crack spread to more socially conservative states, where the abortion rate had also gone up later. So crime was higher in the 1990s in socially conservative states that hadn’t legalized abortion in 1970.
And, the crack wave burned out first in the places where it started first, most famously New York City.
We’ve all heard a million arguments about why crime fell in NYC in the 1990s, but the simplest explanation was offered by Knight-Ridder reporter Jonathan Tilove recently: there are today in NYC, 36% more black women alive than black men. Nationally, among all races, there are 8% more women than men alive. Obviously, this gigantic black male shortage in NYC wasn’t caused by abortion — there was virtually no sex selective abortion at the time. No, it was mostly caused by an enormous increase in imprisonment _and_ by the most dangerous black men murdering each other in large quantities. Levitt has never written, as far as I know, about the impact of “selective post-natal abortions,” as it were, on the crime rate, but it’s clearly a substantial factor in a number of big cities that were hit hard by crack.
Moreover, as I pointed out to him in 1999, and as the famous chapter in “Freakonomics” on how badly dealing crack pays confirmed, a lot of the next cohort of urban youths, those born more than a half decade after abortion was legalized in their state, figured out that dealing crack was a stupid career choice. Seeing how their older brothers and cousins were winding up in prisons, wheelchairs, and cemeteries, they became more law-abiding.
But these anti-crime trends in the 1990s happened first where crack happened first, which tended to also be where legal abortion happened first.
So, for this controversy, the crucial issue is The Burden of Proof. Dr. Levitt has tried hard to hand the Burden of Proof off to his skeptics, claiming that he’s looked at all other possible causes of the 1990s crime decline, and they aren’t adequate, so abortion must be the cause of the remainder. Of course, in reality, he hasn’t looked at all the causes — for example, I’ve never seen him take into account “selective post-natal abortions” of the most dangerous gangstas by other gangsta, nor the social learning impact on the next cohort.
But, moreover, there’s an old saying that large assertions require large evidence. Clearly, the burden of proof rests on Dr. Levitt.
There’s also an old idea in science called Occam’s Razor, which more or less says that scientists should be biased toward simplicity in explanations. Throughout this six year controversy, Dr. Levitt has consistently gone for the most complicated, hard-to-understand, and (as we’ve seen this week) hard-to-check-upon models. Maybe that was necessary — social phenomena are terribly complicated. But the impact of his behavior on the public and on much of his profession has been to encourage _not_ a critical engagement with historical and sociological record, but an attitude of faith, a warm feeling that this really smart guy has Figured It All Out using Really Complicated Statistics and we should just take his word for it. As a marketing strategy, this oracular approach of “Freakonomics” has been mind-bogglingly successful, but perhaps I may be forgiven for not thinking it advances the cause of good social science.
— SteveSailerVocê leu aqui antes….
…e eu li no Marginal Revolution. Seguinte: encontraram uns furos econométricos no paper do Levitt sobre aborto e criminalidade. Imagino que isso vai fazer algum barulho na imprensa e no público em geral que não entende como a Ciência funciona….
— De Gustibus Non Est Disputandum[...] This isn’t the first time that Levitt’s conclusions have come under attack, but this is the first time a high-profile economist has taken on Levitt at length. Levitt says that he hasn’t changed his stance and that the abortion effect holds true in Canada and Australia. Levitt himself has responded to these charges on his blog, noting that he will post a lengthy response once he has fully parsed Foote and Goetz’s paper. For the moment, Levitt confesses that he’s embarassed that he forgot the pivotal data, but insists that the data still matches up “when you run the specifications we meant to run.” [...]
— Edward Champion’s Return of the Reluctant » Lies, Damned Lies and Freakanomics[...] Here’s Levitt’s response and a PDF copy of the “debunking” paper. [...]
— La Shawn Barber’s Corner » Steve Levitt Admits Error in ‘Freakonomics’ Abortion Theory“Oops-onomics”
Dec 1st 2005
From The Economist print edition
Did Steven Levitt, author of “Freakonomics”, get his most notorious paper wrong?
http://www.economist.com/finance/displayStory.cfm?story_id=5246700
— SteveSailer[...] Unlike Lott, they immediately admitted making the error. Nor did correcting the error make their results go away, though it did reduce the size of the effect. The Economist article has more on where that leaves their thesis, but what I find very interesting is Lott’s reaction to all this. He writes: [...]
— Deltoid » Lott, Levitt and coding errors[...] Anyway, in the book there is a chapter that unveils a hidden reason behind the fall in criminality in the 80s : the legalization of abortion. This is a very powerful argument against pro-lifers, and a testimony on how incredibly complex human interactions are. However, it appears the authors made errors in their computer code that could mean the results are meaningless ! The Horror ! In a nice example of how science corrects itself, Steven D. Levitt (the main author) explains the consequences of the error, which are fortunately less bad than expected. [...]
— Bloody Fingers » Blog Archive » Freakonomics errata