Or at least that is the impression you might get if you read this article in today’s Wall Street Journal.
I will post a longer blog entry once I have had time to fully digest the working paper by Foote and Goetz which is the basis for the article.
For now, I will say just a few things:
1) It is not at all clear from the WSJ article is that Foote and Goetz are talking about only one of the five different pieces of evidence we put forth in our paper. They have no criticisms of the other four approaches, all of which point to the same conclusion.
2) There was a coding error that led the final table of my paper with John Donohue on legalized abortion to have specifications that did not match what we said we did in the text. (We’re still trying to figure out where we went wrong on this.) This is personally quite embarrassing because I pride myself on being careful with data. Still, that embarrassment aside, when you run the specifications we meant to run, you still find big, negative effects of abortion on arrests (although smaller in magnitude than what we report). The good news is that the story we put forth in the paper is not materially changed by the coding error.
3) Only when you make other changes to the specification that Foote and Goetz think are appropriate, do the results weaken further and in some cases disappear. The part of the paper that Foote and Goetz focus on is one that is incredibly demanding of the data. For those of you who are technically minded, our results survive if you include state*age interactions, year*age interactions, and state*year interactions. (We can include all these interactions because we have arrest data by state and single year of age.) Given how imperfect the abortion data are, I think most economists would be shocked that our results stand up to removing all of this variation, not that when you go even further in terms of demands on the data things get very weak.
Again, as I said, I will post again on this subject once I have had a chance to carefully study the details of what they have done, and after I have been able to go back to the raw data and understand why the results change when one does what Foote and Goetz do.
and 
. You can also watch it on 
Ever since I debated Dr. Levitt in Slate.com in 1999, people have been telling me that my simpleminded little graphs and ratios of national crime trends showing that Dr. Levitt hadn’t met the burden of proof couldn’t possibly be right because Dr. Levitt’s state-level evidence was so much more gloriously, glamorously, incomprehensibly complicated than mine, and Occam’s Butterknife says that the guy with the most convoluted argument wins.
Well, now we now why Dr. Levitt’s abstruse state-level analysis didn’t match up with my straight-forward national-level analysis: because he made two big mistakes in his work.
Ever since I debated Dr. Levitt in Slate.com in 1999, people have been telling me that my simpleminded little graphs and ratios of national crime trends showing that Dr. Levitt hadn’t met the burden of proof couldn’t possibly be right because Dr. Levitt’s state-level evidence was so much more gloriously, glamorously, incomprehensibly complicated than mine, and Occam’s Butterknife says that the guy with the most convoluted argument wins.
Well, now we now why Dr. Levitt’s abstruse state-level analysis didn’t match up with my straight-forward national-level analysis: because he made two big mistakes in his work.
Would it be appropriate to calculate powers for the various specifications for which Foote and Goetz find insignificant coefficients on abortion to determine exactly how demanding of the data the specifications are?
Also, by saying that the specifications are too demanding what assumptions are you making in your model by not including the state-age-year interactions?
Would it be appropriate to calculate powers for the various specifications for which Foote and Goetz find insignificant coefficients on abortion to determine exactly how demanding of the data the specifications are?
Also, by saying that the specifications are too demanding what assumptions are you making in your model by not including the state-age-year interactions?
“For those of you who are technically minded, our results survive if you include state*age interactions, year*age interactions, and state*year interactions.”
Including interaction terms is standard in data analysis BY UNDERGRADUATE PSYCHOLOGY MAJORS but you think it’s high tech. What you consider “technically advanced” and “incredibly demanding of the data” is something psychologists teach undergraduates to do in introductory research methods classes.
Thanks! In just one sentence you do more to support the hypothesis “Whoever called economics the dismal science got it half right” than anyone in recent history.
“For those of you who are technically minded, our results survive if you include state*age interactions, year*age interactions, and state*year interactions.”
Including interaction terms is standard in data analysis BY UNDERGRADUATE PSYCHOLOGY MAJORS but you think it’s high tech. What you consider “technically advanced” and “incredibly demanding of the data” is something psychologists teach undergraduates to do in introductory research methods classes.
Thanks! In just one sentence you do more to support the hypothesis “Whoever called economics the dismal science got it half right” than anyone in recent history.
Technically advanced clearly refers to being technically advanced for the blog audience not for economists. Incredibly demanding of the data is not difficulty in performing the computations, it is having enough variation to have a reasonable chance at finding significant results.
Technically advanced clearly refers to being technically advanced for the blog audience not for economists. Incredibly demanding of the data is not difficulty in performing the computations, it is having enough variation to have a reasonable chance at finding significant results.