Yesterday, I posted about the conclusions that Eric Bradlow, Shane Jensen, Adi Wyner, and I drew from analyzing Roger Clemens‘s career statistics. I thought that it might be useful to show how we got from the findings in the Clemens Report (exonerating him), to our somewhat opposite conclusions. So for budding forensic economists, here is a step-by-step guide, with pictures.
1. The Raw Data
The “Clemens Report” mainly analyzes his earned run average through time. These numbers appear to show no reliable pattern, as they bounce around a lot from season-to-season. At this point, it is hard to see any particularly interesting pattern in the data.
2. An alternative metric, and a fitted curve
The problem with analyzing ERA is that it is affected by a lot of things beyond pitching quality. For instance, defense affects a player’s ERA, and poor pitching is not much impacted if there happen to be no runners on base. Instead, we turn to a more reliable metric – walks plus hits per inning pitched. This metric yields less “bounce,” and a more reliable pattern is revealed. Fitting a curve, we find that Clemens’ performance deteriorated for about a decade, then started to improve for the last decade of his career.
The turning point appears to be at around the age (36-37) in which the Mitchell report suggests he used performance-enhancing drugs. When we analyze other summary measures of his pitching performance, we see a roughly similar pattern, although some look more suspicious and some less suspicious.
3. Creating a Comparison Group
To figure out whether Clemens’s performance is unusual, we needed to compare his career trajectory with other durable pitchers. The Clemens report analyzed Nolan Ryan, and this was a wise choice: Ryan’s performance also improved in the final decade of his career.
But a useful comparison group should involve many other pitchers who have also had long and successful careers. When we examine all 30 other pitchers who, since 1967, have started in at least 10 games in 15 seasons with 3000 innings pitched, we see a pervasive pattern: nearly all of them improve for about a decade, and then their performance deteriorates in the second half. The exceptions to this rule are those pitchers who simply tend to simply get worse through time – and this looked to be Clemens’s trajectory until his mid-30s.
But overall, Clemens’ path looks “upside-down,” as he gets worse first, and then improves later.
4. Clemens’ Career Versus Other Pitchers
We fit a curve that describes the typical career of a durable starting pitcher. Think of this as being like the “control group” in a medical study. Clemens’s career arc looks very different than our control group, suggesting something unusual occurring.
Unfortunately, our statistical analysis cannot pinpoint the precise cause of this unusual pattern. But it is clear that the Clemens report stretches credibility in arguing that his late career was typical. His late-career performance certainly was quite exceptional given the trajectory that he was on in the first half, suggesting that close scrutiny is warranted.
and 
. You can also watch it on 
Thanks for the analysis. I am not going to say that I know all the details in question, specifically the periods of Clemens career that he allegedly used HGH, but I do see some interesting trends in your first and second charts (all trend lines aside).
Both the first and second chart appear to have an improvement in performance from the age of 38 to 45. During that time his ERA and WHIP almost consistently improved year-over-year. Those years would be 1998 to 2006, which coincides with much of the “Steroids Era” and his stint with the New York Yankees.
I would be interested to see statistics for your “other durable pitchers” during the same ages…
Did you consider the fact that Clemens switched from a better hitting league to a worse hitting league later in his career?
The quadratic curve fit on Clemens’ data looks a little simplistic – there’s lots of bouncing around. But it does appear to show a dramatic improvement from age 35 to 36, as well as steady improvement from 38 to 45.
Is there any data on his pitching speeds over time?
I am not a big Clemens fan, but I have to disagree with the use of an “Average Durable Pitcher”. Ryan and Clemens are unique among old school pitchers in that they put an emphasis on strength training, particularly leg and core strength. This training appears to prolong the career of pitchers.
The question of how Clemens was able to keep his strength is debatable, but Ryan was also able to do it, and there is no indication that Ryan used enhancers to do it.
My only issue is that your Clemens curve appears to have a very poor fit. The distances from the line to the data, or the residuals, are large. i’d like to see how good that fit really is. Mind you, I still think he cheated, but I doubt it had that great of an effect.
Clemens worked harder than any pitcher that lived before him. Also, he benefited from legal advances in technology (B12, greenies, etc.).
How about r^2 values for those curve fits? To my eye, it looks like the WHIP curve fits just as badly as a curve drawn through the ERA data.
While this a great article in the abstract, I don’t believe these numbers, either in the form of the ERA or the WHIP, are conclusive. This is as a physicist looking at data, not a Clemens fan or detractor.
The data do not appear to be inconsistent with a zero or even negative curvature fit. This is only based on my experience; I haven’t done the calculation. However, if you provide the data, I will certainly give it a try.
I’d like to see the error bars on the data points (Clemens didn’t pitch exactly the same number of pitches every year, so each point represents only sample of his ability over time) as well as the uncertainty of the quadratic fit before saying Clemens career looks typical or not.