Yesterday, I posted about the conclusions that Eric Bradlow, Shane Jensen, Adi Wyner, and I drew from analyzing Roger Clemens‘s career statistics. I thought that it might be useful to show how we got from the findings in the Clemens Report (exonerating him), to our somewhat opposite conclusions. So for budding forensic economists, here is a step-by-step guide, with pictures.
1. The Raw Data
The “Clemens Report” mainly analyzes his earned run average through time. These numbers appear to show no reliable pattern, as they bounce around a lot from season-to-season. At this point, it is hard to see any particularly interesting pattern in the data.
2. An alternative metric, and a fitted curve
The problem with analyzing ERA is that it is affected by a lot of things beyond pitching quality. For instance, defense affects a player’s ERA, and poor pitching is not much impacted if there happen to be no runners on base. Instead, we turn to a more reliable metric – walks plus hits per inning pitched. This metric yields less “bounce,” and a more reliable pattern is revealed. Fitting a curve, we find that Clemens’ performance deteriorated for about a decade, then started to improve for the last decade of his career.
The turning point appears to be at around the age (36-37) in which the Mitchell report suggests he used performance-enhancing drugs. When we analyze other summary measures of his pitching performance, we see a roughly similar pattern, although some look more suspicious and some less suspicious.
3. Creating a Comparison Group
To figure out whether Clemens’s performance is unusual, we needed to compare his career trajectory with other durable pitchers. The Clemens report analyzed Nolan Ryan, and this was a wise choice: Ryan’s performance also improved in the final decade of his career.
But a useful comparison group should involve many other pitchers who have also had long and successful careers. When we examine all 30 other pitchers who, since 1967, have started in at least 10 games in 15 seasons with 3000 innings pitched, we see a pervasive pattern: nearly all of them improve for about a decade, and then their performance deteriorates in the second half. The exceptions to this rule are those pitchers who simply tend to simply get worse through time – and this looked to be Clemens’s trajectory until his mid-30s.
But overall, Clemens’ path looks “upside-down,” as he gets worse first, and then improves later.
4. Clemens’ Career Versus Other Pitchers
We fit a curve that describes the typical career of a durable starting pitcher. Think of this as being like the “control group” in a medical study. Clemens’s career arc looks very different than our control group, suggesting something unusual occurring.
Unfortunately, our statistical analysis cannot pinpoint the precise cause of this unusual pattern. But it is clear that the Clemens report stretches credibility in arguing that his late career was typical. His late-career performance certainly was quite exceptional given the trajectory that he was on in the first half, suggesting that close scrutiny is warranted.
and 
. You can also watch it on 
yeah I’d like to see your r squared values too
Can you post the raw numerical data here, or on your website?
Using the curve fit seems inappropriate. You’re more likely to be fooled by randomness in this case than find evidence that he used enhancers.
It’s fun to look into performance metrics, but aren’t there easier (and much more accurate) ways of establishing whether or not he used drugs?
A previous commenter mentioned it – but why hasn’t it been taken into consideration that Clemens moved to the NL. A pitcher faces considerably less talent in the NL overall – not to mention facing a pitcher (usually a very poor hitter) 1 out of every 9 at bats.
He also wasn’t putting in a full work load. His partial seasons allowed him to stay stronger for longer.
I think something that would be more representative of whether he declines is the speed of his fast ball. If he was truly using Performance enhancing drugs – wouldn’t his fast ball have picked up MPH? Especially when you consider how much he trains – performance enhancing drugs would, i would opine, allow him to not only negate the effects of aging but overcome them and his fast ball should have picked up some mph that he would have lost otherwise.
I don’t know the stats – but being a Yankee fan – Roger’s fastball had definitely been waning even when he first came here. He just seemed to pitch smarter. But I’ll still await the evidence
What about normalizing Clemens’ WHIP by taking the average of all AL pitchers that year. That may remove variables like a worse crop of hitters or a larger strike zone during the last ten years.
And normalize by park too. Ten years ago is when Clemens left Fenway. Considering the fact that Fenway is a hitter-friendly park, this may actually show a more dramatic turn-around in his pitching.
The point of Clemens’ argument is that while he may have performed better than average, there are some other pitchers with similar career trajectories who were not accused of cheating. I happen to think he’s guilty, but this analysis is not well-done. If Nolan Ryan and other pitchers can steadily improve, this demonstrates it is possible, and that Clemens’ career, while a minor outlier, is not an extreme outlier. On figure 4 above, there are two additional curves, one starting just below where Ryan’s starts and the other ending just below where Clemens and Johnson are nearly colinear, show similar downward-trending shapes.
WHIP has some issues with it, just like ERA. A lot goes into the Hits aspect, including ballpark factors. Try comparing K/9 (in particular) and BB/9 across the pitchers. It eliminates most outside influences and shows that Clemens has maintained his K/9, while guys like Schilling and Johnson have had steep declines since the age of 37/38.
BB/9 has decreased significantly for Schilling and Johnson, while it’s maintained for Clemens.
Off the top of my head, I can see a bunch of things wrong here:
WHIP, like ERA, is subject to many outside factors, defense, park, league, etc. Using WHIP doesn’t make them go away.
Those curves seem to have only a very loose correlation to the actual WHIP, what is the confidence level that there is an _actual_ inflection point.
You have data points at 45 and 46 for Clemens. By standard reckoning (ie, age at the beginning of the season), Clemens did not have an age 45 season. Also, he has not yet turned 46.
I don’t think the Hendricks brothers ever claimed that Clemens was ‘typical’ in any sense. They simply claimed that his longevity was not wholly unprecedented. Put another way, if you were to compare Ruth’s home run hitting patterns to every player that played before him, you could conclude that ‘something unusual [was] occurring’