The NBA draft this year provides a vivid real world test of whether very simple regressions can out-predict experts on a central business decision — the NBA draft.
Chris Doughty, a 2008 industrial and operations engineering graduate of the University of Michigan, pointed me to a cool regression analysis of the great John Hollinger. Using data from current NBA players, Hollinger sees how well the players’ college stats explain their subsequent performance in the NBA.
For example, here’s his analysis of the 2006 draft ranked in descending order of predicted play:
Hollinger’s regression analysis suggests that Rudy Gay, Marcus Williams, and newly minted NBA champion Rajon Rondo should have been the top three picks — where as the actual top three were Adam Morrison (who Hollinger rates as 14th best player), Brandon Roy, and Randy Foye.
Comparing the NBA success of his system’s predictions to the actual draft, Hollinger concludes, “while the system isn’t perfect, it’s a clear improvement on what actually took place.” Even armed with this statistical analysis, many teams continue to go with their gut.
But Hollinger goes further and applies his regression results to the current crop of players in the draft. Here are the regression’s rankings for this year’s draft:
A natural experiment is for us to wait a few years and see whether (once again?) the equation beats the expert.
It’s particularly interesting to see what happens to players like Darrell Arthur who had very different Hollinger and actual draft ratings. Hollinger ranked Arthur 3rd in projected PER (player efficiency rating) but Arthur was taken 27th. (Donte Greene, Kosta Koufus, Roy Hibbert, and Marreese Speights were also undervalued by the regression’s lights).
Instead of predicting the 3rd year PER, I’d also be interested in seeing how college stats predict the amount the player gets paid in his second NBA contract.
Regressions can also help assess which humans are the best prognosticators. Hollinger could also add the predictions of various long-time scouts into his regression and see which scouts best predict future NBA success of draft prospects.
and 
. You can also watch it on 
RE: Colin
Hollinger’s system was applied to college players, not high school or international and they are listed in the order taken as college players (not actual overall draft position). In that particular posting, it lists only perimeter players..which could have been more clearly spelled out.
If i understand it correctly you are using data from the prior season’s draft/regular season record to predict the player’s performance subsequent to the draft. Why wait for a full season? You could figure out if this had any value by doing the regression on prior seasons… too much work?
Drafts are as simple as a ranking of skill level. There is also the question of what a particular team needs, what you don’t want you opponents to get, and all other forms of game theory dynamics.
The name on that list that really stands out is Darnell Jackson, who didn’t go until the 2nd round. THAT would be a great test of this analysis. I would think being able to pick 1st round talent in the 2nd round would be not only easier, but more valuable as well. Or maybe the value in this system is finding players to stay away from. Use the outliers, both positive and negative, as your guide.
Hollinger’s regression is, I assume, based on past seasons, including 2006. So, to oversimplify things, this chart uses data from past seasons to predict outcomes from past seasons. In other words, Hollinger’s regression should predict the best players from past drafts well because that’s what it was based off of; that doesn’t necessarily mean it will predict this year’s draft well. (Now, Hollinger’s a smart guy and it’s a good concept, so this system should be just fine.)
Hollinger is using his special stat (PER) on college careers to predict how those players will do at his special stat (PER) in the pros.
You can’t say Hollinger is “better” at predicting success, WHEN YOU ONLY USE HIS MEASURE OF SUCCESS. Right or wrong, GMs aren’t drafting to PER.
If my little brother predicts the new Bruckheimer film will be better than The Godfather because it has more explosions, and then it does have more explosions, does that make my little brother better at predicting which movies will be good?
Lets jump the gun and look at 2nd year PER for the 2006 perimeter players listed in John’s chart:
Gay – 17.51
M Williams – 11.06
Rondo – 15.63
S Williams (this is the ‘Williams’ in the ‘actual order column) – 12.83
Roy – 19.44
Farmar – 15.43
Adams – DNP
Balkman – 11.55
Morrison – DNP
Foye – 12.77
Redick – 12.78
Brewer – 8.29
Carney – 12.15
In this projection, where being off by 15% is the difference between being a top five pick and a late first round selection, Hollinger was off by 15% on Marcus Williams, Roy, and Balkman, 3 of the 7 players in his top 8 that played in the NBA last season. While I don’t put a lot of stock in the PER rating (which undermines the usefulness of this analysis), a redraft based on last season’s PER would put perimeter players in this order:
Roy
Gay
Rondo
Farmar
S. Williams
Redick
Foye
Carney
Comparing this ordering to the chart above, I don’t think the Hollinger prediction was better than the actual draft. Both had a stinker at the top (Morrison/Marcus Williams). And the actual draft captured 6 of the top 8 2007-2008 perimeter players with the first 8 selections of perimeter players in the 2006 draft.
This article would make a whole lot more sense if the author took the two lists on the 2006 table and actually compared the player’s production based on the order of listing to come up with an objective measure of performance. But just saying “it’s a clear improvement on what actually took place” is ridiculous. It’s not clear to me at all. Aside from Morrison I’d wager the draft order is a better order than Hollinger’s based on a fan’s knowledge.
And then doing this with the ACTUAL draft and not a guard sampling of it would be even better. This seems to me a pretty lazily written piece.