Here are two interesting follow-ups to Tuesday’s post, in which I described how basketball teams who are behind at half-time fare a bit better than might be expected.
First, my friend Lionel Page points me to a related study of his, which analyzes tennis. Lionel uses a similar approach to arrive at a different conclusion, but I think his results point to the same psychological factors. Let me explain.
Lionel analyzes over half a million tennis matches, but focuses on that subset of matches where the first set is played to a tie break (there are about 70,000 of them). In fact, he focuses on those matches where the first set is awfully close to a draw — a long and drawn-out tiebreak, lasting more than 20 points. But eventually someone wins, and it turns out that player is much more likely to win the second set. And so, winning leads to winning in tennis, but losing leads to winning in basketball.
Are these results in conflict? I don’t think so: the effect of falling a little behind in a tennis tiebreak is very different to basketball. In tennis, falling behind in a tiebreak means losing the entire set, which is a big hurdle to overcome. Recall that Berger and Pope ran laboratory experiments in which they tried to figure out the (de-) motivating effect of being a little behind, a long way behind, tied, a little ahead, or a long way ahead. They found that being a little behind led to a burst of extra motivation, which describes the basketball finding nicely. But being a long way behind led to no extra motivation, and so the tennis player who just lost the first set doesn’t get these motivating benefits. In fact, Lionel’s finding suggests that it might actually be de-motivating. Because basketball scores are continuous, and tennis outcomes are so sharply discrete, the psychological impacts of small differences in early play are very different.
Second, Freako commenters showed that they are among the most demanding consumers of statistical analyses, and were pretty vocal in what they perceived to be shortcomings in the Berger-Pope analysis (and see more from Andrew Gelman, here). I passed on your comments to the authors, and here’s their response:
In our sample of N.C.A.A. basketball games, teams that were losing by one point at halftime were more likely to win the game than teams winning by one point. This is indisputable. However, is this finding statistically significant given the noise in the data?
The key issue here is understanding exactly what hypothesis we are testing to see whether losing by a small amount increases motivation. Directly comparing the winning percentage of teams down by one with teams up by one is problematic because these are different types of teams in different situations. Teams are not randomly assigned to be up or down by one. On average, teams down by one tend to be worse (they have a lower season winning percentage). Furthermore, it is mechanically harder for teams down by one to win. They have to score at least two more points than their opponent to win, while their opponents can win even if the teams trade baskets the rest of the game.
This means that we shouldn’t expect teams down by one to win 50 percent of games. What should be expected? This is where reasonable people may begin to differ on the right way to construct a counterfactual. Many different curves can be fitted to the data. One may argue (as many did in the comments) that a linear line should be fitted; Andrew Gelman suggests a logistic function. It ends up that it doesn’t really matter what curve is fit.
For example, consider the figure below (the exact figure requested by Andrew Gelman) which indicates the winning percentage for the home team as the halftime point difference for the home team ranges from -10 to 10. Also, note the inclusion of standard error bars for sophisticated readers. The dotted line represents the fitted curve from a simple logistic function when including the halftime score difference linearly. Focus on the winning percentage when either the away team was losing by a point, or the home team was losing by a point. In both of these situations, the losing team did better than expected. For example, when the home team is ahead by one point, they end up only winning 57.5 percent of games while we would have expected them to win 65.6 percent of games. This difference in actual versus expected performance (8.1 percent) is statistically significant at conventional levels and provides evidence in favor of our hypothesis that losing can be motivating.
This difference persists when controlling for home-team advantage, possession arrow to start the second half, prior season winning percentage, and team fixed effects (see Table 1 of the paper).
Further, supplementary analyses show that teams losing by one point closed the gap the most in the first few minutes after halftime (supporting our motivation hypothesis). Laboratory studies, using random assignment, also demonstrate that merely telling people they are slightly behind halfway through a competition leads them to exert more effort.
Taken together, these findings indicate that being slightly behind motivates people to work harder and be more successful.
Finally, let me say just why I like this paper. It’s easy to mine sports data to find interesting anomalies, but sometimes it is hard to see what it means. And it is easy to get students in an experimental setting to do weird things that have no relevance to the real world. It is the juxtaposition of suggestive data from the field with a well-designed experiment that leads me to conclude there’s some interesting social science here.
My friends at the Association for Professional Basketball Research have collected some interesting discussion threads on this controversy, here, including data suggesting a similar pattern in the NBA.
and 
. You can also watch it on 
Tennis and Basketball are also fundamentally different because one is a team sport typically played in a season and the other is an individual sport typically played in tournaments.
There is a motivation for a basketball coach to play his team at less than 100% in every game except the championship, because he wouldn’t want to risk injuring his best players. In other words; if you’re coaching the old-school Bulls and your up by enough, why even put Jordan in the second half? Particularly because a single loss won’t ruin your season.
What motivation would a tennis player have to ever play at less than 100%, when if they lose any match they are out of the tournament?
This graph is much, much better.
However, there are several other points that deviate from the fit by comparable amounts to the +1 and -1 points.
Agree with comments 1 and 10. I think the author is missing the important point. This plot, though it represents a great improvement over the original, does not present compelling evidence that the regression discontinuity model does a better job of explaining the data than the logistic model. Several points, not just +/- 1, appear to be outliers in want of explanation.
Chad Orzel wrote an extremely good response excoriating the earlier graph:
http://scienceblogs.com/principles/2009/03/teams_who_are_ahead_win_more_f.php
About the second paragraph, kman, I have my reserves. Because basketball`s own dynamics, extra possession really doesn`t matter because of the steals in a game, which can turn a game around in a matter of seconds, even between possessions.
In this sense, I think a good variable to consider is the amount of steals in a game for the winning teams that trail by one point. I haven’t read the paper yet and don’t know if this statistic is considered, but I think a motivated team would increase efforts when trailing by one point, so the amount of steals (and maybe rebounds) will increase as well.
The study itself may well be statistically accurate and the authors may have a valid conclusion, at least with respect to the dynamics of NCAA basketball games. Many of the comments to the original article, mine included, point out suspected statistical errors. In truth, my statistical skills are too rusty for me to fairly evaluate the technical merits of the study. The authors are much better statisticians than I and probably much better than most of the others who commented. They will certainly have reasonable responses to any technical objections to the statistics, as today’s article demonstrates.
On the other hand, it is clear to me is that the authors very deliberately presented the data so as to emphasize their conclusion. They consciously decided to exclude 0 from the Trend line data. They used a 5th order polynomial when a lower order polynomial probably would have fit just as well but would not have shown such a dramatic discontinuity. They projected the Trend lines to -.5 and +.5 when they had actual data for -1, 0, and 1, again increasing the appearance of a discontinuity. In graph 1B of the academic paper they chose to display the range -3 to +3, which just happens to be the range that maximizes the appearance of discontinuity. While their conclusion may be perfectly valid statistically, the added emphasis in the presentation suggests authors who are more interested in supporting their thesis than objectively reporting the results of their study.
The authors would be well served to recall that statistical significance does not prove causation in fact. At most, the data suggests there might be a very small winning effect from being behind by one. Given the weakness of the findings and noting that there are other outlier points on the graph which are not explainable by the authors’ thesis, it seems that the most that could be said is that there is a possibility that something interesting might be going on.
On the other hand this might just be noise even if it is statistically significant. The most that should come out of these studies is an observation that further study might be fruitful. Instead, the authors trumpeted their conclusion in the New York Times and claimed that it has significant application to situations that are far beyond the scope of their study. Perhaps they are so “bright” that they see significance in the results that the rest of us aren’t smart enough to see. Or perhaps they were lured by the prospect of a catchy headline and failed to see how marginal their conclusion really might be. That is not the standard that I would hope for from “two of the brightest young behavioral economists around”.
I have 25,000 college basketball games in my database, with scores in the halves as well as pointspread lines. I do not have the same results as the authors of this working paper. Here are my resutls (no home/away breakdown)
Team down by 5 wins game 27.4%
down by 4 wins 34.9%
down by 3 wins 38.1%
down by 2 wins 42.5%
down by 1 wins 48.0%
tied wins 50%
There were an average of 975 games in each line.
Then I looked at games where the pointspread for the game was 7.5 or less. There were fewer games – only about 640 games per line.
Team down by 5 wins game 29.7%
down by 4 wins 38.1%
down by 3 wins 40.9%
down by 2 wins 45.0%
down by 1 wins 48.1%
tied wins 50%
I think I have more data and games than the authors of the paper, but they have more information on each game. It is possible my data is flawed, I am not in academics so I am not as worried about accuracy as people in academics may be. However I do my best to gather accurate data and I have no reason to believe my data is not accurate.
For the academics out there, what are the chances that the data in the working paper and the data I have can be reconciled by the difference in sample size?.
“what are the chances that the data in the working paper and the data I have can be reconciled by the difference in sample size?”
Pretty good. Brian Burke, using another NCAA dataset, also finds that +1 teams win about 52% of the time. See his article with link to spreadsheet at Wages of Wins:
http://dberri.wordpress.com/2009/03/05/modeling-win-probability-for-a-college-basketball-game-a-guest-post-from-brian-burke/.
I think it probably is true, looking at all this data, that a 1 point lead is not half as valuable as a two point lead. But it’s very unlikely that being one point behind is actually better than being tied or up one. And to the extent there is a “-1 effect,” it’s not at all clear this has anything to do with “effort.”