Here’s the latest guest post from Yale economist and law professor Ian Ayres. His past posts can be found here and here.
In a recent post, I mentioned that when playing poker, I use my watch as a crude random number generator to tell me when to bluff. While there are lots of sports in which it’s best to play a somewhat random strategy, that doesn’t mean that every possible play is equally likely. But it does mean, for example, that when it’s third-and-2 in football, the offense wants to have some possibility of passing to keep the defense honest.
Levitt and others have tested the degree to which professional tennis and soccer players are successful at playing randomized strategies. But it remains a mystery to me why coaches don’t have random number generators (any laptop would do) to help them pick the next pitch in baseball, or the next play they will call in football. Norv Turner would pick the probability of running or passing, and then let the computer decide which it would be.
But an even bigger puzzle is why teams don’t exploit the other powerful use of randomization. To my knowledge, no sports team in the history of humankind has ever run a random control trial to figure out which strategies work the best. (I make this extravagant claim in hopes of provoking you all into providing some counterexamples.) Randomized studies are the gold standard of medical testing, and they’re now the hottest thing in Internet ads.
Want to know whether your Web banner for beer should say “Tastes Great” or “Less Filling”? Run a randomized test in which half the people see one and half the people see the other at random, and then sit back and watch whether one ad generates more sales. I ran just this kind of test on Google Adwords to help choose the title of a book (shameless pitch) I was writing. When I started writing it, I loved the title, “The End of Intuition.” But in a randomized test, “Super Crunchers” had a 63 percent higher click-through rate.
So why don’t sports teams run (more) randomized experiments? The Boston Red Sox are famous for relying on number crunching to gain a competitive edge. But why don’t they proactively make some powerful data by creating randomized treatment and control groups? They could use their minor league teams, for instance, to figure out whether catchers or pitchers make better calls.
They could even have a randomized trial of randomization — they could randomly assign the pitches for half the at-bats to be called in the traditional way (by the coach or the catcher) and the other half could be called by a random strategy established in advance. It would be a double-blind study, because neither the pitcher nor the hitter would need to know which system called the pitch.
If it turned out that the random strategy reduced the batting average of your opponents, that would be pretty strong evidence that it was a better strategy.
Or you could run an experiment to find out whether football teams should go for it more often on fourth down. Economist David Romer has crunched numbers to suggest that professional football teams should go for it fourth down a lot more than they currently do. His proposed optimal strategy is summarized in the following graph, found at the end of his paper:
Amazingly, the data suggests that if it’s fourth down and your team has the ball on the opponent’s 33-yard line, you should go for it even if you have 9 yards left for a first down. NFL coaches have resisted Romer’s advice (though Pulaski Academy has started acting on it). But this is another area in which a little randomized testing could go a long way to help figure out what works. There are thousands upon thousands of college and high school games, but we collectively go for decades without figuring out whether simple changes in strategy could really produce better outcomes.
If you know of any randomized tests of sports strategy, please let me know. And if you are a coach and want to run a test, feel free to contact me. I’d be happy to help design and evaluate a test.
and 
. You can also watch it on 
Randomization has it’s limits though, doesn’t it? Each play is not necessarily as effective as others and therefore, by opting for a random alternative instead of a strategic one could illicit negative results that could cost the game.
In football, going for it on fourth down in your opponents territory would be a good thing because, the net gain from punting would be minimal and the more opportunities you have to score 7 points rather than three would more than likely increase thus increasing your chances to win. But going for it in your territory could reward your opponent with prime field position. I think a team like New England – with it’s high powered offense – can go for it on 4th down (and this year they have) a lot more than most. A struggling offensive team might not be so successful. If you’ve gained 6 yards on your first three plays – you might not want to risk gaining the needed 4 on one final play.
But in baseball, most starting pitchers have three strong pitches and two that aren’t as strong. While some hitters are guess hitters, most adapt to the actual pitch based on release point, speed, etc – which means that the effectiveness of the pitch isn’t based as much on the element of surprise as it is on the actual strength of the pitchers skill level. Having it go at random would leave a pitcher more likely to give up hits and runs, leaving his team more likely to lose.
In basketball – the current Golden State Warriors basically play as random as you can get on offense. They just shoot whenever they can – bunking conventional wisdom (ie: shooting in a 1 on 4 situation, shooting within 8 seconds on the shot clock, forcing the tempo when they don’t have “numbers”, etc) . They are currently 11-9, although 8-2 in their last 10.
Randomization has it’s limits though, doesn’t it? Each play is not necessarily as effective as others and therefore, by opting for a random alternative instead of a strategic one could illicit negative results that could cost the game.
In football, going for it on fourth down in your opponents territory would be a good thing because, the net gain from punting would be minimal and the more opportunities you have to score 7 points rather than three would more than likely increase thus increasing your chances to win. But going for it in your territory could reward your opponent with prime field position. I think a team like New England – with it’s high powered offense – can go for it on 4th down (and this year they have) a lot more than most. A struggling offensive team might not be so successful. If you’ve gained 6 yards on your first three plays – you might not want to risk gaining the needed 4 on one final play.
But in baseball, most starting pitchers have three strong pitches and two that aren’t as strong. While some hitters are guess hitters, most adapt to the actual pitch based on release point, speed, etc – which means that the effectiveness of the pitch isn’t based as much on the element of surprise as it is on the actual strength of the pitchers skill level. Having it go at random would leave a pitcher more likely to give up hits and runs, leaving his team more likely to lose.
In basketball – the current Golden State Warriors basically play as random as you can get on offense. They just shoot whenever they can – bunking conventional wisdom (ie: shooting in a 1 on 4 situation, shooting within 8 seconds on the shot clock, forcing the tempo when they don’t have “numbers”, etc) . They are currently 11-9, although 8-2 in their last 10.
I saw this on an earlier post about clock management, but it seems like video games, such as Madden, would be a perfect place to study this possibility in earnest. No simulation is perfect, but the benefit in increased number of games to test on is impossible to ignore.
I’m no economist, but couldn’t you run 10000 simulation games of identical teams playing each other (how about Vikings vs. Vikings, so Adrian Peterson is always on the field)? One team is programmed never to punt, the other to follow its built-in ultra-conservative mindset. Couldn’t we expect some interesting results?
I saw this on an earlier post about clock management, but it seems like video games, such as Madden, would be a perfect place to study this possibility in earnest. No simulation is perfect, but the benefit in increased number of games to test on is impossible to ignore.
I’m no economist, but couldn’t you run 10000 simulation games of identical teams playing each other (how about Vikings vs. Vikings, so Adrian Peterson is always on the field)? One team is programmed never to punt, the other to follow its built-in ultra-conservative mindset. Couldn’t we expect some interesting results?
Good point Barb. Given what everyone knows about the Patriots’ offense, why did Tomlin kick a field goal when the Steelers were down 14-10 in the first half of the Patriots’ game? Did he have that much faith in his defense? In fact, consensus is that the Patriots have a defense that is merely good, with aging players who might not stand up to really getting pushed around in a game. So why kick a field goal when your offensive line was stilling show signs of winning the line-of-scrimmage battle?
Good point Barb. Given what everyone knows about the Patriots’ offense, why did Tomlin kick a field goal when the Steelers were down 14-10 in the first half of the Patriots’ game? Did he have that much faith in his defense? In fact, consensus is that the Patriots have a defense that is merely good, with aging players who might not stand up to really getting pushed around in a game. So why kick a field goal when your offensive line was stilling show signs of winning the line-of-scrimmage battle?
I thought the Red Sox were famous for spending lots of money to gain a competitive edge?
I thought the Red Sox were famous for spending lots of money to gain a competitive edge?