My son took the SSAT exam this past Saturday. And while I was sitting in the Choate athletic facility waiting for him to finish, I remembered that Avinash Dixit and Barry Nalebuff‘s new book, The Art of Strategy, has a great example concerning standardized testing. Game theory is so powerful it can help you figure out the correct answer without even knowing what the question is.
Consider the following question for the GMAT (the test given to MBA applicants). Unfortunately, issues of copyright clearance have prevented us from reproducing the question, but that shouldn’t stop us.
Which of the following is the correct answer?
a) 4p sq. inches
b) 8p sq. inches
c) 16 sq. inches
d) 16p sq. inches
e) 32p sq. inches
O.K., we recognize that you’re at a bit of a disadvantage not having the question. Still, we think that by putting on your game-theory hat you can still figure it out.
Before reading their analysis, take a shot at trying to reason your way to the correct answer.
Here’s what they said:
The odd answer in the series is c. Since it is so different from the other answers, it is probably not right. The fact that the units are in square inches suggests an answer that has a perfect square in it, such as 4p or 16p.
This is a fine start and demonstrates good test-taking skills, but we haven’t really started to use game theory. Think of the game being played by the person writing the question. What is that person’s objective?
He or she wants people who understand the problem to get the answer right and those who don’t to get it wrong. Thus wrong answers have to be chosen carefully so as to be appealing to folks who don’t quite know the answer. For example, in response to the question: “How many feet are in a mile?” an answer of “Giraffe,” or even 16p, is unlikely to attract any takers.
Turning this around, imagine that 16 square inches really is the right answer. What kind of question might have 16 square inches as the answer but would lead someone to think 32p is right? Not many. People don’t often go around adding p to answers for the fun of it. “Did you see my new car — it gets 10p miles to the gallon.” We think not. Hence we can truly rule out 16 as being the correct solution.
Let’s now turn to the two perfect squares, 4p and 16p. Assume for a moment that 16p square inches is the correct solution. The problem might have been: “What is the area of a circle with a radius of 4?” The correct formula for the area of a circle is pr2. However, the person who didn’t quite remember the formula might have mixed it up with the formula for the circumference of a circle, 2pr. (Yes, we know that the circumference is in inches, not square inches, but the person making this mistake would be unlikely to recognize this issue.)
Note that if r = 4, then 2pr is 8p, and that would lead the person to the wrong answer of b. The person could also mix and match and use the formula 2pr2, and hence believe that 32p or e was the right answer. The person could leave off the p and come up with 16 or c, or the person could forget to square the radius and simply use pr as the area, leading to 4p or a. In summary, if 16p is the correct answer, then we can tell a plausible story about how each of the other answers might be chosen. They are all good wrong answers for the test maker.
What if 4p is the correct solution (so that r = 2)? Think now about the most common mistake: mixing up circumference with area. If the student used the wrong formula, 2pr, he or she would still get 4p, albeit with incorrect units. There is nothing worse, from a test maker’s perspective, than allowing the person to get the right answer for the wrong reason. Hence 4p would be a terrible right answer, as it would allow too many people who didn’t know what they were doing to get full credit.
At this point, we are done. We are confident that the right answer is 16p. And we are right. By thinking about the objective of the person writing the test, we can suss out the right answer, often without even seeing the question.
Now, we don’t recommend that you go about taking the GMAT and other tests without bothering to even look at the questions. We appreciate that if you are smart enough to go through this logic, you most likely know the formula for the area of a circle. But you never know. There will be cases where you don’t know the meaning of one of the answers or the material for the question wasn’t covered in your course. In those cases, thinking about the testing game may lead you to the right answer.
If you want a fun way to learn a ton of useful game theory, this is the book for you. How good is it? Steve Levitt has a blurb on the book saying it’s so good, he read it twice.
and 
. You can also watch it on 
There is an SAT guide written specifically for smart kids on how to get a better score called “Up Your Score.” It was written by a bunch of kids from Ithaca HS in Ithaca NY. They have a guessing strategy that is much simpler but based in the same general theory, that still gets you to 16pi. Their strategy is to pick the answer that is most like the other answers.
So looking at the answers above, you would immediately eliminate c) because all the other answers contain pi. However, 16 also appears in 2 answers (none of the other numbers appear more than once), so you would guess 16pi as the right answer. That takes about 2 seconds to do- much quicker than Dixit & Nalebuff’s strategy, which although more thorough, wastes precious time that you could be using to figure out questions that you don’t need to blindly guess an answer.
You can immediately throw out c because it doesn’t have a pi, but it also indicates that 16 is in the answer. You see this all the time in standardized tests. Also if you add the coefficient of the first answer to itself you get the second answer and if you square it you get the 3rd. You see this a lot as well and it usually means that one of the two are correct. Since we are talking about squares and with the help of answer C, you can reasonably assume the answer is D. It’s not as detailed as your description, but a hell of a lot faster. I tihnk it would be fun to take the SATs without the questions.
I have to agree with some previous commentators. The technique would not be usable in an actual test because 1) to make it work you need to understand the common errors test-makers are looking for. If your understanding is that deep, you don’t need the game-theory. And 2) it would take way to long to answer a significant number of questions this way.
It is interesting to think of tests this way, and to try to figure out the correct answer without seeing the question. However, could there be alternative questions which lead to different correct answers within the same set of choices? Is ‘d’ the exclusively correct answer?
Ahhhh, Up Your Score. I usually say The Princess Bride is my favorite book ever, but Up Your Score might top it. My father bought it for me when I was in middle school because for some reason I got to take the SAT’s in 8th grade to see if I qualified for some smart-kids program at Johns Hopkins. I didn’t. But I learned a ton from that book, and read it probably once a year throughout high school. I’d bet it added 50 points to my SAT score. It’s such a smart, well-written book, with some pretty good humor in it too, to keep it interesting. And it’s probably going to improve one’s SAT score by more than a Kaplan course (or the like). When you figure it’s also roughly 1/50th of the cost, it’s a no-brainer!
I like the idea that this broken, farcical test can be broken, but I believe any high schooler with an understanding of game theory would probably do OK on his/her own.
And #2 and #3 are correct… if you used the correct formula to inform this decision making process, you probably should have just used it to solve the problem in the first place. Hence, the Up Your Score strategy (which I call the Sesame Street “one of these things is most like the other”) is the quickest and easiest.
Up Your Score: 1, Freakonomics: 0
If you’re using good test taking strategy and skipping what you don’t know this method is indeed useful, and there is plenty of time available contrary to what the posters have posted. It takes much longer to read it than think it through.
It’s amazing how many simple, little standardized test tricks there are. When used together, you do not have need anywhere near a mastery of the material to ace the exam.
Of course it helps if you have above average intelligence to make connections that the average person will miss.
I remember teaching myself math in high school by reverse solving the questions on the exams (I wasn’t a good student and often slept through class).