The movie “Snakes on a Plane” had enormous internet buzz before being released , but fizzled at the box office.
This has led to a great deal of discussion in both the traditional media and online about what does or does not make internet buzz translate into commercial success.
One reasonable answer to that question may be that when the buzz is faked/manufactured, commercial success will not follow.
Was the buzz around “Snakes on a Plane” artificially manipulated by people involved with the movie? Economist Cyril Morong, who teaches at San Antonio College, thinks the answer to that question may be “yes.” He has assembled some data to try to make his point.
What follows is his analysis, in the form of an email he sent to me and consented to have me post here on the blog.
From Cyril:
Over at the Internet Movie Database (IMBD), you can rate a movie from 1 to 10 with 10 being the best. One commentor said it was strange how the percentage of people giving the movie [Snakes on a Plane] 10 out of 10 was much, much higher than the percentage of people giving it 9 out of 10 (65.7% vs. 9.2%). They suggested that this was a big dropoff and maybe something was not quite kosher about the ratings (maybe a bunch of phony 10 out of 10 votes were cast).
So I looked at the other top box office movies for Friday and for the whole year to see how they did (from a website called “The Numbers”). Here are their ratios in terms of what % of the voters gave it 10 and what % gave it 9. “Snakes” is very unusual, with the 10% being about 7 times higher than the 9%. The first number is what % gave the movie 10 and the number after the / is what % gave in 9.
Snakes on a Plane 65.7/9.2
Invincible 28.4/13.5
Talladega Nights: The Ballad of Ricky Bobby 20.9/11.2
Beerfest 33.4/14.2
Idlewild 30.7/9.4
Accepted 27.3/10.2
Step Up 39.2/7
Little Miss Sunshine 52.7/25.3
World Trade Center 31.2/13.3
Barnyard 17.0/5.3
Superman Returns 28.2/15.2
Pirates of the Caribbean: Dead Man’s Chest 34.5/15
Cars 32.6/18.7
X-Men 21.8/13.1
Da Vinci Code 19.2/8.9
Ice Age 19.5/10.8
Over the Hedge 20.8/14.5
Click 34.3/12.7
Mission: Impossible 15.9/11.1
The Devil Wears Prada 20.6/14.0
Notice that the 10% to 9% ratio is usually around 2 to 3, sometimes even less. So “Snakes” is unusual. I am not sure if it statistically significant, though. But it would not surprise me.
Also, I found the rating given to each of these movies by the “top 1000 voters.” I guess those are the people who vote the most. The top 1000 only gave “Snakes” a 5.9. That dropoff of 2.0 is large, but not the largest. But given that 7.9 is normally a very good rating, and 5.9 not too good, it is interesting.
Below are how these movies did overall and then with the top 1000 voters. The number in parantheses is how many votes. Notice that “Little Miss Sunshine” also drops off quite a bit. Alot of movies drop off by less than 1 going to the top 1000 voters.
Snakes on a Plane 7.9 (19,072)-5.9(49)
Invincible 7 (356)-7.8 (5)
Talladega Nights: The Ballad of Ricky Bobby 6.6 (7329)-6.0 (49)
Beerfest 6.8 (613)-6.8 (8)
Idlewild 5.7(192)-no votes from top 1000 voters
Accepted 6.2 (1187)-3.9(13)
Step Up 4.8 (1953)-3.4 (18)
Little Miss Sunshine 8.4 (5193)-6.0(45)
World Trade Center 7.1(4587)-6.8(39)
Barnyard 4.5 (546)-3.3(15)
Superman Returns 7.1 (36575)-6.4(179)
Pirates of the Caribbean: Dead Man’s Chest 7.3/10 (44,012)-6.7 (185)
Cars 7.7 (11,435)-7.0 (106)
X-Men 7.0 (37,669)-6.8 (226)
Da Vinci Code 6.5 (38,383)-5.8(186)
Ice Age 6.9 (11,672)-6.3(117)
Over the Hedge 7.2 (6,584) 6.7(93)
Click 6.9 (8,905)-5.3 (56)
Mission: Impossible 6.8(21,591)-6.2(161)
The Devil Wears Prada 7.0 (5,598)-6.2(48)
I wouldn’t say this is a smoking gun, although it is suggestive that something is amiss. Hat’s off to Cyril for going out and getting the data to test an interesting hypothesis. Do any other blog readers have ideas about how to figure out whether the buzz for Snakes may not be exactly what it seems?
21 Comments
How big was the buzz exactly among bloggers? I know there was a ton of talk, but it seemed more of a meme that genuine interest.
— scott cunninghamFirst of all I think that anything that is hyped so much is likely to receive a bunch of phony ratings, that it is unlikely to be an orchestrated effort by anyone. Regardless of which it was, the ratings on one site alone is not enough determine whether it was real or fake buzz. I think a lot of the issue is that bloggers (and especially prominent blogs like Gawker) talk about something a lot because its the talk of the moment but aren’t actually going to leave their computers to see the movie.
And here’s a theory I’d like to throw out there, entirely unsubstantiated:
Maybe because the buzz around Snakes on a Plane was so internet based that like many tech savvy users those involved in the buzz merely viewed clips on YouTube or downloaded the movie illegally.
Just a thought.
— onlineodditiesVery interesting! If this kind of bought-hype marketing becomes more commonplace I bet detecting it will become a much more sophisticated and nuanced thing. Same principle as data-mining I guess–the richness of information on the internet makes it a much more traceable entity (like that AOL user whose searches identified her).
— queenofshebaI think the idea of “Snakes on Plane” was hysterical, but no one actually wanted to see the movie. It was more fun inventing the movie ourselves, andthe movie was sure to disappoint in comparison.
I did go and see it with a good friend, and it was awfully, hilariously bad. But you have to go with a friend in on the joke.
— Gail LOK, it’s a blog analysis, but still there should be some standards. Note that if the movie is rated a “10″ by 65.7% of the voters, there are only 34.3% left. If 9.2% rated it a “9″, that’s 9.2/34.3 = 26% of those remaining.
Similar numbers of “9″ voters as a percent of “not 10″ voters — for Invincible (19%) or Talladega (14%) — show that these movied had even more of a “dropoff”.
That’s not surprising — if there are bunch of people rating a movie “10″, one would expect a higher than normal level of “9″’s because the overall rating is so high.
It may well be that the voting was manipulated, but the “dropoff” analysis doesn’t show it.
The second set of data is more suggestive of manipulation. Note that the number of “Top 1000 voters” rating Snakes was 49 - practically the same number as rated “The Devil wears Pravda”, “Click”, “Little Miss Sunshine”, and “Talladega”. Yet the number of total votes is much larger for “Snakes” than for these other movies.
So, “Snakes” gets more total votes, and this plethora of votes is much more positive — that’s a better case for “buzz marketing”.
— zbicyclistI think the Snakes rating is more indiciative of human psychology than corporate cleverness.
The reason it got all the internet hype is the concept — and no matter how good or bad the film was, the concept would still be there. Moreover, these people excited about the concept are already on the internet, and thus more likely to vote on IMDB, and just throw down a 10 vote. Geek-friendly films tend to do well on IMDB, especially in the first week. V for Vendetta received very high ratings on IMDB then lowered somewhat over the following weeks — much like Snakes on a Plane. Even more arty films like Little Miss Sunshine have some geek appeal due to their indie cred.
Finally, IMDB registers over 12,000 votes fo 10 for Snakes — that seems like an awful lot of paychecks for the studio to cut without someone blabbing somewhere on the internet that they got paid for voting.
One thing that woudl be interesting is if we could find out how many of those 10 votes came from people who hadn’t even seen the film, or how many of those votes came from people who have only voted just once on IMDB.
— sppetersonUm, the internet buzz was about how ridiculous the movie was, and how the premise was outlandish, not necessarily about how the movie seemed fun, interesting, or worthwhile. A large part of the buzz also just constituted of people quoting what has become the catch-phrase from the movie “there’s motherf-ing snakes on this motherf-ing plane”
— trueenuffI agree with the last couple of replies. It has to do with the specific nature of this buzz. I mean, this movie was about snakes, and they were on a plane! Snakes on a freaking plane! With Samuel L!
It doesn’t get any better than that. This movie doesn’t deserve 8 out of 10, 9 out of 10, even a 10 out of 10 is just an expression for the unexpressible, when you’re dealing with Snakes on a Freaking Plane.
Hopefully it’s clear what happened now.
— rdolivawI think some people may have entered 10 out of 10 simply because it is funny to think of “Snakes on a Plane” as the greatest movie of all time. The last commenter said it best. The movie was about snakes of a freaking plane. “Snakes on a Plan” isn’t a good movie. That is why no one votes it at an eight or nine. It’s so bad, that it’s the greatest movie. It’s like Rosie O’Donnell making the 100 sexiest celebrities list. As a joke, a tabloid might rank her as number one. But no one has any reason to rank her at number 50.
That being said, I am inclined to believe you are right. I got a call from a Samuel L Jackson recording, giving me a 2-minute lecture to see Snakes on a Plane and bring my “homeboys.” Obviously, there has been an online effort to promote the movie.
Then again, a group of people may be promoting Snakes on a Plane as the greatest movie of all time as a joke, rather than people getting rich from the movie.
— SaavedraInteresting comments. Maybe if there is no funny business going on. The numbers I found may show how people view a movie (like how campy they think it is) instead of a conspiracy.
If you go to IMDB and look at the worst movies ever
http://www.imdb.com/chart/bottom
You will find they have a high percentage of votes giving them 10 out of 10 (at least relative to the number of 9’s). Maybe some people think it is fun to give a lousy movie a 10 (why?). But so far “Snakes” does not have a very low rating-it has a good one.
The IMDB does not reveal their formula, to prevent stuffing the ballot box. Here is what they say:
“IMDb publishes weighted vote averages rather than raw data averages. Various filters are applied to the raw data in order to eliminate and reduce attempts at ‘vote stuffing’ by individuals more interested in changing the current rating of a movie than giving their true opinion of it.
The exact methods we use will not be disclosed. This should ensure that the policy remains effective. The result is a more accurate vote average.”
But still, whatever method is used is used for all movies. So something different is going on for “Snakes.” I agree that it might not necessarily mean that someone intentionally stuffed the ballot box.
— Cyril MorongI disagree that it “fizzled at the box office”. To Fizzle implies it should have done well. But come on, it is Snakes.
On a plane.
How lame is that? IMHO, ANY, and I mean ABSOLUTELY ANY, revenue it generated at teh box office is more than it would have made as straight to DVD release, which is what the idea justified.
In my mind, and I admit I have no idea how much more a cinema release makes over straight to DVD, but Snakes On A Plane has a whole entire revenue source (Cinema Takings) it would otherwise be denied as a singular result of the internet buzz.
If anyone can tell me that isn’t significant, and a massive change / opportunity / development, I will eat a snake. On a plane.
— mikey22Snakes on a Plane had a budget of 35 million dollars, so I don’t think it was ever planned as direct to DVD.
— sppetersonI like the point that zbicyclist made about the percentages and the ratio of the 10s to the 9s.
We could do the reverse of this (sort of). Instead, we can see what share the 1’s had of the non 10s. As of now, 64.4% of the users have given “Snakes” a 10. That means the rest of the ratings are not 10 (35.6%). 6.6% gave the movie a 1. So the ratio of 1s to the non 10s is 18.5%. For some other movies this ratio was
Talladega Nights 8.5%
Superman Returns 8%
The Devil Wears Prada 3.3%
So compared to these movies, “Snakes” is getting alot of 10s but alot of 1s. Is “Snakes” some kind of extreme movie? If so, why? I will try to do this ratio for more movies and post in the next few days.
If you look at the top 10 all-time at IMDB (like the Godfather, Lord of the Rings), not a single one got 10 in even 60% of the votes. Some were under 50.
Another odd thing is that the movies I listed all seem to have more 10s than 9s. Is that evidence of abuse being normal? How well does the secret IMDB formula thwart abuse?
In the top 10 all time, very often, if not in all cases, the % of voters that gave the movie a 1 was more than the % giving it 2,3, and 4 combined. Do people have some wish to stick it to the great movies?
As I mentioned earlier, the IMDB has a secret formula to prevent abuse. So do people try to help or hurt a movie anyway? Or do they just not know?
I want to look at which movies have the biggest difference between the arithmetic mean (the average with none of the adjustments that the IMDB makes) and their actual rating. Will the ones with the biggest differnce be the ones that had the most abuse?
— Cyril MorongOne thing occurs to me about the following I wrote above:
“In the top 10 all time, very often, if not in all cases, the % of voters that gave the movie a 1 was more than the % giving it 2,3, and 4 combined. Do people have some wish to stick it to the great movies?”
Maybe some of the people giving a movie a 1 really want to give it a 0, or a -1, -2, etc. But they can’t go any lower than 1. So all of the really low ranks have to settle for 1. Maybe the % of true 1’s is really lower than the % of true 2s and 3s. Same thing on the upside? Alot of movies have more 10s than 9s. Did some of those 10s really want to give it an 11, 12, etc?
— Cyril Morong“Snakes on a Plane had a budget of 35 million dollars”
Before or after the hype?
They shot 5 extra days of scenes in post production to appease fans. One assumes some of the budget was given as a result of the fan hysteria, not before it.
— mikey22http://en.wikipedia.org/wiki/Direct-to-video
— mikey22“Such films can cost as much as $20 million, just under a third of the average cost of a Hollywood release”
So, Snakes cost 35 Million, around about the figfure one would assume for straigth-to-video + some kicked in for the hype, no?
You guys are all missing the point.. going to the movies is a DATE experience.. I couldn’t’ get any of my friends who are girls to go see a movie that had snakes in it. WOMEN do not like snakes, and geting them to go see a movie about snakes is practially impossible.
If it had been any other subject, then the movie would have been a smash hit.
— chazzzzyAre some movies marketed as date movies? I can’t imagine that “Snakes” was.
Alot more men than women have rated “Snakes” at IMDB. The first number is men, the second is women. For comparisons, I list “Prada,” which I assume is a chick flick, but I could be wrong about that.
Snakes on a Plane 17360-2211
The Devil Wears Prada 2874-2241
Here are the numbers for the current top 10 bosx office movies as of today
Invincible 780-153
Talladega Nights 1588-851
Little Miss Sunshine 4342-1827
Beerfest 1370-133
World Trade Center 3667-800
Accepted 1032-287
Step Up 870-1116
Idlewild 292-119
Barnyard 411-134
So it looks like the ratio of men to women is unusually high who have rated “Snakes.” One thing that might be interesting is that women gave “Step Up” a 7.0 and “Snakes” a 7.9.
— Cyril MorongIn an earlier post I mentioned finding the % of “1″ votes or ranks as a % of all of the non “10″ votes. It first looked like “Snakes” ranked really high here. I thought that it suggested something bimodal going on-alot of 1s and alot of 10s. If a movie has a lot of 10s, it starts to limit how many 1s it can have. That is why I took the % of non 10s. It seems if a movie has a high ratio here, and if the votes are honest, then those top 4 movies have alot of people loving them and alot of people hating them.
So I took the list of movies I originally looked at and I ranked them from highest to lowest in this ratio. 39.6% of the voters gave “Step Up” a 10. That leaves another 60.4%. But 21.9 gave it a 1. So the ratio is 36.3% (21.9/60.4). That was by far the highest. Snakes was only 4th, but way ahead of those below it.
Step Up 36.3
Barnyard 20
World Trade Center 19.4
Snakes on a Plane 18.5
Idlewild 16.8
Beerfest 9.6
Da Vinci Code 8.9
Talladega Nights 8.4
Accepted 8.4
Superman Returns 8.2
Mission: Impossible 6.7
Click 6.5
XMen5.6
Pirates of the Caribbean 5
Invincible 4.1
Cars 4
The Devil Wears Prada 3.2
Little Miss Sunshine 2.3
Ice Age 1.7
Over the Hedge 1.7
There is a huge dropoff after “Step Up.” Then a huge dropoff after “Snakes.” Also, the top 4 movies seem to have nothing in common.
About 13% gave “World Trade Center” a 1. It has an overall rating of 7.1. So that is farily good. Remember that’s the official IMDB rating that uses a secret formula that tries to thwart any ballot stuffing (I guess it tries to stop people from both artifically helping and/or hurting a movie). Did so many people really hate this movie so much?
— Cyril MorongI found out how much each movie had earned at the box office then divided that by the number of votes it had gotten at IMDB. “Snakes” by far had the lowest rate. My guess is that it means that the people who saw it felt very intensely about it or people voted who did not see it. Maybe this does not mean anything, but it seems interesting. Here are the dollar per vote totals
Barnyard $129,386
Ice Age $78,220
Over the Hedge $56,816
Cars $42,329
Step Up $36,369
The Devil Wears Prada $29,416
Click $28,528
Invincible $27,712
Accepted $21,431
Talladega Nights $21,345
Pirates of the Caribbean $19,993
Mission: Impossible $17,127
Da Vinci Code $16,976
World Trade Center $16,970
Idlewild $16,435
Xmen $14,102
Superman Returns $10,113
Beerfest $7,767
Little Miss Sunshine $4,833
Snakes on a Plane $2,149
I used “The Numbers” site to get the dollar amount, as of yesterday. This could be a little bit of a problem since I took the number of votes from today. As soon as “The Numbers” posts data through today, I will do this again. Also, I only used US data for both votes and dollar amounts.
— Cyril MorongI think there’s a couple of other problems here. First of all, as Tuffley on my staff pointed out, they upped the film to an R rating despite the fact that the folks most likely going to see the film (and who were probably blogging the most about it) were underage and now unable to get in. This was a big mistake, as what was in the film that seemed R-worthy (a bit of nudity) was truly superfluous. What they should have done is shot for R, released PG-13 to cinemas, then followed up with PG-13 on DVD, then the inevitable UNRATED edition (which, generally, is just the regular movie plus four minutes of additional footage they were too lazy to pass by the MPAA).
Also of note is that we’ve seen this before: SERENITY had huge buzz from the sci-fi community and then had a poor showing at the box office.
I think a lot of it has to do with general laziness. Blogging about something is free…but going to see it or snagging it on DVD costs money. And SOAP had “rental” written all over it. I think, sadly, people clamor for better films or to have Hollywood listen to them, but in the end, they won’t back up the right decisions with coin. So Hollywood lumbers along, confused as heck, and cinemagoers just continue to complain.
Regardless, what’s been proven is that net-buzz cannot open a movie.
— Widgett