In my last post, I promised to say a bit more about prediction markets at Google. Google has been running internal prediction markets for a couple of years, and Eric Zitzewitz and I were fortunate enough to team up with Google whiz Bo Cowgill to analyze these markets.
Ask any economist about the “theory of the firm” (what firms do and don’t do), and they will emphasize the importance of information. So we decided to move beyond asking, “Do prediction markets work?” and instead use them as a tool for better understanding how information flows within a (very cool) corporation. Information is terrific to theorize about, but hard to measure. That’s where the prediction market is useful: if you and I trade similarly on a market, then we can infer that you share similar information.
Our research uses this insight to try to understand which employees trade in correlation with which other employees, and, hence, to measure how information flows within Google. We came up with some pretty interesting findings:
Sitting within a few feet of a workmate has a big effect. (Our data includes the exact GPS coordinates of each person’s desk, as well as their previous desks.)
Beyond this, sitting on the same floor as someone barely has any effect.
There is some evidence that organizational proximity (such as working for the same managers) also plays a role.
Professional relationships are quite important, although having been assigned to the same cross-departmental project as another person didn’t have any effect.
Shared interests (such as being on similar e-mail lists) also yield similar trading behavior.
Yet when Googlers were asked who their friends were, the relationships they reported did not explain trading behavior at all.
Demographic similarity was surprisingly unimportant, except where two colleagues shared a native non-English language.
We also have some interesting insights about biases in prediction markets. If there’s interest, I’ll return to this in a future post.
The overwhelming importance of “micro-geography” was quite striking, particularly as this is the sort of organization in which Instant Messaging and e-mail (plus blogs and wikis) might have otherwise suggested the death of distance. Certainly this research changed my mind about the importance of open-plan seating. This isn’t a lesson lost on Google either, as cube-mates are kept in close proximity, and Googlers are asked to move desks approximately once every three months. Interestingly, personal relationships persist once these moves have occurred, and people tend to trade in a way correlated with that of their cube-mate from three months ago; although, reassuringly, they do not trade in a way correlated with their future cube-mate. (I say “reassuringly” because this is a useful way of testing whether our results reflect Google seating people with similar opinions near each other, rather than people near each other influencing the opinions of others.)
I don’t know about your firm, but we academics are too self-important to ever sit in cubicles. Our research suggests that this may be unfortunate, and perhaps many of the best ideas in economics never occur, because the idea is waiting for us at a water cooler conversation at which we never arrive. I would love to see my colleagues brainstorm more often and more freely. If we can’t tear down the physical walls between our offices, how can we all change our workplaces to encourage the free flow of information and ideas? Comments are open.
Related links: click here for the full research paper; a companion post at the Google blog; a recent write-up in the Times. Other commentary: Marginal Revolution, Andrew Gelman, Zubin Jelveh at Portfolio, Justin Lahart at the Wall Street Journal, BloggingStocks.
18 Comments
Very interesting study. I’d definitely be interested in hearing more about your findings on biases in prediction markets.
— RSAt a conference I spoke to last fall I heard a talk by Mark Lentczner, director of Studio Icehouse at Linden Lab (the creators of Second Life) in which he addressed that same issue of how to best “pack in” knowledge workers.
As he explained it, Second Life grew very fast from a few dozen to a few hundred on staff, and in the process, as people became more specialized, the free-wheeling exchange of ideas was lost. People started to relate more and more to just those in their project team.
Second Life used a very different strategy than Google’s to re-inject serendipity in the exchange of ideas. Recognizing that you exchange the most information with your office mates, Second Life decided to physically separate team members, so that everyone would be sharing offices with people not on their own team. At the same time, they used Second Life itself to create virtual shared office space for members of the same team. So, each person physically sits in an office with other people assigned to unrelated projects, but also sits virtually in an SL virtual office frequented by just their team members.
Best of both worlds, literally.
— Emile Servan-SchreiberBrief aside: ‘Googler’ is the in-house appellation for a Google employee? It’s reasonable enough, but it clashes with the (my?) more generic “Googler as one who searches with Google” (or even more generally, “one who uses search engines” or just “one who digs into things”).
Advantage goes to Google’s internal culture, ultimately, but it does jangle in the ear a bit.
— Josh MillardReading from the posts describing the markets, it seems there is no cost for being wrong / no huge incentive to be correct (a synthetic currency with top heavy payouts, isnt exactly a balanced surrogate for money). Eliminating the actual ‘market’ from these essentially moves them from being predictive markets, to dynamic polls (with ‘prizes’), which as seen from NH primary is prone to large margins of errors.
— MichaelI think a good way to encourage open flow of information might be for a group of employees (especially a group of professors/grad student research assistants) to set up an online chat room. I’ve always thought that it would be helpful if I could just send everybody an IM with a question or suggestion… since it’s less intrusive than a phone call and more low-key than an email, it would be a great mode of communication.
— ShanFor instance, I work in academic research (I actually work at a lab, but that’s probably irrelevant here) and I am always pondering ideas that I’m sure somebody else has already answered. If I could just send an IM to one of my fellow researchers, it would be much more efficient than looking at previously-published papers. For some reason, it seems that I’d be “bothering” my colleagues if I were to send them an email or call them on the phone (not to mention that I’d have to do it individually for each of them), but it’s much less invasive if I just send an IM.
Somebody should do a study… if I were a professor, this is how I’d set up my lab. Unfortunatetly, I’m just a lowly student and my boss would just laugh at an idea like that.
As someone who works on a trade floor where everyone simply has a bank of monitors and filing cabinet at their workspace and there are not even cube walls between desks, this is hardly surprising. The information flow between those 8-12 people within easy talking distance is rapid and non-stop. Sit more than a row and a half away, though, and I’m lucky to see you, much less share ideas.
— MitchI think seating proximity also affects the quality of relationships. If people sit next to each other they get along better. If there is someone in another building they must work with, via phone or email, they are more likely to express frustration with that person, even making up insulting nicknames for the person, and also having genuine work related problems. But if the company has that person move so all now sit together, don’t be surprised if within a short time the former enemies are spending a lot of time talking about the work related things they have in common, even quickly becoming fast friends.
— GregI’d love to hear more on biases. And it is interesting eavesdropping via their respective blogs on the watercooler/lunch talk of the George Mason mafia.
— burger flipperAmost nine years ago now I joined an email list of folks that were struggling to learn a failrly intricate editing program from a company called AVID. It was one of the greatest learning experiences of my life. We all would learn at different rates, of course, but if we had a question about how something was done we would send it out to the list and almost instantly an answer would come back from the hundred or so members. The list was populated by workers around the world working to master the same software. the learning curve was unbelievable. Eventually, though, a certain competancy level was reached on the software and boredom must have set in for the posts degenerated to personal attacks on some remark made by the new comer and the list finally broke up. It was great while it lasted though.
— Jace AndersonBefore becoming a lawyer I was a programmer. I worked in a small office with 3-4 other lowly programmers. The more senior programmers had their own offices. I’ll give you a guess to decide which group produced more efficient, productive, and innovative work?
It makes lots of sense. We weren’t isolated doing our own task. Instead, each of us was constantly engaged with 2-3 other people on a variety of topics. We could easily discuss/argue/debate the merits of x over y and then come to result z instead. We got more work done, more quickly and with fewer mistakes than the 7-8 other people with their own offices.
Now I’m a lawyer, in my own office, but I still wander down the hall to “discuss” things with others.
— mmmI want to hear about biases.
And what does it say about me when I almost typed ‘big asses’ instead of ‘biases’.
— BiasesShan - I sympathize with your feeling that your suggestion would be shrugged off if presented to your higher ups. I’ve felt that way before in an office environment, that because I was low on the totem pole, my input had less weight.
What I would recommend you consider is an online forum for your department. It doesn’t require the kind of always on attention that a chat room does, and as long as you don’t require immediate answers, can be a great tool for collaboration without physical proximity. Answers can be expanded and bandied about with a permanent record available so that it can provide a long term resource which will aid people long after you have left your lab for your bright future.
I can’t guarantee your superiors will go for the idea, but it can’t hurt to ask, especially with the small investment required for setup and maintenance.
— Joshua HeggenI look favourably on open spaces in offices for the reason that they appear minimalist hence may be less costly for start ups. The study now suggests that they are also useful for passing around ideas where that is critical for improved perfromance. Are you sure that they would be useful for Law firms for instance?
I would like to hear about biases on information markets too.
— OwinokI agree with Josh (3.) that the clash of meanings of “Googler” is unfortunate and to resolve this propose that employees of Google should be known as “Googlanders”
— Mike PMany words have different meanings, Googler and Googler can be used in different contexts and easily understood.
— AlanI’d like to know more about the market activity. In particular:
What percentage of employees regularly trade?
For each “security”, what’s the daily volume like?
What is the monthly cash value of the outcome to the most successful traders, roughly speaking? $100, $1K, $10K, more?
How seriously does management look at market activity to inform decisions?
Given that this is a free perk, what are the main reasons employees who are not active don’t trade?
Who gets to decide what predictions are put on the market?
How much concern is there about sensitive information being spread through the company, either because of the questions being asked by the predictions or the dynamic price data?
— Rafe FurstThe self-appellation of ‘Googler’ for Google employees is not going to change, clash though it may be.
New Google employees are known as ‘Nooglers’ for a time. Anecdotes can be initiated with “Back when I was a Noogler…”
— MangoHi there folks!
I like this foru msooo much, thanks for the creators.
— nicknfshfdg