Crowdsourcing in Control
Website Magazine recently had the opportunity to speak with James Rubinstein, PM Search Metrics at eBay, in advance of his presentation at Crowdopolis– an event that took place in July of 2012. The conference addressed the future of crowdsourcing in content marketing, advertising and technology and featured speakers from several Fortune 500 companies including GE, Nokia, Amazon, and Ebay.
Crowdsourcing, in the briefest of definitions for those wholly unfamiliar, is the division of labor – taking a large job, breaking it down into component tasks and then distributing it. “By breaking down large tasks into their smallest size, it come be done very quickly,” said Rubinstein.
Rubinstein thinks of crowdsourcing as “many hands make light work” and uses crowdsourcing in a variety of ways at Ebay in his role as a project manager – including testing inventory descriptions, testing search result quality, and measuring the efficiency of the search experience for users.
Rubinstein's presentation at Crowdopolis, “Crowdsourcing Lessons from James Bond” provided a practical view into the role that research plays in successful crowdsourcing, how to use that research to productionalize the process, and how to maintain positive vendor relationships. The following is a roundup of important insights, some digital nuggets of wisdom, that WM gained from our informal discussion with Ebay’s Rubinstein.
- Crowdsourcing is a new space so new and different things need to be tried and tested to figure out what works (e.g. new measurement techniques, new layouts, new metrics).
- Ebay does lots of crowdsourcing with different teams - testing inventory descriptions, addressing search results quality, measuring the search experience.
- Through crowdsourcing Ebay has been able to make a difference in what's been shipped. Other teams have brought down costs, ramped up the workforce, got a job completed.
- There are lots of ways to get answers to questions but crowdsourcing is not always the best. For a lot of questions that don't have direct financial impact, it's tough to read.
- If you can put in sufficient controls, it can be cheaper, faster, and can bring better quality than traditional outsourcing.
- One of the key advantages of crowdsourcing is speed - you want to have prefigured many things so you can quickly get the data, put it into a template and get the right information back.
- Crowdsourcing vendors should have some area of specialization - don't be all things to all people. Some are good at micro-tasks, some are good at content generation or delivery. Need some specialization in the workforce - know your skills.
- Two vendors to consider include oDesk (e.g. for content generation) and uTest (specialize in online testing).