What we provide

We unlock the potential of Amazon’s Mechanical Turk, a marketplace for distributing work among thousands of anonymous web-users. AMT can help your company collect data from a large pool of people, or get a big job done very quickly. But it can also be very difficult to use. Effectively producing reliable data on AMT requires software engineering expertise and a deep knowledge of statistics. That is where Dolores Labs comes in. We set up your task and interface with the AMT system so that you get your data quickly, and perform exhaustive data analysis to make sure that data is quality. We make crowdsourcing easy.

FAQ


What is Mechanical Turk?
Mechanical Turk is a marketplace where businesses post small tasks that can be done without training. It is full of "Turkers," people who sign onto Mechanical Turk, complete tasks and get paid per task.

What kinds of tasks does Turk do well?
Mechanical Turk can be used for a wide range of tasks - from market research to search result annotation to copy-editing. It is especially useful for tasks that are easy for humans but hard for computers, such as image labeling or text extraction.

Who does the tasks on Mechanical Turk?
Since a worker, or Turker, must have a bank account to be paid, almost all of the Turkers live in the United States. Turkers are anonymous, but in order to get a better sense of who Turkers are, Dolores Labs did a demographic survey of Turkers. We found that Turkers come from many different backgrounds, but more than half have a college degree and most are native English speakers.

What advantages does Dolores Labs provide over using Mechanical Turk directly?
  1. We have built expertise and relationships with the "Turker" community. We have had over 1000 workers do work for us on a variety of tasks. Our statistical algorithms and analysis tools help us identify the highest quality workers and give them extra rewards; we also detect scammers and remove them.
  2. We have a convenient interface to Mechanical Turk. You don't need to hire an engineer to handle the batch uploading of large data sets.
  3. We have experience building interfaces that workers like. We collect feedback from them on every project and constantly improve our web-based survey designs to make the data collection faster and of higher quality.

What kinds of tasks can you do?
Our system works best with tasks that can be done by the average person with little training. The output should be something where we can compare two persons' answers and generally expect them to be identical. Our examples page shows the types of tasks we've done.

How quickly can you collect data?
On most labeling tasks we can guarantee that we will collect 1000 labels a day (with up to 5 judges per label). If a customer needs higher throughput, we can potentially achieve that by paying workers an above market rate, posting in message boards and notifying our best workers of the task.

How do we get started?
Contact us for more information -- we're happy to answer any questions you might have!