Amazon's Turker Crowd Has Had Enough

Workers on Amazon Mechanical Turk have long felt ignored and underpaid. A new platform hopes to give them a better home—and could change the future of crowd work.
Creative Art Direction Redindhi Studio   Illustration by Laurent Hrybyk
Illustration by: Laurent Hrybyk; Creative Art Direction: Redindhi Studio

When Manish Bhatia began working on Amazon Mechanical Turk as a side gig in 2010, he was surprised to find himself completely fascinated by the work. Contrary to frequent coverage depicting the piece-work platform as a digital sweatshop offering low-skill tasks, he thought the microtasks were intellectually stimulating. Many involved training machine-learning algorithms to do things like make purchasing recommendations based on past behavior or categorize content by genre; Bhatia enjoyed thinking of himself as the “AI behind the AI” and knowing that he was doing something to shape the future. The only problem was that he wasn’t getting paid.

Bhatia says that whenever Amazon mailed him a check, it got lost in transit to India. And while he could choose instead to receive his pay in Amazon credits, those only went so far. So in December 2014, when hundreds of “Turkers” in similar straits banded together for the first time in collective action against Amazon, Bhatia joined in on one of their projects. Along with dozens of fellow workers, he emailed Jeff Bezos. He thanked the Amazon CEO for creating Mechanical Turk, which he truly loved. He just wished his checks would actually make it to his bank account.

He didn't expect a response. Most other workers who participated in the campaign just received form letters. But to his shock, Bhatia got a personalized email with a promise to improve things. Within six months, Amazon enabled bank transfers for workers in India, and Bhatia’s checks started coming through. His victory was a watershed moment for activists in the crowd-work community. And while that particular campaign petered out fairly quickly, it laid the groundwork for a bigger movement that’s gaining steam today: Some Turkers are beginning to leave Amazon’s service in search of a better home.

To workers on Mechanical Turk (or “MTurk,” in Turker lingo), Amazon—which did not respond to request for comment on this story—has long seemed like an impenetrable wall, offering few updates and little support. Meanwhile, the Turker workforce has proven particularly difficult to organize: MTurk magnifies the challenges of the gig economy, with its isolated workers spread across the globe and hidden behind usernames, performing minute tasks on a platform operated by a massive, wealthy corporation. MTurk is also one of the least consumer-facing corners of the gig economy—so while ethically minded customers have taken Uber, Handy, and the like to task for their treatment of workers, Amazon’s gig-work platform has largely managed to evade public scrutiny for its low pay and reported lack of transparency.

Turkers—Amazon says that there are “over 500,000” of them, but has not updated that number in years—have figured out how to make the best out of a less-than-ideal situation. They’ve created Chrome plugins to sort the good tasks from the bad; they’ve written programs to awaken themselves with a loud chime when a particularly high-paying task goes live in the middle of the night; and they’ve built forums where they can offer one another advice. Those forums have taken on a life of their own, leading to deep networks of support and friendship—and at least one Turker marriage.

Today, MTurk is more important than it’s ever been. Its crowd-work model has been adopted by Silicon Valley’s biggest companies to train AI algorithms, spot fake news, and keep violent content off of social media. In the long run, AI might take these jobs over—but right now humans are very much needed for tasks like cleaning and categorizing data. Turkers know they’re in demand, and some are losing patience with Amazon. For over a decade, activist-minded Turkers have been rallying for change, with little success. Meanwhile, other platforms in the gig economy have begun inching toward improvement—just in the past few months, Uber has added in-app tipping, and Postmates and Lyft have come out in support of legislation to help develop portable benefits programs for workers. Many Turkers feel that it’s long past time for a crowd work overhaul.

That time might be now. A new, worker-friendly platform, dubbed Daemo, is currently under development at Stanford’s Crowd Research Collective, and describes itself as a “self-governed crowdsourcing marketplace”—a utopia, of sorts, for Turkers who have long clamored for more agency. It’s a tall order for anyone—let alone a wonky team of academics—to seriously challenge one of the biggest companies in the world. But Amazon Mechanical Turk, by all accounts, has barely changed in its 12 years of existence. And that might just make it the perfect target for disruption.

Daemo’s origins lie in the difficulty of reforming MTurk from within. In 2014, a group of professors and Turkers started “We Are Dynamo,” a crowd-work “guild” that for the first time united workers in collective action. Its organizers included Stanford’s Michael Bernstein and Niloufar Salehi, UC San Diego’s Lilly Irani (one of the creators of a popular Turker plugin, Turkopticon), and Kristy Milland, a longtime Turker and the community manager of the TurkerNation forum. Those professors helped Dynamo’s 550-plus members recruit over 100 academic researchers to pledge fair payment and open communication policies. And under Milland’s leadership, workers on Dynamo kicked off the letter-writing campaign that ultimately got Manish Bhatia his paychecks—and earned Turker activists their first taste of media attention. “This was when workers realized we could be part of the conversation about us and the work we do,” says Milland.

But only two of those dozens of letters to Amazon received any personalized response, and Dynamo lost steam when Amazon caught wind of what was going on. In order to verify that new members were indeed Turkers, the Dynamo creators asked them to complete a quick task on MTurk—and Amazon eventually shut down the account that Dynamo was using to post those verification tasks, cutting off the group’s access to new blood. (“Nominally it was a terms-of-service thing, though I struggle to see exactly how,” says Bernstein. Amazon did not respond to a request for comment.)

Amazon’s crackdown on Dynamo stoked a “sense of frustration in the community of not being able to affect change within the platform,” as Bernstein describes it. He increasingly came to believe that the only way to help Turkers was to bake a better deal into the DNA of a whole new platform.

So Bernstein set out on an ambitious mission to do just that—to build a new, more equitable crowd-work platform that competes directly with Amazon. A global collaboration of hundreds of workers, requesters, and researchers, including Bernstein and Carnegie Mellon PhD candidate Mark Whiting, Daemo has been in alpha for the past year. It will expand to the public in a matter of weeks—at which point its true mettle will be tested.

Daemo takes aim at two of MTurk's most glaring flaws: pay and communication. On Daemo, tasks are expected to pay $10 an hour—higher than the federal minimum wage, though lower than that in states such as Massachusetts and Washington. (On MTurk, for comparison, Pew Research found that 91 percent of workers make less than $8 an hour.) “It’s tricky to institute these minimum wages without a kind of formal way of measuring work performance,” says Whiting. If workers earn a flat rate per hour, he explains, those who tear through tasks particularly quickly may feel it’s not worth their time, whereas requesters dealing with slower workers may feel they’re not getting their money’s worth. Daemo has settled on a policy of norm-setting, reminding requesters of the $10/hour expectation when they’re setting up their tasks and empowering the worker community to take action if that norm is ignored.

Every new task uploaded to Daemo triggers a new discussion topic on its forum, where workers can suggest improvements to the task’s setup, ask questions if they’re confused, and flag whether they think the pay is unfair. If enough people flag a given task, it’ll be removed. On MTurk, those lowball tasks linger—and are invariably snatched up by someone willing to work for pennies.

Daemo takes the issue of fair pay so seriously that its administrators would only allow me to speak with workers on the platform if I set up my interviews as “tasks” and compensated workers for their time. In the interest of learning whether workers like the platform as much as its creators hope—and with my editors’ signoff—I agreed to pay $10 to any worker who would speak with me for up to an hour (totaling $21.34). I found those who raised their hands to be measured in their praise of Daemo. They were delighted by its high pay and open communication; one worker happily noted that when he made a suggestion for how Daemo could improve its interface, the update came through within a few days. Yet they weren’t sure that Daemo could ramp up a steady enough supply of work to lure a critical mass of Turkers away from Amazon.

“It has to scale up ginormously—but I sure hope it does,” says Gina Bixby, who Bernstein recruited to Daemo from MTurk in May. “On Turk I am completely a number, and if I don’t do the work they know that a different number will. It doesn’t matter to them who does it. I actually feel like it does make a difference to Daemo.”

Daemo’s creators don’t hesitate to admit that it is a work in progress. More than 300 workers have completed some several thousand tasks on the platform—and as Daemo gets closer to launching publicly, the frequency with which new tasks are posted has ramped up—but it’s nowhere near the size of Mechanical Turk, which typically has half a million tasks available per day. Daemo also may never be able to attract what Whiting calls “lone wolf workers,” who aren’t interested in a community-oriented approach and who choose crowd work specifically because of its asocial nature. Meanwhile, Milland, the TurkerNation community manager, is skeptical of Daemo’s academic origins, and would rather see an MTurk competitor be fully worker-owned and operated—something that, she says, the community is raring to build.

It’s also not yet clear how Daemo’s model of transparency and collective governance will scale. At its current size, it's remarkably manageable: Workers are hand-selected, and Bernstein is able to talk to nearly every requester and explain the lay of the land. When I was unsure of whether I’d properly set up my interview task, Bernstein was quick to troubleshoot with me over email; when my task went live, he immediately introduced me to the community and vouched for my legitimacy.

That level of involvement and care is what makes Daemo so attractive—but it’s also what might hold it back. Bernstein acknowledges that: He says that “the real test will come when I can no longer have a call with everyone who wants to use the platform. Then the norms have to carry over to people who are new to the community.”

Daemo offers a fascinating case study for tackling the intractable challenges posed by gig work. In its short lifespan, the platform begun to demonstrate what labor advocates have long known to be true: The Daemo team has observed that higher pay and clearer communication results in higher quality work and more reliable performance—and employers, or, in this case, requesters, seem willing to pay a premium for better work. Daemo has also found, through surveying hundreds of workers, that nearly all of them would be willing to switch to an alternate platform with better pay and communication—as long as there's enough work there for everyone. Milland, skeptical though she may be of Daemo itself, has noticed the same thing.

“If you could create a platform that’s identical in every single way [to MTurk], and add one benefit—it could be as simple as, ‘We’ll listen to you and we’ll update the site per your recommendations’—everyone would move,” she says. “If we could provide them with a platform that does even more than that, it’s a no-brainer. It would kill it.”

There are plenty of reasons to doubt that a bootstrapped effort can challenge Amazon. After all, the many grassroots competitors to gig economy giants like Uber and Airbnb have failed so far to pose any real threat. But unlike Uber or Airbnb, MTurk is not constantly coming out with exciting new features or giving its participants incentives to stick around. Anyone who can improve upon MTurk’s sorry functionality and lack of transparency might actually stand a real chance of wooing people away to a shinier, more user- and worker-friendly competitor.

That competitor may be Daemo. It may be the worker-owned cooperative that Milland would like to see, or it may be a brand new, VC-funded startup altogether. Any of these has a shot at rehabilitating one of the most dysfunctional corners of the gig economy by actually listening to workers’ concerns. And if they succeed, they might just prove that the future of work isn’t so hopeless, after all.