Former TikTok moderators sue over emotional toll of 'extremely disturbing' videos

When Ashley Velez accepted a job last year reviewing videos for TikTok, "We were told we would be the front line of defense from protecting children from seeing violence," she said.

But the Las Vegas mother of two boys, ages 8 and 17, said she was stunned when she discovered what the position entailed.

"We would see death and graphic, graphic pornography. I would see nude underage children every day," Velez said. "I would see people get shot in the face, and another video of a kid getting beaten made me cry for two hours straight."

Velez worked for TikTok from May to November 2021, one of some 10,000 content moderators worldwide who police videos on the site, making sure it remains an endless feed of lighthearted content, rather than a cesspool of violent and disturbing videos.

Now, Velez and another former TikTok moderator, Reece Young, have filed a federal lawsuit seeking class-action status against the video-sharing app and its parent company, ByteDance.

The Chinese-owned app is the envy of Silicon Valley social media giants and has more than 1 billion monthly active users. Its success depends in no small part on the work of moderators like Velez, toiling behind the scenes to scrub TikTok of distressing content before the masses see it.

While the plight of moderators is often absent from fights over what content social media platforms allow and what they ban, there is a growing movement to hold tech giants accountable for the welfare of these workers on the frontlines of that debate. On Thursday, Velez and Young sought to do just that.

Velez: 'Someone has to suffer and see this stuff'

Their lawsuit accuses TikTok of negligence and says it broke California labor laws by allegedly not protecting Velez and Young from the emotional trauma caused by reviewing hundreds of "highly toxic and extremely disturbing" videos every week, including of animal cruelty, torture and even the execution of children.

"Underage nude children was the plethora of what I saw," said Velez, who now works as an independent contractor for vacation home rental site Boutiq. "People like us have to filter out the unsavory content. Somebody has to suffer and see this stuff so nobody else has to."

According to the suit, Young and Velez were exposed to an unsafe work environment because TikTok did not provide adequate mental health treatment to help deal with the anxiety, depression and post-traumatic stress associated with reviewing graphic videos.

Young and Velez were both contractors, not employees of TikTok. Young worked for the New York company Atrium; Velez was hired by Telus International, a publicly-traded Canadian tech firm. The suit says TikTok and ByteDance controlled the day-to-day schedule of Young and Velez by directly tying their pay to how well they moderated content in TikTok's system and by pushing them to hit aggressive quota targets.

Moderators like Young and Velez are expected to review videos "for no longer than 25 seconds" and decide with more than 80% accuracy whether the content breaks one of TikTok's rules, according to suit. In order to meet quotas, the suit alleges, moderators often watch multiple videos at once.

Young and Velez were allowed two 15-minute breaks and an hour-long lunch break over a 12-hour workday. If they took any other breaks, they risked losing pay, the suit says.

That amounts to punishing the content moderators by "making them extremely ill-equipped to handle the mentally devastating imagery their work required them to view without any meaningful counseling or meaningful breaks during their work," wrote lawyer Joseph Saveri and other attorneys for the plaintiffs in the suit.

A TikTok spokeswoman declined to comment on the lawsuit, but said the company "strives to promote a caring working environment for our employees and contractors."

TikTok moderators, according to company, are offered "a range of wellness services so that moderators feel supported mentally and emotionally."

Telus International said in a statement that it "has a robust resiliency and mental health program in place to support all our team members, as well as a comprehensive benefits program for access to personal health and well-being services."

Velez said she did set up a meeting with a Telus counselor, who spoke to her for 30 minutes. "They saw so many people that it didn't seem like they had time to actually help you with what you were suffering with," she said. "It would have been nice if they would even acknowledge that the videos were causing a problem in the first place."

TikTok suit comes after $54 million Facebook settlement

Social media companies, including TikTok, use artificial intelligence to screen millions of videos of disturbing content, but the technology cannot catch everything so human moderators remain critical to keeping the platforms safe.

Yet people can act only so quickly. While moderators are pushed to act fast, they also are tasked with analyzing multiple aspects of each video, the suit states. There are now 100 different "tags" moderators use when indicating a video violates a rule, like whether a video showed the torso of a minor, — up from 20 tags — and moderators are expected to analyze what's happening in the background of each video.

It is unclear when, exactly, the new moderation standards went into effect. Since Russia invaded Ukraine, TikTok has been under new pressure as it attempts to stay ahead of a flood of misleading videos about the war.

In the suit, lawyers for Young and Velez argue their clients' psychological trauma stems not just from videos that are violent, but also those that spread conspiracy theories, including recordings suggesting that the COVID-19 pandemic is a fraud or denying the Holocaust ever happened.

The National Center for Missing and Exploited Children has developed best practices for content moderators for images and videos of exploited minors. Those measures include blurring parts of an image or superimposing a grid over the image to lessen its potential emotional impact on the people reviewing it.

The suit says that although TikTok is a member of the center's Technology Coalition, which also includes Google and Facebook, it has not adopted the group's recommendations, instead focusing on hitting video-review quotas above all else. A spokeswoman for TikTok did not return a request for comment on to the allegation.

The lawyers representing the TikTok content moderators sued Facebook several years ago on behalf of thousands of moderators who said they experienced emotional distress on the job. In May 2020, Facebook agreed to settle the suit for $52 million. Individual moderators who were part of the class action were eligible for at least $1,000, depending on the severity of their emotional disorders related to their Facebook moderation work.

The same lawyers filed a similar suit against TikTok in December on behalf of moderator Candie Frazier. It was dropped last month before reaching a settlement after her lawyers say she was fired. [Copyright 2022 NPR]