TikTok secret backtest 15 million users

by Huda Thomas: News

TikTok ran a backtest on about 15 million Americans in 2021 without telling them. The company tweaked its recommendation algorithm to cut down on harmful content for most U.S. users, but kept 10% on an older version to compare. None of those in the control group knew they were part of an experiment. One of them was 16-year-old Chase Nasca, who died by suicide in February 2022. He had no history of mental health issues. After a Bloomberg Businessweek reporter asked about him in March 2023, TikTok's algorithm and safety teams reconstructed his account and discovered he'd been placed in that control group. Even though he mostly searched for sports and comedy, his feed kept serving him suicide and other dangerous content. The internal findings were written up in a 14-page report on youth safety and filter bubbles, which was shared with 10 teams including communications, legal response, policy, security, and search. The report suggested two fixes: shrink the control group from 10% of U.S. users to under 1%, and reduce the time a user stays in it from up to a full year to just seven days. But it also flagged that even at 1%, at least 1.8 million users would still miss out on the safety update, leaving them "at risk for falling into bubbles." A former trust and safety team leader at TikTok said he'd never heard of the experiment, and that the control group was way too big and should have excluded minors and suicide content. TikTok maintains the 2022 test was routine and meant to verify that safety changes worked as intended. Chase's parents filed a wrongful death lawsuit, but a judge dismissed it last year under Section 230; an appeal is pending. All big platforms run similar tests, but none disclose how many users are stuck in the old version at any given time.