Facebook's own data is not as conclusive as you think about teens and mental health

On Tuesday, Facebook whistleblower Frances Haugen testified before a Senate panel. The hearing's focus was advertised as "protecting kids online."

"I believe that Facebook's products harm children," she said in her opening statement, saying that the documents she published proved that Facebook's "profit optimizing machine is generating self-harm and self-hate — especially for vulnerable groups, like teenage girls." Facebook spokesperson Andy Stone noted on Twitter during the hearing that Haugen "did not work on child safety or Instagram or research these issues and has no direct knowledge of the topic from her work at Facebook."

Researchers have worked for decades to tease out the relationship between teen media use and mental health. Although there is debate, they tend to agree that the evidence we've seen so far is complex, contradictory, and ultimately inconclusive. That is equally true of Facebook's internal marketing data, leaked by Haugen, as it is of the validated studies on the topic.

Opinion versus fact

The leaked Facebook research consists of opinion surveys and interviews. Facebook asked teens about their impressions of Instagram's effect on their body image, mental health, and other issues.

That reliance on self-reporting — the teens' own opinions — as a single indicator of harm is a problem, says Candice Odgers, a psychologist who studies adolescence at University of California, Irvine and Duke University. That's because teenagers are already primed by media coverage, and the disapproval of adults, to believe that social media is bad for them.

Odgers was a coauthor of a study conducted in 2015 and published in 2020 that found exactly this. "If you ask teens if they are addicted/harmed by social media or their phones, the vast majority say yes," she tells NPR. "But if you actually do the research and connect their use to objective measures ... there is very little to no connection." With the exception of a small increase in behavior problems, her study found no real world connections between smartphone or social media use and several different measures of psychological distress and well-being. "At the population level," the paper concluded, "there was little evidence that digital technology access and use is negatively associated with young adolescents' well-being."

Small numbers

Odgers' paper was peer-reviewed. It had 2,100 participants. It's just one of hundreds of studies published over decades on children and adolescents' media use and well-being. This research started with radio, moved on to television, video games and now social media. All along the way, large peer-reviewed studies have found few correlations. "It's mostly null," Odgers says.

The Facebook research was not peer-reviewed or designed to be nationally representative, and some of the statistics that have received the most attention were based on very small numbers.

According to Facebook's own annotations of the leaked slides, the finding broadly reported as "30% of teen girls felt Instagram made them feel worse about their bodies" was based on 150 respondents out of a few thousand Instagram users surveyed. They only answered the question about Instagram's role if they had already reported having body image issues. So the finding does not describe a random sampling of teenage girls, or even all the girls in the survey. It's a subset of a subset of a subset.

In another of the Facebook surveys, out of more than 2,500 teenage Instagram users surveyed in the U.K. and U.S., 16 total respondents reported suicidal thoughts that they said started with Instagram. Because of the way this data was sliced and diced in Facebook's internal slides, those 16 people, less than 1% of all respondents, became the ultimate source of stories that reported 6% of teens in the U.S. and 13% in the U.K. blamed Instagram for suicidal thoughts.

Vicious and virtuous circles

Vicky Rideout is an independent researcher who has published more than two dozen studies on young people and media use. She says it's "a useless distraction" to compare the confrontation with Facebook to the showdown over Big Tobacco, as senators have been doing at these hearings. That's for two reasons: because the evidence is nowhere near as strong, and because social media — unlike cigarettes — can be beneficial as well as harmful.

One of Rideout's 2021 studies on teens, unlike Facebook's internal findings, used a nationally representative sample and used a recognized scale to measure depression. In her study, 43% of respondents said using social media usually makes them feel better — not worse — when they're depressed, stressed, or anxious. Less than half as many, 17%, said it usually makes them feel worse. The rest said it makes no difference either way.

Rideout's research suggests that there is a small group of severely depressed teenagers for whom social media has a bigger impact for better and for worse. She thinks they should be a focus of future research.

Both Rideout and Odgers say that rather than get stuck in an endless loop of doomscrolling over small, inconclusive results, the public conversation on social media and teens needs to move toward solutions. They would like to see companies like Facebook put resources toward designing and testing positive interventions.

Some ideas researchers are currently looking at: connecting young people with information about mental wellness or health; promoting accounts that have been shown to make people feel better about themselves; or prompting teens to check in with peers who are having a rough day.

"There really are a lot of teens suffering from depression, and they really do use a lot of social media, and social media really does play an outsized role in their lives," says Rideout. "If there are concrete steps that Instagram or any other social media company can take to elevate the positive and diminish the negative aspects of their platforms, that's something we should support."

Editor's note: Facebook is among NPR's financial supporters and since publishing her book, The Art of Screen Time, Kamenetz's husband took a job with Facebook. He works in an unrelated division. [Copyright 2021 NPR]