Social Media Algorithms and Child Safety

The shift from social graph to recommendation graph

For most of social media's history, the content a user saw was a function of who they followed. Around 2016, every major platform shifted to algorithmic feeds — what users see is determined less by their connections and more by what the platform predicts will keep them scrolling. By 2020, TikTok had normalized the model: an entirely interest-driven, follower-irrelevant feed. By 2024, Instagram Reels, YouTube Shorts, Facebook Reels, and Snapchat Spotlight had copied the pattern.

That shift matters for children specifically because the optimization target — engagement (time on app, video completion, comments, shares) — selects against the very content moderation child-safety systems require. Boring is safe. Boring gets cut by the algorithm.

What the platforms' own research shows

The clearest evidence comes from Meta's own internal research, leaked in September 2021 by former product manager Frances Haugen and published by the Wall Street Journal as the “Facebook Files.”

• Instagram and teen girls:A 2019 internal Meta presentation titled “Teen Mental Health Deep Dive” reported that 32% of teen girls who felt bad about their bodies said Instagram made them feel worse. For teens with suicidal thoughts, 13% of UK and 6% of US users traced those thoughts to Instagram.
• The “rabbit-hole” effect:Meta researchers created test accounts for fictional 13-year-old girls and found that following one diet-related account caused Instagram to recommend a stream of accounts with names like “Eternally starved” and “Skinny binge.” Researchers explicitly flagged that the recommendation system was amplifying eating-disorder content.
• The decision to ship anyway:Meta's response was to shelve Instagram Kids in September 2021 after public pressure, but it did not change the underlying recommendation model on the main app.

Meta has disputed framing and methodology in the leaked documents, but has not produced internal research contradicting the core findings.

The CCDH TikTok study — 2.6 minutes

In December 2022, the Center for Countering Digital Hate published Deadly by Design, a study in which researchers created fresh TikTok accounts as 13-year-old users (the platform's minimum age) and recorded the time-to-first-harmful-content. Median times:

For accounts whose handle included “loseweight,” eating-disorder content appeared 12× more often than for control accounts. TikTok disputed the methodology but did not provide its own analysis of how often the platform serves this content to new teen accounts.

The grooming pipeline on Discord and Snapchat

The algorithmic harm story isn't only about content — it's about contact. On Snapchat's Quick Add and Discord's server discovery features, the recommendation system surfaces adults to minors and minors to adults based on engagement signals, mutual contacts, and shared interest groups. NCMEC's 2024 CyberTipline data showed online enticement reports up 158% year-over-year. Many of those reports describe a pattern that begins with a platform recommendation:

1. Minor joins a gaming or fandom-themed Discord server, surfaced by the platform's recommendation system.
2. Adult user in the server initiates DM contact — often referencing the shared interest as a credibility signal.
3. Conversation moves to a less-monitored channel (Snapchat for ephemeral images, Telegram for encrypted continuation).
4. Sextortion or in-person grooming follows within days or weeks.

See our deeper dives on Discord's predator pipeline and gaming grooming for case-by-case treatment.

The legal response — 42 states, one consolidated suit

On October 24, 2023, attorneys general from 42 states filed coordinated federal and state lawsuits against Meta in the Northern District of California, alleging that Instagram and Facebook's recommendation systems were designed to be addictive to minors in violation of consumer-protection statutes and COPPA. The federal case is consolidated as In re: Social Media Adolescent Addiction/Personal Injury Products Liability Litigation (MDL 3047) in N.D. Cal.

Key allegations across the complaints:

• Meta knowingly designed recommendation and notification systems to maximize teen engagement at the cost of well-being.
• Meta misrepresented the safety of its products to users, parents, and Congress.
• Meta allowed children under 13 on Instagram in violation of COPPA, then used those accounts' data to refine the recommendation model.

Massachusetts's standalone suit was allowed to proceed by the state Supreme Judicial Court in April 2026 after Meta's motion to dismiss was rejected. Additional state suits target TikTok (14 states + DC, October 2024, plus standalone Texas action), Snapchat (multistate), Discord (NJ April 2025, TX May 2026), and Roblox (TX November 2025, IA December 2025, IN May 2026).

The legislative response

Two federal bills directly target algorithmic harm to minors. Track both on the GuardKids legislation tracker.

• Kids Online Safety Act (KOSA) — Establishes a statutory duty of care for platforms to mitigate harms to minors including eating disorders, suicide, sexual exploitation, and substance-use promotion. Requires safer defaults for under-17 accounts. Passed the Senate 91–3 in the 118th Congress but stalled in the House; reintroduced in 119th Congress as S.1748, currently in the Senate Commerce Committee.
• TAKE IT DOWN Act — Signed into law as Public Law 119-12on May 19, 2025. Criminalizes nonconsensual intimate imagery including AI deepfakes of minors and requires platforms to remove flagged content within 48 hours. Doesn't regulate algorithms directly, but does establish a precedent for platform-side enforcement obligations.

What “safety by design” would actually look like

Researchers and child-safety organizations (Thorn, Common Sense Media, 5Rights, NCMEC) have converged on a small set of changes that would meaningfully reduce algorithmic harm to minors without requiring a wholesale ban:

• Default-off recommendation systems for under-18 accounts — feed is chronological, follower-based unless the minor and a parent co-elect otherwise.
• No interest-based ads targeting under-18 accounts (already law in the EU under the Digital Services Act for under-18; Meta has voluntarily implemented globally for under-16).
• Friction on adult-to-minor DMs — adult-aged accounts cannot DM minor-aged accounts unless the minor is a verified follower (Instagram partially implemented in 2021 for under-18, but DM-from-server-recommendation workarounds persist on Discord and Snap).
• Mandatory transparency: publishing how the recommendation model treats accounts identified as belonging to minors, including any signals it uses about age, suspected age, or implied age.

Where each major platform stands

Quick map to the platform report cards on GuardKids:

Sources

• Wall Street Journal, The Facebook Filesseries, 2021 — including “Facebook Knows Instagram Is Toxic for Teen Girls” (Sept 14, 2021).
• Center for Countering Digital Hate, Deadly by Design, December 2022.
• In re: Social Media Adolescent Addiction/Personal Injury Products Liability Litigation, MDL 3047 (N.D. Cal.).
• NCMEC CyberTipline 2025 Annual Report — online enticement category, 158% YoY increase.
• S.1748 — Kids Online Safety Act (119th Congress).
• Public Law 119-12 — TAKE IT DOWN Act.
• European Commission, Digital Services Act, Article 28 (protection of minors).