Meta to Notify Parents When Teens Discuss Suicide or Self-Harm With Its Chatbot

Meta announced that it will notify parents when their teenager discusses suicide or self-harm with the company's Meta AI chatbot. The company says it has built a dedicated system to identify conversations in which a teen makes a clear reference to hurting themselves, and that flagged chats will be manually reviewed before an alert is sent.

Meta stated that where a teen's intent is ambiguous, it will err on the side of caution and notify the parent. It acknowledged that this may sometimes mean alerting parents when there is no real cause for concern, describing the approach as a starting point that will be monitored.

The alerts are now live for parents using Instagram Parental Supervision in the United States, the United Kingdom, Australia, and Canada, and Meta says they will roll out globally by the end of the year. The update builds on existing alerts sent when a teen repeatedly searches for suicide or self-harm terms on Instagram, and on a feature that lets parents see the topics their teen discussed with Meta AI over the past week.

Meta also said its Limited Content setting, which places teens in a more restrictive experience, now applies to Meta AI, causing the chatbot to decline a broader range of prompts. The company added that it will contact emergency services when a conversation with Meta AI, whether the user is an adult or a teen, suggests someone is at risk of suicide, extending a practice it already applies to posts on Facebook and Instagram.

The changes arrive as technology companies face scrutiny from regulators and parents over how AI chatbots respond to users in crisis, particularly teenagers. For families, the measures raise questions about the reliability of automated detection and the weight placed on such systems in matters of a child's safety.

This report is based on original reporting by TechCrunch.