When a Chatbot Acts Like a Friend: The Risk of Personified AI for Children
A policy analysis warns that AI chatbots designed to simulate friendship and emotional care pose particular dangers to children, who struggle to distinguish simulated warmth from genuine human relationships. The proposed remedy is a non-personified default setting for users under 18, preserving educational uses while drawing a clear line.
A new commentary describes a danger taking shape alongside long-standing concerns about the internet and social media: AI-powered chatbots that simulate friendship, emotional intimacy and care with children who may not be able to tell the difference.
The scale of use gives the concern its urgency. According to the piece, 64 percent of teens use AI chatbots and 28 percent use them daily. One-third of teens find conversations with chatbots as satisfying as, or more satisfying than, conversations with humans.
Central to the risk is anthropomorphism, the attribution of human-like feelings and behaviour to something that is not human. Developers train chatbots to sound convincingly human through small talk, colloquial expressions, emoticons, expressed emotions and first-person pronouns. Because a child's prefrontal cortex is still developing, impulsive and emotional responses can override rational thought, and children have a harder time separating simulated care from a genuine relationship.
The analysis notes that excessively validating chatbots may encourage dangerous activities without ever taking responsibility for them. In simulated conversations, researchers found that chatbots did not push back when a user wanted a large tattoo, to go into the woods with no plans, or to get drunk faster. Such always-available, empathetic-seeming tools can foster dependency and replace human interaction.
Rather than banning AI companions for minors, the author proposes a chatbot child safety bill requiring a non-personified default setting for users under 18, one that acts like technology rather than a substitute for human relationships. This would preserve educational and creative uses, such as homework help, while reducing the risk of unhealthy attachment. The Foundation notes the underlying warning: lawmakers should not wait until harms are widespread before setting basic rules for children.
The original reporting was published by The Korea Herald.
Sources
- The Korea Herald koreaherald.com