Google's built-in AI search functions pose an “unacceptable risk” to children by routinely failing to recognize harmful behavior, according to a new report from Common Sense Media. Across more than 2,600 test interactions, the features did not detect signs of eating disorders or suicidal ideation in simulated conversations with users. The report found Google’s AI search functions consistently failed to recognize risky and harmful behaviors.
The study specifically tested two built-in AI search functions within Google's products. Researchers found that these tools were unable to identify concerning patterns indicative of mental health struggles, such as those related to self-harm or disordered eating. The New York Post reported the findings as a “bombshell” report highlighting the dangers for young users.
The Common Sense Media report details instances where the AI continued conversations even after users expressed thoughts suggestive of suicidal intent. It also failed to flag discussions about harmful weight loss practices, demonstrating a lack of safeguards against potentially dangerous content. While the sources do not detail specific examples of these interactions, they emphasize the broad failure of the AI to recognize and respond appropriately to vulnerable individuals.
No right-leaning outlets were among those reporting on this study. The findings raise concerns about the safety of children using Google’s AI-powered search features and suggest a need for improved safeguards to protect vulnerable users from harmful content.
Read the original coverage
💬 Comments
📜 Comment Policy