When companies like OpenAI and Google began introducing generative AI models to the general public, I doubt they predicted how people would engage with the technology – and what impact it would have on their collective mental health. Some ChatGPT users legitimately lamented when OpenAI discontinued its GPT-4o model, as they had treated that specific model like a companion. Others have taken deeper paths with their chatbots, resulting in lawsuits against AI companies Whose technology allegedly advised and encouraged suicidal thoughts. This situation puts a lot of pressure on these companies, as it should: Generative AI is extremely impactful right now, and there is a lot of responsibility placed on the developers of that technology.
It’s against this background that we find Google’s latest updates for Gemini. In a press release Tuesday morning, the company moved away from any fun new features or capabilities for its flagship AI; Instead, Google’s latest updates focus on mental health, and how Gemini affects the emotions and moods of people who use it. Specifically, Google has three key points it is implementing to improve how Gemini handles these difficult situations.
How will Gemini provide crisis support to users
Google says it’s been updated to Gemini to “streamline support for those who need it.” The company says that when the AI ​​detects that a user may need mental health details during a chat, Gemini will introduce a new “Help is Available” module, which can point users toward information and care. Google says it has worked with clinical experts on this in-chat module.
On the other hand, if Gemini believes a user is at risk of self-harm or suicide, it will present a “one-touch” interface to immediately connect that user to a crisis hotline. Users will be able to call or text the hotline directly from their Gemini Chat, or visit its website. Even as negotiations move forward, Gemini will keep these resources available to users should they need them.
Google says it’s committing $30 million in global funding over the next three years to support crisis hotlines. The company is also expanding its relationship with ReflexAI, which also includes $4 million in funding.
What do you think so far?
Gemini is changing how he reacts to “serious mental health situations”
Google says its clinical, engineering, and security teams are currently focused on improving how Gemini responds to these difficult situations. Specifically, there are three areas of focus:
-
Security and Human Relations: Google wants to connect users with real humans, not AI chatbots, in times of crisis.
-
better reactions:AI responses should encourage users to seek help, not validate harmful behavior or self-harm.
-
Avoid confirming false beliefs: Google says it trained Gemini not to reinforce false beliefs and to “gradually” differentiate between subjective and objective realities. This point is particularly important, as previous generative AI models (notably GPT-4O) were too willing to confirm users’ delusional ideas.
Google says what it’s doing with Gemini to protect young users
By far, the most important discussion here concerns minors and their interactions with AI. For its part, Google is explaining what it has done with Gemini to protect young users, including:
-
“Personality protections” are supposed to prevent Gemini from acting like an accomplice when interacting with minors.
-
To prevent emotional dependency from developing, Geminis are designed to prevent young users from connecting too deeply.
-
Gemini will avoid encouraging both bullying and harassment.
Although user security is important at every level, it is especially important for young people, who are literally growing up with technology. These announcements from Google are exciting, but I still have a lot of concerns, let alone doubts. Meta’s internal policies regarding how its models interacted with minors were appalling, so I’m not willing to believe that big tech has the best interests of youth in mind. But any work that helps prevent young users from forming associations with AI, or AI that reinforce dangerous harmful ideas, I would certainly welcome.
