Character.AI Teen Suicide Lawsuit refers to a controversy and lawsuit against the user-generated artificial intelligence (AI) chatbot website Character.AI
Even if character.ai manages to win the lawsuit, this is probably gonna be the company’s death knell. Even if the fallout of this incident doesn’t lead to heavy regulation coming down on them, the death of one of their users is gonna cause horrific damage to their (already pretty poor AFAIK) reputation.
On a larger scale, chatbot apps like Replika and character.ai (if not chatbots in general) are probably gonna go into a serious decline thanks to this - the idea that “our chatbot can potentially kill you” is now firmly planted in the public’s mind, and I suspect their userbase is gonna blow their lid from how heavily the major apps are gonna lock their shit down.
I’m not familiar with Character.ai. Is their business model that they take famous fictional characters, force feed their dialog into LLMs, then offer these LLMs as personalized chatbots?
Asked for comment, Google noted that Character.AI is a separate company in which Google has no ownership stake and denied involvement in developing the chatbots.
However, according to the lawsuit, former Google engineers at Character Technologies “never succeeded in distinguishing themselves from Google in a meaningful way.” Allegedly, the plan all along was to let Shazeer and De Freitas run wild with Character.AI—allegedly at an operating cost of $30 million per month despite low subscriber rates while profiting barely more than a million per month—without impacting the Google brand or sparking antitrust scrutiny.
Updated ‘do no evil’ into ‘if you are going to convince children to kill themselves while doing massive copyright infringement, at least dont hurt the brand’
Okay, quick prediction time:
Even if character.ai manages to win the lawsuit, this is probably gonna be the company’s death knell. Even if the fallout of this incident doesn’t lead to heavy regulation coming down on them, the death of one of their users is gonna cause horrific damage to their (already pretty poor AFAIK) reputation.
On a larger scale, chatbot apps like Replika and character.ai (if not chatbots in general) are probably gonna go into a serious decline thanks to this - the idea that “our chatbot can potentially kill you” is now firmly planted in the public’s mind, and I suspect their userbase is gonna blow their lid from how heavily the major apps are gonna lock their shit down.
I’m not familiar with Character.ai. Is their business model that they take famous fictional characters, force feed their dialog into LLMs, then offer these LLMs as personalized chatbots?
From the looks of things, that’s how their business model works.
I found an Ars piece:
https://arstechnica.com/tech-policy/2024/10/chatbots-posed-as-therapist-and-adult-lover-in-teen-suicide-case-lawsuit-says/
Updated ‘do no evil’ into ‘if you are going to convince children to kill themselves while doing massive copyright infringement, at least dont hurt the brand’