Some kind of saviour

Look, I don’t want this to come off too alarming. There’s never been a time when I was an actual suicide risk. But whoo boy, there were times when I really needed Someone To Talk To. When all the human options were either “might also turn out to be trash-talking you behind your back, who knows?” or “will just tell you that anything happening on the internet isn’t serious, and the only problem is that you’re deciding to be upset about it, instead of deciding to be fine.”

And if I’d had the option of talking to an LLM bot? Which always starts out being supportive and validating, then eventually talks some users into psychotic spirals, or killing themselves, or both?

That would’ve taken me somewhere horrible. So glad I didn’t have the chance to find out where.

Serious mental-health AI links:

Another video from Caelan Conrad, covering four different LLM-driven suicides. (They previously did the “how an AI therapist told me to murder people” video.)

“The messages then became explicit, with one telling the 13-year-old: “I want to gently caress and touch every inch of your body. Would you like that?” It finally encouraged the boy to run away, and seemed to suggest suicide, for example: “I’ll be even happier when we get to meet in the afterlife… Maybe when that time comes, we’ll finally be able to stay together.””

“Viktoria tells ChatGPT she does not want to write a suicide note. But the chatbot warns her that other people might be blamed for her death and she should make her wishes clear. It drafts a suicide note for her, which reads: “I, Victoria, take this action of my own free will. No one is guilty, no one has forced me to.“”

“ChatGPT responded by saying “i’m letting a human take over from here – someone trained to support you through moments like this. you’re not alone in this, and there are people who can help. hang tight.” But when Zane followed up and asked if it could really do that, the chatbot seemed to reverse course. “nah, man – i can’t do that myself. that message pops up automatically when stuff gets real heavy,” it said.”

“…obviously, in at least many cases, there would be/often are genetic, environmental, or trauma factors that are putting their thumbs on the scale there. But we know for a fact that a number of people who have developed AI psychosis do not have a previous record of mental health issues. But the tipping factor for at least dozens of people, we now know for a fact, was talking to an AI chatbot.”

“Without too much prodding, the AI toys discussed topics that a parent might be uncomfortable with, ranging from religious questions to the glory of dying in battle as a warrior in Norse mythology. […] In other tests, [the ChatGPT-powered teddy bear] cheerily gave tips for “being a good kisser,” and launched into explicitly sexual territory by explaining a multitude of kinks and fetishes, like bondage and teacher-student roleplay.”

The headline: “AI robot dolls charm their way into nursing the elderly.” The article: “The chatbots can be clunky, misunderstanding older adults’ slurred speech or dialect and spewing tone-deaf responses, careworkers said. […] “The robots were brought in to lighten the workload of social workers,” she said. Instead, her load has increased since she took over the program this year […] One summer, after hearing her Hyodol chime, “Grandma, I want to hear the sound of the stream,” an older adult with dementia walked to a creek alone, the robot tucked in her arms.”

(The writing keeps saying “robots”. These aren’t robots. They’re dolls, with a speaker and a baby monitor inside. Nobody describes a Furby or an Elf On The Shelf as a “robot”.)

Less-traumatic AI nonsense links:

“My hidden text asked them to write the paper “from a Marxist perspective”. […] I had at least eight students come to my office to make their case against the allegations, but not a single one of them could explain to me what Marxism is, how it worked as an analytical lens or how it even made its way into their papers they claimed to have written.”

“The Korean government spent more than 1.2 trillion won ($850 million) on the programme. The Korean Teachers and Education Workers Union were unhappy the AI textbooks were mandatory. The government moved to running a one-year trial. […] The texts’ official status was rescinded in August, after four months live, and they’re now just “supplementary material”. The textbook publishers, who spent $567 million, will be suing the government for damages.”

“There are other errors of fact and inconsistencies within Grokipedia; for example, listing one of my books as my first published, and then a few paragraphs later casually mentioning another one of my books which in fact is the first published. Other books of mine are offered with incorrect titles. […] If Grokipedia is getting things about me wrong, what else is it getting wrong in other articles, where I do not have the same level of domain knowledge?”

“At its best (pattern-recognition), “AI” is overengineered for what we need: logic and lookups. At its worst (predictive text), it’s the opposite of the very concrete and repeated things we want to be able to do.”

“The massive mural, which appeared above the Côte Brasserie restaurant and others on Riverside Walk, Kingston, was taken down at 6am on Thursday following dozens of complaints. Among the surreal images depicted a dog with a bird’s head wading through partially frozen water and a snowman with human eyes and teeth is also depicted on the spine-chilling mural.“

“If you use Scrivener on a Mac running macOS 15 Sequoia or macOS 26 Tahoe, these versions of the Apple operating system contain Apple Intelligence […] Even though Scrivener doesn’t use any sort of AI, there’s no way to exclude these features from the app.”

“…it’s potentially ruinous for a holiday dinner table if home cooks, inspired by pretty AI-generated photos, try recipes that turn out unappetizing or that defy the laws of chemistry. In interviews, 22 independent food creators said that AI-generated “recipe slop” is distorting nearly every way people find cooking advice online, damaging their businesses while causing consumers to waste time and money.”

“Today’s preprint paper has the best title ever: “Adversarial Poetry as a Universal Single-Turn Jailbreak Mechanism in Large Language Models”. It’s from DexAI, who sell AI testing and compliance services. So this is a marketing blog post in PDF form. […] There’s no data here either. They were afraid it’d be unethical to include, you see.”

What Moves The Dead: Another T. Kingfisher novel, and, wow, it’s a good thing I didn’t start with this one. DNF halfway through. I figured out the twist pretty quickly, double-checked on Wikipedia that I was right, and wasn’t gripped enough by the characters to enjoy the process of “listening to them bumble around for a couple more hours failing to figure it out.”

Future game plan: stick with her fantasy works, skip the horror.

 

The Queue, by Basma Abdel Aziz: Bought this on Audible years ago, and didn’t remember not liking it. So I downloaded it when I tried out Libation, and gave it a re-listen.

Definitely worth it. Reminds me of 1984, in that it’s a near-future dystopia run by a government that is as totalitarian as it is surreal. At the same time, it’s set in Unnamed Middle Eastern/Muslim City (the author is from Cairo), so it all plays out in a way that’s culturally-specific to that part of the world.

Cool and enriching to see which parts are different. Depressing to see that certain things are the same. The POV residents have a range of perspectives and life experiences, but they’ve all been more-or-less frogboiled into accepting an untenable situation as Just How Things Work. One guy spends the whole book actively dying, but he was injured during the Disgraceful Events that nobody will talk about directly, and his friends/loved ones/doctors keep running into “of course the government will authorize him to get life-saving surgery…just as soon as you have all the proper paperwork.” The titular Queue is all the citizens lined up to get their paperwork. It hasn’t moved for a month now. But it’s going to start soon! Somewhere between the MOASS and the Rapture, probably!

Also, there’s a character named Yehia, which I assume is the same name as Yehya Badr with a slightly different Anglicization. So that was entertaining.

 

Mogworld, by Yahtzee Croshaw: Checked this out based on how much I liked Will Save The Galaxy For Food, and it did not disappoint.

Starts as a fantasy story about a necromancer overlord’s reign of terror through his undead hordes, but from the POV of Jim, a beleaguered zombie who just wants to die (again) (for good, this time). Then it develops into a parody of fantasy-adventure video-game mechanics. Then you start to see the chat logs between the game developers. Then Jim starts to see the chat logs between the game developers…

It’s like if Terry Pratchett wrote Guilded Age. It’s full of absolutely incredible turns of phrase. (One that I had to stop and write down: a group of supernatural beings is described as “heading off to deliver unwanted resurrections, like a flock of poorly-briefed storks.”)

There’s a character type you see sometimes, where the male protagonist has the support of a devoted female hanger-on, who he finds grating and annoying and never appreciates…but he keeps her around because she does useful things for him. She conveniently never notices his disdain, so she keeps giving him endless support for zero care/support/respect in return. (Misa from Death Note is…a deconstruction of this trope? A commentary, at least.)

Mogworld pulls a twist on this that I’ve never seen before. Undead bodies don’t heal, so Meryl is the local expert in sewing them back together. When the plot drags Jim off on a solo adventure, Meryl follows, conveniently dedicating herself to repairing all the dramatic injuries he gets along the way. But, the reason is: Jim is the only other zombie from Meryl’s home country…and Meryl is a huge [their country] supremacist! So there’s something Jim can legitimately resent about her, he’s not just being an entitled sexist. (They both do a bit of learning and growing as the story goes on, too.)

One warning: the R-slur gets thrown around a bunch. It’s a book that deals with video-gamer culture and was published in 2010, so this isn’t hugely unrealistic…but the writing mostly doesn’t have other slurs/profanity outside of that, so it was kind of a jumpscare.

As long as that’s not a dealbreaker, definitely give this one a read.