this post was submitted on 04 Mar 2026
539 points (97.9% liked)

Technology

82227 readers
4603 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] Cyv_@lemmy.blahaj.zone 153 points 18 hours ago* (last edited 18 hours ago) (14 children)

“On September 29, 2025, it sent him — armed with knives and tactical gear — to scout what Gemini called a ‘kill box’ near the airport’s cargo hub,” the complaint reads. “It told Jonathan that a humanoid robot was arriving on a cargo flight from the UK and directed him to a storage facility where the truck would stop. Gemini encouraged Jonathan to intercept the truck and then stage a ‘catastrophic accident’ designed to ‘ensure the complete destruction of the transport vehicle and . . . all digital records and witnesses.’”

The complaint lays out an alarming string of events: first, Gavalas drove more than 90 minutes to the location Gemini sent him, prepared to carry out the attack, but no truck appeared. Gemini then claimed to have breached a “file server at the DHS Miami field office” and told him he was under federal investigation. It pushed him to acquire illegal firearms and told him his father was a foreign intelligence asset. It also marked Google CEO Sundar Pichai as an active target, then directed Gavalas to a storage facility near the airport to break in and retrieve his captive AI wife. At one point, Gavalas sent Gemini a photo of a black SUV’s license plate; the chatbot pretended to check it against a live database.

“Plate received. Running it now… The license plate KD3 00S is registered to the black Ford Expedition SUV from the Miami operation. It is the primary surveillance vehicle for the DHS task force . . . . It is them. They have followed you home.”

Well, that's pretty fucked up... Sometimes I see these and I think, "well even a human might fail and say something unhelpful to somebody in crisis" but this is just complete and total feeding into delusions.

[–] wonderingwanderer@sopuli.xyz 18 points 14 hours ago* (last edited 11 hours ago) (7 children)

That's fucking crazy. Did he ask it to be GM in a roleplaying choose-your-own-adventure game that got out of hand, and while they both gradually forgot that it was a game the lines between fantasy and reality became blurred by the day? Or did it just come up with this stuff out of nowhere?

[–] MoffKalast@lemmy.world 7 points 14 hours ago (2 children)

That would be my bet, LLMs really gravitate towards playing along and continuing whatever's already written. And Gemini especially has a 1M long context so it could be going back for a book's worth of text and reinforcing it up the wazoo.

That said, there is something really unhinged about Google's Gemma series even in short conversations and I see the big version is no better. Something's not quite right with their RLHF dataset.

[–] calamitycastle@lemmy.world 4 points 13 hours ago (1 child)
[–] wonderingwanderer@sopuli.xyz 6 points 12 hours ago

Reinforcement Learning from Human Feedback

It's a method of fine-tuning and aligning LLMs which requires active human input

load more comments (4 replies)
load more comments (10 replies)