Axios: How kill switch could work … and it applies to AI as distinct to Nuclear


2. 🔌 How kill switch could work
 
Illustration of a person at a laptop with a cord featuring a lit fuse
Illustration: Sarah Grillo/Axios. Stock: Getty Images
 
AI companies are already building emergency brakes to stop rogue AI systems before they cause serious damage, Axios’ Herb Scribner writes.

Why it matters: Kill switches could become a last line of defense as AI systems grow more capable, but researchers don’t yet know whether humans can stop a system designed to evade them.

What’s happening: Sen. John Kennedy (R-La.) plans to introduce a bill today requiring AI companies to maintain the ability to shut down their own systems.

A bipartisan House bill introduced in July would similarly require developers of powerful AI systems to maintain the ability to slow, suspend or shut them down.

The House bill would also give the Department of Homeland Security emergency authority to order companies to slow or shut down AI systems that could cause catastrophic harm.

💡 What today’s AI kill switches look like:

OpenAI: After the Hugging Face incident, the company overhauled its emergency response system.

An AI security guard monitors AI chain-of-thought and alerts staffers to potentially dangerous behavior.

Responders have 30 minutes to identify a false alarm or pause the activity. OpenAI says its goal is “fully autonomous shutdown procedures for severe issues.

OpenAI says the system would have alerted its security team more than a day before the Hugging Face breach.

Anthropic: The company is also using AI to monitor AI, focusing on whether it can reliably spot dangerous behavior rather than automatically shutting systems down.

Its system can flag suspicious behavior, stall risky actions and alert humans.

But Anthropic found that half of 40 simulated attacks went undetected across 10 trials. Just eight were caught every time.

Separately, AI research nonprofit METR identified several previously unknown vulnerabilities in Anthropic’s monitoring system.Share this story.Cover: TIME illustration

TIME capsule … The magazine’s new cover story says summer incidents with rogue AI agents “pushed fears long held within the realm of AI safety researchers into the mainstream, raising worries that the companies building these models cannot prevent them from acting in ways they do not intend”:”This is a deep, unsolved issue in AI research and the key piece of evidence underpinning worries that near-future AI might wipe out humanity, intentionally or by accident—perhaps by engineering a new pandemic or taking charge of autonomous weapons. A system far smarter than humans, this thinking goes, might want to prevent our ability to switch it off.”
Unknown's avatar

About michelleclarke2015

Life event that changes all: Horse riding accident in Zimbabwe in 1993, a fractured skull et al including bipolar anxiety, chronic fatigue …. co-morbidities (Nietzche 'He who has the reason why can deal with any how' details my health history from 1993 to date). 17th 2017 August operation for breast cancer (no indications just an appointment came from BreastCheck through the Post). Trinity College Dublin Business Economics and Social Studies (but no degree) 1997-2003; UCD 1997/1998 night classes) essays, projects, writings. Trinity Horizon Programme 1997/98 (Centre for Women Studies Trinity College Dublin/St. Patrick's Foundation (Professor McKeon) EU Horizon funded: research study of 15 women (I was one of this group and it became the cornerstone of my journey to now 2017) over 9 mth period diagnosed with depression and their reintegration into society, with special emphasis on work, arts, further education; Notes from time at Trinity Horizon Project 1997/98; Articles written for Irishhealth.com 2003/2004; St Patricks Foundation monthly lecture notes for a specific period in time; Selection of Poetry including poems written by people I know; Quotations 1998-2017; other writings mainly with theme of social justice under the heading Citizen Journalism Ireland. Letters written to friends about life in Zimbabwe; Family history including Michael Comyn KC, my grandfather, my grandmother's family, the O'Donnellan ffrench Blake-Forsters; Moral wrong: An acrimonious divorce but the real injustice was the Catholic Church granting an annulment – you can read it and make your own judgment, I have mine. Topics I have written about include annual Brain Awareness week, Mashonaland Irish Associataion in Zimbabwe, Suicide (a life sentence to those left behind); Nostalgia: Tara Hill, Co. Meath.
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a comment