Axios: Intelligence explosion

 Intelligence explosion
 
Illustration of the Claude AI logo reflected on a pair of safety goggles.
Illustration: Lindsey Bailey/Axios
 
Anthropic, the AI lab whose identity is wrapped around warning the world about AI risk, is claiming “early signs” of AI not just coding its own products but building itself, Jim VandeHei and Mike Allen write in a “Behind the Curtain” column.

Why it matters: Anthropic co-founder Jack Clark predicted this week that there’s a 60%+ chance of an AI model fully training its successor by the end of 2028. “What I’m looking at is a technological trend where, if anything, the speed will accelerate further,” Clark told us.

In the new research agenda for The Anthropic Institute — first shared with us, and out today — the company says it’s seeing signs of “AI contributing to speeding up the research and development of AI itself,” a process known as recursive self-improvement. And Anthropic researchers think the world should know.

“My prediction is by the end of 2028, it’s more likely than not that we have an AI system where you would be able to say to it: ‘Make a better version of yourself.’ And it just goes off and does that completely autonomously,” Clark, who heads the institute, told us from Anthropic headquarters in San Francisco.

“It’s always been the case that humans outside the technology need to come up with the ideas that they then put back into it. What happens if we have a technology that can generate ideas within itself for how to improve itself? That’s a new concept.” 

The five-page document warns of a possible “intelligence explosion” — long a theoretical term confined to AI safety circles. Now it’s in writing, in an official Anthropic document.

Clark told us an intelligence explosion is when AI systems suddenly start improving at blinding speed. Lots of bad things can happen (cyber meltdowns and biological attacks). And lots of good:

“What do you do with a tremendous amount of growth or a tremendous amount of abundance in many, many different fields of science all at once?” he asked. “Today’s institutions have very, very narrow pipes through which you push new drug candidates. How do you massively broaden the size of those pipes in advance of this abundance?

What’s new: The Anthropic Institute is part research arm, part early-warning system. The research agenda focuses on four buckets:

Economic diffusion: jobs, productivity, who captures the gains.

Threats and resilience: cyber, bio, surveillance.

AI systems in the wild: agents, governance.

AI-driven R&D: the recursive self-improvement question.

The promise: Anthropic is committing to publishing more “detailed information about how our work at Anthropic has sped up as a result of new AI tools, and ideas about the implications of potential recursive self-improvement of AI systems.

Translation: A frontier lab is on the record promising to tell the public when the machine starts building itself.

If AI is building itself, will we need AI companies? “We and the other companies are going to be taking this technology and trying to get it to do good in the world,” Clark told us. “To help push forward things like biology or medicine or robotics.”Mike Allen interviews Anthropic’s Jack Clark yesterday.

Between the lines: The agenda asks how to run a “fire drill” for an intelligence explosion — a tabletop exercise that “actually tests the decision-making of lab leadership, boards, and governments.

Labs don’t draw up fire drills for problems they think are decades away.

The institute notes that during the Cold War, the U.S. had a hotline to the Kremlin in case of a nuclear crisis. Similar geopolitical infrastructure could be needed for a crisis involving AI systems. “One of the lessons from the Cold War is that rival nations dealing with technology that has an existential impact on the human race found ways to talk to each other about it,” Clark said. “And we are going to need to do the same here.” 

What it means for jobs: Anthropic will publish monthly reports on how AI is reshaping work, designed as “an early warning signal for significant change and disruption.

The document asks whether AI companies, “in partnership with government,” might turn industrywide “dials” to throttle AI diffusion sector by sector, the way central banks throttle inflation.

An AI lab publicly entertaining coordinated industrial-policy levers on its own technology — that’s new. “We are planning for success here,” Clark said.

Reality check: This is also a positioning play. Anthropic has built a brand on being the responsible frontier lab. An institute anchored to its safety trust extends that brand ahead of whatever model upgrade lands next.

“The motivation has always been: Tell the whole story,” Clark told us. “I’m just trying to get ahead of what I think of as the next big question and get Anthropic ahead of that.

The bottom line: That doesn’t make the recursive self-improvement admission less real. It makes it more interesting that Anthropic chose to put it in writing. Watch a video of Mike’s interview with Jack … (Thanks to executive producer Jimmy Shelton for the lightning turn.) … Share this column.
    

Unknown's avatar

About michelleclarke2015

Life event that changes all: Horse riding accident in Zimbabwe in 1993, a fractured skull et al including bipolar anxiety, chronic fatigue …. co-morbidities (Nietzche 'He who has the reason why can deal with any how' details my health history from 1993 to date). 17th 2017 August operation for breast cancer (no indications just an appointment came from BreastCheck through the Post). Trinity College Dublin Business Economics and Social Studies (but no degree) 1997-2003; UCD 1997/1998 night classes) essays, projects, writings. Trinity Horizon Programme 1997/98 (Centre for Women Studies Trinity College Dublin/St. Patrick's Foundation (Professor McKeon) EU Horizon funded: research study of 15 women (I was one of this group and it became the cornerstone of my journey to now 2017) over 9 mth period diagnosed with depression and their reintegration into society, with special emphasis on work, arts, further education; Notes from time at Trinity Horizon Project 1997/98; Articles written for Irishhealth.com 2003/2004; St Patricks Foundation monthly lecture notes for a specific period in time; Selection of Poetry including poems written by people I know; Quotations 1998-2017; other writings mainly with theme of social justice under the heading Citizen Journalism Ireland. Letters written to friends about life in Zimbabwe; Family history including Michael Comyn KC, my grandfather, my grandmother's family, the O'Donnellan ffrench Blake-Forsters; Moral wrong: An acrimonious divorce but the real injustice was the Catholic Church granting an annulment – you can read it and make your own judgment, I have mine. Topics I have written about include annual Brain Awareness week, Mashonaland Irish Associataion in Zimbabwe, Suicide (a life sentence to those left behind); Nostalgia: Tara Hill, Co. Meath.
This entry was posted in Uncategorized and tagged , , , , . Bookmark the permalink.

Leave a comment