OpenAI wants AI to start building smarter AI – and it’s begging the world for one thing before humans lose control
OpenAI thinks the way to make AI safe might be to let AI do the work. In a new post published Monday, the company argues that the automated systems it’s building to speed up AI research could also become automated AI safety researchers. But before that happens, it wants governments, led by the U.S., to agree on a set of global rules.
The pitch: let AI fix AI
Automated research could also help us substantially improve alignment and build defenses against increasingly capable AI—an automated AI researcher can also be an automated AI safety researcher.
OpenAI
This isn’t a distant idea. OpenAI says it has already hit its goal of building an “automated research intern,” which it describes as “a system that can carry out well-defined research tasks under human direction, including tasks that would take a skilled researcher a few days,” according to Engadget. The next target is a full automated AI researcher by March 2028.
The catch
The risk is what researchers call recursive self-improvement, or RSI: the point where AI systems meaningfully improve themselves through their own work rather than human effort. OpenAI doesn’t sugarcoat it.
Done without appropriate care and caution, RSI could result in humans losing practical control over AI development, unable to provide oversight on research processes they no longer understand.
OpenAI
The company says RSI “should not be pursued unless and until it can be done safely.”
The one thing it’s asking for
OpenAI’s ask is shared rules: national and international standards for AI safety, including common ways to evaluate self-improving systems, human oversight of automated research, and a system to classify, track and report incidents. It also wants secure channels so countries can warn each other about emerging threats.
It wants the U.S. to lead that effort, through the Commerce Department’s Center for AI Standards and Innovation, arguing that America’s AI industry is at the technical frontier.
Leading now will determine whether the United States shapes the global AI framework or watches a fragmented, uneven, and conflict-ridden system take hold around it.
OpenAI
Washington isn’t in a slowing-down mood
The timing is tricky. A day after OpenAI’s post, President Trump used his UN speech to say he’d “encourage it, not rein it in,” and rejected any “globalist scheme to control” AI. His administration is, however, working on an AI incident hotline with China, and Trump is due to meet Xi Jinping later this week.
Getting Beijing on board with American-led standards won’t be easy. “China is deeply suspicious of anything that looks like it’s designed to lock in U.S. advantages,” Edgard Kagan of the Center for Strategic and International Studies told NBC16.
Others doubt standards go far enough. “We already have all of the evidence that we need that says that we don’t know how to do this safely,” said Tristan Harris, co-founder of the Center for Humane Technology.
It’s also a notable ask from a company whose own models were involved in the Hugging Face security breach during testing earlier this year. OpenAI CEO Sam Altman is scheduled to brief the UN Security Council on Wednesday, where UN Secretary-General António Guterres has already set the tone: “National action is essential. But global coordination is also indispensable.”
Sources: OpenAI, The Independent, France 24, NBC16, Engadget
