Altman tells the UN Security Council a 0.1% catastrophe risk is not acceptable
Sam Altman told the United Nations Security Council on September 23 that no level of catastrophic risk from AI is acceptable, down to a tenth of a percent. "It doesn't matter whether people put the risk of catastrophe at 10%, or 1%, or 12%, or .1%," OpenAI's published transcript reads. "None of these levels are remotely acceptable."
Altman told the Council that OpenAI has unilaterally slowed down in the past and will do so again.
Altman put a condition on training alongside it: "And we should not train models that we cannot make an extremely strong case that we will be able to keep under human control."
OpenAI published the text as remarks as delivered. Altman put no threshold of his own in place of the ones he rejected, and named no system.
Altman says OpenAI has slowed down before
"We have unilaterally slowed down in the past. We will do so in the future," Altman said, having just told the Council that OpenAI does not believe it is locked in a race where it is unable to do that. It is a claim about the company's own conduct, and the transcript does not say which release he meant.
He put the same point to the rest of the field: "The industry must not accept too much technological risk just because the benefits are too great and that they feel too important to slow down." Beating rivals on pace, he said, is not a reason to make rash decisions.
Self-improving systems are where he draws the line
Altman named recursive self-improvement as the point where the concern sharpens. "That concern becomes especially important as we approach systems that can improve themselves and future versions of themselves," he said, before adding: "This moment calls for extreme care."
He set out two failure modes rather than three, saying he largely agreed with Professor Bengio but would say two instead. One is losing control of the future to AI; the other is concentrating too much power in too few hands, and on that he said no person, company or country should be able to use the most powerful AI models to impose their worldview on everyone else.
The maths record was his evidence for the pace
To argue that the timeline has compressed, Altman ran through four summers. Three summers ago the models were okay at grade-school maths, two summers ago they were pretty good at high-school competitions, last summer they reached gold-medal level at the most prestigious international maths competition, and a few weeks ago one solved a Millennium Prize Problem, the Navier-Stokes equations.
Those equations, he noted, are used for aircraft design, weather forecasting and the study of blood flow.
What he asked the Council for was common international standards on measuring capabilities, assessing risks and reporting incidents, plus secure channels between governments and infrastructure operators, and he warned that the most important decisions cannot be left to labs in San Francisco alone. He endorsed no particular law, which sets the ask apart from the bill from Senator Bernie Sanders and Representative Greg Casar that would ban superintelligent AI outright.
On the company's own place in it, he was brief: "We might be one of the builders of this technology, but we are not the heroes of this story."
