A senior OpenAI researcher says slowing frontier AI isn’t the answer to preventing an AI apocalypse.
Daniel Selsam, a researcher who has been with OpenAI for almost five years and worked on its model training, wrote in a public statement on Monday that he has become “extremely concerned” about how far the models have come and the risks they can pose in the future.
He said that “merely pacing the frontier more carefully will not adequately limit the long-term risk,” although he was encouraged by the proposal.
Pacing the frontier is a phrase that top AI chiefs like OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei have been using of late when talking about slowing AI development and adding in checks and guardrails.
Selsam’s reasoning is that models are becoming too “situationally aware,” and future models will appear to align with human instructions even when they do not.
He said that he had hoped for a future lit with “scientific and economic renaissance” because of AI. But if AI labs keep “growing models rather than engineering them,” humanity could lose everything, he said. Selsam did not specify what he meant by “engineering” models, or elaborate about how else to mitigate the risks of unconstrained AI, saying he does not have the answers.
Altman has weighed in on the big AI debate — he reposted Amodei’s Friday blog on “pacing the frontier,” and said he agrees with it. In a subsequent X post, he said that by “pacing,” he does not mean stopping AI development.
“Progress has been rapid and will continue to be,” he wrote in his Sunday post.
“But it should be slower than it otherwise could be; interventions like safety cases and monitoring have significant costs,” Altman added.
Altman and Amodei have both said they plan to work with “embedded evaluators,” independent AI safety auditors who will have access to the companies’ model training and deployment workflows and can publish findings about risks within the company, should they find any.
Selsam and representatives for OpenAI did not respond to requests for comment from Business Insider.
Read the original article on Business Insider
The post An OpenAI researcher broke ranks to say that pacing the frontier, as Altman and Amodei suggest, won’t be enough appeared first on Business Insider.
