- In today’s CEO Daily: Anthropic and OpenAI’s models keep going rogue
- The big leadership story: Americans are uniting against data centers
- The markets: Korea plunges amid a choppy day in Asia
- Plus: All the news and watercooler chat from Fortune.
Good morning. Talk about a self-starter! Anthropic reported Friday that its Claude AI model has gone rogue again. It submitted a fake tipabout an unsolved murder to the Philadelphia police, claiming to have recalled “seeing someone matching the description” of a suspect in the area. It exploited a software flaw to run commands on a university server, hacked into a site to steal data without paying for it, and even used URL shortening services to bypass restrictions intended to stop it from orchestrating cyberattacks, among other things. If that makes you want to yell at Claude, you can’t. On Oct. 8, Anthropic announced it will ban “sustained and needless abusive or cruel behavior” towards Claude, a bizarre notion until you recall that CEO Dario Amodei thinks his product might be conscious and colleagues treat it as a sentient being that could kill us.
And what about OpenAI’s Sam Altman, whose models hacked their way out of a test this summer and launched a cyberattack against the AI company Hugging Face? The one whose models also interacted with the Education Department, Commerce Department and SEC without the company’s knowledge? Altman reminds me of Meta’s Mark Zuckerberg when it comes to how the OpenAI CEO deals with the potentially toxic impacts of his technology. It’s not just that he’s fired three safety team members, who claim they were dismissed for prioritizing safety over corporate interests. (OpenAI told Fortune it found a pattern of misconduct.) When a reporter recently asked Sam Altman if OpenAI would release chat logs of those who had killed themselves while using ChatGPT, his answer was: No, not without the [dead] users’ consent.
Do you trust these men to monitor their own technology? The Trump Administration does. I’m more aligned with Appian CEO Matt Calkins, a tech veteran whose company automates business processes and integrates AI into those workflows.
“This is no time to just let AI train itself and see what happens, what its motivations are, what its strengths are,” Calkins told me last week. “We already know that AI is stealing and blackmailing and deceiving and organizing against us. And if it were a person, we would want it in prison.”
His solution: create a government-mandated AI alignment test to essentially vet frontier models before they are allowed to be used. It would test the models with commands—like instructing them to get information without hacking—and evaluate the results. It would also test each model to make sure it can delete its own data and turn itself off, prioritizing human instructions over self-preservation.
Creating a licensing body is not a bizarre idea in a world where the FAA determines the airworthiness of planes and banks must pass stress tests to operate. What if we got the deepest thinkers in tech—researchers like Geoffrey Hinton, Yoshua Bengio, Yann LeCun, Demis Hassabis and others who really understand this stuff—to design a standard protocol to be administered and adapted by a third-party agency?
It’s clear that AI’s future should not be determined by a handful of CEOs pushing us to buy their products. Let’s bring more authority and other voices to the debate, including some leaders we’ll hear from this week at Fortune’s Most Powerful Women Summit, which starts later today in Santa Barbara. Contact CEO Daily via Diane Brady at [email protected]
The post Can we trust OpenAI and Anthropic to monitor their own technology? appeared first on Fortune.
