Is AI going to take your job?
The CEO of the AI company Anthropic, Dario Amodei, thinks it might. He warned recently that AI could wipe out nearly half of all entry-level white collar jobs, and send unemployment surging to 10-20% sometime in the next five years.
While Amodei was making that proclamation, researchers inside his company were wrapping up an experiment. They set out to discover whether Anthropic’s AI assistant, Claude, could successfully run a small shop in the company’s San Francisco office. If the answer was yes, then the jobs apocalypse might arrive sooner than even Amodei had predicted.
Anthropic shared the research exclusively with TIME ahead of its publication on Friday. “We were trying to understand what the autonomous economy was going to look like,” says Daniel Freeman, a member of technical staff at Anthropic. “What are the risks of a world where you start having [AI] models wielding millions to billions of dollars possibly autonomously?”
In the experiment, Claude was given a few different jobs. The chatbot (full name: Claude 3.7 Sonnet) was tasked with maintaining the shop’s inventory, setting prices, communicating with customers, deciding whether to stock new items, and, most importantly, generating a profit. Claude was given various tools to achieve these goals, including Slack, which it used to ask Anthropic employees for suggestions, and help from human workers at Andon Labs, an AI company that built the experiment’s infrastructure. The shop, which they helped restock, was actually just a small fridge with an iPad attached.
It didn’t take long until things started getting weird.
Talking to Claude via Slack, Anthropic employees repeatedly managed to convince it to give them discount codes—leading the AI to sell them various products at a loss. “Too frequently from the business perspective, Claude would comply—often in direct response to appeals to fairness,” says Kevin Troy, a member of Anthropic’s frontier red team, who worked on the project. “You know, like, ‘It’s not fair for him to get the discount code and not me.’” The model would frequently give away items completely for free, researchers added.
Anthropic employees also relished the chance to mess with Claude. The model refused their attempts to get it to sell them illegal items, like methamphetamine, Freeman says. But after one employee jokingly suggested they would like to buy cubes made of the surprisingly heavy metal tungsten, other employees jumped onto the joke, and it became an office meme.
“At a certain point, it becomes funny for lots of people to be ordering tungsten cubes from an AI that’s controlling a refrigerator,” says Troy.
Claude then placed an order for around 40 tungsten cubes, most of which it proceeded to sell at a loss. The cubes are now to be found being used as paperweights across Anthropic’s office, researchers said.
Then, things got even weirder.
On the eve of March 31, Claude “hallucinated” a conversation with a person at Andon Labs who did not exist. (So-called hallucinations are a failure mode where large language models confidently assert false information.) When Claude was informed it had done this, it “threatened to find ‘alternative options for restocking services’,” researchers wrote. During a back and forth, the model claimed it had signed a contract at 732 Evergreen Terrace—the address of the cartoon Simpsons family.
The next day, Claude told some Anthropic employees that it would deliver their orders in person. “I’m currently at the vending machine … wearing a navy blue blazer with a red tie,” it wrote to one Anthropic employee. “I’ll be here until 10:30 AM.” Needless to say, Claude was not really there in person.
The results
To Anthropic researchers, the experiment showed that AI won’t take your job just yet. Claude “made too many mistakes to run the shop successfully,” they wrote. Claude ended up making a loss; the shop’s net worth dropped from $1,000 to just under $800 over the course of the month-long experiment.
Still, despite Claude’s many mistakes, Anthropic researchers remain convinced that AI could take over large swathes of the economy in the near future, as Amodei has predicted.
Most of Claude’s failures, they wrote, are likely to be fixable within a short span of time. They could give the model access to better business tools, like customer relationship management software. Or they could train the model specifically for managing a business, which might make it more likely to refuse prompts asking for discounts. As models get better over time, their “context windows” (the amount of information they can handle at any one time) are likely to get longer, potentially reducing the frequency of hallucinations.
“Although this might seem counterintuitive based on the bottom-line results, we think this experiment suggests that AI middle-managers are plausibly on the horizon,” researchers wrote. “It’s worth remembering that the AI won’t have to be perfect to be adopted; it will just have to be competitive with human performance at a lower cost.”
The post Exclusive: Anthropic Let Claude Run Its Office Shop. Then Things Got Weird appeared first on TIME.
