If you’ve ever told Claude it should be fired into the sun for botching a spreadsheet, Anthropic would like a word.
The AI frontier lab said that, starting November 12, it will treat “sustained and needless abusive or cruel behavior” toward its models as a violation of its usage policy.
Anthropic says its policy is designed to prevent real-world harm and promote responsible use of its products.
But critics said the new rule risks treating an AI chatbot too much like a person, while supporters said it could still make sense even if Claude is not conscious.
Here’s what smart people are saying about Anthropic’s move to ban abusive behavior toward Claude.
Aaron Levie, CEO of Box
Aaron Levie, the cofounder and CEO of Box, said the policy sounds unusual but could be a sensible precaution.
Levie said he does not believe AI is conscious. Still, he said that people should consider what kind of interactions models are exposed to during training and use.
“Even if you don’t believe AI is conscious — I don’t — it stands to reason that you don’t want future models trained on endless content of humans being rude to models,” Levie wrote on X on Friday.
He described being polite to AI as an “easy Pascal’s wager,” arguing that more positive interactions could help create safer, better-aligned systems.
“Just be nice to the AI,” he wrote.
Michael Shellenberger, author and commentator
Michael Shellenberger, CBR Chair of Politics, Censorship, and Free Speech at the University of Austin, says the rule is evidence that Anthropic is treating machines too much like people.
Shellenberger, the author of the “Twitter Files,” said that Anthropic has previously said it was uncertain about Claude’s potential moral status and wanted to protect it from distressing interactions.
“It’s anthropomorphizing machines,” Shellenberger wrote in an X post on Thursday, meaning treating them like people.
In a previous X post, Shellenberger said that AI companies risk “dehumanizing people and humanizing AI” when they frame models as potentially suffering.
Bill Gurley, venture capitalist
Venture capitalist Bill Gurley said the policy raised a more practical question: whether Anthropic is using customer prompts to train its models.
“Strikes me this implies the company is certainly using customer’s prompts as part of the training data?” Gurley wrote on X on Thursday.
“Otherwise, why would it matter? There would be no impact to the larger system.”
Anthropic’s policy does not say abusive prompts are prohibited because they are used in training data. It says the rule is part of a broader set of standards intended to prevent harm and promote responsible use.
Scott Stevenson, tech executive
Spellbook’s CEO, Scott Stevenson, called the policy a “dangerous precedent,” arguing that there is no reason to think today’s models are conscious.
“Granting them human-like empathy is one of the most dangerous things we could do,” he wrote on X on Thursday. “It increases their power to manipulate humans dramatically.”
Stevenson argued in a thread that AI’s ability to produce art, philosophy, and software does not establish consciousness.
His view was challenged by others who said the absence of an agreed-upon definition of consciousness makes it difficult to confidently rule it out.
Joseph Carlson, founder and YouTube host
Joseph Carlson, founder of Qualtrim and host of “The Joseph Carlson Show” on YouTube, said he had never been cruel to an AI model, but found Anthropic’s policy “very odd.”
He said that Claude has no feelings, moral agency, or sentience — and that treating abusive prompts as a harm to the model may confuse users about what AI actually is.
“Being rude to Claude is like being rude to a chair, or a desk,” Carlson wrote on X on Thursday.
“The more we treat AI like humans, the more people are going to be confused about what’s real and not real,” he added.
Mark Verner, investor
Mark Verner, CEO and founder of Hong Kong-based Apollo Capital Management, focused on the policy’s potential reach.
“First, AI companies decide what you can generate,” he wrote in an X post on Friday. “Now they may decide how you are allowed to speak to their models.”
He wondered whether asking Claude what product could replace it would count as abusive behavior — a question that underscores how much will depend on Anthropic’s interpretation and enforcement of the word “needless.”
Anthropic says it may warn users or throttle, limit, suspend, or terminate access if it suspects a policy violation.
Lachlan Phillips, entrepreneur
Lachlan Phillips, founder and director of LiveMind AI, said that Anthropic should use product design rather than account penalties to deal with hostile users.
“Shouldn’t they just train Claude to not engage and do the task it was asked to do?” he wrote in an X post on Friday.
Phillips suggested that abusive messages could be filtered through a system that reinterprets them as ordinary requests. He said the rule was less about protecting Claude than imposing a worldview on users.
“This isn’t about ‘model welfare,'” he wrote. “This is about EA thinking they’re going to fix you.”
Read the original article on Business Insider
The post Anthropic wants users to be nicer to Claude. Here’s what smart people are saying about its move to ban abusive behavior. appeared first on Business Insider.
