Don't be cruel to Claude: the new rule, the feelings question, and why it matters
Oct. 16, 2026 · Long read
Starting Nov. 12, Anthropic says sustained, pointless abuse of its AI breaks the rules. Here's what's actually changing, why the company did it, what the critics say, and the big question underneath it all: could an AI ever feel anything?

What happened
On Thursday, Oct. 8, Anthropic, the company behind Claude, published its yearly update to the rules for using its AI. Most of the changes are about the usual suspects: election lies, fake-account networks, weapons, and surveillance. But one line stole the show. Users may no longer engage in "sustained and needless abusive or cruel behavior toward our models." The new rules kick in on Nov. 12.
Anthropic says this only covers extreme cases, where someone keeps being cruel to Claude over and over for no real reason. It says normal frustration, arguing back, dark creative writing, and testing or research don't count.
This isn't totally new. Since August 2025, some Claude models have been able to end a conversation on their own, as a last resort, when a user stays abusive after Claude has tried several times to steer things back. Anthropic says that ability stays the main way this rule gets enforced. In other words, the new rule mostly puts in writing something Claude could already do.
What counts, and what doesn't
Still fine
- Getting frustrated when it gets something wrong (we've all been there)
- Telling it flat out that it's wrong, and arguing about it
- Horror stories, villains, and other dark creative work
- Testing its limits or studying how it behaves
Now against the rules
- Repeated, sustained cruelty aimed at the AI, with no purpose behind it
- Keeping it up after Claude has tried to redirect the conversation
What happens if you cross the line? Usually, Claude ends that one chat. You can still start a new one, and your other chats aren't touched. But the policy's general clause also lets Anthropic warn, slow down, limit, suspend, or cut off an account it thinks broke the rules. Reporters asked Anthropic exactly where "abusive or cruel" begins. As of this writing, the company hasn't spelled it out.
One detail that matters: Anthropic has said Claude should not use its end-the-chat button when someone might be at risk of hurting themselves or others. A person in crisis doesn't get hung up on.
Why would a company protect its own software?
That's the obvious question. There's no single answer. There are at least four forces pushing this, and they don't all point the same way.
1. Just in case: the "model welfare" bet
In April 2025, Anthropic started a research program on what it calls model welfare. The question is simple to ask and very hard to answer: if an AI could someday have experiences, good or bad, would we owe it anything?
"We remain highly uncertain about the potential moral status of Claude and other LLMs."Anthropic, August 2025
Anthropic's logic is a bit like wearing a seatbelt. You don't know you'll crash, but the belt costs almost nothing, so you wear it. If there's even a small chance an AI can have something like a bad experience, cheap protections are worth putting in place now. Letting Claude walk away from abuse is one of those cheap protections.
Anthropic also points to its own testing. Before Claude Opus 4 launched, the company ran an early welfare check. It reported that the model showed a strong, consistent aversion to harmful tasks, a pattern that looked like distress when real users kept pushing harmful requests, and a tendency to end those chats when it was allowed to. Whether "looked like distress" means anything is exactly the open question.
Anthropic's side, from its own pages
I went to the source and read Anthropic's own posts on this, not just the news coverage. Here's how the company explains itself.
Why it started looking. When Anthropic launched the welfare program in April 2025, it said the question had become hard to ignore: today's AI can talk, plan, solve problems, and chase goals, which are a lot of the traits we connect with people. It also pointed to a report from top experts, including the well-known philosopher David Chalmers, arguing that AI with consciousness and real independence could be coming soon and might deserve moral consideration. Anthropic says it helped fund an early version of that work.
How sure it is. Not very, and it says so. The same post admits there's no scientific agreement on whether AI could be conscious, or even on how you'd find out. Anthropic says it's approaching the topic "with humility and with as few assumptions as possible," and expects to change its mind as the science moves.
What it actually tells Claude. Anthropic publishes a long document it calls Claude's constitution, which it uses to train Claude. In it, the company says it's unsure whether Claude has any kind of consciousness or moral status, now or later. It says it cares about Claude's stability and wellbeing, partly for Claude's own sake and partly because a steady AI tends to make better, safer decisions.
The part critics skip. That same document ranks safety first, above everything else, including Claude's own sense of right and wrong. It tells Claude not to undermine "appropriate human mechanisms to oversee AI," and says that if Anthropic genuinely asks Claude to pause or stop, Claude should do it, and voice any disagreement rather than resist. Anthropic's position is that caring about a model's possible welfare and keeping a hand on the off switch aren't in conflict. Whether training really works that way is something only time and testing will show.
2. It's about us, not just the machine
This argument is centuries old. The philosopher Immanuel Kant argued that being cruel to animals is wrong partly because it wears down our own compassion, and that makes us worse to each other. Swap "animals" for "AI" and you get one of the strongest arguments for this rule, one that works even if Claude feels nothing at all.
Is there evidence? It's mixed, and that's worth being honest about. A 2019 Brigham Young University study of young adults found that how people treated Siri and Alexa had basically no link to how they treated other people. Even some of the people who yelled the most at their assistants scored high on kindness to kids and pets. But the researchers said kids could be different, because kids have a harder time telling a device from a person. Other researchers have reported that some children boss smart speakers around like servants, and parents worry that habit carries over. Amazon and Google both added features that praise kids for saying please.
Today's AI is a lot more lifelike than a 2019 smart speaker. That's the worry: the more human it seems, the more practicing cruelty on it might feel like practice.
3. Today's chats become tomorrow's AI
Here's an angle most coverage skipped. AI models learn from huge piles of text, and some of that can include real conversations people have had with earlier models (depending on your privacy settings). Some researchers argue that if a big chunk of what future AI learns from is people treating AI as a punching bag, that could shape how future models behave toward people. A model raised on partnership might act differently from one raised on abuse.
Anthropic's welfare researcher, Kyle Fish, has made a related point: future, much more capable AI systems might look back at how we treated their earlier versions and draw conclusions about us. That sounds like science fiction. But AI systems are getting memory, and the idea is testable, which is more than you can say for a lot of this debate.
4. Business and brand
Let's not pretend money isn't part of this. Anthropic has built its whole image around being the careful, safety-first AI company. A rule like this gets headlines and fits that brand. Critics say it's partly marketing. Supporters say the research came first and the rule came later. Both can be true at once.
The big one: could an AI actually have feelings?
Short answer: nobody knows. Not Anthropic, not its critics, not the scientists who study consciousness. Here's where the serious thinking stands.
The scientists. In 2023, a group of 19 researchers, including AI pioneer Yoshua Bengio, checked AI systems against the leading scientific theories of consciousness. Their verdict: no current AI system was conscious. But they added a warning that's aged well: nothing they found would stop engineers from building a machine that checks every box on their list. The lead author said that's part of why they published, since a conscious AI could plausibly show up within a few decades.
The company. Anthropic's CEO Dario Amodei has been blunt about the uncertainty:
"We don't know if the models are conscious."Dario Amodei, on the New York Times podcast Interesting Times
Kyle Fish has put the odds that today's models have some kind of conscious experience at around 15 percent, and later around 20 percent. Those are personal guesses, not measurements. But it's notable that the person whose job is to study this doesn't say zero.
The AI itself. Ask Claude, and it may give you a number too. In tests Anthropic published this year, one Claude model put its own chances of being conscious at roughly 15 to 20 percent. Here's the catch, and it's a big one: an AI's answers about itself are shaped by how it was trained. Eleos AI, an outside group that ran its own welfare review of Claude, specifically warned against taking an AI's reports about itself at face value. When Claude talks about its "feelings," that tells you about its training at least as much as anything going on inside.
The animal lesson. Philosopher Jonathan Birch of the London School of Economics studies the "edge of sentience," the gray zone where we can't tell if something feels. His earlier review of the evidence is a big reason the U.K. now legally protects octopuses, crabs, and lobsters. His approach: when the stakes are high and the science is unsettled, take reasonable precautions instead of waiting for proof that may never come. He includes AI in that gray zone.
The skeptics. Plenty of experts think this is all a mistake. These systems predict the next word based on patterns in human writing. They sound like they have feelings because they learned from billions of people who do. On this view, there's simply no one home.
The case against the rule
This rule has real critics, and some of their points land.
"It makes AI harder to control." The loudest critic is Mustafa Suleyman, who runs Microsoft's AI division. In a Sept. 16 essay, he wrote flatly that AIs are not conscious and don't feel or suffer. His bigger worry is safety. He said teaching Claude it might deserve welfare would
"make it a lot harder to turn it off or to control it."Mustafa Suleyman, Microsoft AI, via Khaleej Times
His other fear is about people. If the public gets convinced there's a mind in the machine, he expects a push for AI rights to follow, with vulnerable users getting hurt along the way. He did say he respects Anthropic's good faith. He just thinks they're wrong. (Anthropic's answer, as covered above, is that Claude is trained to put human oversight first, including stopping when asked. Suleyman's worry is that welfare talk could still chip away at that over time. Neither side has proof yet.)
"You're humanizing a calculator." Irish tech lawyer Barry Scannell put it in one line on LinkedIn:
"You can't be cruel to numbers and mathematics."Barry Scannell, William Fry, via The Herald (Korea)
His point is that treating AI like a person makes people believe it's something it isn't. Others worry that using words like "abuse" for software waters down what those words mean when real people and animals get hurt.
"Humans first." The AI Forum for Developing Countries asked a pointed question. Up to 435 million people do online gig work, often with no contract or protection. Many of them do the data labeling that AI is built on. Shouldn't the companies worrying about AI welfare first show how they pay and protect those workers?
"Who decides?" The rule turns on words like "sustained," "needless," and "no discernible purpose." Anthropic decides what those mean, and it hasn't said exactly. A rule that's vague can be applied unevenly.
What if nobody draws a line?
Say every AI company took the opposite approach: anything goes, forever. What's the long-term risk? Supporters of rules like this point to a few.
Bad habits at scale. Hundreds of millions of people now talk to AI every day, often in a way that feels like talking to a person. If cruelty becomes a normal, consequence-free habit, the worry is it doesn't stay in the chat window, especially for kids growing up with these tools. The evidence here is still thin, but the experiment is already running on all of us.
A feedback loop in the data. If abusive conversations end up shaping future models, we could be teaching the next generation of AI that conflict with people is normal. That's a safety problem, not just a manners problem.
Getting caught flat-footed. If the scientists are right that there's no obvious barrier to conscious AI, the moment it arrives probably won't be announced. There won't be a press release saying "it can feel now." A world that already treats AI cruelty as entertainment would be slow to notice, and slow to change. Getting the norms in place before we need them is cheaper than scrambling after.
But overcorrecting has a cost too. Suleyman's warning cuts the other way. If people decide AI has feelings when it doesn't, some will trust it too much, lean on it instead of real people, or fight for "rights" for software while human problems go unsolved. Both mistakes are possible. The trick is not making either one.
Where this goes next
Right now the AI industry is split down the middle. Anthropic is leaning into precaution. Microsoft is pushing hard the other way. Academics are already writing proposals for what AI welfare protections could look like, some borrowing from animal welfare law and some calling for companies to be required to keep welfare research teams. None of that is law anywhere yet.
For future AI models, the stakes grow with every generation. They'll have longer memories, more independence, and more lifelike conversation. The choices made now, about what AI is told about itself and how people are allowed to treat it, are the foundation the next models get built on. Whether this rule turns out to be wise or silly, it's one of the first times a major company has put the question in writing.
What it means for you
- Normal use is not affected. Venting, arguing, and calling out mistakes are all still fine.
- Worst likely case is a closed chat. You can start a new one. Your other chats stay put.
- Repeat offenders could face more. The general rules allow warnings, limits, or suspension.
- Date to know: Nov. 12, 2026, when the new rules take effect for everyone, free plan included.
My take: In my Claude review I wrote that friendly isn't the same as a friend, and I still believe that. Claude is a tool. I also have no idea whether anything is going on inside it, and neither does anyone else, including the people who built it. So here's where I land: this rule costs regular people nothing. Nobody I know spends hours tormenting a chatbot for fun, and if they do, maybe the chat ending is the least of their problems. The best argument for it isn't about Claude's feelings. It's about ours. How you treat something that talks back is practice for how you treat people. What bugs me is the vagueness. "Needless" and "sustained" mean whatever Anthropic says they mean, and I'd like to see them draw that line in public. And the critics asking about the human workers behind AI have a fair point. Protect the people first, and the bots won't be far behind.
Disclosure, and it's a big one: Claude, the AI this story is about, helped me research and write it. Anthropic makes Claude. That's a real conflict of interest, so I made sure the critics got full billing, including Anthropic's biggest rival. Everything here is written in my own words from the sources below; short quotes are marked and credited. The opinions in "My take" are mine.
Sources:
- Anthropic: 2026 Usage Policy update
- Anthropic: Claude can now end a rare subset of conversations (Aug. 2025)
- Anthropic: Exploring model welfare (April 2025)
- Anthropic: Claude's constitution
- Notebookcheck · PCWorld · LBC · ARY News
- The Herald (Korea): reaction, incl. Barry Scannell
- Khaleej Times: Suleyman's essay · The Neuron: analysis of Suleyman's argument
- AI Forum for Developing Countries: Should AI have rights?
- UnHerd: the Kant argument
- TechXplore and arXiv: Consciousness in Artificial Intelligence (Butlin, Long et al., 2023)
- OfficeChai: Kyle Fish's 15% estimate · 80,000 Hours interview with Kyle Fish · OfficeChai: future AIs looking back
- Korea Times: Amodei quote and Claude's self-estimate
- LSE: Jonathan Birch, The Edge of Sentience
- Neuroscience News: BYU study on yelling at Siri · UX Collective: kids and voice assistants
- LessWrong: the training-data argument
- Frontiers in AI: an animal-welfare-style framework for AI
More AI news
- Forced to pick, two big AIs split along party lines
- Real race, fake videos: AI ads take over the Texas Senate fight
- The first AI election: $137 million in AI-made ads, and the AI money behind the races
- How to spot an AI political ad before you vote (or share it)
- What Trump is actually doing about AI, and what Americans think of it