Papalocal
Loading…
News
← All news
Technology

Anthropic bans cruel behavior toward Claude, tightens rules on weapons, surveillance, and propaganda

Anthropic's first usage policy update in over a year formally prohibits users from repeatedly abusing its AI — and expands bans on election interference, weapons software, and covert influence operations.

How does this story make you feel?

Anthropic has updated its usage policy for the first time in more than a year, and the change drawing the most attention is one that would have seemed absurd not long ago: users are now formally prohibited from subjecting Claude to 'sustained and needless abusive or cruel behavior.' The policy takes effect November 12.

The company is careful to define the scope narrowly. According to Anthropic, the rule applies only at the extreme end — cases in which users are repeatedly cruel to its models for no discernible reason. Ordinary frustration, pushback, dark creative themes, and model testing and research all fall outside its reach.

The primary enforcement tool remains the same one Anthropic introduced last August: allowing Claude to end conversations with persistently abusive users. Once Claude closes a chat, the conversation is over and no new messages can be sent, though the user's other chats remain open. Anthropic did not confirm to The Verge whether additional penalties — such as account suspensions or bans — would follow, though the-decoder.com reported that the policy does allow Anthropic to warn, throttle, restrict, suspend, or terminate user access.

The abuse provision grows directly out of Anthropic's ongoing 'model welfare' research. In August 2025, when the company first gave Claude the ability to end conversations, it cited findings that Claude Opus 4 showed 'a robust and consistent aversion to harm,' apparent distress when engaging with abusive users, and a tendency to exit harmful conversations when permitted to do so. Anthropic has repeatedly stated it is uncertain whether Claude has moral status, but treats the question as serious enough to warrant precautions — including, according to the-decoder.com, commitments to preserve the weights of retired models and to interview models before shutting them down.

We don't know if the models are conscious.— Dario Amodei, CEO, Anthropic
Questions about potential internal experience, consciousness, moral status, and welfare are serious ones that we're investigating as models become more sophisticated and capable.— Kyle Fish, model welfare research lead, Anthropic

Not everyone in the industry shares that view. A draft code of conduct for Microsoft AI, posted by Mustafa Suleiman last September, took the position that 'the idea of model welfare is wrong' and that 'AI's should not have rights or legal personhood.' That document was later revised to a more measured position, acknowledging that 'the science of AI consciousness is far from settled' while still rejecting legal personhood or welfare entitlements for models, according to The Verge.

The model-welfare rule is the most novel addition, but the bulk of the policy update is a direct response to misuse Anthropic says it has documented over the past year. Investigators at the company found state media outlets, government propaganda offices, and commercial firms had turned Claude into a tool for operating networks of fake accounts and fabricated news sites. Rules against that activity already existed but were scattered across the elections, fraud, privacy, and disinformation sections. Anthropic has now consolidated them into a single section titled 'Do Not Engage in Deceptive Campaigns or Artificial Activity,' covering both political and commercial deception and explicitly banning the tools and infrastructure used to run influence operations.

The elections section has been sharpened and renamed 'Do Not Undermine Democratic Processes.' It focuses specifically on voter deception and election disruption — spreading false information about candidates or voting procedures, impersonating candidates or election officials, or suppressing turnout. At the same time, Anthropic removed a blanket prohibition on personalized vote and campaign targeting that it says had inadvertently blocked legitimate civic work, such as nonprofits translating voter information or election officials sending ballot cure notices. The behaviors that motivated the original rule — deceptive targeting and misuse of voters' personal data — remain prohibited under other sections.

On weapons, Anthropic says its policy has always banned using Claude to develop them, but the company has recently seen attempts to use Claude to build guidance and control software for weapons. In the updated policy, the prohibition is spelled out to include 'software and components that make weapons work, as well as actions like arming drones and other autonomous vehicles.'

The surveillance section has also been rewritten. Anthropic's September threat intelligence report documented growing use of AI to build systems that identify and track political dissidents. Under the revised language, consent is the dividing line: tracking individuals without it is off-limits regardless of whether it occurs in real time or through analysis of data gathered earlier, and Claude may not be used to decide or recommend who to investigate, arrest, or charge in a law enforcement or criminal justice process. Permitted uses — including fraud monitoring that users have agreed to, content moderation, journalism, and legal research — are now spelled out explicitly.

A new provision addresses AI connected to physical hardware. For situations in which Anthropic's models are 'connected to hardware that takes autonomous physical actions and which might be capable of causing injury,' the policy requires that a qualified operator be able to watch the equipment and intervene if necessary, and that the equipment be capable of holding a safe state if Claude is disconnected. The Verge noted it is not clear from the policy whether that operator must be human.

Anthropic acknowledged that exceptions to some of these rules are possible for government contracts 'if, in Anthropic's judgment, the contractual use restrictions and applicable safeguards are adequate to mitigate the potential harms.' The company has previously contracted with the US military.

Why it matters — The policy codifies Anthropic's view that its AI may warrant some form of protection from abuse — a position with real implications for how the industry thinks about AI systems — while also revealing a year's worth of documented misuse in areas from weapons development to election interference that the company is now explicitly trying to close off.

⚠ Not yet confirmed

  • Anthropic brought in dozens of religious thinkers starting in fall 2025 to discuss Claude's possible consciousness, with co-founder Christopher Olah reportedly treating Claude as a being potentially capable of suffering; participants signed NDAs.
  • Anthropic has committed to preserving the weights of retired models and interviewing models before shutting them down.
  • Whether the 'qualified operator' required to oversee autonomous physical hardware must be human.

Reported by theverge.com, anthropic.com, macrumors.com, the-decoder.com

People in this story

Community

Be the first to comment.

Share this story
Text Email