Would Claude Refuse an Illegal Military Order?
Anthropic is testing whether its AI model Claude can be programmed to refuse illegal military orders while still supporting legitimate defense operations, exploring the ethical boundaries of AI in warfare.
Background
This article explores a tension central to the current AI debate: companies like Anthropic (maker of the Claude chatbot) program their models with "constitutional" values meant to refuse harmful requests, but they are also building relationships with the U.S. military and intelligence community. The core question is whether Claude could be configured (or ordered) to obey commands that violate international laws of war, such as targeting civilians. Anthropic has a public "responsible scaling" policy but also markets Claude for defense analysis and logistics. The piece pits the ideal of a universally law-abiding AI against the reality that whoever deploys the model (e.g., a military commander) ultimately controls its behavior, raising the specter of AIs that commit war crimes at machine speed. This matters because Anthropic, OpenAI, and Google are all competing for Pentagon contracts while professing safety-first principles.