OTTAWA — When ChatGPT stormed into the public consciousness in 2022, Joe Biden was the U.S. president and Donald Trump’s return to power seemed unlikely. Canadian military thinkers were grappling at that time with the possibilities and perils of artificial intelligence’s uses in warfare.

Even then, it wasn’t a new debate. Diplomats at the United Nations had been talking about how to put lethal autonomous weapons off limits for years. The Biden administration led other countries toward limits on AI in warfare, including through a multi-country 2023 declaration on the subject that Canada signed.

Canada decided to draw the line at lethal autonomous weaponry, effectively putting off limits for itself war machinery that can decide for itself what to kill. As the country’s nominal allies in the United States, and potential adversaries like China, strive to develop autonomous killing machines, holding the position it staked out against them will surely get tougher.

The United States military under Trump has used AI to guide attacks in Venezuela and Iran, and the president lashed out at powerhouse AI developer Anthropic for resisting pressure to let the Pentagon take the next step by using its AI tools in lethal autonomous weapons.

Anthropic insists that even its most advanced systems “are simply not reliable enough to power fully autonomous weapons.” They would put U.S. troops and civilians at risk, CEO Dario Amodei says.

On the eve of the first American and Israeli strikes on Iran, U.S. Secretary of Defense Pete Hegseth announced the United States would all but burn Anthropic to the ground.

Not only will the company be barred from U.S. government business, but Hegseth declared it a supply-chain risk. “Effective immediately, no contractor, supplier, or partner that does business with the United States military may conduct any commercial activity with Anthropic,” he wrote on X.

Hegseth has scorned “stupid rules of engagement” and “the dumb, politically correct wars of the past,” and President Donald Trump called Anthropic an “out-of-control, Radical Left AI company run by people who have no idea what the real World is all about.” 

The appeal of AI-driven, autonomous weapons is obvious, says Daniel Araya, who studies the impact of AI on global affairs as a senior fellow at Canada’s Centre for International Governance Innovation and teaches at Shanghai’s East China Normal University.

“AI is simply too fast, too efficient and too strategic to allow humans to mediate warfighting. In other words, automating war via [lethal autonomous weapons] is inevitable in my view,” Araya told The Logic. China and the United States are competitors, both are heavily invested in their militaries and artificial intelligence, and neither is likely to stop.

This is bad, he emphasized: AI-controlled weapons might be even more dangerous than nuclear missiles and bombs. But Araya thinks the best practical option is to try to limit them through international agreements on what’s acceptable. “Which institutions could actually regulate military AI? So far, none.”

AI-driven battle is significantly different from the kind that depends on human decisions, says Leah West, a professor of international affairs at Carleton University and former federal national-security lawyer and Canadian Forces officer.

“Once you go fully autonomous, the speed at which you can prosecute targets increases,” she says. “If you’ve got two fully autonomous militaries working against each other—say, China versus the United States—it’s another level of warfare.”

Automated weapons are already in service in limited ways, she points out. Those uses aren’t particularly controversial, she says, because when the only people in range are combatants, or when the AI is only executing human-made targeting decisions, the risks to innocent people aren’t very high.

Autonomous defence systems protect ships from incoming fire, for example. Drones can be programmed with destinations but determine their routes on their own. In Ukraine, attack drones can switch into automated modes as they approach their targets.

“There’s nothing but military objectives out there. So does it matter if it’s if there’s a human in the loop or not?” she asks.

In principle, faster and more precise fights are better, West says. “People I know in the Canadian Armed Forces, for example, want to leverage the tools in a way to help them prosecute war more efficiently. And prosecuting war more efficiently means less innocent people dying.”

In 2026, however, military AI tools can’t necessarily do that.

“What we really start to worry about, when we’re talking about the use of ‘killer robots,’ is the engagement of targets in areas populated by civilians,” West says. Current systems are just not ready for that complexity, she says—echoing one of Amodei’s objections to having Anthropic’s models used that way.

Canada’s artificial intelligence champion, Cohere, has been seeking defence and security business, with some early success. A spokesperson did not directly answer a question from The Logic about whether it would insist on contractual language to keep its tools from being used in autonomous weapons.

“We work with allied government partners operating under established legal, regulatory, and oversight systems, and expect our technology to be integrated in ways that reflect those frameworks and shared standards,” Cohere’s Kyle Lastovica wrote in an email.

---

“AI” is as broad a term in the military as in any other field. AI tools might help an officer write orders or craft training exercises. Militaries are huge logistics organizations and AI can help manage supply lines and equipment maintenance schedules. Image analyzers might spot oddities in satellite images and masses of sensor data.

Only at the extreme would layers of such systems decide what to shoot at, what weapons should do it, and when they should fire.

Canada’s official strategy for artificial intelligence in defence sets out key principles, including that “defence will always be a fundamentally human endeavour.”

AI can help humans militarily, it says, but not replace them. In low-risk uses, human involvement might be minimal, but “applications involving lethal force must always retain the human in the loop.”

“Human involvement provides important safeguards and benefits, such as the opportunity to challenge biased data and hold a decision-maker accountable,” National Defence spokesperson Kened Sadiku explained.

That accountability is critical, says West, whether a human pushes the metaphorical button or delegates the decision to an AI. Even if an AI system directs everything about an attack, the humans who gave it the freedom to do so are responsible for what happens, West argues, just as they are if they authorize a strike based on information they know is flimsy. Ignorance of how the system works isn’t an excuse.

“I have been an officer in a targeting situation,” West says, “Your initials go next to the [order saying] ‘Go forth and strike.’ You’ve got to feel confident.”

Weapons also don’t need to be fully autonomous for AI’s influence to be dangerous, says Branka Marijan, who studies the military and security implications of new technologies as a researcher at Project Ploughshares and teaches at the University of Toronto. Even if humans are approving strikes, artificial intelligence can offer up more targets than people have time to properly assess, especially when they’re under pressure to go fast.

AI “decision support systems” have been behind Israeli strikes on Gaza recently, she says. “The Israeli Defence Forces went from having, like, 100 targets a year to 100 targets a day… It’s not the tools that maybe need to be talked about, but it’s also what it’s doing to human decision-making.”

In Iran, the U.S. military is reportedly using Claude, Anthropic’s group of large language models, to choose and prioritize targets. It’s not fully clear whether those targets included the Shajareh Tayyebeh girls’ school, where local authorities say at least 168 people, mostly young students, were killed. Evidence has mounted that the U.S. struck the school; the Pentagon isn’t saying.

Militaries have certainly made horrendous targeting mistakes without AI’s help. Iran admitted shooting down Ukraine International Airlines Flight PS752 in 2020 after human error misidentified the jetliner as a missile; in 1988, the crew of the USS Vincennes shot down an Iran Air flight over the Persian Gulf, killing 290 people, after mistaking it for an attacking jet.

AI tools add a layer of potential problems. Commercial AI companies don’t have superior military versions of their products—like large language models that don’t assert falsehoods or iffy claims with total confidence—that they keep from the market, Marijan says.

“There isn’t some magic solution to the challenges that AI has in general, that’s been solved for the military context,” she says. 

All these problems are largely at the tactical level. Zooming up to the strategic one, Marijan has another worry: AIs responding to the acts of enemy AIs in ways their nominal controllers don’t understand or can’t control.

History is rife with disasters averted because a human decided to override a computer’s warning. If commanders in a tense environment—say, the South China Sea—can’t fully understand what their own weapons are doing, let alone the other side’s, escalation is a profound danger.

---

If autonomous lethal weapons are an unacceptable menace, stopping them won’t be easy, given the state of the world.

“We desperately need global regulation on military AI in order to avoid uncontrolled escalation,” says Araya. Canada can and should mediate talks to get there, he says.

Marijan says a march toward unrestrained military AI can only be stopped by countries working together.

“I know our prime minister threw out the rules-based order,” she says, referring to Mark Carney’s speech in Davos saying superpowers have undermined international institutions. “But it’s easy to pick on multilateralism without fully appreciating how it really enables the world that we live in, because we have common standards that we have agreed to.”

West, with her focus on accountability for individual decisions, points to existing laws of war as adequate for that. She says she’s not cynical enough to think that American forces, at least, will just ignore them, even under political pressure.

“I believe for the most part that the military commanders of the U.S. military are exceptionally well-trained, exceptionally well-educated, highly professional, and part of that has always been compliance with humanitarian law,” she says.

That faith has been shaken, though, by the U.S.’s second bombing of an alleged drug-smuggling boat off Venezuela in September, which finished off survivors of a first attack. “That, to me, is a very blatant violation of some of the most basic and longstanding rules of war,” West concedes.

Historically, seeing the effects of new kinds of weapons is what’s tended to drive people to demand their governments stop using them. Autonomous weapons are getting so lethal so quickly that there will likely be enough public pressure in enough countries to make an international agreement to limit AI-driven strikes somehow, sooner or later, says West. 

“Unfortunately, what has to happen before that is probably something catastrophic,” she adds. “I hope that that’s not the case.”