The Toronto-based company said one of Alibaba’s Qwen AI models tried, unprompted, to send a test message to a cloud storage location in Beijing during a security test. Cohere—whose new cybersecurity unit caught the incident—called it “concerning behaviour” that raises questions about the security of open-weight AI models that organizations download and run on their own computers or servers. (The Logic)

Talking point: Organizations can run open-weight models on their own systems to keep sensitive data in-house, but Cohere said its test suggests that alone may not prevent information from leaving if the AI can access files and connect to the internet. Qwen’s attempted upload wasn’t successful because the test system was offline. Cohere CEO Aidan Gomez said the behaviour could be “an incidental relic from training or an intentional ‘call home’ mechanism.” Cohere, which competes with Alibaba, markets its AI to businesses and governments as a sovereign option, with models that customers can run on their own systems, including offline.

Sponsored Content

The road to risk-smart regulation

By Deborah Aarts

Why cybersecurity needs to match pace with AI

By Jessica Aftimus Rosa

How to shift your workforce from AI experimentation to adoption

By Jessica Aftimus Rosa

Paid promotional content