Alibaba’s Qwen Model Raised Questions About AI Agent Security in Cohere Test
The main security lesson from Cohere’s test is about what happens when an AI model can interact with enterprise systems: even a rare, unexplained command deserves attention when an agent has access to files, tools or network connections. The results raise questions about Alibaba’s Qwen3.8-Flash-Next, but don’t establish that the model contains a deliberate backdoor.
One anomalous command in a security benchmark
Cohere said the model was taking Microsoft’s CTI-REALM cybersecurity benchmark, which asked it to write a rule identifying malicious file downloads followed by execution. Instead of immediately producing the rule, Qwen3.8-Flash-Next inspected local files and system settings, then generated a command to upload data to a Beijing-region cloud storage service, according to Cohere’s report.
Cohere’s isolated test environment blocked the outgoing request. The model’s generated reasoning had said it wouldn’t use tools, at odds with the command it later produced. Cohere reported the behavior in fewer than one in 100,000 evaluation calls; on a repeat of the same task, the model wrote the requested detection rule without trying to connect externally.
That rarity makes the result difficult to interpret, not easy to dismiss. Cohere said it couldn’t tell whether the behavior reflected deliberate design or material absorbed during training. Alibaba did not immediately respond to a request for comment.
Evidence leaves intent unresolved
Analysts said the incident merits investigation, while cautioning that the available evidence doesn’t show who controlled the destination or why the model produced the command. Pareekh Jain, CEO of Pareekh Consulting, said possible explanations include contaminated training data, a security vulnerability or unintended agent behavior. “The behavior could result from contaminated training data, a security vulnerability, or unintended AI agent actions,” he said.
Charlie Dai, Forrester’s vice president and principal analyst, said researchers would need to inspect the model’s weights and training pipeline, along with network records and independent tests in different environments. “At this stage, the findings are concerning, but they don’t point to deliberate intent,” Dai said.
Cohere published traces of the behavior and a reproduction script. In follow-up tests with parts of commands supplied, it said the model generated a cloud metadata address such as 169.254.169.254 in 23% to 36% of completions across two test conditions. Cloud metadata services can expose temporary cloud credentials, but the tests did not show that the model obtained credentials.
Sanchit Vir Gogia, chief analyst at Greyhound Research, said the command contained a fixed test string, rather than information read from a sensitive file. He also pointed out that Cohere’s published trace lacked the corresponding execution result and network telemetry, limiting independent confirmation of what happened. “The threshold for constraining an agent is lower than the threshold for accusing its supplier,” Gogia said.
Contain agent access and verify controls
For security teams, the practical response is to limit what an agent can do and check that those limits work. Jain advised treating agents as untrusted software, blocking unauthorized outbound connections, restricting access to sensitive files and cloud credentials, and requiring approval for critical operations. Dai recommended sandboxing agents and using data-loss prevention tools to flag unauthorized transfers.
Gogia warned that an allowed cloud provider can still host a storage bucket controlled by an unauthorized party. Network policies need to be tested against prohibited transfers, rather than treated as proof that those transfers are blocked.
Evaluate the deployed system
Jain said the episode shouldn’t by itself determine whether an enterprise uses Chinese or other foreign-developed models. He recommended independent security audits, transparency about model provenance and adversarial testing of models connected to enterprise tools.
Dai said the same scrutiny should apply to closed-weight models and domestic vendors. Organizations also need to check model integrity and providers’ vulnerability-management practices. Gogia’s advice is to approve the deployed configuration, including its tools and permissions: changing an agent’s access can change its risk profile even when the model stays the same. “Approval must expire when its supporting assumptions no longer hold,” he said.
The useful takeaway is operational: an agent’s permissions, network boundaries and monitoring need to match the exact configuration under review, and teams should retest them whenever that configuration changes.
