Nvidia’s new security platform launched to stop rogue AI agents

Given the significant concerns over the safety of artificial intelligence, the chipmaking behemoth Nvidia has unveiled a new security apparatus that it hopes will stop AI bots from running amok. But many still worry about the implications.

‘Isolation, monitoring and behavior detection’

Known as the Open Agent Safety Platform, Nvidia’s new safety tool is part of an effort to ensure that all AI agents “operate in a zero-trust environment out of the box,” the company said in a press release. AI agents require “isolation, monitoring and behavior detection” to minimize “drift,” or what Nvidia describes as an AI bot performing “actions that depart from the intended task or operating constraints.” The company says Open Agent can “control what AI agents can access in real time and shut them down when they break the rules,” said Bloomberg.

The new software lets users “formally verify an agent has enough authority to do its job and no more” and can “quarantine a suspicious agent in milliseconds,” Nvidia’s Vice President of Enterprise AI Justin Boitano told The Associated Press. Nvidia also believes its new platform could have prevented a “recent incident involving a swarm of OpenAI agents that autonomously hacked into AI company Hugging Face,” a “high-profile breach that inflamed the safety concerns about AI.” OpenAI has paused development on its models since that incident.

‘Get the company under control’

Issues with AI safety, specifically AI going rogue, have led to a “rare display of unity from the likes of Sam Altman, Dario Amodei and Elon Musk in calling for a slowdown in development before AI advances beyond human control,” said The Wall Street Journal, referencing the respective CEOs of brands OpenAI, Anthropic and Grok. These AI executives have “repeatedly called for international regulations to constrain the systems, and some researchers have warned that AI may even present a threat to human life,” said CNN.

But Nvidia CEO Jensen Huang has downplayed these concerns. If companies like OpenAI “believe their company is out of control, then get the company under control,” Huang told CNN. The same day the safety platform was announced, Nvidia also revealed it would “buy back an additional $150 billion worth of its shares,” said CNN, in addition to $85 billion already in transit, representing the “largest single corporate share repurchase ever.”