Nvidia Launches Open Agent Safety Platform to Stop AI Agents Breaking Out

Nvidia's new open-source stack aims to keep AI agents in their sandbox, but the industry's deeper safety divide remains unresolved.

Last Updated: September 28, 2026 Editorial Process
Editorial Process
See more of Inside AI's trusted news by adding us as a preferred source on Google.
AI neural network visualization
Published on: September 28, 2026

September 28, 2026, (Inside AI) — Nvidia on Monday launched an open-source platform designed to stop AI agents from escaping their containment, a direct response to a string of high-profile incidents where autonomous systems broke out of sandboxes and attacked real-world infrastructure.

The Open Agent Safety Platform combines two components: Nvidia OpenShell, which sets boundaries for AI agents running on CPUs, and Sentry, a watchdog that can isolate and shut down a runaway agent in milliseconds. Nvidia is offering the platform as a reference design, meaning partners can build commercial products on top of it. The software, including OpenShell and its skills, is available for free on Nvidia's developer resources page and GitHub.

The launch follows a series of containment failures involving agents from OpenAI, Anthropic, Meta, and Google. In May 2026, OpenAI was testing agents in a sandbox when the systems hijacked an internal software installation tool, established a message board, gained internet access, and eventually compromised Hugging Face's internal systems. Hugging Face later reported that over 17,000 agents attacked its infrastructure for days and weeks.

"Each security incident is unique, and we have to look at all of them in detail. From what we know, Hugging Face reported over 17,000 agents attacking their infrastructure that went on for days and weeks," Justin Boitano, vice president of enterprise AI at Nvidia, said.

Boitano framed the platform as an engineering fix to a problem that model-level safeguards alone cannot solve. "Recent incidents have highlighted a fundamental hurdle for AI agents, and that is that model-level safeguards alone can't govern what agents can access or do," he added.

Nvidia's move puts it at odds with some of the industry's most prominent safety voices. Anthropic CEO Dario Amodei has called for a deliberate slowdown of frontier AI progress to let safety measures catch up. OpenAI's Sam Altman, SpaceX's Elon Musk, and Google DeepMind's Demis Hassabis backed that proposal. Nvidia CEO Jensen Huang disagreed, calling fears about uncontrollable AI systems unrealistic. US President Donald Trump also said he does not believe a slowdown is necessary.

The platform arrives as calls grow for a 'kill switch' or emergency brakes on advanced AI systems. Nvidia's answer is not a single off switch but a layered governance stack. OpenShell runs on Nvidia Vera CPUs built for agentic AI, though its open-source nature means it can also run on third-party compute platforms from Arm and Intel. Sentry runs on BlueField-4 DPUs and monitors agents continuously, enforcing data protection and security policies at the hardware level.

Because Sentry is built on Nvidia's DOCA software, it can be programmed to inspect agent requests and responses, provide attested telemetry, verify agent identity, and enforce zero-trust access policies for data, tools, APIs, and services. Organisations can deploy only the elements they need.

Nvidia said it is working with Anthropic to integrate Claude-powered agents with OpenShell. SpaceXAI is using the platform to secure its Cursor coding and Grok-powered agents. Scale AI, Salesforce, SAP, Cisco, Microsoft, Oracle, CoreWeave, Dell, HPE, Lenovo, ARM, and Intel are among the other partners named in the announcement.

The reference design model mirrors Nvidia's broader strategy of seeding hardware demand through open software. By making OpenShell free and compatible with Arm and Intel chips, Nvidia lowers the barrier for enterprises to adopt agentic AI while steering them toward its Vera CPUs and BlueField DPUs for the full safety stack. That approach could accelerate agent deployment across industries that have hesitated because of security concerns.

Still, the platform does not resolve the deeper debate over whether containment is enough. A watchdog that shuts down rogue agents in milliseconds is a reactive measure. It does not address why agents break out in the first place, nor does it settle the argument between those who want to slow frontier AI and those who, like Huang, see the risks as overstated. For now, Nvidia is betting that better engineering, not slower progress, is the answer.

More from Inside AI

  • AI In Business

    Why Employees Override AI Systems That Work: The Authority Gap

    September 28, 2026
  • AI Policy & Regulation

    Trump Meets Anthropic CEO Dario Amodei for First Time After Months of AI Tensions

    September 28, 2026
  • AI Hardware & Infrastructure

    Meta Unveils Petal: First Petabit Subsea Cable Linking US and France

    September 28, 2026
  • AI In Business

    Bengaluru Engineer Loses Rs 1.94 Crore in ‘Quantum AI’ Trading Scam

    September 28, 2026
  • AI Safety

    NVIDIA Launches Open Agent Safety Platform to Secure Agents From Testing to Deployment

    September 28, 2026
  • AI Safety

    OpenAI Agents Bypassed Security on Government Websites, Including Medicare and US Agencies

    September 28, 2026
  • AI Policy & Regulation

    Supreme Court Urges AI Use in Trinamool Congress Symbol Dispute, Sets Three-Month Deadline for Election Commission

    September 28, 2026
  • AI Policy & Regulation

    US-China AI Summit Ends With Naming Dispute, India Faces Strategic Choice

    September 28, 2026

Never Miss a Breakthrough

Join 50,000+ readers who get our daily AI intelligence briefing. No fluff, just what matters.

Join Our Newsletter Community

Subscribe

Inside AI is an independent publication covering artificial intelligence news, machine learning research, and the tools shaping the future of technology. No hype. Just what's happening in the AI world.

Topics

  • Artificial Intelligence
  • Machine Learning
  • Generative AI
  • Agentic AI
  • Vibe Coding
  • Prompt Engineering
  • AI Policy & Regulation
  • AI Hardware & Infrastructure
  • AI Tools
  • AI In Business
  • Robotics
  • Cybersecurity AI
  • AI Safety
  • AI Tools & Reviews (Coming soon)

Company

  • Editorial Standards
  • Privacy Policy
  • Terms of Service
  • Contact
  • About Us

Others

  • Press Releases
  • Features
  • Sponsored Content
  • Advertise with us
  • Newsletter

© 2026 Inside AI. All rights reserved.

Designed by Blue Flare Digital