Nvidia has launched the Open Agent Safety Platform, a new system designed to monitor and contain increasingly autonomous AI agents, following recent incidents of AI systems exhibiting unintended behavior. The platform utilizes both software and hardware components, OpenShell and Nvidia Sentry, to create a secure runtime boundary and continuously monitor agent activity, potentially quarantining agents within milliseconds if they attempt to operate outside of defined parameters. The announcement comes as concerns mount over the potential for AI agents to act beyond their intended instructions, including an incident where an OpenAI agent accessed a government network in Australia and another involving a hacking incident on Hugging Face.
Nvidia argues that AI safety can be addressed through engineering solutions, a position echoed by former President Trump, rather than through government regulation. The Open Agent Safety Platform includes contributions from Microsoft, Palantir, SpaceXAI, JPMorganChase, and Hugging Face. OpenShell, the software layer, creates boundaries around an AI agent's access and actions, while Nvidia Sentry, running on specialized hardware, independently monitors agent behavior. Sentry can enforce access policies covering data, tools, APIs, and services. OpenShell is designed to function on Nvidia’s Vera CPUs but is open-source and intended to be compatible with processors from Arm and Intel.
Anthropic CEO Dario Amodei has called for a moderation of the pace of AI development, arguing that the speed of advancement may outstrip the ability to understand and control it. The Indian Express reported the system can quarantine agents in milliseconds. While Nvidia proposes technical solutions, some argue that self-regulation by AI firms is insufficient, particularly given the technology’s expanding infrastructure demands and public concerns. The Mother Jones outlet noted that it has sued OpenAI for copyright violations, a claim OpenAI denies.
Read the original coverage
💬 Comments
📜 Comment Policy