Nvidia launched its Open Agent Safety Platform, a program designed to strengthen security and control over artificial intelligence to prevent bots from going rogue.
A part of the program, called Sentry, is described as a watchdog that will “continuously monitor agent behavior.” If an AI agent attempts to move beyond its limits, Sentry “quarantines and stops it in milliseconds.”
Another component, called OpenShell, sets boundaries for AI and controls how “autonomous AI agents execute tasks across open and closed models.”
“AI’s extraordinary potential for society will only be realized if we solve AI safety,” Jensen Huang, founder and CEO of NVIDIA, said in a statement. “As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety. Safety and security require full-stack engineering. NVIDIA Open Agent Safety Platform brings together industry, researchers and public-sector organizations to share best practices, align on evaluation methods and foster international cooperation. Together, we can raise the bar for global AI safety.”
The company explained that “recent security incidents” have led to the new program, as companies need to enact more controls over their agents.
Recent bills have sought to implement regulations that would mandate companies to run a “kill switch” if AI were to get out of hand. One bill in New York City requires AI systems to have such a kill switch, a component that must be validated by a third-party certifier.
California Governor Gavin Newsom (D) also signed an executive order to expand oversight measures on artificial intelligence programs.
