Jensen Huang’s next goal is to give robots a brain. Thibaud Moritz/AFP via Getty Images
The A.I. boom has helped make Nvidia the world’s most valuable publicly traded company with a market cap of $4.3 trillion. But CEO Jensen Huang envisions the company’s future to center around physical A.I. At its SIGGRAPH conference on Aug. 11, Nvidia unveiled a suite of A.I. models aimed at advancing physical A.I. systems, including humanoid robots and self-driving cars. The centerpiece is Cosmos Reason, a vision-language-action (VLA) model that the company says can give robots human-like reasoning abilities.
“Computer graphics and A.I. are converging to fundamentally transform robotics,” said Rev Lebaredian, Nvidia’s vice president of Omniverse and simulation technologies, in a statement. “By combining A.I. reasoning with scalable, physically accurate simulation, we’re enabling developers to build tomorrow’s robots and autonomous vehicles that will transform trillions of dollars in industries.”
Cosmos Reason is a 7-billion-parameter model that boasts an understanding of prior knowledge, physics and common sense in how to operate amongst the real world. When applied to physical A.I. systems, it will act “as the brain for deliberate, methodical decision-making,” said Nvidia in a blog post.
The model can also automate dataset curation and annotation, a capability already being used by Uber as well as Nvidia’s own robotics and self-driving teams. Magna, a mobility tech company, is incorporating Cosmos Reason to help vehicles adapt faster to new environments. Other adopters include VAST Data, Milestone Systems and Linker Vision, which are using it for traffic monitoring and safety improvements in urban and industrial settings.
While companies like OpenAI have also developed VLA models for robotics, Nvidia claims Cosmos Reason stands out for its ability to handle multistep tasks, adapt to unfamiliar situations and navigate ambiguity.
In addition to Cosmos Reason, Nvidia earlier this week also introduced Cosmos Transfer-2, an upcoming model that will let robot developers create photorealistic data from 3D simulation scenes and spatial control inputs. Its distilled version, Cosmos Transfer, offers faster performance and has already been adopted by robotics firm Lightwheel and medical robotics company Moon Surgical.
These releases are part of Huang’s broader plan to push Nvidia beyond its GPU roots. For now, the physical A.I. segment remains small. Nvidia’s automotive and robotics division, despite growing by 72 percent in revenue between February and April to $567 million, only accounted for around 1 percent of its total quarterly sales. Evn so, Huang, who declared in June that Nvidia “stopped thinking of ourselves as a chip company long ago,” believes physical A.I. will one day define the company’s future.
