TORONTO — Some of AI’s most prominent researchers believe the field’s next big thing will be so-called world models that allow automated tools to simulate what happens next and become more effective at solving problems in real life.

World models give AI-powered systems like self-driving cars and household robots “the ability to imagine the future,” said Danijar Hafner, a research scientist at Google DeepMind. These systems could help realize some of AI’s biggest promises, like actually helpful android helpers, and perhaps even machines with human-level smarts. 

The new models are part of an AI branch called reinforcement learning, which trains systems by having them attempt tasks, then rewarding them when they succeed. Instead of trial and error in real conditions, world models plan and play out their approach in an internal simulation, said Hafner—the equivalent of humans using their imagination. 

Startups are already putting these new systems to work in virtual spaces. In December, San Francisco-based World Labs showed off a system that generates a 3D environment from a flat image, which it said could be used to help make video games and movies. AI pioneer Fei-Fei Li is CEO of the startup, which has raised US$230 million from backers including Toronto’s Radical Ventures. Two other companies in the field, Higgsfield AI and Runway, use similar technology to produce videos.

Companies building autonomous vehicles are also experimenting with world models, according to Hafner. Reinforcement learning requires “a lot of data,” he said, but, he added, it’s not feasible to have cars randomly driving around to work out how to stay in a lane. By using world models, developers can help their systems learn the rules of the road in simulations where no one can get hurt. 

Longer term, Hafner said world models could help robots take on a variety of tasks like helping with household chores, moving boxes around a warehouse or working a manufacturing line. 

Despite big AI advances, the futuristic-looking robots of today have limited capabilities. When Tesla showed off its Optimus prototype at an investor event last October, it reportedly had employees remotely controlling some of its actions, suggesting there’s still a big gap between AI dream and reality. “World models are the most promising technology to get us there,” Hafner said.

World models could also help businesses manage more of their work. New York-based Skyfall AI wants to create “a completely automated enterprise,” where AI agents handle everyday tasks in IT and other departments, said CEO Sam Pasupalak. 

While a self-driving car reacts to inputs from cameras and sensors, Skyfall’s world models train using the many actions workers take in their firm’s software systems. “What they’re trying to get is the cause and effect,” said Kaheer Suleman, the startup’s chief AI officer.

World models could also help get AI closer to human-level smarts, according to some researchers. To match people’s real-life capabilities, machines must reason, plan and understand the physical environment, according to AI pioneer Yann LeCun. They need “mental models of how the world works,” he said in a talk last October. “Every animal has one.” 

But not everyone in the field agrees. World models will help anywhere that AI needs to interact with the real world, said Hafner. But “it’s still pretty unclear to what extent that will help with general reasoning.”

Whatever the future holds for world models, they’ve opened up new opportunities for AI founders and investors. Large language models (LLMs) have dominated the popular and corporate conversation around AI since ChatGPT debuted two years ago. But LLMs are expensive to build, with leading startups OpenAI, Anthropic and Cohere raising and spending tens of billions between them on processing power alone. Some new AI firms are instead focusing on smaller models targeted at specific sectors. 

World models require “nowhere near” the amount of compute it takes to train the most advanced LLMs, said Hafner. He predicted that the two systems—frontier and world models—will eventually be joined, giving generative tools a better grasp of the real world.

Pasupalak said the ultimate cost of world models will depend on their size. He and Suleman previously started Maluuba, an AI startup based in Waterloo, Ont., that they sold to Microsoft in January 2017. They returned to AI with Skyfall to chase a new market opportunity. “The LLMs are hitting a peak,” Pasupalak claimed, citing their high cost and tendency to hallucinate. Skyfall, he added, is “taking a bet for what’s going to be big three to four years from now.”