Google's new AI model creates playable 3D worlds from images
What's the story
DeepMind, Google's artificial intelligence (AI) research division, has unveiled a new model called Genie 2.
The advanced AI can generate an infinite number of interactive 3D worlds from just a single image and text description.
The model succeeds the original Genie, which was released earlier this year. It is similar to models being developed by World Labs and Israeli start-up Decart.
Advanced features
Genie 2's capabilities and training
Genie 2 aims to generate a diverse set of rich 3D worlds, where you can jump, swim, and do a lot more with a mouse or keyboard.
The model has been trained on videos, which allows it to simulate object interactions, animations, lighting effects, physics principles, reflections as well as the behavior of non-player characters (NPCs).
Most of the simulations created by Genie 2 look like high-quality video games.
AI interaction
Genie 2's intelligent response and consistency
Genie 2 can create consistent worlds with varying perspectives, such as first-person and isometric views, for as long as a minute.
"Genie 2 responds intelligently to actions taken by pressing keys on a keyboard, identifying the character and moving it correctly," per DeepMind's blog post.
Unlike other world models that often fail with artifacting, consistency, and hallucination-related issues, Genie 2 can remember parts of a simulated scene that aren't in view and render them accurately when they become visible again.
Research tool
Genie 2's potential applications and future development
DeepMind envisions Genie 2 as a research and creative tool for prototyping interactive experiences and evaluating AI agents.
"Thanks to Genie 2's out-of-distribution generalization capabilities, concept art and drawings can be turned into fully interactive environments," the company stated.
Although still in its infancy, DeepMind believes Genie 2 will be critical in developing future AI agents. This is supported by Google's increased investment in world model research, the next big thing in generative AI.