A man played a musical instrument by a river, and music began to flow. When the prompt ‘a whale’s tail rises behind the person’ was entered into the input box, a whale’s tail instantly burst out of the river water. When the command ‘it starts to snow’ was subsequently added, snowflakes fluttered, and snow began to gently accumulate on the man’s head and clothes. Much like inside a video game, pressing directional keys and specific buttons also changed the viewpoint and camera position.
This was part of a video unveiled by Google DeepMind on the 27th (local time) when it introduced its new world model, Google Genie 4. Genie 4 has drawn industry attention because, going a step further from traditional video AI that generates videos all at once from a single prompt, it allows real-time intervention in the on-screen environment through prompts. Google has currently released Genie 4 as a research version to select researchers.
Genie 4 is Google’s world model. A world model is an AI that predicts and implements what will happen next based on surrounding environments and actions. Just as Large Language Models (LLMs) predict the next word based on preceding context, world models predict the environment that will change according to the user’s actions. It works by generating how the field of view changes when walking around a room, or how a ball rolls when touched.
This latest version of Genie 4 has become the first of Google’s real-time world models capable of self-generating sound. It produces not only sound effects such as wind blowing through trees and beach waves, but also dialogues between characters. The characters react in real-time to user prompts or events within the virtual world. For example, in a video made from the first-person perspective of a knight riding a dragon, when the prompt ‘the knight shouts as he attacks the castle, and another knight warns that it is a trap’ was entered, the characters flying on the dragon looked at each other and shouted.
World models like Genie 4 are expected to be utilized in content production such as games and movies, as well as in robot training. Game developers can create prototypes containing explorable game worlds by describing backgrounds, characters, and desired interactions in words. Google stated, “Game developers are testing visual elements, interaction methods, and more by inputting text prompts before writing any code.” In the film production sector, it is also being tested as a tool to preview scenes by changing camera angles, lighting, dialogue, and actions in real-time.
In the physical AI sector, such as robots and autonomous vehicles, world models can serve as virtual training grounds. For robots to operate in unfamiliar environments, they must learn actions and outcomes across diverse situations. Collecting real-world data requires significant time and cost, and repeatedly recreating high-risk accident scenarios is difficult. Seo Hyung-joo, Chief Technology Officer (CTO) at physical AI company Carbon Six, noted, “It is difficult to implement tasks like folding laundry with existing simulations. Utilizing a world model will make it possible to handle situations that were previously hard to express with traditional simulations over the long term.”
The key challenge lies in the accuracy and consistency of the virtual world. To be used for robot training, the visuals must not only look plausible, but the outcomes of actions must also align with reality. In games, spaces and objects must be maintained even after prolonged movement. Improving this aspect remains a task ahead. Google stated, “As simulations grow longer or complex modifications are repeated, the appearance and state of objects may change.”
