Google recently dropped its AI text-to-video model, Lumiere. The results are putting existing models to the test. With Lumiere, Google’s AI video generation model, a quick text prompt can create seamless and realistic video.
Google just announced its latest AI video model that is capable of creating realistic, diverse, and coherent motion. Known as Lumiere, the latest offering from Google is a text-to-video and image-to-video model. In simple words, you input text or image and the AI neural networks translate it into a video. Based on recent reports, Lumiere is much beyond the simple text-to-video functionality.
The tool allows users to animate existing images, and create videos in the format of an input image or painting. It also allows video in painting and creating specific animation in sections within an image.
How does Lumiere create videos?
Google’s research titled ‘Lumiere: A Space-Time Diffusion Model for Video Generation’ offers the scientific details behind Lumiere. Lumiere generates videos by leveraging advanced AI algorithms. The model utilizes neural networks to interpret and translate input text or images into dynamic and realistic video content. The intricate mechanisms behind Lumiere’s video creation go beyond simple processes, showcasing the model’s sophisticated capabilities in transforming textual or visual inputs into engaging and coherent videos.
The main innovation here is the SpaceTime Diffusion model that generates the entire temporal duration of the video at once. In contrast, existing AI video models synthesize distant key frames one at a time. With Lumiere, Google aims to offer global temporal consistency ensuring coherent representation across different frames.
Watch this video to know how “Google’s Lumiere Works!”
Note:
The model has yet to be released to the general public; however, if you are interested in learning more or watching the models in action, you can visit the Lumiere website, where you can see plenty of demos of the model performing the different tasks.