Friday, October 4, 2024
HomeAI NewsGenerative AIGoogle Showcases the Impressive AI Video Editor Dreamix

Google Showcases the Impressive AI Video Editor Dreamix

Published on

Related

AI App Review: Pixverse.AI

Text-to-Video generative AI apps like Pixverse are on the rise and our social timelines...

Spotify AI DJ: Impressive Personalization With AI-Powered DJ

Spotify just announced Spotify AI DJ, its AI assistant that will take your music...

Alongside a paper published jointly with the The Gebrew University of Jerusalem, Google Research Group recently showcased its newest AI tool, Dreamix on its Github page.

Dreamix, a text-driven video diffusion model, as Google puts it, takes either a video or an image as its input, and works its magic to create a video after you direct it with your text prompts.

What is a “Diffusion Model”?

We have been hearing a lot about diffusion models when it comes to image / video creation and you may be wondering what is it. In simple terms, diffusion models in AI are generative models, and they are trained by first destroying an image they are feed by creating noise. After the destruction, they try to rebuild the images from the noise. And they do it millions of times to improve their creation abilities. Dreamix is a text-conditioned video diffusion model, meaning it uses both text and videos/images to generate outputs. In the published paper, researches note possible future applications as motion interpolation between images and text guided painting.

But what does that mean for us mere mortals. The premise is pretty simple, really. Do you want to make a video of a teddy bear running on the street? No problem, input a photo of a teddy bear into the tool and write the prompt “a bear walking”, and voilà, your walking teddy bear video is ready. Well, not quite yet. Although the tool really makes your favorite teddy bear to walk on a street with a prompt, the tool is not yet publicly available.

The reason is pretty simple (and kind of scary, really): Media generation tools are pretty new and potential malicious use cases are not thoroughly researched. In the research paper, the contributors to this tool raised their concerns:

Malicious parties may try to use edited videos to mis-lead viewers or to engage in targeted harassment. Future research must continue investigating these concerns.

A sample video generated by Dreamix with a text prompt. Now, how malicious this can be?

Now the good news! Dreamix has definitely a lot of potential to be helpful to many people a great deal. Video editing has somewhat been an area with a steep learning curve, and it is only becoming mainstream in the last few years due to many easy-to-use video editing apps popping up on iOS / Android App Stores. And now with the ability to generate videos with just 1 picture and a text prompt takes this ease-of-use to next level.

Dreamix accepts videos and images to supplement the text prompt, but here are some creative examples from the demo page on Github:

Creative Uses of Dreamix

1. Enhance a Video Scene

Dreamix can be used to enhance a scenery. No we’re not talking about Facetune type of beauty make-up. Dreamix properly re-creates the scene, and blows life into videos, as shown in the tropical island video below:

Impressive object addition / enhancement by Dreamix. Swans are not real, we think.

2. Animate Still Objects

Do you remember mobile apps that allows you take many videos of your Lego characters and then stitches them to make a short animation. Well, Dreamix can do the video with just a few pictures. And, as the icing on the cake, it can add elements that are not in the original photos. Take a look at this example of a figure doing weight-lifting:

A typical fireman.

3. Switch the Subject Animal With Another

Did you ever wish your cat to make the suave dancing moves of your neighbour’s dog? Well, it seems that you might just be able to create a video of. See Dreamix at work:

We will definitely use Dreamix for this.

4. Change the Subject in A Video

This one is particularly interesting as it might literally change the food preparation videos. See below how a pot full of onions are switched with onions:

Dreamix turning onions into noodles. One pot to rule them all.

5. Create Timelapse Videos

Another particular video we were impressed by Dreamix is how it can create a timelapse video a plant growing.

Dreamix giving life to plants, figuratively of course.

We have seen a lot of text generators (ChatGTP), and image creators (Dall-E, Midjourney) in the last couple of years, but this is the first tool that uses text prompts to create and direct videos. While this tech is new, it is still a bit far from mainstream distribution, mainly first because of the computation costs (a short video will need 30 minutes of processing time), and second the ethical considerations as we discussed above. Still we are curious what these types of tools will bring to the table (of Youtube creators mostly?)

SourceGithub

Author

Editorial Team
Editorial Teamhttps://thebase.ai
Bringing you curated news about AI and latest tech.

Latest articles

AI App Review: Pixverse.AI

Text-to-Video generative AI apps like Pixverse are on the rise and our social timelines...

Spotify AI DJ: Impressive Personalization With AI-Powered DJ

Spotify just announced Spotify AI DJ, its AI assistant that will take your music...

What is ChatGPT: The Powerful AI Assistant

If you're reading this, chances are you've heard about ChatGPT and might be wondering...

LLM Meaning: Definition and 5 Use Cases of the Exciting Large Language Models

Large Language Models (LLMs) are a type of artificial intelligence (AI) systems that have...

AI Art Generator: Best 24 AI Image Generators, Free and Paid

AI Art Generators are the hottest topics in town right now, and for a...

More like this

AI App Review: Pixverse.AI

Text-to-Video generative AI apps like Pixverse are on the rise and our social timelines...

Spotify AI DJ: Impressive Personalization With AI-Powered DJ

Spotify just announced Spotify AI DJ, its AI assistant that will take your music...

What is ChatGPT: The Powerful AI Assistant

If you're reading this, chances are you've heard about ChatGPT and might be wondering...