Adobe's NEW AI Video Features are AMAZING! (Let’s Recreate Them)

Curious Refuge
22 Apr 202424:46

TLDRThis week's AI film news is packed with exciting developments in AI video editing. Microsoft's Vasa 1 tool is set to revolutionize entertainment by converting images and audio into realistic videos with controllable eye movements. Adobe announced new AI features for Premiere Pro, including video painting, object removal, and generative AI video, though they won't be available until the end of the year. However, workarounds using current technology are presented. Meta's new AI assistant, a competitor to Chat GPT, offers free and impressive capabilities, including real-time image generation and multimodal experiences. Additionally, researchers have developed a way to turn any video into a video game, and the episode features unique AI films, such as 'Amy' with a 1950s TV show aesthetic and 'The Fliff Atrre,' an AI documentary. The community is also encouraged to subscribe for the latest AI tutorials and news.

Takeaways

  • 🎥 Microsoft released a tool called Vasa 1 that can turn images and audio clips into realistic videos with controllable eye movements and character types.
  • 🚀 Vasa 1 has the potential to revolutionize entertainment and advertising by creating personalized media content based on the consumer.
  • 🎨 Adobe announced new AI video editing features for Premiere Pro, including video painting and object removal, but they won't be available until the end of the year.
  • 🧩 Pik Labs can be used today to perform similar video painting tasks as Adobe's upcoming features, though with slightly lower fidelity.
  • 🔍 Runway offers an object removal tool similar to Adobe's, capable of automatically removing objects from footage in near real-time.
  • 📐 Adobe's generative expand feature can extend video clips using AI, but the integration of live-action footage with AI-generated video needs improvement.
  • 🔄 A workaround for generative expand can be achieved by exporting a frame from the video editing software and using Runway's image to video feature.
  • 🌐 Adobe is opening up its API to integrate other AI models like Sora and Runway directly into Premiere Pro for more creative possibilities.
  • 🏆 Meta released a free, competitive AI assistant that can generate real-time images and has a multimodal experience, although it's not as creative as chat GPT.
  • 🎮 Researchers developed a way to turn any video into a video game by scanning the environment and creating interactive games based on the video content.
  • 📽️ AI film news highlighted several AI-generated films, showcasing the increasing realism and creativity in AI-driven storytelling and film production.
  • 🎉 The AI creativity tournament in Las Vegas was a success, indicating a growing interest and community around AI in creative fields.

Q & A

  • What is the name of the new tool released by Microsoft that has implications for entertainment and society?

    -The new tool released by Microsoft is called Vasa 1. It allows users to upload an image and an audio clip, which Vasa 1 will then turn into a realistic video with the unique ability to control and change the direction of eye movement.

  • How does the Vasa 1 tool differ from other AI tools in terms of character animation?

    -Vasa 1 has the ability to create animations that are nonhuman but in a very realistic way. It also allows for the projection of the same movement on different types of characters and provides more control over the 3D dynamics of the character, including the orientation during the animation process.

  • What is the significance of the real-time rendering feature of the Vasa 1 tool?

    -The real-time rendering feature of Vasa 1 allows users to upload an image and an audio clip and see the footage played back immediately. This has significant implications for the future of live-action entertainment and other areas of society, such as AI therapy and education.

  • What new AI video tools did Adobe announce for Premiere Pro?

    -Adobe announced several new AI video tools for Premiere Pro, including video in painting, which allows users to create a mask and prompt what they want to see, an object removal tool with a content-aware feature for automatic rotoscoping, and generative expand, which extends the length of a clip using artificial intelligence.

  • How can users currently achieve video inpainting without waiting for Adobe's Premiere Pro update?

    -Users can achieve video inpainting using third-party tools like Pika's website (Pika.doart), where they can upload a video, select the region they want to change, and generate new content for that region.

  • What is the potential issue with Adobe's generative expand feature when it comes to integrating AI video with live-action footage?

    -The generative expand feature may result in a static and slow push-in effect when cutting from the original clip to the AI video, making it feel disjointed and less cohesive. There can also be inconsistencies in lighting and object appearance that reveal the AI-generated segment.

  • What is the significance of Adobe opening up their API to other tech companies?

    -Opening up their API allows other tech companies to integrate their AI models directly into Premiere Pro, expanding the creative possibilities and efficiency for users by enabling the generation of various AI video content within the software.

  • What was the outcome of the AI creativity tournament held in Las Vegas?

    -The AI creativity tournament in Las Vegas was a success, with 450 people attending. The winner of the Apple Vision Pro competition was Jagger, who also received a custom crochet trophy from Curious Refuge.

  • How does Meta's new AI assistant compare to Chat GPT in terms of creativity?

    -Meta's AI assistant is as creative as Chat GPT 3.5 but not as advanced as version 4. It can generate ideas and images in real-time and offers a multimodal experience, allowing for text-based interactions as well as image generation and animation.

  • What is the innovative feature of the tool that turns any video into a video game?

    -The tool converts any video into a Nerf scanned environment, allowing users to play various games based on the scanned environment. It can transform videos into 3D scans where users can interact with objects in a virtual setting.

  • What are the three AI films featured in the AI film news of the week?

    -The three AI films featured are 'Amy,' which has a 1950s TV show style aesthetic and uses Dr. Seuss rhyming; 'The Fliff Atrre,' an AI documentary by The Real Robot; and a project from PZF AI that creates a very realistic AI film with convincing shots and sound design.

Outlines

00:00

🚀 Microsoft's Vasa 1: AI Video Tool Revolution

Microsoft has unveiled Vasa 1, a groundbreaking AI tool that can transform still images and audio clips into realistic videos. Vasa 1's unique feature is its ability to control eye movements and apply the same movements across different character types. This has significant implications for the future of entertainment and advertising, offering personalized media experiences. The tool can also animate non-human entities in a realistic manner, as demonstrated by the Mona Lisa singing scenario. Additionally, it provides 3D dynamics control for character orientation during animation. Vasa 1 renders videos in real-time, which could impact live-action entertainment and other societal applications, such as AI therapy and education.

05:01

🎨 Adobe Premiere Pro's Upcoming AI Features

Adobe has announced new AI video tools for Premiere Pro, set to be released by year's end. The first feature is 'video in painting,' which allows users to create a mask and transform objects within a video, as illustrated by changing a briefcase's contents. Although these features are not yet available, workarounds using current tools like Pik Labs are presented. Pik Labs enables users to modify video regions and generate new content, such as replacing diamonds with potatoes or changing a watch face. Adobe also demonstrated an object removal tool that operates like automatic rotoscoping, with Runway offering similar functionality currently. Lastly, Adobe introduced 'generative expand,' which extends video clips using AI, with varying results in integration and realism.

10:03

🧩 Extending Video Clips with AI Tools

The video discusses the ability to extend video clips using AI, as showcased by Adobe's generative expand feature. The speaker tests the integration of live-action footage with AI-generated video and notes a lack of cohesion, suggesting that current AI tools may not yet produce seamless results. However, a method to extend clips using Runway's 'image to video' feature is demonstrated, showing a relatively realistic outcome. The video also critiques Adobe's generative extend model, pointing out inconsistencies and suggesting that while the tool could be useful in certain contexts, it is not without flaws.

15:04

🤖 Meta's AI Assistant and Creativity in AI

Meta has released a free AI assistant, a competitor to Chat GPT, which is integrated into the Meta Suite and accessible through a desktop application. The tool provides up-to-date search results and can generate real-time images based on prompts. It is compared to Chat GPT, with the latter being deemed more creative for professional use, while Meta's tool is considered the best free option. The AI can also create multimodal experiences, generating artwork and animations based on textual prompts, offering a creative partnership for users.

20:07

🎮 AI Film News and Community Highlights

The video concludes with a discussion on AI film news, highlighting a community event in Las Vegas and the release of Meta's AI tool. It also showcases three AI films: 'Amy,' a 1950s TV show-style narrative about a robot seeking a heart; 'The Fliff Atrre,' an AI documentary about a fictional creature; and a project from PZF AI that mimics a live-action film with convincing shots and sound design. The video encourages viewers to subscribe for AI tutorials and news, and thanks the community for their support.

Mindmap

Keywords

💡AI Video Features

AI Video Features refer to the new capabilities being developed by technology companies to enhance video editing and creation using artificial intelligence. In the video, Adobe's announcement of AI video editing features in Premiere Pro is a central topic, highlighting how these features will revolutionize the way content is produced.

💡Reverse Engineering

Reverse engineering in this context means taking apart an existing system or product to understand how it works, in order to replicate or improve upon its design. The video discusses how to reverse engineer Adobe's new AI video features using current technology.

💡Vasa 1

Vasa 1 is a new tool released by Microsoft that has significant implications for entertainment and beyond. It can transform an uploaded image and audio clip into a realistic video, with unique features such as controlling eye movement and applying the same movement to different character types. It's showcased as a powerful example of AI's potential in creative fields.

💡Video In Painting

Video in painting is a feature that allows users to create a mask around an object in a video and then change the contents of that masked area. Adobe demonstrated this by transforming a briefcase with a few diamonds into one with many diamonds, illustrating the potential for altering video content in creative ways.

💡Object Removal Tool

An object removal tool is a software feature that can automatically detect and remove unwanted objects from video footage. Adobe's announcement includes a tool that pairs with a content-aware feature for seamless object removal, akin to automatic rotoscoping, which is a significant time-saver in post-production.

💡Generative Expand

Generative expand is a technology that extends the length of a video clip using artificial intelligence. It's designed to create a more extended, seamless sequence that maintains the original clip's style and motion, which is particularly useful for filmmakers looking to add duration to a scene without reshoots.

💡Generative AI Video

Generative AI video involves using AI to create new video content from scratch, often by generating images or scenes that did not exist in the original footage. Adobe's update to Premiere Pro includes the integration of generative AI video, which is set to save time and open up new creative possibilities.

💡API Access

API, or Application Programming Interface, access refers to the ability for different software or tech companies to interact with each other's services. Adobe's announcement about opening up their API means that other companies can integrate their AI models directly into Premiere Pro, expanding its capabilities.

💡Meta's Creative AI Assistant

Meta's Creative AI Assistant is a new tool that competes with chat GPT, offering free services for generating text, images, and potentially more. It's integrated into Meta's suite of applications and is designed to be a versatile and up-to-date tool for creative tasks.

💡AI Creative Esports League

The concept of an AI Creative Esports League suggests the development of a competitive environment where AI technologies are used to create content, possibly in a gaming context. It indicates a future where AI is not only a tool for individual creators but also a platform for collaborative and competitive events.

💡Multimodal Experience

A multimodal experience in the context of AI refers to the ability of an AI system to engage with users through multiple senses or modes, such as text, images, and potentially sound or touch. Meta's AI tool is described as offering a multimodal experience, suggesting it can generate and interact with various types of content.

Highlights

Microsoft released a new tool called Vasa 1 that can turn images and audio clips into realistic videos with controllable eye movements.

Vasa 1 allows for the creation of animations in various styles, including non-human characters like the Mona Lisa singing.

The tool can render animations in real-time, with potential applications in live-action entertainment and AI therapy.

Adobe announced new AI video editing features for Premiere Pro, including video painting and object removal, but they won't be available until the end of the year.

Pika and Runway can be used today to achieve similar effects to Adobe's upcoming features, such as video inpainting and object removal.

Adobe's generative expand feature can extend video clips using AI, but the integration with live-action footage can be static and slow.

Runway can be used to extend video clips with AI, offering a quick fix until Adobe's feature is available.

Adobe's update includes generative AI video directly in Premiere Pro, with an open API for third-party tools like Sora and Runway.

The integration of AI-generated video with live-action footage can be challenging and may not always appear cohesive.

Meta released a free creative AI assistant that can generate real-time images and has a multimodal experience for conversations and artwork generation.

Meta's AI tool is less creative than chat GPT but offers a free alternative with the ability to create movie ideas and visualize them.

Researchers have developed a method to turn any video into a video game by scanning the environment and creating interactive games.

AI Film News showcased three AI films this week, including 'Amy' with a 1950s TV show aesthetic, 'The Fliff Atrus' as an AI documentary, and a realistic AI project from PZF AI.

The AI creativity tournament in Las Vegas attracted 450 attendees and featured competitions and panels discussing the future of creative technology.

Adobe's new features aim to democratize creative processes by integrating AI tools directly into video editing software.

The current state of AI-generated content integration in film making is promising but still faces challenges in achieving seamless realism.

The community is encouraged to subscribe to AI Film News for the latest tutorials and news on AI in film making.