Adobe's NEW AI Video Features are AMAZING! (Let’s Recreate Them)
TLDRThis week's AI film news is packed with exciting developments in AI video editing. Microsoft's Vasa 1 tool is set to revolutionize entertainment by converting images and audio into realistic videos with controllable eye movements. Adobe announced new AI features for Premiere Pro, including video painting, object removal, and generative AI video, though they won't be available until the end of the year. However, workarounds using current technology are presented. Meta's new AI assistant, a competitor to Chat GPT, offers free and impressive capabilities, including real-time image generation and multimodal experiences. Additionally, researchers have developed a way to turn any video into a video game, and the episode features unique AI films, such as 'Amy' with a 1950s TV show aesthetic and 'The Fliff Atrre,' an AI documentary. The community is also encouraged to subscribe for the latest AI tutorials and news.
Takeaways
- 🎥 Microsoft released a tool called Vasa 1 that can turn images and audio clips into realistic videos with controllable eye movements and character types.
- 🚀 Vasa 1 has the potential to revolutionize entertainment and advertising by creating personalized media content based on the consumer.
- 🎨 Adobe announced new AI video editing features for Premiere Pro, including video painting and object removal, but they won't be available until the end of the year.
- 🧩 Pik Labs can be used today to perform similar video painting tasks as Adobe's upcoming features, though with slightly lower fidelity.
- 🔍 Runway offers an object removal tool similar to Adobe's, capable of automatically removing objects from footage in near real-time.
- 📐 Adobe's generative expand feature can extend video clips using AI, but the integration of live-action footage with AI-generated video needs improvement.
- 🔄 A workaround for generative expand can be achieved by exporting a frame from the video editing software and using Runway's image to video feature.
- 🌐 Adobe is opening up its API to integrate other AI models like Sora and Runway directly into Premiere Pro for more creative possibilities.
- 🏆 Meta released a free, competitive AI assistant that can generate real-time images and has a multimodal experience, although it's not as creative as chat GPT.
- 🎮 Researchers developed a way to turn any video into a video game by scanning the environment and creating interactive games based on the video content.
- 📽️ AI film news highlighted several AI-generated films, showcasing the increasing realism and creativity in AI-driven storytelling and film production.
- 🎉 The AI creativity tournament in Las Vegas was a success, indicating a growing interest and community around AI in creative fields.
Q & A
What is the name of the new tool released by Microsoft that has implications for entertainment and society?
-The new tool released by Microsoft is called Vasa 1. It allows users to upload an image and an audio clip, which Vasa 1 will then turn into a realistic video with the unique ability to control and change the direction of eye movement.
How does the Vasa 1 tool differ from other AI tools in terms of character animation?
-Vasa 1 has the ability to create animations that are nonhuman but in a very realistic way. It also allows for the projection of the same movement on different types of characters and provides more control over the 3D dynamics of the character, including the orientation during the animation process.
What is the significance of the real-time rendering feature of the Vasa 1 tool?
-The real-time rendering feature of Vasa 1 allows users to upload an image and an audio clip and see the footage played back immediately. This has significant implications for the future of live-action entertainment and other areas of society, such as AI therapy and education.
What new AI video tools did Adobe announce for Premiere Pro?
-Adobe announced several new AI video tools for Premiere Pro, including video in painting, which allows users to create a mask and prompt what they want to see, an object removal tool with a content-aware feature for automatic rotoscoping, and generative expand, which extends the length of a clip using artificial intelligence.
How can users currently achieve video inpainting without waiting for Adobe's Premiere Pro update?
-Users can achieve video inpainting using third-party tools like Pika's website (Pika.doart), where they can upload a video, select the region they want to change, and generate new content for that region.
What is the potential issue with Adobe's generative expand feature when it comes to integrating AI video with live-action footage?
-The generative expand feature may result in a static and slow push-in effect when cutting from the original clip to the AI video, making it feel disjointed and less cohesive. There can also be inconsistencies in lighting and object appearance that reveal the AI-generated segment.
What is the significance of Adobe opening up their API to other tech companies?
-Opening up their API allows other tech companies to integrate their AI models directly into Premiere Pro, expanding the creative possibilities and efficiency for users by enabling the generation of various AI video content within the software.
What was the outcome of the AI creativity tournament held in Las Vegas?
-The AI creativity tournament in Las Vegas was a success, with 450 people attending. The winner of the Apple Vision Pro competition was Jagger, who also received a custom crochet trophy from Curious Refuge.
How does Meta's new AI assistant compare to Chat GPT in terms of creativity?
-Meta's AI assistant is as creative as Chat GPT 3.5 but not as advanced as version 4. It can generate ideas and images in real-time and offers a multimodal experience, allowing for text-based interactions as well as image generation and animation.
What is the innovative feature of the tool that turns any video into a video game?
-The tool converts any video into a Nerf scanned environment, allowing users to play various games based on the scanned environment. It can transform videos into 3D scans where users can interact with objects in a virtual setting.
What are the three AI films featured in the AI film news of the week?
-The three AI films featured are 'Amy,' which has a 1950s TV show style aesthetic and uses Dr. Seuss rhyming; 'The Fliff Atrre,' an AI documentary by The Real Robot; and a project from PZF AI that creates a very realistic AI film with convincing shots and sound design.
Outlines
🚀 Microsoft's Vasa 1: AI Video Tool Revolution
Microsoft has unveiled Vasa 1, a groundbreaking AI tool that can transform still images and audio clips into realistic videos. Vasa 1's unique feature is its ability to control eye movements and apply the same movements across different character types. This has significant implications for the future of entertainment and advertising, offering personalized media experiences. The tool can also animate non-human entities in a realistic manner, as demonstrated by the Mona Lisa singing scenario. Additionally, it provides 3D dynamics control for character orientation during animation. Vasa 1 renders videos in real-time, which could impact live-action entertainment and other societal applications, such as AI therapy and education.
🎨 Adobe Premiere Pro's Upcoming AI Features
Adobe has announced new AI video tools for Premiere Pro, set to be released by year's end. The first feature is 'video in painting,' which allows users to create a mask and transform objects within a video, as illustrated by changing a briefcase's contents. Although these features are not yet available, workarounds using current tools like Pik Labs are presented. Pik Labs enables users to modify video regions and generate new content, such as replacing diamonds with potatoes or changing a watch face. Adobe also demonstrated an object removal tool that operates like automatic rotoscoping, with Runway offering similar functionality currently. Lastly, Adobe introduced 'generative expand,' which extends video clips using AI, with varying results in integration and realism.
🧩 Extending Video Clips with AI Tools
The video discusses the ability to extend video clips using AI, as showcased by Adobe's generative expand feature. The speaker tests the integration of live-action footage with AI-generated video and notes a lack of cohesion, suggesting that current AI tools may not yet produce seamless results. However, a method to extend clips using Runway's 'image to video' feature is demonstrated, showing a relatively realistic outcome. The video also critiques Adobe's generative extend model, pointing out inconsistencies and suggesting that while the tool could be useful in certain contexts, it is not without flaws.
🤖 Meta's AI Assistant and Creativity in AI
Meta has released a free AI assistant, a competitor to Chat GPT, which is integrated into the Meta Suite and accessible through a desktop application. The tool provides up-to-date search results and can generate real-time images based on prompts. It is compared to Chat GPT, with the latter being deemed more creative for professional use, while Meta's tool is considered the best free option. The AI can also create multimodal experiences, generating artwork and animations based on textual prompts, offering a creative partnership for users.
🎮 AI Film News and Community Highlights
The video concludes with a discussion on AI film news, highlighting a community event in Las Vegas and the release of Meta's AI tool. It also showcases three AI films: 'Amy,' a 1950s TV show-style narrative about a robot seeking a heart; 'The Fliff Atrre,' an AI documentary about a fictional creature; and a project from PZF AI that mimics a live-action film with convincing shots and sound design. The video encourages viewers to subscribe for AI tutorials and news, and thanks the community for their support.
Mindmap
Keywords
💡AI Video Features
💡Reverse Engineering
💡Vasa 1
💡Video In Painting
💡Object Removal Tool
💡Generative Expand
💡Generative AI Video
💡API Access
💡Meta's Creative AI Assistant
💡AI Creative Esports League
💡Multimodal Experience
Highlights
Microsoft released a new tool called Vasa 1 that can turn images and audio clips into realistic videos with controllable eye movements.
Vasa 1 allows for the creation of animations in various styles, including non-human characters like the Mona Lisa singing.
The tool can render animations in real-time, with potential applications in live-action entertainment and AI therapy.
Adobe announced new AI video editing features for Premiere Pro, including video painting and object removal, but they won't be available until the end of the year.
Pika and Runway can be used today to achieve similar effects to Adobe's upcoming features, such as video inpainting and object removal.
Adobe's generative expand feature can extend video clips using AI, but the integration with live-action footage can be static and slow.
Runway can be used to extend video clips with AI, offering a quick fix until Adobe's feature is available.
Adobe's update includes generative AI video directly in Premiere Pro, with an open API for third-party tools like Sora and Runway.
The integration of AI-generated video with live-action footage can be challenging and may not always appear cohesive.
Meta released a free creative AI assistant that can generate real-time images and has a multimodal experience for conversations and artwork generation.
Meta's AI tool is less creative than chat GPT but offers a free alternative with the ability to create movie ideas and visualize them.
Researchers have developed a method to turn any video into a video game by scanning the environment and creating interactive games.
AI Film News showcased three AI films this week, including 'Amy' with a 1950s TV show aesthetic, 'The Fliff Atrus' as an AI documentary, and a realistic AI project from PZF AI.
The AI creativity tournament in Las Vegas attracted 450 attendees and featured competitions and panels discussing the future of creative technology.
Adobe's new features aim to democratize creative processes by integrating AI tools directly into video editing software.
The current state of AI-generated content integration in film making is promising but still faces challenges in achieving seamless realism.
The community is encouraged to subscribe to AI Film News for the latest tutorials and news on AI in film making.
