A Legit Sora Competitor is Here + Testing AI Actor Emotions

Curious Refuge
4 May 202419:48

TLDRThis week in AI film news, a new Sora competitor called Vu emerges, offering video generation through prompts with up to 16 seconds of 1080P quality. Adobe's Video Giga Gan tool is highlighted for its ability to upres footage realistically, enhancing facial textures and details. Synthesia's expressive AI avatars are showcased, which respond emotionally to text inputs. The team at Nvidia introduces large-scale NeRF scans, allowing real-time rendering of areas up to 25 square km. Adobe and UCSD present prompt-based image editing, while Render's AI image editing tools are demonstrated, including changing clothing and objects in images. A team from China presents a method to edit 3D models using prompts. The community is invited to AI events, including the Can Film Festival and AI on the Lot. The episode concludes with a discussion on AI films, including 'Dear Me,' which won the Beijing Film Festival 2024, and the AI trailer competition in partnership with Submachine.

Takeaways

  • 🚀 A new Sora competitor named Vu has emerged, offering video generation based on prompts with up to 16 seconds in 1080P quality.
  • 🎥 Adobe's Video Giga Gan tool can upscale low-resolution footage to high resolution while maintaining realistic facial textures and details.
  • 🤖 Synthesia's Expressive AI Avatars can now respond emotionally to text prompts, changing their facial structure based on the words spoken.
  • 📈 Synthesia recently raised $90 million, indicating significant investment in the development of AI avatars.
  • 🧑🏻‍💼 LinkedIn's creator, Reid Hoffman, cloned himself digitally and had a conversation with his AI counterpart, showcasing the future of AI in society.
  • 🌐 Nvidia's research team has developed the ability to perform large-scale NeRF scans, allowing areas up to 25 square kilometers to be scanned and rendered in real time.
  • 🎨 Adobe and the University of California San Diego have partnered to demonstrate prompt-based image editing, where images can be contextually altered with AI based on textual descriptions.
  • 👕 Render's online application allows users to change clothing in images using AI prompts, with the example given of changing a jacket to a red one seamlessly.
  • 🦕 A team from China has developed a method to edit 3D models using prompts, including generating models through conversation and making edits by painting.
  • 🎬 "Dear Me," an AI-assisted film that integrated live-action footage with AI techniques, won the Beijing Film Festival 2024.
  • 📚 Chat GBT can now store information typed in as a memory to assist with future prompts, reducing the need for repetitive reprompting.

Q & A

  • What is the new Sora competitor mentioned in the video?

    -The new Sora competitor mentioned is called Vu, which allows users to generate videos based on text prompts.

  • How does Adobe's Video Giga Gan tool enhance video footage?

    -Video Giga Gan is an AI tool that upscales low-resolution footage to a higher resolution in a realistic way, improving facial textures, details in hair, and overall image quality without making it look plastic or overly smooth.

  • What is Synthesia's expressive AI avatars feature?

    -Synthesia's expressive AI avatars are AI actors that can change their facial expressions and emotional responses based on the text or words inputted into the system.

  • What is the significance of the large scale NeRF scans developed by Nvidia?

    -Nvidia's large scale NeRF scans allow for the scanning of areas up to 25 square kilometers and render them in real time on a computer. This technology can be used for virtual production sets in Hollywood, video games, and other applications, potentially changing the entertainment industry.

  • How does the AI image editing tool from Render work?

    -Render's AI image editing tool allows users to upload an image and make specific changes to it using AI prompts. For example, users can change the clothing of a person in an image or edit other elements within the scene.

  • What is the process like when using Sora to create AI videos?

    -Using Sora is described as similar to a slot machine experience where you input a prompt and hope for the desired outcome. It involves extensive post-processing, including color changes and rotoscoping. The render-to-use ratio can be as high as 300:1, meaning 300 shots are generated for every one used in the final film.

  • What is the AI film news of the week featuring?

    -The AI film news of the week features advancements in AI video generation, including the new competitor to Sora called Vu, as well as various AI tools for image and video editing, and the exploration of AI avatars by Synthesia.

  • How does the AI tool from China allow for 3D model editing?

    -The AI tool from China enables users to generate 3D models through conversational prompts and allows for the editing of these models by simply painting or drawing the desired changes onto the model.

  • What is the AI on the Lot event?

    -AI on the Lot is an AI film-making event that showcases the use of AI in the film industry. It is recommended for those in the Los Angeles area who are interested in the intersection of AI and filmmaking.

  • What is the significance of the AI film 'Dear Me'?

    -Dear Me is an AI-assisted film that won the Beijing Film Festival 2024. It integrates live-action footage with style transfers and other AI concepts to create a stylized film.

  • What is the role of the AI community in the development of AI film-making?

    -The AI community plays a crucial role in the development of AI film-making by networking artists, sharing the latest trends and techniques, and fostering a collaborative environment for innovation in the field.

Outlines

00:00

🎥 AI in Filmmaking: New Tools and Avatars

The video script introduces a new competitor to Sora, highlighting advancements in AI where actors can change emotions based on dialogue. It discusses Adobe's Video Giga Gan tool, which upscales footage to high resolution while maintaining realism. The script also covers Synthesia's expressive AI avatars that respond to text prompts with emotional expressions. Other advancements include AI video tools like Vu, which can generate short, high-quality videos from prompts, and the use of AI in post-processing for color correction and asset comping, as demonstrated by the creators of an AI video for Sora.

05:04

🌐 Large-Scale 3D Scanning and Image Editing with AI

The script discusses Nvidia's ability to perform large-scale 3D scans, covering up to 25 square kilometers, and render them in real-time, which could revolutionize virtual production in Hollywood and video games. It also covers Adobe and UCSD's research on prompt-based image editing, allowing users to contextually change images through text prompts. Render's online application is highlighted for its AI-driven image editing capabilities, such as changing clothing or objects in an image with high accuracy and realism.

10:05

📚 AI Video Generation and 3D Model Editing

The script introduces a new AI video tool called Vu, which can generate short videos from text prompts, competing with Sora. It also discusses a team from China's ability to edit 3D models using prompts, allowing for simple modifications like opening a dinosaur's mouth or converting a banana into a whale. The tool's potential for combining models and creating dynamic AI movement is explored, along with the challenges of rendering and post-processing as shared by the creators of an AI film.

15:08

🚀 AI Filmmaking Courses, Events, and Community

The script promotes an AI advertising and filmmaking course, highlighting networking opportunities with artists from major studios and a welcoming AI community. It invites viewers to AI events worldwide, including meetups and the Cannes Film Festival. The script also mentions an interview with Nicholas Newbert, an AI art director, discussing his work and the impact of AI on his career. It concludes with a showcase of AI-assisted films, including award winners and innovative short films, demonstrating the growing capabilities and creative potential of AI in filmmaking.

Mindmap

Keywords

💡Sora

Sora is an AI video tool that allows users to generate videos from text prompts. It is mentioned as a benchmark for comparison in the video, where a new competitor, Vu, is described. Sora is highlighted for its ability to create dynamic AI movement and cinematic scenes, which is a central theme of the video discussing advancements in AI video generation technology.

💡AI Actor Emotions

AI Actor Emotions refer to the capability of AI to simulate and change emotions based on the text or dialogue input. This is showcased through Synthesia's expressive AI avatars, which can respond emotionally to the text they are given. This technology is significant as it brings AI-generated characters closer to human-like expressiveness, which is a key aspect of the video's exploration of AI in film and entertainment.

💡Video Giga Gan

Video Giga Gan is a tool developed by Adobe's research team that upres (up-scales resolution) footage in a highly realistic manner. The video script provides examples where low-resolution footage is transformed into high-quality images with detailed facial textures and hair details. This tool is significant as it represents a leap in AI's ability to enhance visual content, which is a major theme in the video about the evolving landscape of AI in media production.

💡Synthesia

Synthesia is a company that has developed expressive AI avatars. These avatars can change their facial structure and express emotions based on the text they are programmed with. The script mentions that Synthesia has raised significant funding, indicating the potential and investment in AI-driven character animation, which is a key point in the video's discussion on the future of AI in filmmaking.

💡Nvidia's Large Scale NeRF Scans

Nvidia's Large Scale NeRF (Neural Radiance Fields) Scans is a technology that allows for the scanning and real-time rendering of areas up to 25 square kilometers. The video discusses how this technology can be applied to create detailed virtual environments for use in film production, video games, and other entertainment mediums. This represents a significant development in the integration of AI with physical environments, which is a central topic in the video.

💡Prompt Based Image Editing

Prompt Based Image Editing is a technique where AI uses textual prompts to make specific changes to images. The video mentions a collaboration between Adobe and the University of California San Diego to showcase this technology. It is exemplified by the ability to move objects within an image or change the context of a scene based on a description. This keyword is important as it highlights the growing sophistication of AI in understanding and manipulating visual content.

💡Render

Render is an online application that provides advanced AI image editing features through a user-friendly platform. The video script describes how users can change clothing or other elements in an image by simply providing a prompt. This tool is significant as it demonstrates the practical applications of AI in everyday image editing tasks, which is a theme of making AI technology accessible and useful for creative work.

💡3D Model Editing with Prompts

3D Model Editing with Prompts is a technology that allows users to generate and edit 3D models through conversational prompts. The video script explains how this can be done by painting or describing the desired changes. This technology is noteworthy as it represents a new approach to 3D modeling that simplifies the process and makes it more intuitive, aligning with the video's focus on AI's role in simplifying and enhancing creative processes.

💡Vu

Vu is an AI video tool that competes with Sora, allowing users to generate videos from text prompts. The video discusses the quality of videos produced by Vu and compares them to Sora, noting that while not as high quality, Vu is a significant competitor. The mention of Vu is important as it underscores the video's theme of competition and innovation in the field of AI video generation.

💡AI Filmmaking

AI Filmmaking refers to the use of AI technologies in the creation and production of films. The video script discusses various AI tools and techniques being used in this context, such as AI video generation, character animation, and image editing. AI Filmmaking is a central theme of the video, which explores how these technologies are shaping the future of film and entertainment.

💡AI Advertising

AI Advertising involves the use of AI to create or enhance advertising content. The video script mentions an AI advertising and filmmaking course, indicating the application of AI in advertising as a growing field. This keyword is significant as it reflects the broader application of AI beyond entertainment, into the realm of commercial content creation.

Highlights

A new Sora competitor, Vu, has emerged in the market, offering AI-generated videos up to 16 seconds long in 1080P.

Adobe's Video Giga GAN tool allows for upscaling low-resolution footage to high-resolution with impressive facial and hair detail.

Synthesia's Expressive AI Avatars can now respond emotionally and change their facial structure based on the text input.

Nvidia's research team has developed the ability to perform large-scale NeRF scans, allowing for real-time rendering of scanned areas up to 25 square kilometers.

Adobe and the University of California San Diego have partnered to showcase prompt-based image editing, enabling contextual changes to images through text descriptions.

Render's online application allows users to change clothing and other elements in images using AI prompts.

A team from China has developed a method to edit 3D models using prompts, allowing for simple modifications like opening a dinosaur's mouth or transforming objects.

The film 'To Dear Me' won the Beijing Film Festival 2024, showcasing the integration of live-action footage with AI style transfers.

The film 'Son of Life' demonstrates consistency between shots with a black and white color grade and film grains for a stylized look.

The concept film 'Cator' presents a humorous take on a cat gladiator with high-quality, realistic shots.

AEL art's scene 'The Dinner' cleverly uses AI and 3D tools to create dynamic camera movements in a five-character scene.

Synthesia raised $90 million on a $1 billion evaluation, indicating significant investment in their AI avatar technology.

An interview with the creators of the AI video used to demo Sora reveals the extensive post-processing required and the trial-and-error nature of content creation with AI.

An AI trailer competition has been launched in partnership with Submachine, offering the chance to win an Apple Vision Pro.

Hyper has released a new gallery showcasing impressive AI projects, with the ability for users to submit their own films.

Enrollment is open for an AI advertising and filmmaking course covering the latest trends and techniques in the field.

Curious Refuge is hosting meetups around the world, including at the Cannes Film Festival, to foster the AI filmmaking community.

Chat GBT now has the ability to store information from prompts as memory, assisting users in creating future prompts without repetition.

Create Stunning AI Art Instantly

No login. No cost. Just incredible AI images !

Create Stunning AI Art InstantlyGenerate Now