Gaussian Splatting | Luma AI plugin for UE5

miwa maroon
14 Nov 202312:36

TLDRIn this tutorial, the host introduces Gaussian Splatting, a technique for Unreal Engine 5 powered by Luma AI that significantly improves the quality of 3D scans. The process is outlined in three steps: scanning with Luma AI, importing the scan into Unreal Engine 5, and setting up the project to work with Luma AI files. The host also discusses the benefits of using Gaussian Splatting over traditional photogrammetry, such as reduced noise in the scans. Additionally, the tutorial provides insights into using sample scenes for practice and the differences between Luma interactive scenes and Luma field files. The summary concludes with a teaser for the next video, which will cover dynamic lighting.

Takeaways

  • 🤖 **Gaussian Splatting in UE5**: A technique introduced by Luma AI that revolutionizes the way high-quality scans are implemented in Unreal Engine 5.
  • 🚀 **Easy to Use**: Contrary to initial impressions, Gaussian Splatting is not as complex as it seems and can be easily integrated into UE5.
  • 🎨 **Photogrammetry Alternative**: Gaussian Splatting offers a superior alternative to traditional photogrammetry, reducing noise and enhancing image quality.
  • 📸 **Scanning Process**: Utilizes Luma AI for scanning, which includes moving slowly and capturing from as many unique angles as possible for better results.
  • ☕ **Time-Consuming Upload**: After scanning, uploading the master piece to Luma AI takes about 20-30 minutes, a good time to grab a coffee.
  • 📂 **Sample Scenes**: Luma AI provides sample scenes for practice, which can be downloaded and explored without needing to scan physical objects.
  • 🔍 **Different File Types**: There are two main types of Gaussian Splatting files for UE5: Luma Interactive Scene and Luma Field, each with its own benefits.
  • 📁 **File Size and Performance**: The .py file format is lighter and easier on the system, allowing for smoother performance and easier manipulation.
  • 💻 **Project Setup**: For working with Luma AI files, it's recommended to create a C++ project in UE5 to avoid potential errors during packaging.
  • 🔄 **Dynamic vs. Baked**: Gaussian Splatting files can be either baked (static lighting) or dynamic (affected by lighting changes), offering flexibility in scene design.
  • 🌟 **TAA Feature**: The TAA (Temporal Anti-Aliasing) option is available for a lighter system load, improving performance without sacrificing visual quality.

Q & A

  • What is the topic of the tutorial?

    -The tutorial is about Gaussian Splatting in Unreal Engine 5, a feature provided by Luma AI.

  • Why did the author find Gaussian Splatting interesting?

    -The author found Gaussian Splatting interesting because it offers a high-quality alternative to traditional photogrammetry, which can be noisy.

  • What is the first step in using Gaussian Splatting?

    -The first step is scanning, which can be done using Luma AI and their U fling feature.

  • How should one move while scanning with Luma AI?

    -One should move slowly and capture from as many unique angles as possible for a better scan.

  • What does the author suggest doing while waiting for the scan to upload?

    -The author suggests having a coffee as the upload process takes about 20 to 30 minutes.

  • What are sample scenes and how can they be accessed?

    -Sample scenes are pre-capture environments provided by Luma AI for practice. They can be accessed and downloaded from the provided notion link.

  • What are the two types of Gaussian Splatting files for Unreal Engine mentioned in the script?

    -The two types are Luma Interactive Scene and Luma Field files.

  • Why is the .py file recommended over the .pei file?

    -The .py file is recommended because it is lighter on the system and easier to work with, despite offering similar functionalities.

  • What is the recommended method for creating a project for Luma AI Gaussian Splatting in Unreal Engine 5?

    -The recommended method is to select C++ when creating the project to avoid errors when packaging the project later.

  • How does the author suggest importing the scan into Unreal Engine 5?

    -The author suggests logging into Luma AI, going to 'My Captures', and downloading the Gaussian Splat .py file optimized for UE5.

  • What is the difference between 'Baked' and 'Dynamic' in the context of Gaussian Splatting?

    -Baked splats are not affected by lighting changes as they are pre-baked, while Dynamic splats do respond to lighting changes.

  • What does TAA stand for and what is its role in the context of Gaussian Splatting?

    -TAA stands for Temporal Anti-Aliasing. It is used to make the file lighter and easier to work with, which can help with performance issues.

Outlines

00:00

🎥 Introduction to Gaussian Splatting in Unreal Engine 5

The video script introduces the concept of Gaussian Splatting in Unreal Engine 5, a technique that significantly improves the quality of 3D scans. The speaker, Maron, expresses excitement about the topic and mentions that it was initially perceived as complex due to a Unity post. However, Maron clarifies that it's not as complicated as it seems and is quite easy to implement. The script outlines the process of scanning and importing into Unreal Engine 5, highlighting the use of Luma AI for scanning, which offers a user-friendly interface and sample scenes for practice. Maron also discusses the process of capturing scenes or objects by moving slowly and capturing from various angles. After the capture, uploading the work to Luma AI is required, which takes about 20 to 30 minutes. During this time, Maron suggests exploring sample scenes available on the Luma AI website.

05:02

📂 Importing and Tweaking Gaussian Splatting Files

The second paragraph focuses on the importing process of Gaussian Splatting files into Unreal Engine 5. It details the steps to import a Luma file, which includes creating a new level and setting up the dynamic two RP. The script explains that the .py files allow for scene adjustments such as cropping, but cautions that these files can be heavy. The speaker recommends using the optimized .py files for Unreal Engine 5, which are lighter and easier on the system. The paragraph also touches on the use of Dural Bridge physics and Capital Playground for those who do not wish to capture their own scenes. Finally, the speaker guides viewers on how to download and import their scans from Luma AI and how to create a project specifically for Luma AI Gaussian Splatting.

10:05

🖥️ Working with Luma AI Plugins and Dynamic Lighting

The final paragraph delves into the technical aspects of working with Luma AI plugins in Unreal Engine 5. It instructs on how to enable the Luma AI plugin and create a new level for the project. The speaker then discusses the process of importing scanned files, explaining the difference between a baked and dynamic Gaussian Splatting file. The baked version does not change with lighting, while the dynamic version does. The paragraph also mentions the TAA (Temporal Anti-Aliasing) option, which is lighter on the system and can be used for better performance. The speaker wraps up by teasing the next video, which will cover dynamic lighting for Gaussian Splatting and share a pre-scanned file, allowing viewers to skip the scanning process and directly use Gaussian Splatting.

Mindmap

Keywords

💡Gaussian Splatting

Gaussian Splatting is a technique used in 3D graphics and game development for creating high-quality, detailed textures and surfaces. In the context of the video, it is presented as a groundbreaking method that has revolutionized the way textures are applied, particularly in comparison to traditional photogrammetry. The host mentions that Gaussian Splatting has 'changed the game' by reducing noise in scans and offering a more efficient and effective alternative.

💡Luma AI

Luma AI is a software tool mentioned in the video that facilitates the process of scanning and creating 3D models or textures. It is highlighted for its user-friendly interface and the ability to integrate with Unreal Engine 5 through a plugin. The video emphasizes Luma AI's role in making Gaussian Splatting more accessible for users.

💡Unreal Engine 5

Unreal Engine 5 (UE5) is a game engine developed by Epic Games, used for creating high-end video games and other interactive experiences. The video discusses how Gaussian Splatting can be implemented within UE5 to enhance visual effects. UE5 is chosen for its compatibility with the Luma AI plugin and its support for advanced rendering techniques.

💡Photogrammetry

Photogrammetry is a technique that captures photographs of an object or scene from different angles, which are then used to create 3D models or detailed textures. In the video, it is mentioned that the host had tried photogrammetry using Nerf, but encountered issues with noise in the scans. Gaussian Splatting is presented as a solution to these issues.

💡Scanning

Scanning, in this video, refers to the process of capturing detailed images of a real-world object or environment using Luma AI. The host describes the scanning process, emphasizing the importance of moving slowly and capturing from various unique angles to ensure a high-quality result. Scanning is a crucial step before implementing Gaussian Splatting.

💡Importing

Importing, within the context of the video, involves bringing the scanned data or downloaded sample scenes into Unreal Engine 5. The process is described as straightforward, with the host demonstrating how to import Luma AI files and create a project that utilizes Gaussian Splatting.

💡Sample Scenes

Sample scenes are pre-made environments or objects provided by Luma AI for practice purposes. The video suggests downloading these scenes to familiarize oneself with the scanning and importing process without needing to scan a physical object. They serve as a way for users to learn and experiment with Gaussian Splatting.

💡Luma File

A Luma file, as mentioned in the video, is a specific file format associated with Luma AI that contains the data required for Gaussian Splatting. The host discusses importing a Luma file into Unreal Engine 5 and how it allows for adjustments such as exposure and cropping of the scanned scenes.

💡Dynamic vs. Baked Splats

The video differentiates between dynamic and baked splats within the context of Gaussian Splatting. Baked splats are immutable to lighting changes, meaning they remain consistent regardless of lighting conditions. Dynamic splats, on the other hand, react to lighting changes in real-time, providing a more flexible and realistic outcome.

💡TAA (Temporal Anti-Aliasing)

TAA is a technique used in video games to smooth out jagged edges in graphics by using information from the previous frame. The video discusses the option of using TAA in Gaussian Splatting files, which can result in a lighter load on the system and improved performance.

💡C++

C++ is a programming language that the host recommends using when creating a project for Luma AI Gaussian Splatting in Unreal Engine 5. It is suggested to avoid errors when packaging the project later on. C++ is often used for developing complex game features and systems.

Highlights

Gaussian Splatting is a mind-blowing technique introduced by Luma AI for Unreal Engine 5.

The technique was initially perceived as complex to use in Unreal Engine 5 but is actually super easy.

Gaussian Splatting has revolutionized the field of photogrammetry by reducing noise in scans.

The tutorial provides a step-by-step guide on scanning and importing into Unreal Engine 5.

Luma AI offers a scanning tool with a user-friendly interface and sample scenes for practice.

Scanning tips include moving slowly and capturing from as many unique angles as possible.

The scanning process takes about 20 to 30 minutes to complete.

Sample scenes are available for download to practice without scanning actual objects.

Luma AI provides two types of Gaussian Splatting files: Luma Interactive Scene and Luma Field.

The .py file format is optimized for Unreal Engine 5 and recommended for its efficiency.

Importing the scan involves logging into Luma AI, downloading the scan, and following simple steps.

Creating a project for Luma AI Gaussian Splatting in Unreal Engine 5 involves selecting C++ to avoid errors.

The Luma AI plugin is essential for working with their files and needs to be installed and enabled.

Different options are available for handling lighting in the scene, such as baked splats and dynamic splats.

The tutorial demonstrates how to adjust exposure and crop scenes using the Luma AI interface.

The tutorial also covers the use of TAA (Temporal Anti-Aliasing) for smoother visuals and performance.

The next video will focus on dynamic lighting for bridges and sharing the project files.

Gaussian Splatting allows for high-quality scanning without the need for physical scanning by the user.

Create Stunning AI Art Instantly

No login. No cost. Just incredible AI images !

Create Stunning AI Art InstantlyGenerate Now