How to AI Upscale with ControlNet Tiles - High Resolution for Everyone!

Sebastian Kamph
4 May 202318:16

TLDRThe video tutorial demonstrates how to upscale low-resolution images to high-definition formats like 4K and 8K using a PC with a GPU that has at least 4GB of VRAM. The process leverages ControlNet Tiles and an extension called Ultimate Stable Fusion Upscale. The host guides viewers through installing the necessary extensions, setting up the image-to-image process with denoising strength adjustments, and using ControlNet for coherence across tiles. The video also explores the use of different models, such as the 4X Ultra Sharp model, and discusses the trade-offs between upscaling in multiple steps versus a single large step. The host shares their workflow, which includes testing various settings and comparing results, and concludes by suggesting that working in fewer, larger steps might help retain more image texture and detail. The video is an informative guide for users looking to enhance the resolution of their images without needing high-end hardware.

Takeaways

  • 🚀 **Upscaling Capability**: You can upscale low-resolution images to high resolutions like 4K and 8K using a GPU with at least 4GB of VRAM.
  • 🔍 **Software Requirement**: To perform the upscaling, you need to have ControlNet installed and the Ultimate Stable Fusion Upscale extension.
  • 📌 **Image Preparation**: Start with a low-resolution image (e.g., 512x512) and use ControlNet tiles along with the Ultimate Stable Fusion Upscale.
  • 🛠️ **Denoising Strength**: Adjust the denoising strength for the first and second pass; a higher value for the first pass retains more details.
  • 🧩 **Tiling Process**: The upscaling process involves generating multiple tiles that are then merged to form the final high-resolution image.
  • 🔄 **Seams Fixing**: Experiment with different settings to reduce visible tile seams, but the effectiveness may vary from image to image.
  • 📈 **Model Selection**: Use the Control 1.1 SD 1.5 tile model, which is crucial for maintaining the coherence of the upscaled image.
  • 🔍 **Detail Retention**: Higher denoising strength can lead to loss of detail; ControlNet helps retain the original image's spatial coherence.
  • ⏱️ **Rendering Time**: Upscaling to higher resolutions takes longer due to the increased number of tiles that need to be processed.
  • 🔗 **Workflow Preference**: Some users prefer a step-by-step upscaling process to immediately scaling to the final size for better control over the outcome.
  • 📚 **Testing and Research**: The presenter combined various tips, tricks, and personal testing to develop an effective upscaling workflow.

Q & A

  • What is the minimum GPU VRAM required to run stable Fusion for upscaling images?

    -The minimum GPU VRAM required to run stable Fusion for upscaling images is 4 gigabytes.

  • How can one install the 'ultimate stable Fusion upscale' extension?

    -To install the 'ultimate stable Fusion upscale' extension, go to your extensions, check available, press the load from search button, search for 'upscale', find 'ultimate as the upscale', and then install and apply the extension, followed by a restart of the UI.

  • What is the recommended denoising strength for the first pass of upscaling an image?

    -For the first pass of upscaling an image, a denoising strength between 0.1 and 0.4 is recommended.

  • What is the role of ControlNet in the upscaling process?

    -ControlNet helps to maintain the coherence and spatial consistency of the upscaled image, ensuring that the details and features are retained across the different tiles generated during the upscaling process.

  • What is the 'Control 1.1 SD 1.5 tile' model used for in the upscaling process?

    -The 'Control 1.1 SD 1.5 tile' model is used as a preprocessor in the upscaling process to ensure that the upscaled image retains its spatial consistency and coherence across tiles.

  • How does the 'ultimate SD upscale' script affect the upscaling process?

    -The 'ultimate SD upscale' script is used to change the target size type to scale from image size, allowing the image to be resized by a specified multiple while ignoring other settings.

  • What is the significance of using ControlNet in conjunction with the upscaling model?

    -Using ControlNet with the upscaling model ensures that the upscaled image retains the original image's coherence and spatial consistency, preventing the loss of detail and the creation of a disjointed final image.

  • What is the maximum resolution that can be achieved using the described upscaling process?

    -The maximum resolution that can be achieved using the described upscaling process is 8K, although the quality may degrade with each upscaling step.

  • How can one reduce the visible tiling seams in the upscaled image?

    -The video suggests experimenting with different settings such as band pass, padding, and half tile offset. However, the speaker found that using 'none' for seams fix provided the best results in their testing.

  • What is the recommended approach for upscaling an image to a higher resolution?

    -The recommended approach is to upscale in steps, doubling the resolution at each step, which helps to retain more texture and detail in the final image.

  • What are the potential downsides to upscaling an image directly to a very high resolution in one step?

    -Upscaling an image directly to a very high resolution in one step can result in a loss of detail, the introduction of 'fake detail', and a smoother, less realistic texture in the final image.

  • How does the upscaling process affect the detail and texture of the upscaled image?

    -The upscaling process can lead to a loss of detail and a smoother texture in the upscaled image, especially when scaling up to very high resolutions in fewer steps.

Outlines

00:00

🚀 Upscaling Low Resolution Images with Control Net

The first paragraph introduces the topic of upscaling low resolution images to high resolutions like 4K and 8K using a PC with a GPU that has at least 4GB of VRAM. The speaker recommends using Control Net and Ultimate Stable Fusion Upscale, and provides a step-by-step guide on how to install necessary extensions and set up the process. The workflow involves loading an image, applying denoising strength, using Control Net with specific settings, and scaling up the image in stages to achieve high resolution.

05:00

🔍 Examining the Effects of Tiling and Seams

The second paragraph discusses the challenges of tiling in upscaling, such as visible tile lines and seams. The speaker shares their testing experience with various settings to minimize these issues but found that none of the tested settings significantly improved the results over using no seams fix at all. The paragraph also explains the process of generating larger images by combining multiple tiles and the trade-offs between speed and quality when choosing the number of tiles.

10:04

📈 Incremental Upscaling and Detail Preservation

The third paragraph focuses on the process of incremental upscaling and the importance of detail preservation. It contrasts the original low-resolution image with upscaled versions, noting the loss of detail as the resolution increases. The speaker emphasizes the role of Control Net in maintaining coherence across tiles and preserving facial features despite high denoising strength. The paragraph ends with a comparison of direct upscaling versus step-by-step scaling, highlighting the trade-offs between the two methods.

15:04

🎨 Final Thoughts on High Resolution Image Generation

The final paragraph wraps up the discussion by demonstrating the generation of an 8K image from a 512x512 original. The speaker notes the increased rendering time and the appearance of 'fake detail' at extreme upscaling. They reflect on the potential of the workflow for generating high-resolution images on lower-end GPUs and invite viewers to share their preferred methods. The paragraph concludes with a reminder that the technology is evolving and encourages viewers to experiment and share their findings.

Mindmap

Keywords

💡AI Upscale

AI Upscale refers to the process of increasing the resolution of an image using artificial intelligence algorithms. In the context of the video, AI Upscale is used to transform low-resolution images into high-resolution ones, such as 4K or 8K, using ControlNet Tiles and Stable Fusion, which is significant for users with older hardware that would typically struggle with such tasks.

💡ControlNet Tiles

ControlNet Tiles is a feature or tool mentioned in the video that aids in the upscaling process by dividing the image into smaller parts, or 'tiles', and then reconstructing the image at a higher resolution. It is integral to achieving high-resolution outputs from lower-resolution inputs, especially for users with limited GPU capabilities.

💡GPU

A GPU, or Graphics Processing Unit, is a type of computer hardware that renders images, videos, and animations. The video emphasizes the requirement of having a GPU with at least 4 gigabytes of VRAM for running the upscaling process smoothly, indicating the importance of GPU in handling graphic-intensive tasks like image upscaling.

💡VRAM

VRAM, or Video Random Access Memory, is the memory used by the GPU to store image data. The script specifies a minimum of 4 gigabytes of VRAM as a prerequisite for running the upscaling software, highlighting the need for sufficient memory to handle high-resolution image processing.

💡Denoising Strength

Denoising Strength is a parameter in the upscaling process that controls the level of noise reduction applied to the image. The video discusses adjusting this parameter for different passes of the upscaling, where higher values may lead to loss of detail but can help in maintaining the coherence of the image when using ControlNet.

💡Preprocessor

In the context of the video, a preprocessor refers to a specific setting within the upscaling software that prepares the image for further processing. 'Tile resample' is mentioned as a type of preprocessor that is used to handle the tiling of the image before upscaling.

💡Ultimate Stable Fusion Upscale

Ultimate Stable Fusion Upscale is an extension or feature within the software that is used to upscale images. It is mentioned as a key component in the upscaling workflow, working in conjunction with ControlNet Tiles to achieve high-resolution results.

💡Image to Image

Image to Image is a mode within the software where users can input an image and apply various transformations or enhancements, including upscaling. The video script describes loading an image into this mode and using it as part of the upscaling process.

💡Seams Fix

Seams Fix refers to a set of adjustments made to address the visible lines or 'seams' that can appear between tiles in an upscaled image. The video discusses testing different settings for seams fix and concludes that, for the images tested, no significant improvement was found over using no seams fix at all.

💡Tile Lines

Tile lines are the visible lines that can appear in an upscaled image where the tiles meet. They are an undesirable effect of the tiling process and are a focus of the video as the presenter experiments with different settings to minimize their appearance.

💡4096 by 4096 pixels

4096 by 4096 pixels refers to the resolution of an image, with the number indicating the number of pixels in the horizontal and vertical dimensions. The video demonstrates upscaling an image to this high resolution as an example of the capabilities of the upscaling process described.

Highlights

You can upscale low resolution images to 4K, 8K, or even higher resolutions using a GPU with at least 4 gigabytes of VRAM.

ControlNet is used in conjunction with Ultimate Stable Fusion Upscale for high resolution image generation.

The process involves using ControlNet Tiles and a specific model, Control 1.1 SD 1.5 Tile, to maintain image coherence.

The denoising strength plays a crucial role in the first pass of upscaling, with values between 0.1 and 0.4 recommended.

Seams between tiles can be minimized, but not completely eliminated, even with various settings and techniques.

ControlNet ensures that the upscaled image retains the original image's spatial coherence.

The Ultimate SD Up scale model can be used for further upscaling, with a 4X Ultra Sharp model available for download.

Upscaling in steps rather than all at once allows for better control over the process and can result in better image quality.

An 8K image can be generated from a 512x512 image, but the process may introduce 'fake detail' and loss of texture.

ControlNet's role is to maintain the spatial relationships between different parts of the image during the upscaling process.

The tutorial provides a step-by-step guide on installing and using the necessary extensions and models for upscaling.

The presenter discusses the limitations of the tiling workflow and the potential for future improvements.

Different images may require different upscaling settings, and the presenter encourages experimentation to find the best workflow.

The presenter shares their personal workflow and invites viewers to share their preferred methods for upscaling.

Upscaling to 8K from a low-resolution image is possible but may result in a longer rendering time and potential loss of detail.

The presenter demonstrates the upscaling process from 512x512 to 8K resolution, highlighting the differences and challenges.

The video concludes with a comparison between the step-upscaling method and direct upscaling to 8K, showing the trade-offs in image quality and rendering time.