InvokeAI - SDXL Getting Started

Invoke
29 Jul 202306:40

TLDRThe video introduces the release of version 3.0.1 with support for the SDXL model, emphasizing its integration into the linear UI for easy prompt input. It highlights the addition of positive and negative style prompt boxes, the importance of concise style prompts, and the recommendation to use a concatenated basic prompt. The video also addresses the need for a fixed VAE file compatible with FP16 precision to avoid image aberrations. It provides guidance on maintaining similar pixel sizes to the original SDXL training images and discusses the refiner model's role in adding details. The use of Euler schedulers is recommended, and the video ends with a teaser for upcoming SDXL fine tunes and resources for effective prompting.

Takeaways

  • ๐Ÿš€ Introduction of version 3.0.1 with integrated support for the SDXL model.
  • ๐ŸŽจ New UI enhancements for easier input of SDXL prompts and refiner usage.
  • ๐Ÿ”„ Addition of positive and negative style prompt boxes for better model interaction.
  • ๐Ÿ“ Importance of concise and focused style prompts for maintaining coherence.
  • โš ๏ธ Caution against overly detailed negative prompts as they may hinder model performance.
  • ๐Ÿ”ง Recommendation to use the 'concatenate basic prompt with style' button for optimal results.
  • ๐Ÿ” Guidance on using the correct VAE file for FP16 precision to avoid image aberrations.
  • ๐Ÿ“ Suggestion to maintain similar pixel dimensions as the SDXL model's training images.
  • โœจ Tips on using the refiner model for adding details while avoiding potential finickiness.
  • ๐Ÿ”„ Explanation of the refiner's step functions and their impact on image development.
  • ๐Ÿ› ๏ธ Discussion on the use of Euler schedulers and their success in the SDXL model.

Q & A

  • What is the main feature introduced in version 3.0.1?

    -The main feature introduced in version 3.0.1 is the support for the SDXL model, which allows users to input prompts and use the refiner directly from the UI.

  • What are the two additional prompt boxes introduced with the SDXL model?

    -The two additional prompt boxes introduced with the SDXL model are the positive style prompt and the negative style prompt.

  • How does the concatenate basic prompt with style button function?

    -The concatenate basic prompt with style button links the subject and style prompts when passing them to the second encoder, which helps in achieving the best results from the SDXL model.

  • What is the recommended approach for using the SDXL model if you're not familiar with it?

    -If you're not familiar with the SDXL model, it is recommended to use the concatenate basic prompt with style button to ensure the prompts are passed correctly into the text encoders.

  • Why is the style prompt particularly important in SDXL?

    -The style prompt is important in SDXL because having too much content in it can lead to a loss of coherence. It's also crucial to focus on not heavily prompting in the negative, as the model tends to dislike excessive negative prompts.

  • What issue was there with the original VAE file for FP16 precision in the SDXL model?

    -The original VAE file for FP16 precision in the SDXL model was broken, causing weird dramatic aberrations in the images, which some people thought was a watermark. A fixed version is recommended to resolve this issue and support FP16 runs properly.

  • What should be the approximate pixel size for the images when using SDXL?

    -The approximate pixel size for the images when using SDXL should be the same as the original images the model was trained on, such as 1024 by 1024, to ensure coherence in the generated images.

  • What is the refiner model and how does it function?

    -The refiner model is a tool that can add detailed improvements to an image. It functions by running a selected number of steps after the base model has created a structurally mostly complete image, focusing on the finishing touches.

  • What is the recommended range for the refiner start slider?

    -The recommended range for the refiner start slider is around 0.7 to 0.8, which tells the base model to end its processing when there's still some noise in the picture, allowing the refiner to work on the finishing touches.

  • What are Euler and Euler ancestral schedulers mentioned in the context of SDXL?

    -Euler and Euler ancestral schedulers are recommended for use with the SDXL model, as they have been found to be successful. Other schedulers like DPM plus plus may result in less desirable outcomes.

  • How can users get started with SDXL prompting?

    -Users can get started with SDXL prompting by checking out the SDXL prompt Styles reference created by a contributor on GitHub, or by joining the community on Discord for more insights and support.

  • What is the significance of the prompt reference taken from the clip drop website?

    -The prompt reference taken from the clip drop website represents the styles that were used there. By combining the subject prompt with the style, users can recreate a similar positive style for their own prompts.

Outlines

00:00

๐Ÿš€ Introduction to SDXL Model and UI Integration

This paragraph introduces the new 3.0.1 update which includes support for the SDXL model within the linear UI. It emphasizes the ease of use for inputting SDXL prompts and utilizing the refiner directly from the UI. The speaker highlights that SDXL represents a shift in prompting methodology and provides guidance on how to effectively use the UI with the SDXL model. The paragraph also discusses the implementation of additional prompt boxes for positive and negative style prompts and the importance of balancing these prompts for optimal results. It advises on the use of the concatenate basic prompt with style button for best outcomes and warns against overly detailed negative prompts. The speaker also addresses technical aspects such as the need for a fixed VAE file compatible with FP16 precision and the original SDXL base model, recommending a fixed version that resolves previous issues. The paragraph concludes with recommendations on starting image sizes for SDXL and an overview of the refiner model, its benefits, and the importance of adjusting settings for optimal performance.

05:03

๐ŸŒŸ Utilizing SDXL Fine Tunes and Community Resources

In this paragraph, the speaker discusses the anticipated release of several SDXL fine tunes and highlights the availability of the new dream shaper XL model. It provides instructions on how to convert the model using the safe tensors file and the model manager. The paragraph also directs users to a comprehensive SDXL prompt styles reference created by a community contributor on GitHub, which is a valuable resource for beginners. The speaker encourages users to experiment with the new features and provides tips on using specific schedulers, recommending Euler or Euler-ancestral for their effectiveness. The paragraph ends with a teaser for upcoming releases and a call to stay tuned for more updates, reinforcing the community aspect with a mention of Discord and a prompt trial demonstration.

Mindmap

Keywords

๐Ÿ’กsdxl

sdxl stands for Style Diffusion XL, a model used in AI for image generation. It represents a shift in how users interact with AI, focusing on style and subject prompts to create images. In the video, the discussion revolves around the introduction of sdxl support in a 3.0.1 update, highlighting its impact on the UI and the creative process.

๐Ÿ’กUI

UI refers to User Interface, the medium through which users interact with the software. In the context of the video, the UI has been updated to include features that accommodate the sdxl model, making it easier for users to input their prompts and utilize the refiner directly.

๐Ÿ’กpositive style prompt

A positive style prompt is a specific type of input used with the sdxl model to guide the AI towards a desired style or aesthetic. It is one of the two additional prompt boxes introduced in the UI update, allowing users to define what they want to see in the generated image.

๐Ÿ’กnegative style prompt

The negative style prompt is used to specify what the user does not want to see in the generated image. It works in conjunction with the positive style prompt, helping the AI understand what elements to avoid, thus refining the final output.

๐Ÿ’กfp16 Precision

fp16 Precision refers to a method of floating-point arithmetic using 16 bits to represent numbers, which is beneficial for AI models in terms of speed and memory usage. The video mentions the need to download a fixed VAE file that supports fp16 runs for the sdxl model, addressing previous issues with image generation.

๐Ÿ’กVAE

VAE stands for Variational Autoencoder, a type of neural network used for efficient learning and generation of new data. In the video, a fixed VAE file is recommended for use with the sdxl model to improve image quality and resolve previous issues.

๐Ÿ’กrefiner

The refiner is a feature that allows users to add details and refine the output of the AI-generated image. It is toggle-able in the UI and can enhance the image, but may require some trial and error to achieve the best results.

๐Ÿ’กEuler schedulers

Euler schedulers are a type of learning rate scheduler used in training neural networks. They are mentioned in the video as a recommendation for use with the sdxl model, as they have been found to produce better results compared to other schedulers.

๐Ÿ’กSD Excel prompting

SD Excel prompting refers to the technique of using specific styles and subjects to guide the AI in generating images. The video encourages users to check out a reference guide for SD Excel prompting to get started easily with the new features.

๐Ÿ’กfine tunes

Fine tunes refer to the process of making minor adjustments to a neural network model to improve its performance on a specific task. In the video, the anticipation of new sdxl fine tunes becoming available is mentioned, indicating upcoming improvements and releases.

๐Ÿ’กimage to image function

The image to image function is a feature that transforms one image into another, often used in AI for tasks like denoising or style transfer. In the context of the video, it is related to the refiner's ability to take a partially completed image and enhance it with additional details.

Highlights

3.0.1 release introduces support for the sdxl model.

The linear UI has been updated to facilitate easy input of sdxl prompts.

Using the sdxl model requires a shift in the way you prompt.

With sdxl, there are now two additional prompt boxes: positive style prompt and negative style prompt.

For best results, use the 'concatenate basic prompt with style' button to link subject and style prompts.

Be cautious with the content in the negative style prompt to maintain coherence.

When using fp16 Precision, download a fixed vae to avoid issues with the original sdxl base 1.0 model.

The fixed vae version supports fp16 and removes the weird scan line look from images.

For sdxl, start with an image size of 1024 by 1024 or similar pixel count to the original images it was trained on.

The refiner model can be toggled on and off; it adds details but may require fine-tuning.

The number of steps in the UI determines the number of steps run using the refiner.

Euler and Euler ancestral schedulers are recommended for use with sdxl.

DPM plus plus schedulers may produce unusual results in sdxl.

Anticipate more sdxl fine tunes becoming available in the near future.

The new dream shaper XL model is one of the first sdxl fine tunes.

For easy SD Excel prompting, refer to the sdxl prompt Styles reference on GitHub.

The prompt reference includes styles used on the clip drop website.

Combining subject and style prompts has been simplified with the updated UI.