Lora Training using only ComfyUI!!
TLDRMarcus introduces a new method for training Lora models exclusively within ComfyUI, eliminating the need for external platforms like Kaggle or Google Collab. By creating a dataset of images and generating text captions, users can train Lora with a single node. The tutorial covers the setup process, including installing the required Scorch CU 121 version and using the LJR Laura node for training. The result is a fully trained Lora model, directly within the ComfyUI environment.
Takeaways
- ๐ ComfyUI now supports full training of Lora models without the need for external platforms like Kaggle or Google Collab.
- ๐ To begin, create a dataset of images and place them in a folder named 'dataset' followed by the name of your Lora model.
- ๐ผ๏ธ The images in the dataset should be in PNG format and can vary in size.
- ๐ The GitHub link for the Lora training node is provided in the video description for download.
- ๐ Generate text captions for each image in the dataset using the Lora caption node and the W14 tagger.
- ๐ Ensure that Comfort UI is updated to the required Scorch CU 121 version for the training process.
- ๐ Connect the image list to the Lora caption node and set the path to the dataset folder for proper training.
- ๐ The training node will save a Lora model after every 10 images (an epoch), gradually building up the model.
- ๐ฏ The final Lora model will be saved in the 'models/Lora' directory with a name corresponding to the folder name.
- โฑ๏ธ The training process is quick, taking only a few minutes as demonstrated in the video.
- ๐ก The video provides a simple and straightforward workflow for training Lora models within ComfyUI, emphasizing the ease of use and efficiency.
Q & A
What is the main purpose of the video created by Marcus?
-The video is created to demonstrate how users can train Lora models exclusively within ComfyUI, eliminating the need for external platforms like Google Colab or Kaggle.
What is required to start training Loras in ComfyUI according to the video?
-To begin training Loras in ComfyUI, you need a dataset of images. Marcus mentions using at least 25 to 50 images for this purpose, although in his demonstration he uses 24.
What specific version of ComfyUI is needed for training Loras as mentioned in the video?
-A specific version of ComfyUI, Scorch CU 121, is required for training Loras, as using other versions might lead to errors or compatibility issues.
What does the node 'LJR Lora' do in the process?
-The 'LJR Lora' node in ComfyUI seems to be a script or module designed to facilitate the training of Lora models by managing the image and text data during training sessions.
How are text captions created for images in the training dataset?
-Text captions are generated by linking an image list to a tagger node in ComfyUI, which analyzes each image and creates a text file describing the contents, aiding the training process.
What is the role of the 'magic node' as mentioned in the video?
-The 'magic node' is likely a central or key component in ComfyUI that handles the training parameters and processes for Lora models, allowing users to specify details like checkpoint names, paths, and training settings.
What is the purpose of saving 'every an eox' during the training process?
-The option to save 'every an eox' during training allows the user to save intermediate versions of the Lora model after certain numbers of epochs, which helps in tracking progress and ensures that not all progress is lost in case of interruptions.
Can you describe the final steps in the training process as outlined in the video?
-After configuring all settings in the 'magic node', the user simply initiates the training by executing the prompt command. The system then processes the data, saves the Lora models at specified intervals, and completes the training.
What is the significance of the folder naming convention for the dataset in ComfyUI?
-The folder naming convention, such as 'folder five umore s', is crucial as it aligns with the hierarchical structure that the training node in ComfyUI expects to find the data, ensuring the data is correctly accessed and processed.
What does Marcus plan to do with the trained Lora models at the end of the video?
-Marcus mentions displaying a few images trained by the Lora model at the end of the video, showcasing the results of the training session.
Outlines
๐ Introducing a New AI Training Method
In this paragraph, Marcus introduces viewers to a new method of training AI models exclusively within the UI, eliminating the need for external platforms like Kaggle or Google Colab. He emphasizes the simplicity of this process, which involves a single node and a GitHub link for the 'allora trading' node by Larry Jane mine. The process begins with creating a dataset of images, which are then organized into a specific folder structure. Marcus explains the importance of naming conventions for the folders and the need for a specific version of Scorch CU 121 for the training to work correctly. He also mentions the creation of text captions for the images, which are essential for training the AI to understand the content of the images.
๐ Setting Up the Training Environment
Marcus continues by detailing the setup process for training AI models. He explains the use of the 'Lura caption load' node and the 'WG tagger' for processing the image list and generating text descriptions. The paragraph outlines the steps for configuring the nodes, including plugging in the image list, text path, and other necessary parameters. The goal is to create text files that describe each image in the dataset, which will guide the AI during the training process. Marcus also discusses the importance of having a separate, dedicated installation of Comfort UI for training purposes.
๐จ Training a Sketch Style AI Model
In this section, Marcus demonstrates the training of a sketch style AI model using the previously prepared dataset and setup. He introduces the 'magic node,' which is central to the training process in the UI. The paragraph covers the various options and parameters available within the node, such as checkpoint name, image path, batch size, and epochs. Marcus explains the process of saving intermediate models and the final model, which is stored in the 'models Lura' directory. He also shares his personal preference for using the name of the model as a trigger word and provides examples of training different sketches without the need for external resources.
๐ฅ Wrapping Up the AI Training Tutorial
Marcus concludes the tutorial by summarizing the workflow and showcasing the final output. He emphasizes the ease and efficiency of training AI models within the UI, reiterating that the entire process was completed without using any external platforms. The paragraph ends with Marcus promising to include links in the video description for further resources and expressing his enthusiasm for creating more content like this. He signs off, teasing the next AI fuzz video and leaving viewers with a sense of anticipation for future tutorials.
Mindmap
Keywords
๐กComfyUI
๐กLora
๐กDataset
๐กText Captions
๐กGitHub
๐กTraining
๐กCheckpoint
๐กEpochs
๐กPrompt
๐กModels
๐กAI Fuzz
Highlights
ComfyUI now supports full training of Lora models without needing external platforms like Kaggle or Google Collab.
Training begins by creating a dataset of images, which are the core of the training data for the AI.
The images should be in different types of sketches and in PNG format, with no requirement for uniform size.
The dataset folder must be named following a specific structure to work with the training node.
Text captions for each image in the dataset are crucial and are created using a separate node.
A specific Scorch CU 121 version is required for the training process to function correctly.
The Lora caption load node and the W14 tagger are used in conjunction to process the images and text.
The training process involves analyzing each image and correlating it with the corresponding text file.
Default settings on the W14 tagger are sufficient for creating text descriptions of the images.
The magic node, ljr e, is responsible for the actual training of the Lora model within ComfyUI.
Configurable options within the training node include checkpoint name, image set path, batch size, and save settings.
The training process saves a Lora model after every epoch, with the final model named after the dataset folder.
The entire training workflow is contained within ComfyUI, utilizing no external resources.
The prompt system in ComfyUI is used to initiate the training process with a simple command.
The training duration is relatively short, as demonstrated by the 23 minutes and 20 seconds it took to train a model.
The video provides a step-by-step guide on how to train a Lora model using only ComfyUI, which is beneficial for users looking for an all-in-one solution.
