Kasucast #26 - FLUX.1 [dev]: Training LoRA with SimpleTuner and AI-Toolkit

#flux #simpletuner #bghira #ComfyUI #LoRA #finetune #stablediffusion #style #styletraining

This is an FLUX.1 [dev] LoRA training log for art style and concepts. I document my thought process, experiments, mistakes, and analysis of quantitative and qualitative results. Hopefully, this can be a good starting guide for people looking to train LoRAs for FLUX.1 [dev] for art style/concepts/subjects.

I trained two LoRAs, a fantasy art one and a cinematic photo one.

If you want to see how I caption my dataset, you can look at this stream I did: https://youtube.com/live/ifHoPn5oiDc?...

My repository with all training settings: https://github.com/kasukanra/SDXL_tra...

Practical tools:
AI-Toolkit: https://github.com/ostris/ai-toolkit
SimpleTuner: https://github.com/bghira/SimpleTuner
SimpleTuner FLUX quickstart: https://github.com/bghira/SimpleTuner...
SimpleTuner dataloader: https://github.com/bghira/SimpleTuner...
SimpleTuner LoRA rank/alpha: https://github.com/bghira/SimpleTuner...
SD3 ControlNet: https://huggingface.co/InstantX/SD3-C...

Theory/Discussion:
Github issue I opened on SimpleTuner regarding old LoRAs not working properly: https://github.com/bghira/SimpleTuner...

Art in datasets from artists:
Entei Ryu: https://www.artstation.com/badzr
kleinerHai: https://www.artstation.com/kleinerhai

I'm sorry if your artwork is in the dataset and I didn't mention your name. Let me know in the comments and I'll credit you. Sometimes, I get data from Pinterest and assume that it's AI generated.

Timestamps:
00:00 Intro
02:12 SimpleTuner rundown
03:24 Rank 128 LoRA overview
05:16 Unconditional/validation images
06:08 Rank 128 LoRA (female white blouse)
07:30 ComfyUI observation regarding LoRA
07:52 Rank 128 LoRA failed outputs (female white blouse)
08:24 Why not change LoRA strength?
08:48 What does alpha do?
10:10 Rank 128 LoRA (male armor)
10:52 Rank 128 LoRA (castle)
12:28 Rank 128 LoRA (robot)
13:33 Rank 128 LoRA failed outputs (female streetwear)
14:36 Rank 128 LoRA (female knight)
15:04 LoRA rank/alpha dimensions comparison
16:44 Comparing LoRAs (male armor)
17:33 Comparing LoRAs (castle)
18:10 Comparing LoRAs (dragon)
19:12 Comparing LoRAs (robot)
19:56 Comparing LoRAs (female streetwear)
20:20 Comparing LoRAs (female knight)
21:14 Flux guidance value theory
21:56 Training without flux guidance value
25:14 Training with flux guidance value
26:00 Flux guidance scale in relation to CFG
27:14 All+ffs overview
28:28 All vs. All+ffs (male armor)
29:35 All vs. All+ffs (castle)
30:08 All vs. All+ffs (dragon)
30:34 All vs. All+ffs (robot)
31:04 All vs. All+ffs (female knight)
31:38 Flux guidance variation all (male armor)
34:00 Flux guidance variation all+ffs (male)
35:05 Flux guidance variation all+ffs (space dragon)
37:42 Flux guidance variation (robot)
40:19 Flux guidance variation (female knight)
43:08 Cinematic Photo LoRA
47:56 LoKr
52:08 AI-Toolkit overview
53:26 AI-Toolkit overview (praxis)
56:21 Conclusion

Find the artworks in the video on Artstation here: https://www.artstation.com/artwork/2B...

📳My Training Settings and ComfyUI workflows: https://github.com/kasukanra/SDXL_tra...

📳Links to my models/LoRAs: https://pastebin.com/x5tMpe2h

💬 Discord channel:   / discord  
🎭 Patreon:   / kasukanra  
🎁 Tips/Donations: https://streamlabs.com/kasukanra/tip

🎉 Social Media:
📷 Instagram:   / kasukanra  
🎨 Artstation: https://www.artstation.com/kasukanra
🐦 Twitter:   / kasukanra  
🐀 Twitch:   / kasukanra  

📩 art enquiries: [email protected]

Images/processes may be fabricated and therefore not real. I am unaware of any illegal activities. Documentation will not be taken as admission of guilt.