Stable Diffusion: High-Resolution Image Synthesis with Latent Diffusion Models | ML Coding Series
❤️ Become The AI Epiphany Patreon ❤️ https://www.patreon.com/theaiepiphany 👨👩👧👦 Join our Discord community 👨👩👧👦 https://discord.gg/peBrCpheKE If you want to understand how stable diffusion exactly works behind the scenes this video is for you. I do a deep dive into the code behind Stable Diffusion explaining: 1. First stage autoencoder training (autoencoder with KL regularization) 2. Latent Diffusion Model training (UNet + conditioning model) 3. Sampling using PLMS scheduler Stable diffusion directly builds upon the "High-Resolution Image Synthesis with Latent Diffusion Models" paper so I do a deep dive into the code behind this paper. Let me know how you like this one - feedback is welcome as always! ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ✅ Stable diffusion repo: https://github.com/CompVis/stable-diffusion ✅ LDM repo: https://github.com/CompVis/latent-diffusion ✅ LDM paper: https://arxiv.org/abs/2112.10752 ✅ VQ-GAN (taming transformers) paper: https://arxiv.org/abs/2012.09841 ✅ PLMS paper: https://arxiv.org/abs/2202.09778 ✅ Imagenette dataset: https://github.com/fastai/imagenette ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ⌚️ Timetable: 00:00:00 Intro: why is Stable Diffusion important 00:03:50 Background knowledge: VQ-GAN, LDM, PLMS papers 00:09:20 Setup for a minimal code walk-through 00:13:30 Autoencoder with KL regularization training 00:17:15 LPIPS (perceptual loss) with discriminator loss 00:21:30 Loading ImageNet data and PyTorch Lightning training loop 00:26:35 Forward pass through the autoencoder 00:30:12 Loss calculation 00:32:08 Perceptual loss 00:36:30 KL and GAN generator loss 00:40:55 Discriminator loss 00:42:45 Summarizing the autoencoder training 00:45:44 LDM training 00:57:00 Encoding the image into the latent space 01:00:12 Forward pass through the LDM 01:01:22 LDM loss 01:04:08 Integrating conditioning via cross attention 01:10:34 Sampling using PLMS 01:16:02 CLIP 01:19:20 Classifier free guidance 01:22:19 Sampling code 01:26:42 Diffusion connection to differential equations (PLMS paper) 01:37:00 Quick glimpse into the safety check function 01:39:20 Outro ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 💰 BECOME A PATREON OF THE AI EPIPHANY ❤️ If these videos, GitHub projects, and blogs help you, consider helping me out by supporting me on Patreon! The AI Epiphany - https://www.patreon.com/theaiepiphany One-time donation - https://www.paypal.com/paypalme/theaiepiphany Huge thank you to these AI Epiphany patreons: Eli Mahler Petar Veličković ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 📄 Website - https://gordicaleksa.com/ 💼 LinkedIn - https://www.linkedin.com/in/aleksagordic/ 🐦 Twitter - https://twitter.com/gordic_aleksa 👨👩👧👦 Discord - https://discord.gg/peBrCpheKE 📺 YouTube - https://www.youtube.com/c/TheAIEpiphany/ 📚 Medium - https://gordicaleksa.medium.com/ 💻 GitHub - https://github.com/gordicaleksa 📢 AI Newsletter - https://aiepiphany.substack.com/ ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ #stablediffusion #latentdiffusion #imagesynthesis
