Facebook AI's DINO | PyTorch Code Explained
π¨βπ©βπ§βπ¦ JOIN OUR DISCORD COMMUNITY: Discord βΊ https://discord.gg/peBrCpheKE π’ SUBSCRIBE TO MY MONTHLY AI NEWSLETTER: Substack βΊ https://aiepiphany.substack.com/ β€οΈ Become The AI Epiphany Patreon β€οΈ βΊ https://www.patreon.com/theaiepiphany With this video, I kick off a brand new series of coding videos in PyTorch where I'll be explaining some of the most impactful AI research through its code analysis! In this video I cover DINO from the "Emerging Properties in Self-Supervised Vision Transformers" paper. β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ β Paper: https://arxiv.org/abs/2104.14294 β Code: https://github.com/facebookresearch/dino β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ βοΈ Timetable: 00:00 DINO paper - short overview 02:38 Code analysis starts - training arguments (argparse) 07:30 Training main function 08:25 DINO augmentations 11:30 Main function resumed 16:05 DINO head (MLP) 18:05 Main function resumed 20:40 DINO loss overview 21:45 Main function resumed 24:05 Augmentations visualized (matplotlib) 27:00 Main function resumed 28:00 DINO Core part!!! (in-depth shape analysis) 33:45 DINO loss in depth 38:00 Main function resumed 39:50 Visualizing attention script explained 48:15 Outro β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ π° BECOME A PATREON OF THE AI EPIPHANY β€οΈ If these videos, GitHub projects, and blogs help you, consider helping me out by supporting me on Patreon! The AI Epiphany βΊ https://www.patreon.com/theaiepiphany One-time donation: https://www.paypal.com/paypalme/theaiepiphany Much love! β€οΈ Huge thank you to these AI Epiphany patreons: Eli Mahler Petar VeliΔkoviΔ Zvonimir Sabljic β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ π‘ The AI Epiphany is a channel dedicated to simplifying the field of AI using creative visualizations and in general, a stronger focus on geometrical and visual intuition, rather than the algebraic and numerical "intuition". β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ π CONNECT WITH ME ON SOCIAL LinkedIn βΊ https://www.linkedin.com/in/aleksagordic/ Twitter βΊ https://twitter.com/gordic_aleksa Instagram βΊ https://www.instagram.com/aiepiphany/ Facebook βΊ https://www.facebook.com/aiepiphany/ π¨βπ©βπ§βπ¦ JOIN OUR DISCORD COMMUNITY: Discord βΊ https://discord.gg/peBrCpheKE π’ SUBSCRIBE TO MY MONTHLY AI NEWSLETTER: Substack βΊ https://aiepiphany.substack.com/ π» FOLLOW ME ON GITHUB FOR ML PROJECTS: GitHub βΊ https://github.com/gordicaleksa π FOLLOW ME ON MEDIUM: Medium βΊ https://gordicaleksa.medium.com/ β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ #dino #pytorch #coding
![Tensor Logic "Unifies" AI Paradigms [Pedro Domingos]](https://img.youtube.com/vi/4APMGvicmxY/mqdefault.jpg)
![Transformers Need Glasses! [Federico Barbero]](https://img.youtube.com/vi/FAspMnu4Rt0/mqdefault.jpg)
