DeepMind DetCon: Efficient Visual Pretraining with Contrastive Detection | Paper Explained
π’ SUBSCRIBE TO MY MONTHLY AI NEWSLETTER: Substack βΊ https://aiepiphany.substack.com/ π¨βπ©βπ§βπ¦ JOIN OUR DISCORD COMMUNITY: Discord βΊ https://discord.gg/peBrCpheKE β€οΈ Become The AI Epiphany Patreon β€οΈ βΊ https://www.patreon.com/theaiepiphany In this video, I cover DetCon: Efficient Visual Pretraining with Contrastive Detection - a novel self-supervised method that achieves SOTA results on various transfer learning tasks. The main idea is to add the semantic segmentation information into the contrastive objective (they used various heuristics to obtain semantic segmentation information in an unsupervised fashion). β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ β Paper: https://arxiv.org/abs/2103.10957 β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ βοΈ Timetable: 00:00 News: Discord and AI newsletter! 01:24 Self-supervised learning, BYOL, and SimCLR 03:50 DetCon method overview 11:35 Semantic segmentation heuristics 12:45 Overview of BYOL and SimCLR 20:15 Results 24:10 Impact of segmentation heuristics 27:20 Outro 28:10 Turn on the notification bell, much love! β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ π° BECOME A PATREON OF THE AI EPIPHANY β€οΈ If these videos, GitHub projects, and blogs help you, consider helping me out by supporting me on Patreon! The AI Epiphany βΊ https://www.patreon.com/theaiepiphany One-time donation: https://www.paypal.com/paypalme/theaiepiphany Much love! β€οΈ TODO: Huge thank you to these AI Epiphany patreons: Eli Mahler Petar VeliΔkoviΔ Zvonimir Sabljic β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ π‘ The AI Epiphany is a channel dedicated to simplifying the field of AI using creative visualizations and in general, a stronger focus on geometrical and visual intuition, rather than the algebraic and numerical "intuition". β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ π CONNECT WITH ME ON SOCIAL LinkedIn βΊ https://www.linkedin.com/in/aleksagordic/ Twitter βΊ https://twitter.com/gordic_aleksa Instagram βΊ https://www.instagram.com/aiepiphany/ Facebook βΊ https://www.facebook.com/aiepiphany/ π¨βπ©βπ§βπ¦ JOIN OUR DISCORD COMMUNITY: Discord βΊ https://discord.gg/peBrCpheKE π’ SUBSCRIBE TO MY MONTHLY AI NEWSLETTER: Substack βΊ https://aiepiphany.substack.com/ π» FOLLOW ME ON GITHUB FOR ML PROJECTS: GitHub βΊ https://github.com/gordicaleksa π FOLLOW ME ON MEDIUM: Medium βΊ https://gordicaleksa.medium.com/ β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ #detcon #selfsupervised #contrastive
