ResNet Strikes Back! | Patches Are All You Need? | Papers Explained

❤️ Become The AI Epiphany Patreon ❤️ ► https://www.patreon.com/theaiepiphany 👨‍👩‍👧‍👦 JOIN OUR DISCORD COMMUNITY: Discord ► https://discord.gg/peBrCpheKE 📢 SUBSCRIBE TO MY MONTHLY AI NEWSLETTER: Substack ► https://aiepiphany.substack.com/ In this video I cover 2 recent papers: * ResNet strikes back: An improved training procedure in timm * Patches Are All You Need? The ResNet paper proposes a novel training procedure and boosts ResNet-50 top-1 accuracy to over 80%! The "Patches Are All You Need" paper shows that the template (using patch embeddings + isotropic model) may be playing a more important role than the transformer block itself. ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ✅ ResNet paper: https://arxiv.org/abs/2110.00476 ✅ All You Need paper: https://openreview.net/forum?id=TVHS5Y4dNvM ✅ timm lib: https://github.com/rwightman/pytorch-image-models/ ✅ My transformer implementation: https://github.com/gordicaleksa/pytorch-original-transformer ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ⌚️ Timetable: 00:00 Intro 00:50 Paper 1: high-level overview 02:50 What caused the accuracy boost? 05:45 Data augmentations explained 07:05 Regularization explained 08:55 Optimizers explained 09:50 Experiments 12:50 Inherent result variance 13:55 Training procedure generalization to other datasets 15:10 Making wrong conclusions is easy 16:30 Paper 2: high-level overview 17:35 Architecture explained 22:25 A note on paper length 23:05 My thoughts 24:05 Single tweet implementation ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 💰 BECOME A PATREON OF THE AI EPIPHANY ❤️ If these videos, GitHub projects, and blogs help you, consider helping me out by supporting me on Patreon! The AI Epiphany ► https://www.patreon.com/theaiepiphany One-time donation: https://www.paypal.com/paypalme/theaiepiphany Much love! ❤️ Huge thank you to these AI Epiphany patreons: Eli Mahler Petar Veličković Bartłomiej Danek Zvonimir Sabljic ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 💡 The AI Epiphany is a channel dedicated to simplifying the field of AI using creative visualizations and in general, a stronger focus on geometrical and visual intuition, rather than the algebraic and numerical "intuition". ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 👋 CONNECT WITH ME ON SOCIAL LinkedIn ► https://www.linkedin.com/in/aleksagordic/ Twitter ► https://twitter.com/gordic_aleksa Instagram ► https://www.instagram.com/aiepiphany/ Facebook ► https://www.facebook.com/aiepiphany/ 👨‍👩‍👧‍👦 JOIN OUR DISCORD COMMUNITY: Discord ► https://discord.gg/peBrCpheKE 📢 SUBSCRIBE TO MY MONTHLY AI NEWSLETTER: Substack ► https://aiepiphany.substack.com/ 💻 FOLLOW ME ON GITHUB FOR ML PROJECTS: GitHub ► https://github.com/gordicaleksa 📚 FOLLOW ME ON MEDIUM: Medium ► https://gordicaleksa.medium.com/ ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ #resnet #patches #allyouneed