Fastformer: Additive Attention Can Be All You Need | Paper Explained
π¨βπ©βπ§βπ¦ JOIN OUR DISCORD COMMUNITY: Discord βΊ https://discord.gg/peBrCpheKE π’ SUBSCRIBE TO MY MONTHLY AI NEWSLETTER: Substack βΊ https://aiepiphany.substack.com/ β€οΈ Become The AI Epiphany Patreon β€οΈ βΊ https://www.patreon.com/theaiepiphany In this video I covered "Fastformer: Additive Attention Can Be All You Need" paper introducing a novel, linear complexity, transformer model! β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ β Paper: https://arxiv.org/abs/2108.09084 β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ βοΈ Timetable: 00:00 Intro 01:00 Previous work and problems 03:10 Fastformer method explained 07:10 Param sharing and complexity 09:10 Results, Fastformer is effective 11:20 Results, Fastformer is efficient 12:45 Ablations 14:10 Outro β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ π° BECOME A PATREON OF THE AI EPIPHANY β€οΈ If these videos, GitHub projects, and blogs help you, consider helping me out by supporting me on Patreon! The AI Epiphany βΊ https://www.patreon.com/theaiepiphany One-time donation: https://www.paypal.com/paypalme/theaiepiphany Much love! β€οΈ Huge thank you to these AI Epiphany patreons: Eli Mahler Petar VeliΔkoviΔ Zvonimir Sabljic β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ π‘ The AI Epiphany is a channel dedicated to simplifying the field of AI using creative visualizations and in general, a stronger focus on geometrical and visual intuition, rather than the algebraic and numerical "intuition". β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ π CONNECT WITH ME ON SOCIAL LinkedIn βΊ https://www.linkedin.com/in/aleksagordic/ Twitter βΊ https://twitter.com/gordic_aleksa Instagram βΊ https://www.instagram.com/aiepiphany/ Facebook βΊ https://www.facebook.com/aiepiphany/ π¨βπ©βπ§βπ¦ JOIN OUR DISCORD COMMUNITY: Discord βΊ https://discord.gg/peBrCpheKE π’ SUBSCRIBE TO MY MONTHLY AI NEWSLETTER: Substack βΊ https://aiepiphany.substack.com/ π» FOLLOW ME ON GITHUB FOR ML PROJECTS: GitHub βΊ https://github.com/gordicaleksa π FOLLOW ME ON MEDIUM: Medium βΊ https://gordicaleksa.medium.com/ β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ #fastformer #transformers #linearcomplexity
