Non-Parametric Transformers | Paper explained
❤️ Become The AI Epiphany Patreon ❤️ ► https://www.patreon.com/theaiepiphany ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ In this video I do a deep dive of the "Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning" paper. ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ✅ Non-Parametric Transformers paper: https://arxiv.org/abs/2106.02584 ✅ Jay Alammar's BERT blog: http://jalammar.github.io/illustrated-bert/ ✅ My LinkedIn post (Judea Pearl): https://www.linkedin.com/posts/aleksagordic_pearl-causality-intelligence-activity-6807985607432785920-Zxn2 (also check out my other posts I made related to this) ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ⌚️ Timetable: 00:00 Key ideas of the paper 01:40 Abstract 02:55 Note on k-NN (non-parametric machine learning) 04:30 Data and NPT setup explained 06:15 NPT loss is inspired by BERT 08:20 A high-level architecture overview 11:30 NPT jointly learns imputation and prediction 12:50 Architecture deep dive (input embeddings, etc) 20:45 More details on the stochastic masking loss 23:30 Connections to Graph Neural Networks and CNNs 29:45 NPT achieves great results on tabular data benchmarks 34:25 NPT learns the underlying relational, causal mechanisms 39:40 NPT does rely on other datapoints 42:10 NPT attends to similar vectors 45:15 Conclusions ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 💰 BECOME A PATREON OF THE AI EPIPHANY ❤️ If these videos, GitHub projects, and blogs help you, consider helping me out by supporting me on Patreon! The AI Epiphany ► https://www.patreon.com/theaiepiphany One-time donation: https://www.paypal.com/paypalme/theaiepiphany Much love! ❤️ Huge thank you to these AI Epiphany patreons: Petar Veličković Zvonimir Sabljic ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 💡 The AI Epiphany is a channel dedicated to simplifying the field of AI using creative visualizations and in general, a stronger focus on geometrical and visual intuition, rather than the algebraic and numerical "intuition". ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 👋 CONNECT WITH ME ON SOCIAL LinkedIn ► https://www.linkedin.com/in/aleksagordic/ Twitter ► https://twitter.com/gordic_aleksa Instagram ► https://www.instagram.com/aiepiphany/ Facebook ► https://www.facebook.com/aiepiphany/ 👨👩👧👦 JOIN OUR DISCORD COMMUNITY: Discord ► https://discord.gg/peBrCpheKE 📢 SUBSCRIBE TO MY MONTHLY AI NEWSLETTER: Substack ► https://aiepiphany.substack.com/ 💻 FOLLOW ME ON GITHUB FOR COOL PROJECTS: GitHub ► https://github.com/gordicaleksa 📚 FOLLOW ME ON MEDIUM: Medium ► https://gordicaleksa.medium.com/ ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ #nonparametric #transformers #relationalreasoning
