State Space Models w/ Albert Gu & Karan Goel (Cartesia AI)

Become a Patreon: https://www.patreon.com/theaiepiphany 👨‍👩‍👧‍👦 Join our Discord community: https://discord.gg/peBrCpheKE Albert & Karan joined us to talk state space models. They are the authors of the seminal S4 paper, Albert is the main author of thee Mamba paper - and together they're pioneering the SSM research and competing against transformer supremacy! :) Recently they posted super cool demos demonstrating just how much more real-time these models are compared to the baselines. ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ check out Cartesia AI: https://cartesia.ai/ ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ⌚️ Timetable: 00:00:00 - 00:01:56 Intro 00:01:56 - 00:03:37 Hyperstack GPUs! (sponsored) 00:03:37 - 00:09:45 Cartesia AI, real-time intelligence 00:09:45 - 00:12:05 text-to-speech demo 00:12:05 - 00:36:35 state space models (SSMs) 00:36:35 - 00:38:30 bitter lessons transformers vs SSMs 00:38:30 - 00:41:30 free lunch? what are the actual tradeoffs? 00:41:30 - 00:42:55 Computing SSMs convolutionally 00:42:55 - 00:50:00 What is the state (RNNs, Attention, Conv, Neural ODEs) 00:50:00 - 00:55:30 Mamba 2, discrete vs continuous spectrum 00:55:30 - 01:03:39 Q&A: scaling laws ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 💰 SPONSOR The AI Epiphany - https://www.patreon.com/theaiepiphany One-time donation - https://www.paypal.com/paypalme/theaiepiphany Huge thank you to these AI Epiphany patreons: Eli Mahler Petar Veličković ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 💼 LinkedIn - https://www.linkedin.com/in/aleksagordic/ 🐦 Twitter - https://twitter.com/gordic_aleksa 👨‍👩‍👧‍👦 Discord - https://discord.gg/peBrCpheKE 📺 YouTube - https://www.youtube.com/c/TheAIEpiphany/ 📚 Medium - https://gordicaleksa.medium.com/ 💻 GitHub - https://github.com/gordicaleksa 📢 AI Newsletter - https://aiepiphany.substack.com/ ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ #statespacemodels #ssms #transformer #albertgu #karangoel #cartesiaai