How does Groq LPU work? (w/ Head of Silicon Igor Arsovski!)
Become a Patreon: https://www.patreon.com/theaiepiphany π¨βπ©βπ§βπ¦ Join our Discord community: https://discord.gg/peBrCpheKE I invited head of silicon at Groq, Igor Arsovski, to share the nitty-gritty details behind Groq's LPUs! LPU or language processing units showed promising results (understatement of the year) on inference benchmarks compared to NVIDIA GPUs. β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ βοΈ Timetable: 00:00:00 - 00:00:42 Intro 00:00:42 - 00:02:23 Hyperstack GPU cloud (sponsored) 00:02:23 - 00:12:18 What does Groq do? 00:12:18 - 00:26:40 Hardware 00:26:40 - 00:37:52 Software 00:37:52 - 00:45:09 Power control 00:45:09 - 00:54:22 Networking 00:54:22 - 00:59:30 AllReduce comparison between GPUs and LPUs 00:59:30 - 01:04:10 GPU vs LPU 01:04:10 - 01:11:45 Outro β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ π° SPONSOR The AI Epiphany - https://www.patreon.com/theaiepiphany One-time donation - https://www.paypal.com/paypalme/theaiepiphany Huge thank you to these AI Epiphany patreons: Eli Mahler Petar VeliΔkoviΔ β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ πΌ LinkedIn - https://www.linkedin.com/in/aleksagordic/ π¦ Twitter - https://twitter.com/gordic_aleksa π¨βπ©βπ§βπ¦ Discord - https://discord.gg/peBrCpheKE πΊ YouTube - https://www.youtube.com/c/TheAIEpiphany/ π Medium - https://gordicaleksa.medium.com/ π» GitHub - https://github.com/gordicaleksa π’ AI Newsletter - https://aiepiphany.substack.com/ β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬ #groq #lpu #aichip #ai
