Why AI Models Run Faster on Some Chips: The KV Cache Effect

Learn more about KV Cache here → https://ibm.biz/~6LPSw4Y8N The same AI model can run differently across chips. Kaoutar El Maghraoui explains how GPUs, TPUs, and AI accelerators store and move KV cache during LLM inference. Learn why memory architecture and data movement can shape AI performance. 00:00 – Understand Why LLMs Need a KV Cache 00:46 – See How KV Cache Reuses Keys and Values 01:39 – Balance KV Cache Capacity and Fast Memory 02:41 – Follow KV Cache Through GPU Memory 04:41 – Compare TPU Memory and Compiler Orchestration 06:44 – Explore Dataflow Accelerator Memory 08:34 – Compare How AI Accelerators Move KV Cache 09:05 – Connect KV Cache Movement to LLM Performance Continue your education in the course Memory Systems in Dataflow Architectures on Coursera here → https://ibm.biz/~I7KAjOCAb AI news moves fast. Sign up for a monthly newsletter for AI updates from IBM → https://ibm.biz/~modQneKWV AI was used in the creation of the transcript and metadata for this video. #kvcache #llm #aimodels #aiaccelerator --------------------------------------------------------------------------------------------------------- Find us on YouTube: 🔵 IBM Technology: https://www.youtube.com/@IBMTechnology 🔵 IBM: https://www.youtube.com/@IBM 🔵 IBM Developer: https://youtube.com/@IBMDeveloperAdvocates