They solved AI’s memory problem!
Attention Residuals by Kimi AI. Adaptive, continuous learning AI models. #ai #ainews #llm #airesearch #agi Thanks to our sponsor Wondercraft. Use my code AI-SEARCH to get $25 OFF! https://www.wondercraft.ai/?via=ref Original paper: https://arxiv.org/abs/2603.15031 Transformers explainer: https://youtu.be/U2hZFMVNSE0 0:00 Intro 0:27 AI’s amnesia problem 1:50 Design of deep AI models 3:19 Residual connections 6:50 The genius of current language models 9:03 Applying attention to residuals 13:05 Wondercraft 15:22 Infra problems 17:46 Compute results 18:50 Performance results 20:45 Wider or deeper 22:22 From static to adaptive Newsletter: https://aisearch.substack.com/ Find AI tools & jobs: https://ai-search.io/ Support: https://ko-fi.com/aisearch Here's my equipment, in case you're wondering: Lenovo Thinkbook: https://amzn.to/4jWeKwH Dell Precision 5690: https://www.dell.com/en-us/dt/ai-technologies/index.htm?utm_source=AISearchTools&utm_medium=youtube&utm_campaign=precisionai#tab0=0 GPU: Nvidia RTX 5000 Ada https://nvda.ws/3zfqGqS Mic: Shure SM7B https://amzn.to/3DErjt1 Audio interface: Scarlett Solo https://amzn.to/3qELMeu
