This 284B Model Shouldn't Fit On Your Laptop. It Does
In this video, we will look at DS-4, that enables DeepSeek V4 Flash to run on a 128GB unified memory. I this video I run it on DGX Spark thanks to @NVIDIADeveloper LINK: https://github.com/antirez/ds4 Blogpost: https://engineerprompt.ai/writing/dwarfstar-4/ My voice to text App: whryte.com Website: https://engineerprompt.ai/ RAG Beyond Basics Course: https://prompt-s-site.thinkific.com/courses/rag Signup for Newsletter, localgpt: https://tally.so/r/3y9bb0 Let's Connect: 🦾 Discord: https://discord.com/invite/t4eYQRUcXB ☕ Buy me a Coffee: https://ko-fi.com/promptengineering |🔴 Patreon: https://www.patreon.com/PromptEngineering 💼Consulting: https://calendly.com/engineerprompt/consulting-call 📧 Business Contact: engineerprompt@gmail.com Become Member: http://tinyurl.com/y5h28s6h 💻 Pre-configured localGPT VM: https://bit.ly/localGPT (use Code: PromptEngineering for 50% off). Signup for Newsletter, localgpt: https://tally.so/r/3y9bb0 00:00 Can It Run Locally 01:42 Why Big Models Don’t Fit 02:39 Quantization Limits Explained 04:26 Dwarf Star Selective Quant 07:56 Calibration and Validation 09:42 SSD Streaming Ends RAM Cliff 15:34 Benchmarks Demo and Wrap
