One Mac Can Run Kimi K3 Locally. Don't Trust It Yet
Kimi K3 compressed to 594GB runs on a Mac, but tool-calling breaks before the model fails. What actually kills local agents. Kimi K3, a 2.8-trillion-parameter mixture-of-experts model from Moonshot AI, shrinks from 1.56TB to 594GB via Unsloth's dynamic 1-bit quantization while retaining 78.9% top-1 accuracy, small enough to run on a Mac Studio with 128GB RAM. The catch: fitting a model and trusting it with file operations are entirely different problems. This video walks through the real failure modes that builders encounter when running local agents, starting with why a model that scores 88.3 on Terminal-Bench (beating Claude Opus 4.8) still crashes on a multi-line git commit message. You'll see how the wire between the model's output and your agent's action-parser breaks silently across llama.cpp, vLLM, and Moonshot's own kimi-cli, malformed JSON, chat-template drops, schema mismatches, control-character parsing failures, each one closing issue after issue while the same layer fails on different models two years later. The video examines what actually changes when you compress a model (spoiler: coding drops, tool-calling stays broken), why the packer of your quantized file matters more than the model's benchmark scores, and the one deliberate test that tells you whether your setup can handle write operations. For builders evaluating local frontier-scale models, agents, and agentic coding workflows, the practical boundary is clear: read-only operations work today; commits require proof the tool-call path is solid. Chapters: 0:00 The 594GB breakthrough that fits a Mac 1:10 Why compression keeps the teeth 2:36 When benchmark scores tell the truth 3:55 The unmerged fork nobody warns about 5:22 How a newline crashed the vendor 6:40 Format drift in a long conversation 8:02 The template that silently vanishes 9:36 When servers pass garbage downstream 10:58 One flag that breaks everything 12:32 This broke on different models first 14:10 Turn off the feature that works 15:47 Paying for size, not safety 17:05 Who packed your file matters most 18:48 What you can safely run today 21:25 The one fix that changes everything Tools & resources mentioned: - Unsloth Kimi K3 GGUF Quantization: https://huggingface.co/unsloth - llama.cpp: https://github.com/ggml-org/llama.cpp - vLLM: https://github.com/vllm-project/vllm - LM Studio: https://lmstudio.ai - Open WebUI: https://openwebui.com - Ollama: https://ollama.ai - oh-my-pi Agent Framework: https://github.com/oh-my-pi - Kimi-cli: https://github.com/moonshot-ai/kimi-cli About The Stack The Stack helps you build with AI. Each video takes one tool, model, or workflow and shows how it works in a few focused minutes, with the real benchmarks and real costs. We go deep on Claude Code and Cursor for AI coding, AI agents and MCP servers, the open-source AI tools and GitHub repos most people miss, RAG and vector search, fine-tuning, and running local LLMs on your own machine with Ollama and LM Studio. We compare models like ChatGPT and Claude, test AI automation with Zapier, Make, and n8n, and flag the tools that actually ship. Subscribe for new breakdowns: https://www.youtube.com/@the-stack-ai?sub_confirmation=1 #kimi-k3 #local-llm #ai-agents
