TurboFieldFare Runs a 26B LLM on a MacBook Air (Run AI Locally)
TurboFieldfare runs a 26B mixture-of-experts local LLM on an 8GB MacBook Air using ~2GB RAM by streaming experts from SSD TurboFieldfare is a custom Swift and Metal engine that runs Gemma 4 26B-A4B, a mixture-of-experts local LLM, on an 8GB M2 MacBook Air using roughly 2GB of RAM. Built by independent developer Andrey Mikhaylov, it keeps a 1.35GB shared model core plus a KV cache resident in memory and streams the other 120 idle experts off the SSD per token, hitting 5.1-6.3 tokens per second on that base machine and 24-35 tok/s on a 24GB M5 Pro. This video breaks down the actual architecture behind the trick. You'll see how the 8-bit router picks 8 of 128 experts plus one shared expert per token, why a 16-slot least-frequently-used cache keeps SSD reads survivable, and why the model's 256K context window quietly eats into the same tiny RAM budget. It also covers the real tradeoffs: an independent LM Studio coding benchmark scored this exact model 33.33% with all 21 test cases failed, a 2026 arXiv paper on SSD-offloaded mixture of experts found energy costs can jump nearly 10x, and the same model already ran on 16GB Macs via plain llama.cpp at comparable speeds without any custom engine. TurboFieldfare isn't a general MLX or llama.cpp wrapper, it's model-specific by design, built only for Gemma 4 26B-A4B. The bigger point: as open models move to MoE architecture, the real constraint becomes RAM to hold weights you rarely touch, not raw compute, and Apple's own MLX-LM still lacks native expert-streaming support that one developer's hundred failed experiments figured out first. For builders who want to own your AI stack, understand local LLM tradeoffs, and see real prompt engineering and AI architecture decisions before trusting a benchmark headline. Chapters: 0:00 The Model That Shouldn't Fit 0:48 One Fifth Of A Second, Explained 1:48 The Hundred Idle Experts 3:00 The Two Gigabyte Accounting Trick 4:34 The Bet Behind Every Token 5:39 The Coding Test It Failed Badly 7:03 The Bill Your Terminal Never Shows 8:04 The Laptop That Was Never The Point 9:21 What A Hundred Failures Actually Proved Tools & resources mentioned: - TurboFieldfare (GitHub): https://github.com/drumih/turbo-fieldfare - TurboFieldfare Benchmarks: https://github.com/drumih/turbo-fieldfare/blob/main/docs/BENCHMARKS.md - Gemma 4 26B-A4B-it (Hugging Face): https://huggingface.co/google/gemma-4-26B-A4B-it - Gemma 4 Model Card: https://ai.google.dev/gemma/docs/core/model_card_4 - LM Studio - Open WebUI - mlx-lm MoE feature request: https://github.com/ggml-org/llama.cpp/discussions/795 - Show HN: TurboFieldfare: https://news.ycombinator.com/item?id=49098510 About The Stack The Stack helps you build with AI. Each video takes one tool, model, or workflow and shows how it works in a few focused minutes, with the real benchmarks and real costs. We go deep on Claude Code and Cursor for AI coding, AI agents and MCP servers, the open-source AI tools and GitHub repos most people miss, RAG and vector search, fine-tuning, and running local LLMs on your own machine with Ollama and LM Studio. We compare models like ChatGPT and Claude, test AI automation with Zapier, Make, and n8n, and flag the tools that actually ship. Subscribe for new breakdowns: https://www.youtube.com/@the-stack-ai?sub_confirmation=1 #turbofieldfare #localllm #mixtureofexperts #aiarchitecture #ownyourai
