NVIDIA's Dory: The 4B Model That Re-Runs Its Own Layers

NVIDIA's Dory 4B model reruns 20 layers 1, 4 times per token, reaching 61.8% on SWE-bench, beating Gemma 26B, without extra weights. Here's what the loops actually cost. NVIDIA released Dory, a four-billion-parameter open model that achieves the work of much larger models by rerunning its middle twenty layers between one and four times per token. On SWE-bench Verified, a test of fixing real GitHub issues, changing the loop setting moved Dory from 50.3% at one pass to 61.8% at four passes, outscoring Google's Gemma 4 26B-A4B at 57.4% in the same test setup. The recurrent architecture splits seventy-two hidden layers into three sections: an eight-pair input block, a twenty-pair recurrent middle block, and an eight-pair output block, allowing the middle section to loop while keeping all roughly four billion parameters in memory just once. Two loops capture 80% of the SWE-bench coding gain and 86% of gains on agent tasks like GDPval, where Dory finished in 49 turns instead of 116. However, every extra pass cuts generation speed, independent tests on RTX 4090 dropped throughput from 188 tokens/second at one loop to 78 at four, and looping did not improve knowledge scores, which stayed around 10% across all loop counts. NVIDIA's headline claim that Dory beats Gemma 26B on average rests primarily on a single non-hallucination benchmark where Dory scores 79.38 versus Gemma's 16.36; removing that column, Gemma leads. Memory scales with loop count unless you manually switch the configuration to last-loop caching, which trades a 2-point accuracy drop for flat cache size. vLLM supports it as of v0.33, and community ports exist for llama.cpp and MLX, though each carries trade-offs. This is for builders optimizing agent tasks and long-document reasoning on tight compute budgets. Chapters: 0:00 Intro 1:19 The Loop 2:54 Why Four 4:10 What It Buys 6:05 The 26B Claim 8:02 Speed 10:17 Memory 12:31 Running It 13:53 Verdict Tools & resources mentioned: - vLLM: https://github.com/vllm-project/vllm - llama.cpp: https://github.com/ggml-org/llama.cpp About The Stack The Stack helps you build with AI. Each video takes one tool, model, or workflow and shows how it works in a few focused minutes, with the real benchmarks and real costs. We go deep on Claude Code and Cursor for AI coding, AI agents and MCP servers, the open-source AI tools and GitHub repos most people miss, RAG and vector search, fine-tuning, and running local LLMs on your own machine with Ollama and LM Studio. We compare models like ChatGPT and Claude, test AI automation with Zapier, Make, and n8n, and flag the tools that actually ship. Subscribe for new breakdowns: https://www.youtube.com/@the-stack-ai?sub_confirmation=1 #nvidia #ai #transformers #machine learning #deep learning