Diffusion Is Coming for Text. Here's NVIDIA's New Model.
NVIDIA’s Two-Tower Diffusion Language Model (Nemotron): Faster Text Generation with Frozen Context In this video, I break down how diffusion is moving into text generation and why it can avoid the compute and memory limits of autoregressive next-token prediction by generating in parallel. I explain NVIDIA’s Nemotron “Two Tower” diffusion language model: two cloned 52-layer towers (Mamba-2 + self-attention + MoE), where one tower is frozen as a left-to-right context model and the other is retrained as a denoiser that fills masked “noise” in 16-token blocks. I cover the layer-by-layer cross-attention “sky bridges,” the diffusion-style timer add-on, quality retention (about 98.7% vs the original), benchmark tradeoffs (math/code drops), ablation results showing freezing is key, and brittleness when changing block size (16 to 64 collapses generation). @NVIDIADeveloper LINKS: https://huggingface.co/nvidia/Nemotron-Labs-TwoTower-30B-A3B-Base-BF16 My voice to text App: whryte.com Website: https://engineerprompt.ai/ RAG Beyond Basics Course: https://prompt-s-site.thinkific.com/courses/rag Signup for Newsletter, localgpt: https://tally.so/r/3y9bb0 Let's Connect: 🦾 Discord: https://discord.com/invite/t4eYQRUcXB ☕ Buy me a Coffee: https://ko-fi.com/promptengineering |🔴 Patreon: https://www.patreon.com/PromptEngineering 💼Consulting: https://calendly.com/engineerprompt/consulting-call 📧 Business Contact: engineerprompt@gmail.com Become Member: http://tinyurl.com/y5h28s6h 💻 Pre-configured localGPT VM: https://bit.ly/localGPT (use Code: PromptEngineering for 50% off). Signup for Newsletter, localgpt: https://tally.so/r/3y9bb0 00:00 Diffusion Nemotron 01:03 Two Tower Big Idea 02:25 Architecture and Benchmarks 03:58 Why Two Towers Work 06:14 Blockwise Diffusion Decoding 09:31 Limits and What’s Next
