Gemini 4: 10M Context Window & The Race to AGI

Link to our newsletter: https://bitbiased.ai/ Sundar Pichai just confirmed that Google is training Gemini 4 — and the most important part may be what he admitted immediately around that announcement. On Alphabet’s Q2 2026 earnings call, Pichai said the next generation of frontier AI will require “much larger base models,” confirmed Gemini 4 is already being trained, and described Google as being “very ambitious” with it. But despite the speculation already building around Gemini 4, Google has revealed almost nothing else: no parameter count, no benchmark scores, no model card, and no release date. That gap between what Google has actually confirmed and what the internet is already claiming is where this story gets interesting. Gemini’s history gives us some clues. Gemini 1.5 brought a Mixture-of-Experts architecture and a million-token context window. Gemini 2.0 pushed into native multimodal output and agentic tool use. Later generations expanded Google’s focus on extended reasoning through Deep Think. If Google continues that pattern, Gemini 4 probably won’t just be about making the model larger — it will need a meaningful capability leap. Google has acknowledged that coding and agentic coding remain areas where it needs to improve. That makes reliable multi-step agents and coding performance particularly important as Gemini 4 moves toward the frontier. The baseline it has to beat is already formidable. Gemini 3.8 Flash, released September 2, 2026, is described by Google as its best reasoning and coding model yet at the same speed and low cost as 3.7. Google reports a 54.9% score on Humanity’s Last Exam, Verified, while pricing remains $0.75 per million input tokens and $3.75 per million output tokens through the end of 2026. Google has also introduced Gemini 3.8 Flash Cyber, a restricted defensive cybersecurity model designed to find and patch software vulnerabilities for vetted government agencies and infrastructure operators through the Fairwind Program. Then there’s the hardware behind all of this. Google’s Ironwood TPUs deliver 4,614 teraflops of FP8 compute per chip with 192GB of high-bandwidth memory. Connect nearly 10,000 of them into a TPU superpod and Google says the system reaches 42.5 exaflops of FP8 compute. Anthropic has also secured access to as many as one million Google TPUs, showing just how enormous Google’s AI infrastructure operation has become. And Google is spending accordingly. Alphabet has poured enormous amounts of capital into AI infrastructure and compute, while Google is even pursuing advanced nuclear power through its partnership with Kairos Power. The planned capacity reaches up to 500 megawatts by 2035, including the Hermes 2 project intended to support electricity demand associated with Google’s data centers. But none of that tells us exactly what Gemini 4 will be. The strongest clue remains Pichai’s emphasis on larger base models combined with Google’s admitted weakness in agentic coding. A Gemini 4 focused on stronger coding, longer autonomous workflows, and more reliable multi-step agents would fit Google’s trajectory — but until Google confirms those capabilities, that remains extrapolation rather than fact. And the competition won’t wait. Claude remains a major force in enterprise coding, while OpenAI continues pushing its own frontier models. Google’s potential advantage may not simply be benchmark leadership, but economics. Gemini 3.8 Flash’s token pricing dramatically undercuts the current frontier pricing cited for OpenAI and Anthropic in this analysis. If Google can preserve anything close to that cost advantage while making Gemini 4 substantially more capable, the real competition may become less about which model wins a benchmark and more about intelligence per dollar. There’s also a much bigger question. Google DeepMind CEO Demis Hassabis has discussed AGI arriving around 2030, give or take, while also pointing to major unsolved problems including continual learning, persistent memory, and grounded understanding of the physical world. Simply making a base model larger does not automatically solve those problems. So the most revealing Gemini 4 story right now isn’t a leaked parameter count or a supposed benchmark. It’s that Google has publicly acknowledged the frontier is moving fast enough that its current models aren’t enough — and it is committing enormous models, custom TPUs, infrastructure spending, and power capacity to what comes next. CHAPTERS 00:00 Google Is Already Training Gemini 4 00:53 The One Confirmed Sentence, And Why It Matters 01:56 How Google Actually Got Here 03:54 What Gemini 4 Actually Has To Beat 06:03 The Hardware Google Is Betting Everything On 07:27 Google Is Spending Like This Is The Whole Company 08:50 So What Might Gemini 4 Actually Be? 10:08 The Competition Isn't Standing Still Either 11:46 When, And What It Actually Means 13:18 The Verdict #gemini4 #google #gemini #googledeepmind #ai