Why Program Synthesis Is Next (Kevin Ellis and Zenna Tavares)
Kevin Ellis (Cornell) and Zenna Tavares (BASIS) argue that the next wave of AI needs to learn like humans do: building abstract models from small amounts of data through active exploration, not just passive pattern matching at scale. The conversation centers on their joint work comparing two fundamentally different ways of solving problems. Induction searches for an explicit program -- something you could write in Python -- that transforms inputs to outputs. Transduction skips the program and directly predicts the answer, the way a neural network would. On the Abstraction and Reasoning Corpus (ARC), these approaches turn out to be complementary: some problems yield to systematic symbolic search, others to neural intuition. The ensemble is stronger than either alone, and the reasons connect to findings in cognitive science about when explicit reasoning helps versus hurts. Kevin explains how his DreamCoder work pioneered a wake-sleep cycle for program synthesis: dream up programs, run them to see what they do, learn the inverse mapping, then wake up and let real-world failures adjust the distribution of dreams. The modern version replaces explicit symbolic libraries with in-context learning over LLM-generated code, keeping the same iterative refinement loop. Zenna introduces his Autumn system for synthesizing the source code of interactive environments from observed behavior -- a form of computational science where the model must also infer hidden state it cannot directly observe. Both researchers converge on the idea that abstraction is the key unsolved problem: real intelligence requires knowing what to ignore, not just what to represent. Zenna frames this through resource rationality -- choosing the right level of abstraction given your computational budget and expected tasks. The discussion closes with Project MARA, their joint effort to build interactive benchmarks that go beyond ARC's static puzzles, requiring agents to actively explore and build world models from scratch. SPONSOR MESSAGES: *** Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. --- REFERENCES: Paper: [00:00:25] DreamCoder: Growing Generalizable, Interpretable Knowledge with Wake-Sleep Bayesian Program Learning https://arxiv.org/abs/2006.08381 [00:01:10] Mind Your Step: Active Search over Compositional Spaces https://arxiv.org/abs/2410.21333 [00:06:05] Bayesian inference in the cognitive sciences https://psycnet.apa.org/record/2008-06911-003 [00:13:00] Induction and Transduction https://arxiv.org/abs/2411.02272 [00:23:15] Neurosymbolic AI: The 3rd Wave https://arxiv.org/abs/2012.05876 [00:38:35] On the Measure of Intelligence (ARC) https://arxiv.org/abs/1911.01547 [00:39:20] Causal Reactive Programs (Autumn) http://www.zenna.org/publications/autumn2022.pdf [00:42:50] MuZero http://arxiv.org/pdf/1911.08265 [00:43:20] VisualPredicator https://arxiv.org/abs/2410.23156 Book: [00:48:55] Bayesian Models of Cognition https://mitpress.mit.edu/9780262049412/bayesian-models-of-cognition/ Essay: [00:49:30] The Bitter Lesson http://www.incompleteideas.net/IncIdeas/BitterLesson.html Project: [01:11:55] Project MARA https://www.basis.ai/blog/mara/ --- LINKS: Full Transcript: https://app.rescript.info/share/e0a208e545cabae728a3d72f76fcd310 Download PDF transcript: https://app.rescript.info/api/public/sessions/f47975e800b064d9/pdf
