World Model · podcast knowledge graph

How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)

2025-04-08 · 77 min · 17 entities

episode page audio

Asserted relationships

  • → discusses Technology company
    0.40
    evidence rules-v4
    Feed category: Technology
  • → references Blog website
    0.38
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": https://basis.ai/blog/mara
  • → references tufalabs.ai website
    0.38
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": https://tufalabs.ai/
  • → references BitterLesson website
    0.38
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": http://incompleteideas.net/IncIdeas/BitterLesson.html
  • → references Record website
    0.38
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": https://psycnet.apa.org/record/2008-06911-003
  • → references ~Ellisk website
    0.38
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": https://cs.cornell.edu/~ellisk
  • → references zenna.org website
    0.38
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": http://zenna.org/
  • → references Bayesian Models Of Cognition person
    0.38
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": https://mitpress.mit.edu/9780262049412/bayesian-models-of-cognition

Entities found in this episode

websites 12

  • mentioned Blog website
    0.45
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": https://basis.ai/blog/mara
  • mentioned tufalabs.ai website
    0.45
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": https://tufalabs.ai/
  • mentioned BitterLesson website
    0.45
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": http://incompleteideas.net/IncIdeas/BitterLesson.html
  • mentioned Record website
    0.45
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": https://psycnet.apa.org/record/2008-06911-003
  • mentioned ~Ellisk website
    0.45
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": https://cs.cornell.edu/~ellisk
  • mentioned zenna.org website
    0.45
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": http://zenna.org/
  • references Blog website
    0.38
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": https://basis.ai/blog/mara
  • references tufalabs.ai website
    0.38
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": https://tufalabs.ai/
  • references BitterLesson website
    0.38
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": http://incompleteideas.net/IncIdeas/BitterLesson.html
  • references Record website
    0.38
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": https://psycnet.apa.org/record/2008-06911-003
  • references ~Ellisk website
    0.38
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": https://cs.cornell.edu/~ellisk
  • references zenna.org website
    0.38
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": http://zenna.org/

companys 2

  • mentioned Technology company
    0.50
    evidence rules-v4
    Feed category: Technology
  • discusses Technology company
    0.40
    evidence rules-v4
    Feed category: Technology

persons 2

  • mentioned Bayesian Models Of Cognition person
    0.45
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": https://mitpress.mit.edu/9780262049412/bayesian-models-of-cognition
  • references Bayesian Models Of Cognition person
    0.38
    evidence rules-v4
    Link in episode "How Machines Learn to Ignore the Noise (Kevin Ellis + Zenna Tavares)": https://mitpress.mit.edu/9780262049412/bayesian-models-of-cognition

podcasts 1

Episode description as stored
Prof. Kevin Ellis and Dr. Zenna Tavares talk about making AI smarter, like humans. They want AI to learn from just a little bit of information by actively trying things out, not just by looking at tons of data. They discuss two main ways AI can "think": one way is like following specific rules or steps (like a computer program), and the other is more intuitive, like guessing based on patterns (like modern AI often does). They found combining both methods works well for solving complex puzzles like ARC. A key idea is "compositionality" - building big ideas from small ones, like LEGOs. This is powerful but can also be overwhelming. Another important idea is "abstraction" - understanding things simply, without getting lost in details, and knowing there are different levels of understanding. Ultimately, they believe the best AI will need to explore, experiment, and build models of the world, much like humans do when learning something new. SPONSOR MESSAGES: *** Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** TRANSCRIPT: https://www.dropbox.com/scl/fi/3ngggvhb3tnemw879er5y/BASIS.pdf?rlkey=lr2zbj3317mex1q5l0c2rsk0h&dl=0 Zenna Tavares: http://www.zenna.org/ Kevin Ellis: https://www.cs.cornell.edu/~ellisk/ TOC: 1. Compositionality and Learning Foundations [00:00:00] 1.1 Compositional Search and Learning Challenges [00:03:55] 1.2 Bayesian Learning and World Models [00:12:05] 1.3 Programming Languages and Compositionality Trade-offs [00:15:35] 1.4 Inductive vs Transductive Approaches in AI Systems 2. Neural-Symbolic Program Synthesis [00:27:20] 2.1 Integration of LLMs with Traditional Programming and Meta-Programming [00:30:43] 2.2 Wake-Sleep Learning and DreamCoder Architecture [00:38:26] 2.3 Program Synthesis from Interactions and Hidden State Inference [00:41:36] 2.4 Abstraction Mechanisms and Resource Rationality [00:48:38] 2.5 Inductive Biases and Causal Abstraction in AI Systems 3. Abstract Reasoning Systems [00:52:10] 3.1 Abstract Concepts and Grid-Based Transformations in ARC [00:56:08] 3.2 Induction vs Transduction Approaches in Abstract Reasoning [00:59:12] 3.3 ARC Limitations and Interactive Learning Extensions [01:06:30] 3.4 Wake-Sleep Program Learning and Hybrid Approaches [01:11:37] 3.5 Project MARA and Future Research Directions REFS: [00:00:25] DreamCoder, Kevin Ellis et al. https://arxiv.org/abs/2006.08381 [00:01:10] Mind Your Step, Ryan Liu et al. https://arxiv.org/abs/2410.21333 [00:06:05] Bayesian inference, Griffiths, T. L., Kemp, C., & Tenenbaum, J. B. https://psycnet.apa.org/record/2008-06911-003 [00:13:00] Induction and Transduction, Wen-Ding Li, Zenna Tavares, Yewen Pu, Kevin Ellis https://arxiv.org/abs/2411.02272 [00:23:15] Neurosymbolic AI, Garcez, Artur d'Avila et al. https://arxiv.org/abs/2012.05876 [00:33:50] Induction and Transduction (II), Wen-Ding Li, Kevin Ellis et al. https://arxiv.org/abs/2411.02272 [00:38:35] ARC, François Chollet https://arxiv.org/abs/1911.01547 [00:39:20] Causal Reactive Programs, Ria Das, Joshua B. Tenenbaum, Armando Solar-Lezama, Zenna Tavares http://www.zenna.org/publications/autumn2022.pdf [00:42:50] MuZero, Julian Schrittwieser et al. http://arxiv.org/pdf/1911.08265 [00:43:20] VisualPredicator, Yichao Liang https://arxiv.org/abs/2410.23156 [00:48:55] Bayesian models of cognition, Joshua B. Tenenbaum https://mitpress.mit.edu/9780262049412/bayesian-models-of-cognition/ [00:49:30] The Bitter Lesson, Rich Sutton http://www.incompleteideas.net/IncIdeas/BitterLesson.html [01:06:35] Program induction, Kevin Ellis, Wen-Ding Li https://arxiv.org/pdf/2411.02272 [01:06:50] DreamCoder (II), Kevin Ellis et al. https://arxiv.org/abs/2006.08381 [01:11:55] Project MARA, Zenna Tavares, Kevin Ellis https://www.basis.ai/blog/mara/