Ep 164: Inside Anthropic with Sholto Douglas & Nicholas Marwell — AGI, Optimism vs Risks, Regulatory Capture, Open Source & More!

Anthropic is one of the fastest-growing companies in history and sits at the forefront of the AI race. This week, we’re joined by two key leaders on Anthropic’s technical team — Sholto Douglas and Nicholas Marwell — for an inside look at the AI frontier. Are we on the cusp of AGI — and what does that mean? What are they most excited about and most concerned about? And where does Anthropic stand on open source, as well as criticisms of its regulatory approach? Sholto Douglas grew up in Sydney, Australia, where he studied robotics and taught himself Machine Learning research; he went on to work at DeepMind before becoming a tech lead in Reinforcement Learning (RL) at Anthropic. Nicholas Marwell, a San Francisco native, studied at Yale and joined Anthropic early in its journey to lead RL Science and now the Horizons team. We begin with the state of AI progress, and why Sholto and Nicholas believe AGI is likely within the next couple of years. We unpack their case for AI optimism, why biology is the most exciting next frontier, and whether a post-scarcity world is truly achievable. Then, they explain why the next decade “belongs to the generalists” and offer advice for young people entering the workforce in the coming years. Then we dig into the hard questions: offense vs. defense in cyber and bio, regulatory capture, open source, distillation, and competition with China. We also discuss the importance of virtue, faith, and institutions, and whether the leading AI researchers in Silicon Valley share the values of the heartland. Finally, they offer a few predictions for the years ahead and whether AI will soon win a Fields Medal or even a Nobel Prize. NOTE: This conversation was recorded about a month ago, and delayed beyond our control, but it’s still timely. 00:00 Episode intro 02:00 Self-taught ML; Unlikely paths to Anthropic 04:13 What are the most exciting things you’re working on? 07:48 Have we reached AGI — and what comes next? 10:10 How to measure AI progress 14:05 Is biology the next great frontier? 16:30 Career advice & the decade of the generalists 19:10 What could go wrong? What are the biggest AI risks? 22:50 Offense vs defense; how technology evolves 24:50 Answering the regulatory capture critique 30:15 Anthropic’s stance on open source 31:30 Would you slow down and let China go ahead? 32:50 Anthropic’s view on distillation 39:50 Is a post-scarcity world achievable? 48:30 How do we preserve values, tradition & community? 54:45 Predictions for 2028 This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit blog.joelonsdale.com