How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal

2026-06-15 · 40 min · 27 entities

episode page audio

Asserted relationships

→ references Tmux product

0.77

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://github.com/tmux/tmux
→ hosted by Claire Vo person

0.55

evidence rules-v5
Feed author/publisher: Claire Vo
→ works at Braintrust company

0.50

evidence rules-v5
CEO of Braintrust
→ discusses Technology company

0.40

evidence rules-v5
Feed category: Technology
→ references penname.co website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://penname.co/
→ references claude.ai website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://claude.ai/
→ references Codex website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://openai.com/codex
→ references chatprd.ai website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://chatprd.ai/
→ references clairevo.com website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://clairevo.com/
→ references Makersschedule website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": http://paulgraham.com/makersschedule.html
→ references braintrust.dev website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://braintrust.dev/
→ references Gpt 5.4 website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://developers.openai.com/api/docs/models/gpt-5.4
→ references Gpt 55 Just Did What No Other Model website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://lennysnewsletter.com/p/gpt-55-just-did-what-no-other-model

Entities found in this episode

websites 18

mentioned penname.co website

0.45

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://penname.co/
mentioned claude.ai website

0.45

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://claude.ai/
mentioned Codex website

0.45

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://openai.com/codex
mentioned chatprd.ai website

0.45

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://chatprd.ai/
mentioned clairevo.com website

0.45

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://clairevo.com/
mentioned Makersschedule website

0.45

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": http://paulgraham.com/makersschedule.html
mentioned braintrust.dev website

0.45

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://braintrust.dev/
mentioned Gpt 5.4 website

0.45

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://developers.openai.com/api/docs/models/gpt-5.4
mentioned Gpt 55 Just Did What No Other Model website

0.45

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://lennysnewsletter.com/p/gpt-55-just-did-what-no-other-model
references penname.co website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://penname.co/
references claude.ai website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://claude.ai/
references Codex website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://openai.com/codex
references chatprd.ai website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://chatprd.ai/
references clairevo.com website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://clairevo.com/
references Makersschedule website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": http://paulgraham.com/makersschedule.html
references braintrust.dev website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://braintrust.dev/
references Gpt 5.4 website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://developers.openai.com/api/docs/models/gpt-5.4
references Gpt 55 Just Did What No Other Model website

0.38

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://lennysnewsletter.com/p/gpt-55-just-did-what-no-other-model

companys 4

mentioned Braintrust company

0.62

evidence rules-v5
CEO of Braintrust
mentioned Technology company

0.50

evidence rules-v5
Feed category: Technology
works at Braintrust company

0.50

evidence rules-v5
CEO of Braintrust
discusses Technology company

0.40

evidence rules-v5
Feed category: Technology

products 2

mentioned Tmux product

0.90

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://github.com/tmux/tmux
references Tmux product

0.77

evidence rules-v5
Link in episode "How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal": https://github.com/tmux/tmux

persons 2

mentioned Claire Vo person

0.70

evidence rules-v5
Feed author/publisher: Claire Vo
hosted by Claire Vo person

0.55

evidence rules-v5
Feed author/publisher: Claire Vo

concepts 1

mentioned Ankur Goyal concept

0.42

evidence rules-v5
How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal

Episode description as stored

In this episode, I sit down with Ankur Goyal , founder and CEO of Braintrust, the AI evals and observability platform used by teams like Notion, Stripe, Vercel, and Zapier. This one is for the senior engineers, staff engineers, VPs of engineering, and CTOs in my audience. We get into how coding agents can take on deeply technical architecture and infrastructure work that no single human engineer could tackle before, and then we demystify evals so you can use them to make your AI products better without touching the implementation. What you’ll learn: How Ankur uses Codex to run week-long benchmark experiments across database indexes, column store formats, and execution engines to speed up slow queries Why he argues there’s no excuse to skip rigorous benchmarking now that agents can run them tirelessly The “agent line” framework: how to decide which decisions, directions, and interactions you can hand off to an agent How I think about the practical vs. theoretical quality of AI on hard technical problems, and why human attention decays on tedious work Why evals are the modern version of a PRD, and how to encode “what good looks like” so a model can figure out the “how” How to build a scoring function live and let an agent improve your prompt inside a safe playground How Ankur turned his designer David’s taste into a repeatable eval so quality scales beyond one person Why fixing your CI is the highest-leverage way to speed up engineering velocity — Brought to you by: Guru —The AI layer of truth Persona —Trusted identity verification for any use case — In this episode, we cover: (00:00) Introduction to Ankur Goyal (03:00) Using AI agents for database optimization (06:10) Running exhaustive benchmarks with coding agents (09:03) Why staff engineers are wrong about AI limitations (11:30) The “agent line” framework for delegation (14:00) Ankur’s workflow: running 4 to 6 concurrent agents (17:16) Technical setup: foreground agents, background agents, and cloud environments (20:32) Spending time with AI tools (23:06) Demystifying evals (26:02) Live demo: Building an eval for documentation answers (30:20) The alternative to evals: vibe checks and whack-a-mole (32:09) Capturing designer taste in scoring functions (33:13) Quick recap (33:44) Managing velocity and throughput (35:40) Why CI/CD investment is critical for AI-accelerated teams (37:30) Ankur’s prompting strategy when agents fail (39:10) Closing thoughts and how to connect — Tools referenced: • Braintrust: https://www.braintrust.dev/ • Codex: https://openai.com/codex/ • GPT 5.4: https://developers.openai.com/api/docs/models/gpt-5.4 • Claude: https://claude.ai/ — Other references: • GPT 5.5 just did what no other model could: https://www.lennysnewsletter.com/p/gpt-55-just-did-what-no-other-model • Paul Graham’s Maker vs. Manager Schedule: http://www.paulgraham.com/makersschedule.html • tmux: https://github.com/tmux/tmux • Chris Tate at Vercel: https://www.linkedin.com/in/ctatedev/ — Where to find Ankur Goyal: LinkedIn: https://www.linkedin.com/in/ankrgyl/ — Where to find Claire Vo: ChatPRD: https://www.chatprd.ai/ Website: https://clairevo.com/ LinkedIn: https://www.linkedin.com/in/clairevo/ X: https://x.com/clairevo — Production and marketing by https://penname.co/ . For inquiries about sponsoring the podcast, email jordan@penname.co.