Why AI Infrastructure must evolve for Agent Experience — Akshat Bubna, Modal CTO

We’ve been running a bit of an Agent Cloud series surveying all the top inference/compute/cloud providers, from Databricks to Daytona to Railway and, even further back, E2B , but we’re excited to conclude this series returning to Modal, which has just raised a monster $355M Series C . The cloud was built for developers. But agents are now changing that. The old infra stack was designed for a human who could read docs, reason through YAML, and understand dashboards to figure out what they need when something broke. While this was painful for developers, it worked since they could fill in missing context in their heads. However, agents don’t have that luxury. Now in this new era of agents, everything has to be tighter. They need a place to write code, run it, inspect the output, change the environment, debug failures, and try again. Fast iteration and feedback loops with all the necessary context are crucial for agents to operate properly. Furthermore, sandboxes are a clear representation of this shift as agents can easily spin up isolated environments. This programmatic infra even extends to research: Two years ago, we were one of the first to cover Modal with CEO Erik Bernhardsson and Alessio designed our favorite LS thumbnail of all time: At the time, Modal was just a teeny little company with a $17M Series A . Today, fresh off their $355M Series C , Modal is one of the clearest examples of the agent cloud future being built in real time: a cloud platform moving past traditional web app assumptions toward the workloads AI actually creates such as elastic inference , sandboxes , GPU burst, post-training, background agents, and infrastructure that agents themselves can operate . In this episode, Modal CTO Akshat Bubna joins swyx and Vibhu to unpack why AI applications don’t fit traditional cloud assumptions, why Kubernetes was never designed for bursty compute-heavy workloads, and why Modal is now shifting from developer experience to agent experience . We go deep on Modal’s AI infra stack: serverless functions, decorator-based infrastructure, elastic inference for custom models , GPU snapshotting, DeFlash, speculative decoding, Auto Endpoints, sandboxes, persistent storage, networked containers, private IPv6, RDMA, multi-node training, and Modal’s capacity pool across 17 cloud providers . Akshat also explains why RL rollouts can require 100,000 sandboxes , why production agents need hard guardrails, why observability may matter more than reading code, and why AI has made infrastructure exciting again. We discuss: * Why Kubernetes wasn’t built for bursty AI workloads * How Modal started as a better runtime before becoming an AI cloud * Why Modal added GPUs before ChatGPT * The shift from developer experience to agent experience * Why observability matters when agents are writing the code * Elastic inference for custom models across audio, video, robotics, and comp bio * GPU snapshotting , cold starts, and why inference workloads are so bursty * Why RL rollouts can require 100,000 sandboxes * DeFlash , speculative decoding, and frontier-level inference performance * Auto Endpoints and making optimized inference easier to deploy * What Modal adds beyond vLLM , SGLang , and raw GPU rental * Modal’s 17-cloud capacity pool and supercloud strategy * Networked sandboxes , sidecars, private IPv6, and RDMA * Serverless multi-node training for post-training and research workloads * Auto-research , model-guided sweeps, and agents launching GPU experiments * Compute strategy , capacity planning, and batch tiers * Why production agents need specialized sandboxes and hard guardrails * Modal’s take on managed agents , CI , Gitpod/Ona, Python, TypeScript, and Modal Bench Akshat Bubna * LinkedIn: https://www.linkedin.com/in/akshat-bubna-188885103 * X: https://x.com/akshat_b Modal * Website: https://modal.com Timestamps 00:00:00 Introduction 00:00:39 Modal’s origin and why Kubernetes wasn’t enough 00:04:32 Developer Experience → Agent Experience