World Model · podcast knowledge graph

AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)

2025-10-04 · 61 min · 22 entities

episode page audio

Asserted relationships

  • → discusses Technology company
    0.40
    evidence rules-v4
    Feed category: Technology
  • → references cyber.fund company
    0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://cyber.fund/
  • → references Ai Investment Principal person
    0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal
  • → references Agentic Misalignment person
    0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://anthropic.com/research/agentic-misalignment
  • → references dagihouse.com website
    0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://dagihouse.com/
  • → references Humandatasurvey website
    0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://prolific.com/humandatasurvey
  • → references iliaishacked.github.io website
    0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://iliaishacked.github.io/
  • → references sequrity.ai website
    0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://sequrity.ai/
  • 0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://app.rescript.info/public/share/dVGsk8dz9_V0J7xMlwguByBq1HXRD6i4uC5z5r7EVGM
  • → references Apache Log4j Vulnerability Guidance website
    0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://cisa.gov/news-events/news/apache-log4j-vulnerability-guidance

Entities found in this episode

websites 12

  • mentioned dagihouse.com website
    0.45
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://dagihouse.com/
  • mentioned Humandatasurvey website
    0.45
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://prolific.com/humandatasurvey
  • mentioned iliaishacked.github.io website
    0.45
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://iliaishacked.github.io/
  • mentioned sequrity.ai website
    0.45
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://sequrity.ai/
  • 0.45
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://app.rescript.info/public/share/dVGsk8dz9_V0J7xMlwguByBq1HXRD6i4uC5z5r7EVGM
  • 0.45
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://cisa.gov/news-events/news/apache-log4j-vulnerability-guidance
  • references dagihouse.com website
    0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://dagihouse.com/
  • references Humandatasurvey website
    0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://prolific.com/humandatasurvey
  • references iliaishacked.github.io website
    0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://iliaishacked.github.io/
  • references sequrity.ai website
    0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://sequrity.ai/
  • 0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://app.rescript.info/public/share/dVGsk8dz9_V0J7xMlwguByBq1HXRD6i4uC5z5r7EVGM
  • 0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://cisa.gov/news-events/news/apache-log4j-vulnerability-guidance

companys 4

  • mentioned Technology company
    0.50
    evidence rules-v4
    Feed category: Technology
  • mentioned cyber.fund company
    0.45
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://cyber.fund/
  • discusses Technology company
    0.40
    evidence rules-v4
    Feed category: Technology
  • references cyber.fund company
    0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://cyber.fund/

persons 4

  • mentioned Ai Investment Principal person
    0.45
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal
  • mentioned Agentic Misalignment person
    0.45
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://anthropic.com/research/agentic-misalignment
  • references Ai Investment Principal person
    0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal
  • references Agentic Misalignment person
    0.38
    evidence rules-v4
    Link in episode "AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)": https://anthropic.com/research/agentic-misalignment

podcasts 1

concepts 1

  • mentioned GDM concept
    0.35
    evidence rules-v4
    GDM
Episode description as stored
Dr. Ilia Shumailov - Former DeepMind AI Security Researcher, now building security tools for AI agents Ever wondered what happens when AI agents start talking to each other—or worse, when they start breaking things? Ilia Shumailov spent years at DeepMind thinking about exactly these problems, and he's here to explain why securing AI is way harder than you think. **SPONSOR MESSAGES** —Check out notebooklm for your research project, it's really powerfulhttps://notebooklm.google.com/ — Take the Prolific human data survey - https://www.prolific.com/humandatasurvey?utm_source=mlst and be the first to see the results and benchmark their practices against the wider community! — cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy Oct SF conference - https://dagihouse.com/?utm_source=mlst - Joscha Bach keynoting(!) + OAI, Anthropic, NVDA,++ Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst Submit investment deck: https://cyber.fund/contact?utm_source=mlst — We're racing toward a world where AI agents will handle our emails, manage our finances, and interact with sensitive data 24/7. But there is a problem. These agents are nothing like human employees. They never sleep, they can touch every endpoint in your system simultaneously, and they can generate sophisticated hacking tools in seconds. Traditional security measures designed for humans simply won't work. Dr. Ilia Shumailov https://x.com/iliaishacked https://iliaishacked.github.io/ https://sequrity.ai/ TRANSCRIPT: https://app.rescript.info/public/share/dVGsk8dz9_V0J7xMlwguByBq1HXRD6i4uC5z5r7EVGM TOC: 00:00:00 - Introduction & Trusted Third Parties via ML 00:03:45 - Background & Career Journey 00:06:42 - Safety vs Security Distinction 00:09:45 - Prompt Injection & Model Capability 00:13:00 - Agents as Worst-Case Adversaries 00:15:45 - Personal AI & CAML System Defense 00:19:30 - Agents vs Humans: Threat Modeling 00:22:30 - Calculator Analogy & Agent Behavior 00:25:00 - IMO Math Solutions & Agent Thinking 00:28:15 - Diffusion of Responsibility & Insider Threats 00:31:00 - Open Source Security Concerns 00:34:45 - Supply Chain Attacks & Trust Issues 00:39:45 - Architectural Backdoors 00:44:00 - Academic Incentives & Defense Work 00:48:30 - Semantic Censorship & Halting Problem 00:52:00 - Model Collapse: Theory & Criticism 00:59:30 - Career Advice & Ross Anderson Tribute REFS: Lessons from Defending Gemini Against Indirect Prompt Injections https://arxiv.org/abs/2505.14534 Defeating Prompt Injections by Design. Debenedetti, E., Shumailov, I., Fan, T., Hayes, J., Carlini, N., Fabian, D., Kern, C., Shi, C., Terzis, A., & Tramèr, F. https://arxiv.org/pdf/2503.18813 Agentic Misalignment: How LLMs could be insider threats https://www.anthropic.com/research/agentic-misalignment STOP ANTHROPOMORPHIZING INTERMEDIATE TOKENS AS REASONING/THINKING TRACES! Subbarao Kambhampati et al https://arxiv.org/pdf/2504.09762 Meiklejohn, S., Blauzvern, H., Maruseac, M., Schrock, S., Simon, L., & Shumailov, I. (2025). Machine learning models have a supply chain problem. https://arxiv.org/abs/2505.22778 Gao, Y., Shumailov, I., & Fawaz, K. (2025). Supply-chain attacks in machine learning frameworks. https://openreview.net/pdf?id=EH5PZW6aCr Apache Log4j Vulnerability Guidance https://www.cisa.gov/news-events/news/apache-log4j-vulnerability-guidance Bober-Irizar, M., Shumailov, I., Zhao, Y., Mullins, R., & Papernot, N. (2022). Architectural backdoors in neural networks. https://arxiv.org/pdf/2206.07840 Position: Fundamental Limitations of LLM Censorship Necessitate New Approaches David Glukhov, Ilia Shumailov, ... https://proceedings.mlr.press/v235/glukhov24a.html AlphaEvolve MLST interview [Matej Balog, Alexander Novikov] https://www.youtube.com/watch?v=vC9nAosXrJw