World Model · podcast knowledge graph

Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai

2025-01-03 · 234 min · 27 entities

episode page audio

Asserted relationships

  • 0.77
    evidence rules-v4
    Link in episode "Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai": https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431
  • → appeared on "The Cognitive Revolution" podcast
    0.68
    evidence rules-v4
    Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai
  • → hosted by Erik Torenberg, Nathan Labenz person
    0.55
    evidence rules-v4
    Feed author/publisher: Erik Torenberg, Nathan Labenz
  • → founded AI company
    0.54
    evidence rules-v4
    founder of AI
  • → founded AI company
    0.54
    evidence rules-v4
    founder of AI
  • → works at AI company
    0.50
    evidence rules-v4
    founder of AI
  • → works at AI company
    0.50
    evidence rules-v4
    founder of AI
  • → discusses Society & Culture concept
    0.40
    evidence rules-v4
    Feed category: Society & Culture
  • → discusses Technology concept
    0.40
    evidence rules-v4
    Feed category: Technology
  • → discusses Business concept
    0.40
    evidence rules-v4
    Feed category: Business
  • → discusses Entrepreneurship concept
    0.40
    evidence rules-v4
    Feed category: Entrepreneurship
  • → hosted by Turpentine company
    0.40
    evidence rules-v4
    Feed author/publisher: Turpentine
  • → references @CognitiveRevolutionPodcast website
    0.38
    evidence rules-v4
    Link in episode "Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai": https://youtube.com/@CognitiveRevolutionPodcast
  • → references CognitiveRevolution website
    0.38
    evidence rules-v4
    Link in episode "Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai": https://80000hours.org/cognitiverevolution
  • 0.38
    evidence rules-v4
    Link in episode "Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai": https://dust-mailbox-c73.notion.site/Vision-Language-Models-11b675d75dd480af994cc474a754bb26

Entities found in this episode

concepts 8

  • mentioned Society & Culture concept
    0.50
    evidence rules-v4
    Feed category: Society & Culture
  • mentioned Technology concept
    0.50
    evidence rules-v4
    Feed category: Technology
  • mentioned Business concept
    0.50
    evidence rules-v4
    Feed category: Business
  • mentioned Entrepreneurship concept
    0.50
    evidence rules-v4
    Feed category: Entrepreneurship
  • discusses Society & Culture concept
    0.40
    evidence rules-v4
    Feed category: Society & Culture
  • discusses Technology concept
    0.40
    evidence rules-v4
    Feed category: Technology
  • discusses Business concept
    0.40
    evidence rules-v4
    Feed category: Business
  • discusses Entrepreneurship concept
    0.40
    evidence rules-v4
    Feed category: Entrepreneurship

companys 7

  • mentioned Turpentine company
    0.70
    evidence rules-v4
    Feed author/publisher: Turpentine
  • mentioned AI company
    0.68
    evidence rules-v4
    founder of AI
  • founded AI company
    0.54
    evidence rules-v4
    founder of AI
  • founded AI company
    0.54
    evidence rules-v4
    founder of AI
  • works at AI company
    0.50
    evidence rules-v4
    founder of AI
  • works at AI company
    0.50
    evidence rules-v4
    founder of AI
  • hosted by Turpentine company
    0.40
    evidence rules-v4
    Feed author/publisher: Turpentine

websites 6

  • mentioned @CognitiveRevolutionPodcast website
    0.45
    evidence rules-v4
    Link in episode "Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai": https://youtube.com/@CognitiveRevolutionPodcast
  • mentioned CognitiveRevolution website
    0.45
    evidence rules-v4
    Link in episode "Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai": https://80000hours.org/cognitiverevolution
  • 0.45
    evidence rules-v4
    Link in episode "Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai": https://dust-mailbox-c73.notion.site/Vision-Language-Models-11b675d75dd480af994cc474a754bb26
  • references @CognitiveRevolutionPodcast website
    0.38
    evidence rules-v4
    Link in episode "Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai": https://youtube.com/@CognitiveRevolutionPodcast
  • references CognitiveRevolution website
    0.38
    evidence rules-v4
    Link in episode "Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai": https://80000hours.org/cognitiverevolution
  • 0.38
    evidence rules-v4
    Link in episode "Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai": https://dust-mailbox-c73.notion.site/Vision-Language-Models-11b675d75dd480af994cc474a754bb26

podcasts 3

  • 0.90
    evidence rules-v4
    Link in episode "Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai": https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431
  • 0.77
    evidence rules-v4
    Link in episode "Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai": https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431
  • appeared on "The Cognitive Revolution" podcast
    0.68
    evidence rules-v4
    Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai

persons 3

  • mentioned Will Hardman of Veratai person
    0.72
    evidence rules-v4
    Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai
  • 0.70
    evidence rules-v4
    Feed author/publisher: Erik Torenberg, Nathan Labenz
  • 0.55
    evidence rules-v4
    Feed author/publisher: Erik Torenberg, Nathan Labenz
Episode description as stored
In this episode of The Cognitive Revolution, Nathan hosts Will Hardman, founder of AI advisory firm Veritai, for a comprehensive technical survey of vision language models (VLMs). We explore the evolution of VLMs from early vision transformers to state-of-the-art architectures like InternVL and Llama3V, examining key innovations and architectural decisions. Join us for an in-depth discussion covering multimodality in AI systems, evaluation frameworks, and practical applications with one of the field's leading experts. Here's to the link to one of the most comprehensive reference documents for VLMs prepared by Will Hardman: https://dust-mailbox-c73.notion.site/Vision-Language-Models-11b675d75dd480af994cc474a754bb26 Help shape our show by taking our quick listener survey at https://bit.ly/TurpentinePulse SPONSORS: Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers. OCI powers industry leaders like Vodafone and Thomson Reuters with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before March 31, 2024 at https://oracle.com/cognitive 80,000 Hours: 80,000 Hours is dedicated to helping you find a fulfilling career that makes a difference. With nearly a decade of research, they offer in-depth material on AI risks, AI policy, and AI safety research. Explore their articles, career reviews, and a podcast featuring experts like Anthropic CEO Dario Amodei. Everything is free, including their Career Guide. Visit https://80000hours.org/cognitiverevolution to start making a meaningful impact today. CHAPTERS: (00:00:00) Teaser (00:00:55) About the Episode (00:05:45) Introduction (00:09:16) VLM Use Cases (00:13:47) Vision Transformers (Part 1) (00:17:48) Sponsors: Oracle Cloud Infrastructure (OCI) (00:19:00) Vision Transformers (Part 2) (00:24:58) OpenAI's CLIP Model (00:33:44) DeepMind's Flamingo (Part 1) (00:33:44) Sponsors: 80,000 Hours (00:35:17) DeepMind's Flamingo (Part 2) (00:48:29) Instruction Tuning with LAVA (01:09:25) MMMU Benchmark (01:14:42) Pre-training with QNVL (01:32:13) InternVL Model Series (01:52:33) Cross-Attention vs. Self-Attention (02:14:33) Hybrid Architectures (02:31:08) Early vs. Late Fusion (02:34:50) VQA and DocVQA Benchmarks (02:40:08) The Blink Benchmark (03:05:37) Generative Pre-training (03:15:26) Multimodal Generation (03:37:00) Frontier Labs & Benchmarks (03:47:45) Conclusion (03:53:28) Outro SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://www.linkedin.com/in/nathanlabenz/ Youtube: https://www.youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk