World Model · podcast knowledge graph

Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan

2024-11-30 · 114 min · 22 entities

episode page audio

Asserted relationships

  • 0.77
    evidence rules-v4
    Link in episode "Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan": https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431
  • → appeared on "The Cognitive Revolution" podcast
    0.68
    evidence rules-v4
    Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan
  • → hosted by Erik Torenberg, Nathan Labenz person
    0.55
    evidence rules-v4
    Feed author/publisher: Erik Torenberg, Nathan Labenz
  • → discusses Society & Culture concept
    0.40
    evidence rules-v4
    Feed category: Society & Culture
  • → discusses Technology concept
    0.40
    evidence rules-v4
    Feed category: Technology
  • → discusses Business concept
    0.40
    evidence rules-v4
    Feed category: Business
  • → discusses Entrepreneurship concept
    0.40
    evidence rules-v4
    Feed category: Entrepreneurship
  • → hosted by Turpentine company
    0.40
    evidence rules-v4
    Feed author/publisher: Turpentine
  • → references @CognitiveRevolutionPodcast website
    0.38
    evidence rules-v4
    Link in episode "Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan": https://youtube.com/@CognitiveRevolutionPodcast
  • → references CognitiveRevolution website
    0.38
    evidence rules-v4
    Link in episode "Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan": https://notion.com/cognitiverevolution
  • → references wandb.me website
    0.38
    evidence rules-v4
    Link in episode "Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan": https://wandb.me/cr

Entities found in this episode

concepts 8

  • mentioned Society & Culture concept
    0.50
    evidence rules-v4
    Feed category: Society & Culture
  • mentioned Technology concept
    0.50
    evidence rules-v4
    Feed category: Technology
  • mentioned Business concept
    0.50
    evidence rules-v4
    Feed category: Business
  • mentioned Entrepreneurship concept
    0.50
    evidence rules-v4
    Feed category: Entrepreneurship
  • discusses Society & Culture concept
    0.40
    evidence rules-v4
    Feed category: Society & Culture
  • discusses Technology concept
    0.40
    evidence rules-v4
    Feed category: Technology
  • discusses Business concept
    0.40
    evidence rules-v4
    Feed category: Business
  • discusses Entrepreneurship concept
    0.40
    evidence rules-v4
    Feed category: Entrepreneurship

websites 6

  • mentioned @CognitiveRevolutionPodcast website
    0.45
    evidence rules-v4
    Link in episode "Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan": https://youtube.com/@CognitiveRevolutionPodcast
  • mentioned CognitiveRevolution website
    0.45
    evidence rules-v4
    Link in episode "Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan": https://notion.com/cognitiverevolution
  • mentioned wandb.me website
    0.45
    evidence rules-v4
    Link in episode "Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan": https://wandb.me/cr
  • references @CognitiveRevolutionPodcast website
    0.38
    evidence rules-v4
    Link in episode "Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan": https://youtube.com/@CognitiveRevolutionPodcast
  • references CognitiveRevolution website
    0.38
    evidence rules-v4
    Link in episode "Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan": https://notion.com/cognitiverevolution
  • references wandb.me website
    0.38
    evidence rules-v4
    Link in episode "Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan": https://wandb.me/cr

podcasts 3

  • 0.90
    evidence rules-v4
    Link in episode "Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan": https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431
  • 0.77
    evidence rules-v4
    Link in episode "Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan": https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431
  • appeared on "The Cognitive Revolution" podcast
    0.68
    evidence rules-v4
    Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan

persons 3

  • mentioned Tan Zhi Xuan person
    0.72
    evidence rules-v4
    Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan
  • 0.70
    evidence rules-v4
    Feed author/publisher: Erik Torenberg, Nathan Labenz
  • 0.55
    evidence rules-v4
    Feed author/publisher: Erik Torenberg, Nathan Labenz

companys 2

  • mentioned Turpentine company
    0.70
    evidence rules-v4
    Feed author/publisher: Turpentine
  • hosted by Turpentine company
    0.40
    evidence rules-v4
    Feed author/publisher: Turpentine
Episode description as stored
In this episode of The Cognitive Revolution, Nathan explores groundbreaking perspectives on AI alignment with MIT PhD student Tan Zhi Xuan. We dive deep into Xuan's critique of preference-based AI alignment and their innovative proposal for role-based AI systems guided by social consensus. The conversation extends into their fascinating work on how AI agents can learn social norms through Bayesian rule induction. Join us for an intellectually stimulating discussion that bridges philosophical theory with practical implementation in AI development. Check out: "Beyond Preferences in AI Alignment" paper: https://arxiv.org/pdf/2408.16984 "Learning and Sustaining Shared Normative Systems via Bayesian Rule Induction in Markov Games" paper: https://arxiv.org/pdf/2402.13399 Help shape our show by taking our quick listener survey at https://bit.ly/TurpentinePulse SPONSORS: Notion: Notion offers powerful workflow and automation templates, perfect for streamlining processes and laying the groundwork for AI-driven automation. With Notion AI, you can search across thousands of documents from various platforms, generating highly relevant analysis and content tailored just for you - try it for free at https://notion.com/cognitiverevolution Weights & Biases RAG++: Advanced training for building production-ready RAG applications. Learn from experts to overcome LLM challenges, evaluate systematically, and integrate advanced features. Includes free Cohere credits. Visit https://wandb.me/cr to start the RAG++ course today. Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers13. OCI powers industry leaders with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before December 31, 2024 at https://oracle.com/cognitive RECOMMENDED PODCAST: Unpack Pricing - Dive into the dark arts of SaaS pricing with Metronome CEO Scott Woody and tech leaders. Learn how strategic pricing drives explosive revenue growth in today's biggest companies like Snowflake, Cockroach Labs, Dropbox and more. Apple: https://podcasts.apple.com/us/podcast/id1765716600 Spotify: https://open.spotify.com/show/38DK3W1Fq1xxQalhDSueFg CHAPTERS: (00:00:00) Teaser (00:01:09) About the Episode (00:04:25) Guest Intro (00:06:25) Xuan's Background (00:12:03) AI Near-Term Outlook (00:17:32) Sponsors: Notion | Weights & Biases RAG++ (00:20:18) Alignment Approaches (00:26:11) Critiques of RLHF (00:34:40) Sponsors: Oracle Cloud Infrastructure (OCI) (00:35:50) Beyond Preferences (00:40:27) Roles and AI Systems (00:45:19) What AI Owes Us (00:51:52) Drexler's AI Services (01:01:08) Constitutional AI (01:09:43) Technical Approach (01:22:01) Norms and Deviations (01:32:31) Norm Decay (01:38:06) Self-Other Overlap (01:44:05) Closing Thoughts (01:54:23) Outro SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://www.linkedin.com/in/nathanlabenz/ Youtube: https://www.youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk