#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY

2023-04-02 · 160 min · episode 112 · 30 entities

episode page audio

Asserted relationships

→ references Connor J Leahy person

0.77

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://linkedin.com/in/connor-j-leahy/https://twitter.com/NPCollapse
→ references Does Not Hate You Superintelligence book

0.77

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://amazon.co.uk/Does-Not-Hate-You-Superintelligence/dp/1474608795
→ references Mysteries Of Mode Collapse person

0.77

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-of-mode-collapse
→ references Alfred Korzybski person

0.77

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://en.wikipedia.org/wiki/Alfred_Korzybski
→ references Instrumental Convergence person

0.77

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://en.wikipedia.org/wiki/Instrumental_convergence
→ references Orthogonality Thesis person

0.77

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://en.wikipedia.org/wiki/Orthogonality_thesis
→ references Epistemological Framing For Ai Alignment Research concept

0.77

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://lesswrong.com/posts/Y4YHTBziAscS5WPN7/epistemological-framing-for-ai-alignment-research
→ references Society Mind Marvin Minsky book

0.77

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://amazon.co.uk/Society-Mind-Marvin-Minsky/dp/0671607405
→ discusses Technology company

0.40

evidence rules-v4
Feed category: Technology
→ references chat.openai.com website

0.38

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://chat.openai.com/
→ references xrai.glass website

0.38

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://xrai.glass/
→ references reducing-suffering.org website

0.38

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://reducing-suffering.org/
→ references Circumventing Interpretability How To Defeat Mind Readers website

0.38

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://alignmentforum.org/posts/EhAbh2pQoAXkm9yor/circumventing-interpretability-how-to-defeat-mind-readers

Entities found in this episode

persons 10

mentioned Connor J Leahy person

0.90

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://linkedin.com/in/connor-j-leahy/https://twitter.com/NPCollapse
mentioned Mysteries Of Mode Collapse person

0.90

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-of-mode-collapse
mentioned Alfred Korzybski person

0.90

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://en.wikipedia.org/wiki/Alfred_Korzybski
mentioned Instrumental Convergence person

0.90

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://en.wikipedia.org/wiki/Instrumental_convergence
mentioned Orthogonality Thesis person

0.90

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://en.wikipedia.org/wiki/Orthogonality_thesis
references Connor J Leahy person

0.77

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://linkedin.com/in/connor-j-leahy/https://twitter.com/NPCollapse
references Mysteries Of Mode Collapse person

0.77

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-of-mode-collapse
references Alfred Korzybski person

0.77

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://en.wikipedia.org/wiki/Alfred_Korzybski
references Instrumental Convergence person

0.77

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://en.wikipedia.org/wiki/Instrumental_convergence
references Orthogonality Thesis person

0.77

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://en.wikipedia.org/wiki/Orthogonality_thesis

websites 8

mentioned chat.openai.com website

0.45

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://chat.openai.com/
mentioned xrai.glass website

0.45

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://xrai.glass/
mentioned reducing-suffering.org website

0.45

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://reducing-suffering.org/
mentioned Circumventing Interpretability How To Defeat Mind Readers website

0.45

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://alignmentforum.org/posts/EhAbh2pQoAXkm9yor/circumventing-interpretability-how-to-defeat-mind-readers
references chat.openai.com website

0.38

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://chat.openai.com/
references xrai.glass website

0.38

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://xrai.glass/
references reducing-suffering.org website

0.38

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://reducing-suffering.org/
references Circumventing Interpretability How To Defeat Mind Readers website

0.38

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://alignmentforum.org/posts/EhAbh2pQoAXkm9yor/circumventing-interpretability-how-to-defeat-mind-readers

concepts 5

mentioned Epistemological Framing For Ai Alignment Research concept

0.90

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://lesswrong.com/posts/Y4YHTBziAscS5WPN7/epistemological-framing-for-ai-alignment-research
references Epistemological Framing For Ai Alignment Research concept

0.77

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://lesswrong.com/posts/Y4YHTBziAscS5WPN7/epistemological-framing-for-ai-alignment-research
mentioned AGI concept

0.35

evidence rules-v4
AGI
mentioned CONNOR concept

0.35

evidence rules-v4
CONNOR
mentioned LEAHY concept

0.35

evidence rules-v4
LEAHY

books 4

mentioned Does Not Hate You Superintelligence book

0.90

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://amazon.co.uk/Does-Not-Hate-You-Superintelligence/dp/1474608795
mentioned Society Mind Marvin Minsky book

0.90

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://amazon.co.uk/Society-Mind-Marvin-Minsky/dp/0671607405
references Does Not Hate You Superintelligence book

0.77

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://amazon.co.uk/Does-Not-Hate-You-Superintelligence/dp/1474608795
references Society Mind Marvin Minsky book

0.77

evidence rules-v4
Link in episode "#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY": https://amazon.co.uk/Society-Mind-Marvin-Minsky/dp/0671607405

companys 2

mentioned Technology company

0.50

evidence rules-v4
Feed category: Technology
discusses Technology company

0.40

evidence rules-v4
Feed category: Technology

podcasts 1

mentioned Machine Learning Street Talk (MLST podcast

0.95

evidence rules-v4
Feed title: Machine Learning Street Talk (MLST)

Episode description as stored

Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 In this podcast with the legendary Connor Leahy (CEO Conjecture) recorded in Dec 2022, we discuss various topics related to artificial intelligence (AI), including AI alignment, the success of ChatGPT, the potential threats of artificial general intelligence (AGI), and the challenges of balancing research and product development at his company, Conjecture. He emphasizes the importance of empathy, dehumanizing our thinking to avoid anthropomorphic biases, and the value of real-world experiences in learning and personal growth. The conversation also covers the Orthogonality Thesis, AI preferences, the mystery of mode collapse, and the paradox of AI alignment. Connor Leahy expresses concern about the rapid development of AI and the potential dangers it poses, especially as AI systems become more powerful and integrated into society. He argues that we need a better understanding of AI systems to ensure their safe and beneficial development. The discussion also touches on the concept of "futuristic whack-a-mole," where futurists predict potential AGI threats, and others try to come up with solutions for those specific scenarios. However, the problem lies in the fact that there could be many more scenarios that neither party can think of, especially when dealing with a system that's smarter than humans. https://www.linkedin.com/in/connor-j-leahy/https://twitter.com/NPCollapse Interviewer: Dr. Tim Scarfe (Innovation CTO @ XRAI Glass https://xrai.glass/) TOC: The success of ChatGPT and its impact on the AI field [00:00:00] Subjective experience [00:15:12] AI Architectural discussion including RLHF [00:18:04] The paradox of AI alignment and the future of AI in society [00:31:44] The impact of AI on society and politics [00:36:11] Future shock levels and the challenges of predicting the future [00:45:58] Long termism and existential risk [00:48:23] Consequentialism vs. deontology in rationalism [00:53:39] The Rationalist Community and its Challenges [01:07:37] AI Alignment and Conjecture [01:14:15] Orthogonality Thesis and AI Preferences [01:17:01] Challenges in AI Alignment [01:20:28] Mechanistic Interpretability in Neural Networks [01:24:54] Building Cleaner Neural Networks [01:31:36] Cognitive horizons / The problem with rapid AI development [01:34:52] Founding Conjecture and raising funds [01:39:36] Inefficiencies in the market and seizing opportunities [01:45:38] Charisma, authenticity, and leadership in startups [01:52:13] Autistic culture and empathy [01:55:26] Learning from real-world experiences [02:01:57] Technical empathy and transhumanism [02:07:18] Moral status and the limits of empathy [02:15:33] Anthropomorphic Thinking and Consequentialism [02:17:42] Conjecture: Balancing Research and Product Development [02:20:37] Epistemology Team at Conjecture [02:31:07] Interpretability and Deception in AGI [02:36:23] Futuristic whack-a-mole and predicting AGI threats [02:38:27] Refs: 1. OpenAI's ChatGPT: https://chat.openai.com/ 2. The Mystery of Mode Collapse (Article): https://www.lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-of-mode-collapse 3. The Rationalist Guide to the Galaxy https://www.amazon.co.uk/Does-Not-Hate-You-Superintelligence/dp/1474608795 5. Alfred Korzybski: https://en.wikipedia.org/wiki/Alfred_Korzybski 6. Instrumental Convergence: https://en.wikipedia.org/wiki/Instrumental_convergence 7. Orthogonality Thesis: https://en.wikipedia.org/wiki/Orthogonality_thesis 8. Brian Tomasik's Essays on Reducing Suffering: https://reducing-suffering.org/ 9. Epistemological Framing for AI Alignment Research: https://www.lesswrong.com/posts/Y4YHTBziAscS5WPN7/epistemological-framing-for-ai-alignment-research 10. How to Defeat Mind readers: https://www.alignmentforum.org/posts/EhAbh2pQoAXkm9yor/circumventing-interpretability-how-to-defeat-mind-readers 11. Society of mind: https://www.amazon.co.uk/Society-Mind-Marvin-Minsky/dp/0671607405