AI Distilled touches on the AI fear movement, but investigating its history, personalities, institutions, and internal disputes was never the purpose of the book. It is included simply because it is such a potent source of misinformation. The metastatic fear movement has come to dominate much of the discussion on artificial intelligence, and it has mounted a strong effort to advocate for legislation. Many people, even prominent ones, have bought into it.
I am loathed to devote much time to it, but misinformation should not be left to fester. Other sectors have developed fanatical fringe movements, such as water fluoridation and nuclear energy. One thing that we should realize from these examples is how important it is to aggressively defend reality and not concede any accordance to nonsense.
Fortunately, other people have already done substantial work. Below are starting points, technical critiques, investigations, firsthand accounts, and resources for people considering leaving these communities.
My advice is to start with AI Panic, Nirit Weiss’s collection of informaiton on the AI fear movement and belief system. She documents its history and its strange ecosystem, extending into Effective Altruism, 80,000 hours, various splinter groups, MIRI, Lesswrong, The Future of Life Institute and the other strange groups and organizations behind the belief that machines are soon to turn super-intelligent and take over the world.
Substantial Websites and Collections
| Resource | What you will find | Best use |
| AI Panic — Nirit Weiss-Blatt | An independent publication examining AI panic, media narratives, existential-risk advocacy, and the people and organizations promoting them. Its longer articles provide extensive links to original material. | Start here for the movement’s history, messaging, and institutional connections. The author is Nirit Weiss-Blatt. (aipanic.news) |
| AI Optimism | Essays and discussions challenging arguments for AI extinction, deceptive behavior, uncontrollability, and catastrophic extrapolations from evolution. Contributors include Quintin Pope and Taylor Belrose. | Start here for direct arguments against central doom premises, rather than reporting about personalities. Specific articles appear below. (optimists.ai) |
| AI as Normal Technology — Arvind Narayanan and Sayash Kapoor · Foundational essay | A substantial alternative to treating AI as an inevitably autonomous force. The foundational essay examines technological development through organizations, deployment constraints, institutions, and human decisions. | Particularly relevant to readers asking why impressive demonstrations do not automatically imply wholesale occupational replacement or uncontrollable technological acceleration. (knightcolumbia.org) |
| Reflective Altruism — David Thorstad | Research-based criticism of singularity arguments, existential-risk estimates, longtermist mathematics, power-seeking claims, and aspects of effective altruism. Organized into accessible series. | One of the strongest collections for sustained philosophical scrutiny. It is critical and reform-oriented, not an indiscriminate rejection of every charitable activity associated with EA. (reflectivealtruism.com) |
| SneerClub | A discussion community sharing criticism and satire of rationalism, associated ideologies, and technology-world grandiosity. | A source of leads and conversation—not a substitute for checking original documents, and not a professional support service. (awful.systems) |
The Incoherent Concept of Superintelligence
| Resource | Type / date | What it challenges |
| The Myth of a Superhuman AI — Kevin Kelly · WIRED version | Extended critical essay, 2017 | The strongest direct match for challenging “superintelligence” as ordinarily presented. Kelly attacks the idea of intelligence as a single ascending scale, the assumption that one system can excel at everything, and the leap from greater capability to effectively unlimited power. Explicitly addresses Bostrom’s framing. (kk.org) |
| Ethical Guidelines for a Superintelligence — Ernest Davis | Critical review of Superintelligence, October 23, 2014; PDF | Despite its neutral-sounding title, this is a pointed critique by a computer scientist. Davis questions treating intelligence as an unbounded, one-dimensional quantity and argues that important conclusions weaken when that idealization is relaxed. He also challenges Bostrom’s treatment of ethical behavior as exceptionally difficult. (cs.nyu.edu) |
| Superintelligence: The Idea That Eats Smart People — Maciej Cegłowski | Long-form skeptical talk, 2016 | A readable, frequently funny examination of the assumptions behind Bostrom-style catastrophe arguments: intelligence, power, self-improvement, and the social appeal of the narrative. . Its contemporary technology examples are historical. (idlewords.com) |
| Deconstructing Bostrom’s Argument for AI Doom — Taylor Belrose and Lance Bush | Recorded discussion, March 10, 2024 | Direct examination of Bostrom’s reasoning, including the relationship between intelligence, goals, and the claimed route to catastrophe. This is an interview/video resource, not a long written paper. (optimists.ai) |
| Against the Singularity Hypothesis — David Thorstad · Accessible article series | Peer-reviewed paper; online 2024, journal volume 2025 | Examines the growth assumptions required for recursive self-improvement to produce an intelligence explosion. Explicitly addresses defenses by Bostrom and David Chalmers and argues that they do not adequately support those assumptions. A particularly strong academic starting point. (link.springer.com) |
| The TESCREAL Bundle: Eugenics and the Promise of Utopia Through Artificial General Intelligence — Timnit Gebru and Émile P. Torres | Scholarly article, April 14, 2024 | Challenges undefined AGI as an engineering objective and argues for specific, testable applications instead. Also presents the authors’ historical interpretation of the ideological traditions surrounding AGI. Useful for both conceptual criticism and intellectual history; its historical thesis should be identified as the authors’ argument. (firstmonday.org) |
| The Thinkism Fallacy — Kevin Kelly | Short conceptual critique, 2021 | Attacks the assumption that enough intelligence can substitute for empirical investigation. Scientific progress also requires experiments, data, physical processes, and elapsed time. Directly relevant to “the superintelligence will simply figure it out.” (kk.org) |
| On the Measure of Intelligence — François Chollet | Research paper, 2019 | Separates performance on particular tasks from the capacity to acquire new skills and generalize. Useful against claims that high benchmark scores establish comprehensive intellectual superiority. Not a rejection of intelligence research: Chollet proposes a more rigorous way to measure it. (arxiv.org) |
| Why AI Is Harder Than We Think — Melanie Mitchell | Research essay, 2021 | Examines recurring misconceptions that lead people to overestimate progress toward broadly capable AI. Useful for understanding why success on selected tasks does not automatically transfer to the broader competence being promised. (arxiv.org) |
| AI Going Rogue? An Integrative Narrative Review of the Tacit Assumptions Underlying Existential AI-Risks — Jascha Bareis, Clemens Ackerl, and Reinhard Heil | Peer-reviewed review, February 9, 2026 | Examines 81 peer-reviewed papers. Identifies frequently unsupported growth claims, speculative catastrophic thresholds, anthropomorphic assumptions, and inadequate attention to infrastructure and social institutions. A substantial recent source for scrutinizing the literature’s foundations. (doi.org) |
Direct Examination of Claims of Self-Improvement, Deception, Power-Seeking
| Resource | What it examines | Why it belongs here |
| Counting Arguments Provide No Evidence for AI Doom — AI Optimism, 2024 | Arguments that dangerous goals or deceptive systems must predominate because there are supposedly many more ways to be dangerous than safe. | Challenges the move from counting imaginable configurations to predicting what training will actually produce. (optimists.ai) |
| AI Is Easy to Control — AI Optimism, 2023 | The authors’ case that controlling AI behavior is more tractable than pessimistic accounts suggest. | A direct counterargument to inevitability-of-uncontrollability claims. The title states the authors’ thesis; it is not a blanket certification of every deployed system. (optimists.ai) |
| Evolution Provides No Evidence for the Sharp Left Turn — Quintin Pope, 2023 | The use of human evolution as an analogy for AI suddenly developing capabilities and behavior incompatible with its training objectives. | Addresses a specific inferential bridge rather than merely expressing optimism. (optimists.ai) |
| My Objections to “We’re All Gonna Die with Eliezer Yudkowsky” — Quintin Pope, 2023 | A detailed response to Yudkowsky’s arguments in a widely circulated interview. | Useful for readers who want particular claims examined individually, rather than another generalized argument about whether AI is good or bad. (optimists.ai) |
| Instrumental Convergence and Power-Seeking — David Thorstad | The assumptions and interpretation of formal arguments about AI systems seeking power. | Helps distinguish what a mathematical result establishes within its model from what people claim it predicts about actual systems. (reflectivealtruism.com) |
| Revisiting the Shutdown Problem — David Thorstad | Arguments that preventing catastrophic behavior through shutdown presents an exceptionally difficult problem. | A focused challenge to the dismissal of “Why couldn’t we turn it off?” as though the answer were already settled. (reflectivealtruism.com) |
| Mistakes in the Moral Mathematics of Existential Risk — David Thorstad | Calculations used to assign extraordinary expected value to preventing existential catastrophe. | Relevant to the argument that hypothetical future populations should dominate present-day charitable or policy priorities. (reflectivealtruism.com) |
Investigations of the Surrounding Communities
| Resource | Coverage | Source / access notes |
| The Rationality Trap — Nirit Weiss-Blatt | A long, extensively linked examination of the rationalist movement, its aspirations, psychological practices, splinter groups, and relationships with AI catastrophe advocacy. | AI Panic, September 12, 2025. A critical synthesis and source-finding resource; follow its links to distinguish original reporting from commentary. (aipanic.news) |
| The Real-Life Consequences of Silicon Valley’s AI Obsession — Ellen Huet · Preserved excerpts and discussion | Accounts of social pressure, psychological consequences, alleged abuse, and the influence of catastrophe beliefs on participants’ lives. Includes people discussed elsewhere in this directory. | Bloomberg, March 7, 2023. Original may be paywalled. The linked discussion preserves substantial excerpts but is not the publisher’s full article. (web.archive.org) |
| TIME investigation: sexual harassment and abuse allegations within EA — Charlotte Alter | Reporting on women’s accounts of harassment, professional/social entanglement, and organizational responses. | February 3, 2023. Verified Internet Archive copy. Read as reporting on specific allegations and responses, not a claim about every participant. (web.archive.org) |
| The Rise of Silicon Valley’s Techno-Religion — Cade Metz | Reporting from Berkeley on Lighthaven, the rationalist community, and its influence within technology circles. | The New York Times, August 4, 2025. Verified Internet Archive copy. Useful for understanding the social infrastructure behind the ideas. (web.archive.org) |
| Why Are There So Many Rationalist Cults? — Ozy Brennan | An insider’s examination of dysfunctional, high-demand groups associated with the rationalist scene, based on interviews and personal familiarity. | Asterisk, 2025. Verified Internet Archive copy. Brennan explicitly identifies as a rationalist and distinguishes the broader community from its dysfunctional subgroups. (web.archive.org) |
| The Delirious, Violent, Impossible True Story of the Zizians — Evan Ratliff | A substantial investigation of the Zizian group, its development, conflicts, and connections to violent events. | WIRED, February 21, 2025. A dated investigation, not a current court-status tracker. The Zizians should not be treated as synonymous with all rationalists or effective altruists. (wired.com) |
Former Insiders, Whistle-Blowers and Others Who Escaped the Movement
| Person / resource | What the account contributes | What not to infer |
| Jessica Taylor: My Experience at and Around MIRI and CFAR — October 2021 | A detailed first-person account of work and social life around the Machine Intelligence Research Institute and Center for Applied Rationality, including psychological practices and difficult experiences. | Personal testimony, including interpretations and allegations—not proof that every claim about every organization is settled. It is not a blanket repudiation of all AI-risk research. (lesswrong.com) |
| Zoe Curzi: My Experience with Leverage Research — October 2021 | An extensive account of life inside Leverage Research, including alleged pressure, unusual psychological practices, and the difficulty of leaving. | Leverage is a particular organization. Its practices cannot automatically be attributed to everyone in adjacent communities. The account contains distressing material. (medium.com) |
| Qiaochu Yuan: 2021 account of disengagement · Longer 2023 account | Describes the effects of organizing life around an enormous world-saving obligation and the importance of reclaiming ordinary personal happiness and autonomy. | He explicitly retained fear of AGI in the 2021 account. Do not present him as a verified example of someone who left and conclusively rejected doom. The longer thread contains distressing personal material. (threadreaderapp.com) |
| Larissa Hesketh-Rowe’s comments, reported by Bloomberg · Accessible excerpts | The former Centre for Effective Altruism CEO questions how some AI work demonstrates results and describes status pressure and conformity. | Valuable criticism from a former organizational leader. Not a verified public renunciation of AI existential risk itself. (greaterwrong.com) |
Resources for those who may be looking to leave the movement
| Resource | What it offers | Important distinction |
| It’s OK to Leave EA — Julia Wise — January 28, 2022 | An explicit acknowledgment that people may need to leave, especially when participation is making them miserable or their social and professional lives have become overly entangled with the community. | Written from within EA, not by an anti-EA exit organization. That makes its recognition of the legitimacy of leaving particularly relevant. (forum.effectivealtruism.org) |
| Counterproductive EA Mental Health Advice—and What to Say Instead — Ada-Maaria Hyvärinen — October 10, 2022 | Critiques advice that evaluates people’s wellbeing primarily through their future usefulness to the cause. Discusses leaving as a real option rather than merely taking a productivity-restoring break. | Community commentary, not individualized clinical advice. Particularly relevant to people who feel guilty about having a life outside the mission. (forum.effectivealtruism.org) |
| Why I’m Leaving — June 21, 2024 | A participant explains leaving the EA label while continuing to donate and support concrete projects, objecting to the increasing prominence of speculative future-impact calculations. | The account is now attributed to a deleted user, so the author’s identity is not independently established. Useful as testimony, not an institutional investigation. (ea.greaterwrong.com) |
| International Cultic Studies Association: resources for survivors, families, and friends | Independent information and referral resources concerning high-control environments, coercive dynamics, leaving, and recovery. | Not AI-specific, and not a finding that an entire movement is a cult. ICSA states that it does not itself provide counseling, accommodation, legal services, or financial support; it connects people with relevant resources. (internationalculticstudies.org) |
If you are part of the Lesswrong community, Effective Altruism or “Rationalist” communities, it can be extremely hard to leave. These groups may have become part of your identity or offer you the feeling of community or importance. That’s entirely understandable. You do not need to leave all at once or announce your departure. The first step can be taking a break from interacting with others in the group for a week or so.
You are stronger on your own. You do not need anyone to tell you how to think, the proper way of looking at the world or validate that you are intelligent. You are most powerful when you are the one calling the shots.
Leave a Reply