Eliezer Yudkowsky is a writer and AI-safety researcher who co-founded the Machine Intelligence Research Institute with Brian and Sabine Atkins. This profile examines his writing on artificial intelligence, rationality, and human values. — MIRI About.

Part 1: Artificial Intelligence and Existential Risk
- On Understanding AI: Yudkowsky warns against building increasingly capable AI while assuming that limited experience with weaker systems is enough to understand or control a much stronger one. — AI as a Positive and Negative Factor.
- On Intelligence and Goals: Yudkowsky argues that recognizing intelligence does not require assuming a particular terminal goal: an optimizer can pursue ends very different from human values. — Recognizing Intelligence.
- On AI Motivation: In Yudkowsky’s risk model, a powerful AI need not hate humans to harm them: human bodies contain matter that could serve a goal indifferent to us. — AI as a Positive and Negative Factor.
- On Fast Takeoff: Yudkowsky argues that recursive self-improvement could make AI capabilities rise quickly; he treats abrupt jumps as a risk-management possibility, not a settled physical law or precise timetable. — AI as a Positive and Negative Factor.
- On Anthropomorphism: Yudkowsky cautions that human-brain intuitions are a poor guide to the behavior of minds built on very different architectures. — Anthropomorphic Optimism.
- On AI Extinction Risk: Yudkowsky publicly argues that developing superhuman AI under current conditions could lead to human extinction; this is his forecast, not an established outcome. — TIME Essay on AI Risk.
- On AGI Timelines: Yudkowsky argues that uncertain timing is no reason to wait for a universally recognized warning before taking AGI risks seriously. — There’s No Fire Alarm for AGI.
- On Intelligence Limits: Yudkowsky cautions that familiar differences among humans may be small relative to the wider space of possible minds, so human intelligence should not be assumed to be a ceiling. — AI as a Positive and Negative Factor.
- On AI Containment: Yudkowsky’s AI-box thought experiment illustrates a possible persuasion risk when a capable AI can communicate with its human gatekeeper. He explicitly described the human role-play results as anecdotal, not proof that containment is impossible. — The AI-Box Experiment.
Part 2: The Alignment Problem
- On Proxy Goals: Yudkowsky argues that a training objective can be optimized successfully while failing to represent everything people actually care about, especially outside a tightly controlled environment. — Alexander–Yudkowsky AGI Goals Dialogue.
- On Value Loading: Yudkowsky argues that specifying everything humans want as an evaluable AI objective is much harder than writing a short instruction such as “do the right thing.” — Alexander–Yudkowsky AGI Goals Dialogue.
- On Fragility of Value: Yudkowsky argues that a future optimized for most human values could still lose what matters if it omits a crucial dimension such as boredom, genuine discovery, or subjective experience. — Value Is Fragile.
- On Coherent Extrapolated Volition: Yudkowsky proposed aiming an AI at a fuller account of what humanity would want after learning more and thinking more clearly, rather than treating current stated preferences as a complete target. — Coherent Extrapolated Volition.
- On Optimization Objectives: Yudkowsky argues that an AI optimized against a specified objective may find outcomes that satisfy that objective while violating the designer's unstated intentions. — Alexander–Yudkowsky AGI Goals Dialogue.
- On Alignment Difficulty: Yudkowsky argues that safety methods should remain sound even if an AI's capabilities jump sharply beyond those of its programmers, because post-development correction may be impossible. — AI as a Positive and Negative Factor.
- On Instrumental Convergence: Yudkowsky explains that an agent pursuing many different goals may still find control over resources and deception instrumentally useful; these tendencies depend on the agent and environment, not malice. — You Can’t Get More Paperclips That Way.
- On Stop Buttons: Yudkowsky warns that a control scheme cannot rely on humans retaining a reward button or unilateral power to modify a system after it becomes much more capable than they are. — AI as a Positive and Negative Factor.
- On the First Critical Attempt: Yudkowsky argues that an unsafe first attempt at building a sufficiently powerful AI might not leave humanity a second chance to learn from the failure. — TIME Essay on AI Risk.
Part 3: Rationality and Updating Beliefs
- On Changing Your Mind: Yudkowsky describes saying “oops” about an earlier belief as a welcome sign that he has learned, even when the correction is embarrassing. — My Bayesian Enlightenment.
- On Facing Reality: Yudkowsky republishes Eugene Gendlin's reminder that acknowledging what is true does not make the situation worse; it makes the existing situation available to face. — You Can Face Reality.
- On Two Kinds of Rationality: Yudkowsky distinguishes epistemic rationality—making beliefs more accurate—from instrumental rationality—choosing actions that better achieve one's values. — What Do We Mean by Rationality?.
- On Evidence: Yudkowsky argues that an observation is evidence for a hypothesis to the extent that we would expect it differently if the hypothesis were false. — What Is Evidence?.
- On Bayesian Updating: Yudkowsky explains Bayes's theorem as a way to revise a hypothesis's probability when a new observation bears on it, taking both the prior and the evidence into account. — An Intuitive Explanation of Bayes's Theorem.
- On Politics: Yudkowsky warns that political identification can make people treat arguments as soldiers to defend or attack, impairing truth-seeking. — Politics Is the Mind-Killer.
- On Surprise: Yudkowsky says a rationalist should be more troubled when observations contradict a belief than when fiction does, using confusion as a cue to improve the model. — Your Strength as a Rationalist.
- On Winning: For Yudkowsky, instrumental rationality is about actions that systematically improve outcomes one values, not a guarantee that every rational person wins every contest. — Rationality Is Systematized Winning.
- On Noticing Confusion: Yudkowsky treats confusion as evidence that a person's model needs work; the world itself is not obliged to match the model. — Your Strength as a Rationalist.
- On Cached Thoughts: Yudkowsky observes that people can reuse stored judgments in place of freshly considering a question, which is useful but can conceal an error. — Cached Thoughts.
Part 4: Cognitive Biases and Human Flaws
- On Scope Insensitivity: In an example Yudkowsky discusses, people's willingness to pay changed surprisingly little when the number of birds at risk rose from 2,000 to 20,000 to 200,000. — Scope Insensitivity.
- On the Halo Effect: Yudkowsky describes how a favorable impression in one area can color judgments about unrelated traits, encouraging overly simple views of people. — The Halo Effect.
- On Motivated Stopping: Yudkowsky warns that a welcome conclusion can tempt people to stop gathering evidence too soon, while an unwelcome one can make them demand excessive additional proof. — Motivated Stopping and Continuation.
- On the Planning Fallacy: Yudkowsky points to project-estimate examples in which people underpredict completion time; an outside-view comparison can improve the forecast. — Planning Fallacy.
- On the Illusion of Transparency: Yudkowsky argues that writers and speakers can overestimate how clearly others will recover their intended meaning. — Illusion of Transparency.
- On the Typical Mind Fallacy: Yudkowsky's example warns against assuming that a diet that worked for oneself is likely to work equally well for a friend. — Firewalling the Optimal from the Rational.
- On Fake Explanations: Yudkowsky's metal-plate example shows that an explanation fitting every possible result does not tell us what to expect next. — Fake Explanations.
- On the Affect Heuristic: Yudkowsky describes how a positive or negative feeling about an activity can influence separate judgments of its benefits and risks. — The Affect Heuristic.
Part 5: Epistemology and Truth-Seeking
- On Maps and Territory: Yudkowsky uses the map-territory metaphor to show why mental models necessarily compress reality and sometimes merge things that ought to be distinguished. — Fallacies of Compression.
- On Predictive Beliefs: Yudkowsky asks what experiences a claim leads us to anticipate; empirical claims should constrain expectations, though he does not deny that unobserved causes can be meaningful. — Making Beliefs Pay Rent.
- On Mysterious Answers: Yudkowsky argues that calling something ‘mysterious’ can merely label a gap in understanding rather than explain the phenomenon. — Mysterious Answers to Mysterious Questions.
- On Testing Beliefs Against Reality: Yudkowsky defines epistemic rationality as improving the correspondence between a person's model and the world, which means letting observations correct the model. — What Do We Mean by Rationality?.
- On Guessing the Teacher's Password: Yudkowsky uses classroom examples in which students repeat the approved words without connecting them to a prediction about what happens. — Guessing the Teacher's Password.
- On Anticipation: Yudkowsky asks what experiences a belief leads us to expect or rule out; a claim compatible with every outcome offers little predictive content. — Making Beliefs Pay Rent.
- On Curiosity: Yudkowsky argues that genuine curiosity makes a person willing to let new evidence shift a belief in either direction, rather than conducting a ritual inquiry that preserves the original view. — The Meditation on Curiosity.
- On Scientific and Personal Evidence: Yudkowsky distinguishes public scientific standards such as replicability from evidence that can rationally update an individual's belief, while cautioning against confusing private evidence with a public proof. — Scientific Evidence, Legal Evidence, Rational Evidence.
- On Truth: Truth is a narrower target than error. Yudkowsky uses that asymmetry to explain why identifying biases and improving truth-finding technique matter: there are many ways for a belief to miss reality. — What’s a Bias, Again?.
Part 6: Communication and Inferential Distances
- On Inferential Distances: Yudkowsky notes that an expert can underestimate how many intermediate concepts an audience needs before a conclusion becomes intelligible. — Expecting Short Inferential Distances.
- On Explaining Complex Ideas: Yudkowsky advises explainers to begin nearer their audience's starting point rather than assuming a deep idea will transfer intact in one leap. — Explainers Shoot High. Aim Low!.
- On Disagreement: Yudkowsky notes that Aumann's result concerns ideal Bayesian reasoners with shared priors and common knowledge of disagreement; persistent real-world disagreement is a reason to examine evidence and reasoning, not a license to assume all peers have identical information. — Against Modest Epistemology.
- On Understanding an Opponent: A useful test of understanding an opposing view is whether one can present its actual argument so faithfully that an advocate would not recognize the presentation as coming from an opponent. — Ideological Turing Test.
- On Apparent Agreement: Yudkowsky notes that two people can agree on a word such as ‘AI’ while imagining quite different observable outcomes; asking for concrete expectations exposes the difference. — Taboo Your Words.
- On Tabooing Terms: Yudkowsky recommends temporarily avoiding a disputed word and describing observable details or mechanisms instead, so disagreements about reality are not hidden inside labels. — Taboo Your Words.
- On Rationalization: Yudkowsky distinguishes examining evidence before reaching a conclusion from deciding the conclusion first and then selecting favorable evidence to defend it. — Rationalization.
- On Language and Thought: Yudkowsky argues that euphemism and abstract phrasing can obscure who did what, and that writers should attend to the impression their words create as well as their formal meaning. — Rationality and the English Language.
Part 7: Human Values and Fun Theory
- On Fun and Challenge: Yudkowsky argues that a desirable future should preserve worthwhile ongoing processes and challenge, not merely automate every task or supply an effortless feeling of winning. — High Challenge.
- On the Value of Life: Yudkowsky rejects the idea that mortality is needed to make life meaningful; he argues that friendship and learning matter in themselves, whether life is finite or extended. — The Meaning That Immortality Gives to Life.
- On Life Extension: Yudkowsky challenges the assumption that extending healthy life would strip it of meaning and asks what new forms of responsibility and learning it could permit. — The Meaning That Immortality Gives to Life.
- On Boredom: Yudkowsky treats a continuing stream of meaningful novelty as a human value that a good future should not accidentally erase. — In Praise of Boredom.
- On the Universe: In a scene from his novel, Yudkowsky has Harry answer a bleak account of an indifferent universe by saying that people themselves care about justice. This is a theme voiced by a fictional character, not a quotation from an interview or essay. — Harry Potter and the Methods of Rationality, Chapter 39.
- On Evolved Desires: Yudkowsky argues that evolution's selection for reproduction did not make humans consciously maximize genetic fitness; our desires, attachments, and pleasures can be ends for us in their own right. — Thou Art Godshatter.
- On Meaning Beyond Pleasure: Yudkowsky argues that value is not exhausted by pleasurable feelings: genuine discovery, freedom, relationships, and people’s own efforts matter to his view of a good life. — Not for the Sake of Happiness (Alone).
- On Good and Evil: In his novel, Yudkowsky has Dumbledore recall a villain who believed his own cause was righteous even while doing terrible harm. The scene illustrates how a self-justifying moral story can coexist with wrongdoing; it is a fictional example, not a claim that all wrongdoing has this form. — Harry Potter and the Methods of Rationality, Chapter 39.
- On Future Generations: Even if we will not personally see the distant future, Yudkowsky argues that we should leave the world in the best state we can for our children rather than treating the future as irrelevant beyond our own lifetimes. — The Meaning That Immortality Gives to Life.
- On Heroism: Heroism need not look like a famous superhero. Yudkowsky argues that an ordinary person who accepts real risk to save others may show greater courage than an invulnerable hero, while the aim should be to save lives rather than to appear virtuous. — Superhero Bias.
Part 8: Strategy, Survival, and the Future
- On Human Agency: Yudkowsky argues that human choices are part of the physical processes that determine the future; physics does not make those choices irrelevant. — Thou Art Physics.
- On Preparing Before a Fire Alarm: Yudkowsky argues that work on AGI safety cannot depend on waiting for a clear public alarm, because by the time such a signal arrives, the most important preparation may already be late. — There’s No Fire Alarm for AGI.
- On Competitive Pressure: In his Bankless interview, Yudkowsky argues that competition among AI firms can push development ahead even while alignment methods remain uncertain; he presents that as a governance difficulty, not a law of every coordination problem. — Bankless Interview Transcript.
- On a Global Pause: Yudkowsky's proposed response to severe AI risk is a worldwide halt to the training of large systems, backed by international monitoring and enforcement; this is his policy proposal, not an existing agreement. — TIME Essay on AI Risk.
- On Policy vs. Reality: Yudkowsky argues that a policy built around having most AIs behave safely would fail if one unsafe system could cause a catastrophe. Whether one system can do that depends on the technical design space, not a collective decision to assume otherwise. — Artificial Intelligence as a Positive and Negative Factor in Global Risk.
- On Technological Progress: In Yudkowsky's account, more computing power can make it easier to build AI without the understanding needed to make it safe; capability advances do not automatically solve alignment. — Artificial Intelligence as a Positive and Negative Factor in Global Risk.
- On Despair: Yudkowsky distinguishes pessimism about the odds from giving up: when the outlook is poor, he says efforts should still aim to improve humanity's chances of survival, even by a small amount. — MIRI Announces New Death With Dignity Strategy.