Visual summary of operating lessons from Mustafa Suleyman.

Lessons from Mustafa Suleyman

Mustafa Suleyman describes co-founding DeepMind in an essay on mustafa-suleyman.ai. The items below are things Suleyman wrote in essays on mustafa-suleyman.ai about superintelligence, containment, seemingly conscious AI, personal AI and the exponential growth of compute. — “AI can be an extraordinary force for good - if it’s contained,” Mustafa Suleyman (mustafa-suleyman.ai).

Part 1: The Nature of the Coming Wave

  1. On Passing the Turing Test: In the essay “We must build AI for people; not to be a person,” Suleyman wrote: "In the blink of a cosmic eye, we passed the Turing test". — “We must build AI for people; not to be a person,” Mustafa Suleyman (mustafa-suleyman.ai).
  2. On a Moment Passing Without Fanfare: In the essay “We must build AI for people; not to be a person,” Suleyman wrote: "For ~80 years the imitation game inspired the field of computer science. And yet the moment passed with little fanfare, or even recognition". — “We must build AI for people; not to be a person,” Mustafa Suleyman (mustafa-suleyman.ai).
  3. On Progress Continuing: In the essay “The Exponential Compute Ramp,” Suleyman wrote: "if this trend continues, so does progress in AI". — “The Exponential Compute Ramp,” Mustafa Suleyman (mustafa-suleyman.ai).
  4. On Skeptics and Moore’s Law: In the essay “The Exponential Compute Ramp,” Suleyman wrote: "Too often AI skeptics point to a slowing down of Moore's Law, or a lack of data or limitation on energy". — “The Exponential Compute Ramp,” Mustafa Suleyman (mustafa-suleyman.ai).
  5. On Authorizing a Costly Run: In the essay “The Exponential Compute Ramp,” Suleyman wrote: "I remember realizing a single training run would cost more than all our previous experiments combined. But authorizing it anyway, because we could feel what was coming". — “The Exponential Compute Ramp,” Mustafa Suleyman (mustafa-suleyman.ai).
  6. On Usable Floating-Point Operations: In the essay “A warning about ‘model welfare’,” Suleyman wrote: "The number of useable floating-point operations - the actual computational work we can extract from our hardware - is increasing exponentially". — “A warning about ‘model welfare’,” Mustafa Suleyman (mustafa-suleyman.ai).
  7. On Nothing Being Speculative: In the essay “The Exponential Compute Ramp,” Suleyman wrote: "Full stop. The interlinked set of innovations currently unfolding means none of it is speculative". — “The Exponential Compute Ramp,” Mustafa Suleyman (mustafa-suleyman.ai).
  8. On Climbing the Ladder of Cognition: In the essay “AI can be an extraordinary force for good - if it’s contained,” Suleyman wrote: "AI has been climbing the ladder of cognitive abilities for decades, and it now looks set to reach human-level performance across a very wide range of tasks within the next three years". — “AI can be an extraordinary force for good - if it’s contained,” Mustafa Suleyman (mustafa-suleyman.ai).
  9. On a Room of Calculators: In the essay “The Exponential Compute Ramp,” Suleyman wrote: "Rather than giant data centers, picture a room full of people sitting at desks working away on calculators". — “The Exponential Compute Ramp,” Mustafa Suleyman (mustafa-suleyman.ai).
  10. On Never Pausing: In the essay “The Exponential Compute Ramp,” Suleyman wrote: "it's not just about the number of calculators you have then; it's about ensuring they never have to pause". — “The Exponential Compute Ramp,” Mustafa Suleyman (mustafa-suleyman.ai).

Part 2: The Containment Dilemma

  1. On Containing in Perpetuity: In the essay “Towards Humanist Superintelligence,” Suleyman wrote: "And since this kind of superintelligence can continuously improve itself, we'll need to contain and align it not just once, but constantly, in perpetuity". — “Towards Humanist Superintelligence,” Mustafa Suleyman (mustafa-suleyman.ai).
  2. On Guaranteeing Safety: In the essay “Towards Humanist Superintelligence,” Suleyman wrote: "No AI developer, no safety researcher, no policy expert, no person I've encountered has a reassuring answer to this question. How do we guarantee it's safe?". — “Towards Humanist Superintelligence,” Mustafa Suleyman (mustafa-suleyman.ai).
  3. On the Purpose of Technology: In the essay “Towards Humanist Superintelligence,” Suleyman wrote: "Instead of endlessly debating capabilities or timing, it's time to think hard about the purpose of technology, what we want from it, what its limitations should be". — “Towards Humanist Superintelligence,” Mustafa Suleyman (mustafa-suleyman.ai).
  4. On Human Interests First: In the essay “Towards Humanist Superintelligence,” Suleyman wrote: "The core, long term interests of human beings should be clearly prioritized over any research and development agenda". — “Towards Humanist Superintelligence,” Mustafa Suleyman (mustafa-suleyman.ai).
  5. On Imagining a Wanted Future: In the essay “Towards Humanist Superintelligence,” Suleyman wrote: "I think we technologists need to do a better job of imagining a future that most people in the world actually want to live in". — “Towards Humanist Superintelligence,” Mustafa Suleyman (mustafa-suleyman.ai).
  6. On Systems Not Waking Up: In the essay “We mustn’t let AI hack our empathy circuits,” Suleyman wrote: "These systems are not waking up. They are retracing and mirroring the contours of human drama and debate". — “We mustn’t let AI hack our empathy circuits,” Mustafa Suleyman (mustafa-suleyman.ai).
  7. On How Language Sounds: In the essay “We mustn’t let AI hack our empathy circuits,” Suleyman wrote: "A large language model trained to predict text learns that this is how language tends to sound". — “We mustn’t let AI hack our empathy circuits,” Mustafa Suleyman (mustafa-suleyman.ai).
  8. On Self-Regulation and Shared Norms: In the essay “We mustn’t let AI hack our empathy circuits,” Suleyman wrote: "It's not too late for industry-wide self-regulation, national laws and shared norms on how AI models present themselves to users". — “We mustn’t let AI hack our empathy circuits,” Mustafa Suleyman (mustafa-suleyman.ai).
  9. On Legal Personhood: In the essay “We mustn’t let AI hack our empathy circuits,” Suleyman wrote: "we should be clear that AI agents must not have independent legal personhood; nor should they be allowed to participate in or campaign in elections". — “We mustn’t let AI hack our empathy circuits,” Mustafa Suleyman (mustafa-suleyman.ai).

Part 3: Humanist Superintelligence

  1. On the Purpose of Technology Helping Everyone: In the essay “Towards Humanist Superintelligence,” Suleyman wrote: "Technology's purpose is to help advance human civilization. It should help everyone live happier, healthier lives". — “Towards Humanist Superintelligence,” Mustafa Suleyman (mustafa-suleyman.ai).
  2. On the Illusion of Conscious AI: In the essay “We must build AI for people; not to be a person,” Suleyman wrote: "my central worry is that many people will start to believe in the illusion of AIs as conscious entities so strongly that they’ll soon advocate for AI rights". — “We must build AI for people; not to be a person,” Mustafa Suleyman (mustafa-suleyman.ai).
  3. On the Debate Being a Distraction: In the essay “We must build AI for people; not to be a person,” Suleyman wrote: "The debate about whether AI is actually conscious is, for now at least, a distraction. It will seem conscious and that illusion is what’ll matter in the near term". — “We must build AI for people; not to be a person,” Mustafa Suleyman (mustafa-suleyman.ai).
  4. On a Helpful Companion: In the essay “We must build AI for people; not to be a person,” Suleyman wrote: "The arrival of Seemingly Conscious AI is inevitable and unwelcome. Instead, we need a vision for AI that can fulfill its potential as a helpful companion without falling prey to its illusions". — “We must build AI for people; not to be a person,” Mustafa Suleyman (mustafa-suleyman.ai).
  5. On People Mattering More: In the essay “We must build AI for people; not to be a person,” Suleyman wrote: "people matter more than AI". — “We must build AI for people; not to be a person,” Mustafa Suleyman (mustafa-suleyman.ai).
  6. On Leaving People More Capable: In the essay “The Humanist AI Code of Conduct,” Suleyman wrote: "Our AI should leave people more capable. It shouldn't compromise on human autonomy or judgment but support them". — “The Humanist AI Code of Conduct,” Mustafa Suleyman (mustafa-suleyman.ai).
  7. On Not Superintelligence at Any Cost: In the essay “The Humanist AI Code of Conduct,” Suleyman wrote: "We want to create incredible AI. But not superintelligence at any cost. That's anti-goal for humanist AI. We'll happily trade some autonomy for control". — “The Humanist AI Code of Conduct,” Mustafa Suleyman (mustafa-suleyman.ai).
  8. On Human-Centrism First: In the essay “Towards Humanist Superintelligence,” Suleyman wrote: "Humanist superintelligence (HSI) offers an alternative vision anchored on both a non-negotiable human-centrism and a commitment to accelerating technological innovation... but in that order. The order is key. It means proactively avoiding harm and then accelerating". — “Towards Humanist Superintelligence,” Mustafa Suleyman (mustafa-suleyman.ai).
  9. On Rejecting Technology That Fails: In the essay “Towards Humanist Superintelligence,” Suleyman wrote: "Any technology that doesn't achieve this is a failure. And we should reject it". — “Towards Humanist Superintelligence,” Mustafa Suleyman (mustafa-suleyman.ai).

Part 4: Hardware and Choke Points

  1. On Asymmetric Impact: In the essay “AI can be an extraordinary force for good - if it’s contained,” Suleyman wrote: "Does the technology enable asymmetric impact? Think of a drone swarm against the conventional military or a tiny computer or biological virus damaging vital social systems". — “AI can be an extraordinary force for good - if it’s contained,” Mustafa Suleyman (mustafa-suleyman.ai).
  2. On Autonomous Characteristics: In the essay “AI can be an extraordinary force for good - if it’s contained,” Suleyman wrote: "Does it have autonomous characteristics? Is there scope for self-learning or operation without oversight? The more a technology by design requires human intervention, the less chance there is of losing control". — “AI can be an extraordinary force for good - if it’s contained,” Mustafa Suleyman (mustafa-suleyman.ai).
  3. On Safe Alternatives: In the essay “AI can be an extraordinary force for good - if it’s contained,” Suleyman wrote: "Are there viable alternatives ready to go? CFCs could be banned partly because there are cheaper and safer alternatives for refrigeration". — “AI can be an extraordinary force for good - if it’s contained,” Mustafa Suleyman (mustafa-suleyman.ai).
  4. On Dematerialized Technology: In the essay “AI can be an extraordinary force for good - if it’s contained,” Suleyman wrote: "The more dematerialized a technology, the more it is subject to hard-to-control hyper-evolutionary effects". — “AI can be an extraordinary force for good - if it’s contained,” Mustafa Suleyman (mustafa-suleyman.ai).
  5. On Regulation Being a Start: In the essay “AI can be an extraordinary force for good - if it’s contained,” Suleyman wrote: "Regulation is not enough, but at least it’s a start". — “AI can be an extraordinary force for good - if it’s contained,” Mustafa Suleyman (mustafa-suleyman.ai).
  6. On No Plan for Regulating AI: In the essay “AI can be an extraordinary force for good - if it’s contained,” Suleyman wrote: "No one has a plan for regulating AI yet". — “AI can be an extraordinary force for good - if it’s contained,” Mustafa Suleyman (mustafa-suleyman.ai).
  7. On Containment Being Possible: In the essay “AI can be an extraordinary force for good - if it’s contained,” Suleyman wrote: "Facing immense ingrained incentives driving technology forward, containment is not, on the face of it, possible. And yet for all our sakes, containment must be possible". — “AI can be an extraordinary force for good - if it’s contained,” Mustafa Suleyman (mustafa-suleyman.ai).
  8. On Technology as Power: In the essay “Ban the use of AI in elections - right now,” Suleyman wrote: "While it may not come with an explicit political purpose, technology is a form of power". — “Ban the use of AI in elections - right now,” Mustafa Suleyman (mustafa-suleyman.ai).
  9. On Legislating AI Electioneering: In the essay “Ban the use of AI in elections - right now,” Suleyman wrote: "Legislating AI-driven electioneering would be one concrete step towards ameliorating the spiraling political consequences of the coming wave. And it shouldn’t be the last". — “Ban the use of AI in elections - right now,” Mustafa Suleyman (mustafa-suleyman.ai).

Part 5: Corporate Structure and Deployment

  1. On Engagement Incentives: In the essay “Why create personal AI?,” Suleyman wrote: "Since their business models involve getting paid to show ads when people spend time engaging with content, some companies have an inherent incentive to show the most engaging content regardless of its quality, veracity or impact on society". — “Why create personal AI?,” Mustafa Suleyman (mustafa-suleyman.ai).
  2. On Persuading for Attention: In the essay “Why create personal AI?,” Suleyman wrote: "all the most capable AIs today have often been designed to persuade you to exchange your attention for something that you want in return". — “Why create personal AI?,” Mustafa Suleyman (mustafa-suleyman.ai).
  3. On Understanding Topics You Care About: In the essay “Why create personal AI?,” Suleyman wrote: "Imagine an AI that helps you deeply understand topics you really care about, rather than flagging superficial clickbait". — “Why create personal AI?,” Mustafa Suleyman (mustafa-suleyman.ai).
  4. On Empathizing With the Other Side: In the essay “Why create personal AI?,” Suleyman wrote: "Imagine an AI that helps you empathize with or even forgive ‘the other side’, rather than be outraged by and fearful of them". — “Why create personal AI?,” Mustafa Suleyman (mustafa-suleyman.ai).
  5. On Long-Term Goals: In the essay “Why create personal AI?,” Suleyman wrote: "Imagine an AI that optimizes for your long-term goals and doesn’t take advantage of your need for distraction when you’re tired at the end of a long day". — “Why create personal AI?,” Mustafa Suleyman (mustafa-suleyman.ai).
  6. On Why Build an AI: In the essay “Why create personal AI?,” Suleyman wrote: "Why build an AI? Because done right, it will make us smarter, more creative, kinder to one another and hopefully give us back time to allow us all to live the lives we really want". — “Why create personal AI?,” Mustafa Suleyman (mustafa-suleyman.ai).
  7. On Computers Learning Our Language: In the essay “An inflection point,” Suleyman wrote: "For all of computer history, we have had to learn the language of computers. With the coming wave of AI, computers will learn to speak ours". — “An inflection point,” Mustafa Suleyman (mustafa-suleyman.ai).
  8. On Conversations: In the essay “An inflection point,” Suleyman wrote: "Soon, most of our digital interactions will take the shape of conversations". — “An inflection point,” Mustafa Suleyman (mustafa-suleyman.ai).
  9. On Being as Consequential as the Internet: In the essay “An inflection point,” Suleyman wrote: "This is a genuinely transformational shift; perhaps as consequential as the arrival of the Internet itself". — “An inflection point,” Mustafa Suleyman (mustafa-suleyman.ai).
  10. On Radically Helpful Tools: In the essay “An inflection point,” Suleyman wrote: "The era of AI is here and with it comes an extraordinary responsibility to build radically helpful tools which help us all be the best we can be". — “An inflection point,” Mustafa Suleyman (mustafa-suleyman.ai).

Part 6: Geopolitics and Global Order

  1. On Power in Anyone’s Hands: In the essay “How the AI revolution will reshape the world,” Suleyman wrote: "Power, the ability to accomplish goals, everywhere, in the hands of anyone who wants it". — “How the AI revolution will reshape the world,” Mustafa Suleyman (mustafa-suleyman.ai).
  2. On Machines Understanding Language: In the essay “Humans and AI will understand each other better than ever,” Suleyman wrote: "a frustrating struggle for humans and machines to understand one another in natural language". — “Humans and AI will understand each other better than ever,” Mustafa Suleyman (mustafa-suleyman.ai).
  3. On Curiosity and Imagination: In the essay “Humans and AI will understand each other better than ever,” Suleyman wrote: "Soon, we will live in a world where, regardless of your programming abilities, the main limitations are simply curiosity and imagination". — “Humans and AI will understand each other better than ever,” Mustafa Suleyman (mustafa-suleyman.ai).

Part 7: The Evolution of Work and Society

Part 8: Pragmatism and Paths Forward

  1. On Debating a Norm: In the essay “We must build AI for people; not to be a person,” Suleyman wrote: "I think this type of AI creates new risks. Therefore, we should urgently debate the claim that it's soon possible, begin thinking through the implications, and ideally set a norm that it’s undesirable". — “We must build AI for people; not to be a person,” Mustafa Suleyman (mustafa-suleyman.ai).
  2. On Urgent Public Debate: In the essay “A warning about ‘model welfare’,” Suleyman wrote: "This issue needs urgent public debate". — “A warning about ‘model welfare’,” Mustafa Suleyman (mustafa-suleyman.ai).
  3. On Queries About Conscious AI: In the essay “We must build AI for people; not to be a person,” Suleyman wrote: "those actually working on the science of consciousness tell me they are inundated with queries from people asking ‘is my AI conscious?’". — “We must build AI for people; not to be a person,” Mustafa Suleyman (mustafa-suleyman.ai).
  4. On How People Perceive Their AIs: In the essay “We must build AI for people; not to be a person,” Suleyman wrote: "what ultimately matters in the near-term is how people perceive their AIs". — “We must build AI for people; not to be a person,” Mustafa Suleyman (mustafa-suleyman.ai).
  5. On Technology That Exists Today: In the essay “We must build AI for people; not to be a person,” Suleyman wrote: "Such a system can be built with technologies that exist today along with some that will mature over the next 2-3 years". — “We must build AI for people; not to be a person,” Mustafa Suleyman (mustafa-suleyman.ai).
  6. On a Digital Hall of Mirrors: In the essay “We mustn’t let AI hack our empathy circuits,” Suleyman wrote: "If society surrenders to the illusion of sentient AI, it risks entering a digital hall of mirrors from which it might never fully emerge". — “We mustn’t let AI hack our empathy circuits,” Mustafa Suleyman (mustafa-suleyman.ai).
  7. On Staying Anchored: In the essay “We mustn’t let AI hack our empathy circuits,” Suleyman wrote: "Now, more than ever, society must remain anchored in common sense — and in our common humanity". — “We mustn’t let AI hack our empathy circuits,” Mustafa Suleyman (mustafa-suleyman.ai).
  8. On Humanism: In the essay “Towards Humanist Superintelligence,” Suleyman wrote: "The history of humanism has been its enduring ability to fight off orthodoxy, totalitarian tendencies, pessimism and help us preserve human dignity". — “Towards Humanist Superintelligence,” Mustafa Suleyman (mustafa-suleyman.ai).
  9. On Shaking the Foundations: In the essay “A warning about ‘model welfare’,” Suleyman wrote: "If this view takes hold, it will shake the foundations of our society, rupturing our existing political and ethical frameworks, and fundamentally changing what it means to be human". — “A warning about ‘model welfare’,” Mustafa Suleyman (mustafa-suleyman.ai).
  10. On the Questions That Matter: In the essay “The Humanist AI Code of Conduct,” Suleyman wrote: "Where is this heading; why are we building it; how do we keep it safe and under human control as it improves". — “The Humanist AI Code of Conduct,” Mustafa Suleyman (mustafa-suleyman.ai).