Visual summary of operating lessons from Jeff Clune.

Lessons from Jeff Clune

Jeff Clune is a professor of computer science at the University of British Columbia and, in the words of the AutoML Podcast's episode description, "previously a Senior Research Scientist at Uber AI, and more recently a Research Team Leader at OpenAI" and "currently a Faculty Member at the Vector Institute." His work centers on open-endedness and what he calls AI-generating algorithms. The items below are words from his AI-GAs paper and interviews. — The AutoML Podcast: The Paths to AGI with Jeff Clune (Sept 2022).

Part 1: Open-Endedness and the Search for Novelty

  1. On static goals: On the TalkRL podcast, he said that "if you have a goal, that probably prevents you from doing things that are really interesting because you get stuck on a local optima or in a dead end". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript.
  2. On Standing on the Shoulders of Human Datasets: On TalkRL, of the lessons of recent machine learning, he said that "AI sees further by standing on the shoulders of giant human datasets to borrow from Newton". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript
  3. On the open-endedness quest: On the TalkRL podcast, he posed the quest this way: "Could we create a system that would forever, as long as we run it, need to innovate, delight, and surprise us in the same way as human evolution and human culture?". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript.
  4. On Learning as Much of the Solution as Possible: On TalkRL, he said the philosophy behind his work is that "we should be learning as much of the solution as possible". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript
  5. On the POET algorithm: In a 2023 MIT talk, he described how POET works: "we're going to generate new learning environments and add them to this growing set" and "if they're not too easy and not too hard for the current agents". — Jeff Clune: Improving Deep Reinforcement Learning via Quality Diversity, Open-Ended and AI-Generating Algorithms (MIT CBMM talk, 2023).
  6. On Generating Training Environments Automatically: On TalkRL, of the POET paper, he said it put on the map "the idea that we should be automatically generating training environments for RL agents rather than hand designing them". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript
  7. On surprise and discovery: On the TalkRL podcast, of an open-ended algorithm, he said "You keep running it and it just keeps surprising you. It keeps innovating, it keeps creating forever". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript.
  8. On Rewarding Exploration: On TalkRL, of the thinking behind Go-Explore, he said: "I should have my own reward for exploring, for going to new states, doing new things, learning how the world works". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript

Part 2: AI-Generating Algorithms (AI-GAs)

  1. On the manual AI approach: In the abstract of his AI-GAs paper, he wrote "The dominant approach in the machine learning community is to attempt to discover each of the pieces that might be required for intelligence, with the implicit assumption that at some point in the future some group will complete the Herculean task of figuring out how to combine all of those pieces into an extremely complex machine. I call this the “manual AI approach.”". — Jeff Clune: AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (arXiv, 2019).
  2. On Returning Before Exploring: On TalkRL, of Go-Explore, he said that "we should first return, then we should explore". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript
  3. On the three pillars: In the abstract of his AI-GAs paper, he wrote "Three Pillars are essential for the approach: (1) meta-learning architectures, (2) meta-learning the learning algorithms themselves, and (3) generating effective learning environments". — Jeff Clune: AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (arXiv, 2019).
  4. On the Mystery of Where Evolution's Creations Came From: On TalkRL, he said that "one of the most profound mysteries in the world is where we came from and also where all of the amazing creations of evolution came from". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript
  5. On scientific automation: In his AI-GAs paper, he wrote that "the stories of science fiction of a scientist starting an experiment, going to sleep, and awakening to discover they have created sentient life are far more conceivable in the AI-GA research paradigm than in the manual path". — Jeff Clune: AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (arXiv, 2019).
  6. On Not Being Interested in White Noise: On TalkRL, he said: "If I just run the system and it's generating more of the same or it's just generating white noise forever, I'm not interested". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript
  7. On learned versus hand-designed solutions: In the abstract of his AI-GAs paper, he wrote "It is based on the clear trend from the history of machine learning that hand-designed solutions eventually are replaced by more effective, learned solutions". — Jeff Clune: AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (arXiv, 2019).
  8. On Bringing Human Judgment to Bear: On TalkRL, he said that "we should bring our human judgment to bear in this field of research, even though scientists don't love hearing that". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript
  9. On bootstrapping intelligence: In the abstract of his AI-GAs paper, he wrote "The idea is to create an AI-generating algorithm (AI-GA), which itself automatically learns how to produce general AI". — Jeff Clune: AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (arXiv, 2019).
  10. On Human Data as an Accelerant: On TalkRL, of models like GPT, he said that "training on that human data, I think it's a fantastic accelerant". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript
  11. On the two paths to general AI: In the abstract of his AI-GAs paper, he wrote that "either the manual or AI-GA approach could be the first to lead to general AI, and that both are worthwhile scientific endeavors irrespective of which is the fastest path". — Jeff Clune: AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (arXiv, 2019).
  12. On Principles That Pay Off Across Problems: On TalkRL, he said that "these principles that we get excited about in one context end up paying off to solve other really hard challenges in machine learning again and again and again". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript

Part 3: Evolutionary Robotics and Biological Inspiration

  1. On nature's algorithm: On the TalkRL podcast, he said that "evolution is 3.5 billion years and running and still surprising us. Why can't we do the same thing with a computer algorithm?". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript.
  2. On recreating open-ended invention in a computer: On the TalkRL podcast, he said of open-ended evolution and culture, "A quest that I've had is, what are the key ingredients that would allow us to create a process like that inside of a computer?". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript.

Part 4: Quality-Diversity and Exploration

  1. On escaping local optima: In his AI-GAs paper, on quality-diversity algorithms, he wrote that "what we want are algorithms that create a diverse set of solutions where each of those solutions is as high-performing as possible for that type". — Jeff Clune: AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (arXiv, 2019).
  2. On abandoning objectives: On the TalkRL podcast, he said "Here's a line from Ken and Joel's book, to achieve your greatest ambitions, you must abandon them". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript.
  3. On stepping stones: On the TalkRL podcast, he said "Just go out and collect a huge diversity of high quality stuff and stepping stones and that will ultimately unlock tremendous progress". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript.
  4. On not setting objectives: On the TalkRL podcast, he said that "if you myopically start giving IQ tests to bacteria, you will never get human beings". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript
  5. On objectives that block discovery: On the TalkRL podcast, of a king who funds only machines that produce more compute, he said "you would never invent the modern computer because to invent the computer you had to have been working on vacuum tubes and electricity, which were things that were not invented because they had immediate benefits for computation". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript.

Part 5: Artificial General Intelligence (AGI) and Ambition

  1. On scientific ambition: On the TalkRL podcast, he said "I love to swing for the fences and I'm most interested by ambitious ideas that might not even be ready for current technology, but you might have to wait a few years or even decades to fully be realized". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript.
  2. On the historic quest: In the abstract of his AI-GAs paper, he wrote "Perhaps the most ambitious scientific quest in human history is the creation of general artificial intelligence, which roughly means AI that is as smart or smarter than humans". — Jeff Clune: AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (arXiv, 2019).
  3. On shifting research investment: In the abstract of his AI-GAs paper, he wrote "I argue that our community should shift a substantial amount of its research investment to the AI-GA approach". — Jeff Clune: AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (arXiv, 2019)

Part 6: The Mechanics of Meta-Learning and Neural Networks

  1. On catastrophic forgetting: In his AI-GAs paper, he described "catastrophic forgetting, a phenomenon wherein neural networks are unable to continuously learn because they learn each newly encountered task by overwriting their knowledge of how to solve all previously learned tasks". — Jeff Clune: AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (arXiv, 2019).
  2. On neuromodulation: In his AI-GAs paper, he wrote "Neuromodulation can help because some neurons in the network can detect which task is currently being performed, and those neurons can turn learning on in the part of the network that performs that task and turn learning off everywhere else". — Jeff Clune: AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (arXiv, 2019).

Part 7: AI Safety, Alignment, and Superintelligence Risks

  1. On the danger of AI-GA research: In his AI-GAs paper, he wrote that "an AI-GA research project with the right ingredients might, and the first scientist to create an AI-GA may not know they have finally stumbled upon the key ingredients until afterwards. That makes AI-GA research more dangerous". — Jeff Clune: AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (arXiv, 2019).
  2. On the responsibility of researchers: Closing the TalkRL podcast, he said "I just think we want to be really conscious of the downstream effects of what we build. I think we want to make sure that we build things as safely as possible and to help humanity as much as possible". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript.
  3. On a Hippocratic oath for the field: Closing the TalkRL podcast, he said "I think that everybody in our field should do the equivalent of taking like a Hippocratic oath to try to do good things with this and not do harm". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript
  4. On humility about downstream effects: Closing the TalkRL podcast, he said "I think we should just be humble about unknown unknowns and potential downstream effects". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript
  5. On value alignment in AI-GAs: In his AI-GAs paper, he wrote that "a major concern with the AI-GA path is that the values of an AI produced by the system are less likely to be aligned with our own" and "One has less control when one is creating AI-GAs than when one is manually building an AI machine piece by piece". — Jeff Clune: AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (arXiv, 2019).
  6. On society's deliberation: On the TalkRL podcast, he said "Society needs to be really deeply thinking about whether we should do this, how fast we should do this, how do we do this safely, and what are the consequences of doing this". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript.
  7. On the pace of progress: On the TalkRL podcast, of progress toward AGI, he said "it's all happening and it's happening at a ridiculously fast pace". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript.
  8. On defining AGI: On the TalkRL podcast, he defined it as something like AI doing the work, and said of the timing "AI doing 50% of the work that got paid for in 2023" and "I say 2030". — TalkRL: The Reinforcement Learning Podcast, Jeff Clune (March 2023), transcript

Part 8: The Philosophy of Scientific Discovery and Automation

  1. On automating science: On Machine Learning Street Talk, describing The AI Scientist and the open-endedness philosophy, he said "you would publish that paper to an internal archive" and "build on its own discoveries forever". — Machine Learning Street Talk: Don't invent faster horses, Prof. Jeff Clune (Jan 2025).