Visual summary of operating lessons from Nathan Lambert.

Lessons from Nathan Lambert

The items below are things Nathan Lambert said on podcasts about open models, post-training and the AI frontier. — Nathan Lambert on The a16z Show, Can Open Models Keep Up With the AI Frontier? (October 2026).

On The Information Bottleneck, Lambert said: "The actual training recipe, which is like a long sequence of complex buttons that you have to push and draw on proprietary data and compute". — Nathan Lambert on The Information Bottleneck, Inside Post-Training and the Open Model Fight (August 2026).

On The Information Bottleneck, Lambert said: "It's really like people need to understand that like just researcher skill alignment and org chart can have like multiples on the output of the model". — Nathan Lambert on The Information Bottleneck, Inside Post-Training and the Open Model Fight (August 2026).

On The Information Bottleneck, Lambert said: "It's just that you need the fully open things to do a lot of types of research that is really deep in the understanding how the model and training works". — Nathan Lambert on The Information Bottleneck, Inside Post-Training and the Open Model Fight (August 2026).

On The a16z Show, Lambert said: "When I think about model progress, it comes down to different forms of capital, and that is compute data and talent". — Nathan Lambert on The a16z Show, Can Open Models Keep Up With the AI Frontier? (October 2026).

On The a16z Show, Lambert said: "There's just like all these really weird dynamics, but fundamentally, like AI progress is about data and about writing better software to serve your models, and those are very different worlds". — Nathan Lambert on The a16z Show, Can Open Models Keep Up With the AI Frontier? (October 2026).

On The a16z Show, Lambert said: "I would say that the method is you buy in mass, and then you figure out what works, and it's not a hundred percent hit rate". — Nathan Lambert on The a16z Show, Can Open Models Keep Up With the AI Frontier? (October 2026).

In the Interconnects essay on post-training in 2025, Lambert wrote: "post-training alone is nowhere near a complete enough lens or taxonomy to study training reasoning language models". — Nathan Lambert, The state of post-training in 2025, Interconnects (January 8, 2025).

In the Interconnects essay on post-training in 2025, Lambert wrote: "Predicting the next token accurately is not something that a user cares about — correct answers and how the answer is presented are". — Nathan Lambert, The state of post-training in 2025, Interconnects (January 8, 2025).

In the Interconnects essay on post-training in 2025, Lambert wrote: "distilling from strong models is a fundamental part of successful post-training". — Nathan Lambert, The state of post-training in 2025, Interconnects (January 8, 2025).

In the Interconnects essay on post-training in 2025, Lambert wrote: "Doing post-training well and having mastery of the techniques seems crucial to making progress on reasoning models like o1". — Nathan Lambert, The state of post-training in 2025, Interconnects (January 8, 2025).

In the Interconnects essay An unexpected RL Renaissance, Lambert wrote: "RLHF was necessary, but not sufficient for ChatGPT". — Nathan Lambert, An unexpected RL Renaissance, Interconnects (February 13, 2025).

In the Interconnects essay An unexpected RL Renaissance, Lambert wrote: "There is just so much low-hanging fruit for improving models with RL". — Nathan Lambert, An unexpected RL Renaissance, Interconnects (February 13, 2025).

In the Interconnects essay How RLHF actually works, Lambert wrote: "I'm narrowing my view of RLHF as a style transfer: RLHF is a topic filter and adds a gentle bug squashing". — Nathan Lambert, How RLHF actually works, Interconnects (June 21, 2023).

In the Interconnects essay How RLHF actually works, Lambert wrote: "This strong need for matching distributions is my best guess as to why we haven't seen that many preference models / RLHF'd models released". — Nathan Lambert, How RLHF actually works, Interconnects (June 21, 2023).

In the Interconnects essay My bets on open models, mid-2026, Lambert wrote: "Only real AI revenue begets more investment". — Nathan Lambert, My bets on open models, mid-2026, Interconnects (April 15, 2026).

In the Interconnects essay My bets on open models, mid-2026, Lambert wrote: "Closed models have certain hard-to-measure qualities that are not well captured in current or past benchmarks". — Nathan Lambert, My bets on open models, mid-2026, Interconnects (April 15, 2026).

In the Interconnects essay on recursive self-improvement, Lambert wrote: "I think the hardest exponential we are fighting is on peak intelligence". — Nathan Lambert, Why I still haven't bought into true RSI, Interconnects (September 19, 2026).

In the Interconnects essay on recursive self-improvement, Lambert wrote: "all of our scaling laws show that you need exponential compute and resources to make linear improvements in intelligence". — Nathan Lambert, Why I still haven't bought into true RSI, Interconnects (September 19, 2026).

In the Interconnects essay on recursive self-improvement, Lambert wrote: "the frontier labs and broader frenetic, competitive culture in the San Francisco AI scene set up an environment that amplifies any AI concern". — Nathan Lambert, Why I still haven't bought into true RSI, Interconnects (September 19, 2026).

In the Interconnects essay on recursive self-improvement, Lambert wrote: "I do buy the cycle of experiment design and testing being 10x faster in the near future, but not hypothesis generation and intuition building". — Nathan Lambert, Why I still haven't bought into true RSI, Interconnects (September 19, 2026).

In the Interconnects essay on recursive self-improvement, Lambert wrote: "lossy self-improvement remains my baseline on the trajectory of progress". — Nathan Lambert, Why I still haven't bought into true RSI, Interconnects (September 19, 2026).

In the Interconnects essay Lessons from the hacks, Lambert wrote: "I think scalable oversight of AI with current AI, as presented today, is a solvable problem, as the models are genuinely useful and follow instructions". — Nathan Lambert, Lessons from the hacks, Interconnects (August 9, 2026).

In the Interconnects essay Lessons from the hacks, Lambert wrote: "the AI industry is wildly, collectively unprepared for handling the next 12-24 months well". — Nathan Lambert, Lessons from the hacks, Interconnects (August 9, 2026).

In the Interconnects essay about writing an AI textbook, Lambert wrote: "Today’s LLMs increase entropy in long-form non-fiction writing". — Nathan Lambert, I wrote an AI textbook, how long until AI can do it better?, Interconnects (August 12, 2026).

In the Interconnects essay about writing an AI textbook, Lambert wrote: "I do not think writing is just ignored, but rather it’s challenging and lacks good training data to specifically intervene on it". — Nathan Lambert, I wrote an AI textbook, how long until AI can do it better?, Interconnects (August 12, 2026).

In the Interconnects essay about writing an AI textbook, Lambert wrote: "I am motivated to write voluminously to learn, to feel, and to express". — Nathan Lambert, I wrote an AI textbook, how long until AI can do it better?, Interconnects (August 12, 2026).

In the Interconnects essay about writing an AI textbook, Lambert wrote: "I see why people turn AI tools into a crutch when most of their writing is just an output to fill space, rather than a means to an end". — Nathan Lambert, I wrote an AI textbook, how long until AI can do it better?, Interconnects (August 12, 2026).

In the Interconnects essay about writing an AI textbook, Lambert wrote: "I so deeply appreciate the cut and dry rule I have for my writing on Interconnects to never use AI outputs in the content". — Nathan Lambert, I wrote an AI textbook, how long until AI can do it better?, Interconnects (August 12, 2026).

In the Interconnects essay about writing an AI textbook, Lambert wrote: "you can only use AI models to save 10-20% of the effort today". — Nathan Lambert, I wrote an AI textbook, how long until AI can do it better?, Interconnects (August 12, 2026).

In the Interconnects essay What comes next with open models, Lambert wrote: "I’ve long believed that the company who will benefit most from the ecosystem of open models is the one who understands it best". — Nathan Lambert, What comes next with open models, Interconnects (March 16, 2026).

In the Interconnects essay What comes next with open models, Lambert wrote: "In a world dominated by coding agents, I want to build open models that Claude Code is desperate to use as a tool". — Nathan Lambert, What comes next with open models, Interconnects (March 16, 2026).

In the Interconnects essay about the post-training textbook, Lambert wrote: "This is the book I wanted to read when I was getting started a few years ago". — Nathan Lambert, 5 useful things you'll learn in my post-training textbook, Interconnects (August 10, 2026).

In the Interconnects essay about the post-training textbook, Lambert wrote: "the time it takes for a research paper to land in a frontier model is 3-9months". — Nathan Lambert, 5 useful things you'll learn in my post-training textbook, Interconnects (August 10, 2026).

In the Interconnects essay about the post-training textbook, Lambert wrote: "it helps you develop research taste". — Nathan Lambert, 5 useful things you'll learn in my post-training textbook, Interconnects (August 10, 2026).

In the Interconnects essay about the post-training textbook, Lambert wrote: "knowing the history of your field better than anyone is what lets you make the best predictions". — Nathan Lambert, 5 useful things you'll learn in my post-training textbook, Interconnects (August 10, 2026).

On Latent Space, Lambert said: "people will see that we don't know that much more, but we have a bit of better way of saying what we don't know". — Nathan Lambert on Latent Space, RLHF 201 (January 11, 2024).

On Latent Space, Lambert said: "any major company that wasn't doing RLHF is now realizing they have to have a team around this". — Nathan Lambert on Latent Space, RLHF 201 (January 11, 2024).

On Latent Space, Lambert said: "the view of RL in language models is pretty contrived already, so it's not like we're doing real RL". — Nathan Lambert on Latent Space, RLHF 201 (January 11, 2024).

On Latent Space, Lambert said: "if you want to be a believer in doing this at scale, these are the sorts of things that you have to accept as preconditions for doing RLHF". — Nathan Lambert on Latent Space, RLHF 201 (January 11, 2024).

On Latent Space, Lambert said: "the notion of preference is trying to stay around correctness and style rather than any meaningful notion of preference". — Nathan Lambert on Latent Space, RLHF 201 (January 11, 2024).

On Latent Space, Lambert said: "we don't have a hundred percent agreement with the reward model and the data and that's fine". — Nathan Lambert on Latent Space, RLHF 201 (January 11, 2024).

On Latent Space, Lambert said: "PPO, proximal policy optimization, it's pretty complicated compared to what you really need to know". — Nathan Lambert on Latent Space, RLHF 201 (January 11, 2024).

On Latent Space, Lambert said: "these are all small gains over just getting the deep understanding of the data in the infrastructure setup". — Nathan Lambert on Latent Space, RLHF 201 (January 11, 2024).

On Latent Space, Lambert said: "RLHF is not an easy tool to make numbers go up with". — Nathan Lambert on Latent Space, RLHF 201 (January 11, 2024).

On Latent Space, Lambert said: "we are starting to come to a point where it's just impossible for manual human preference data collection to scale". — Nathan Lambert on Latent Space, RLHF 201 (January 11, 2024).

On Latent Space, Lambert said: "I really think people in open source and academics are going to figure out how to use any preference data on any model just because they're scrappy". — Nathan Lambert on Latent Space, RLHF 201 (January 11, 2024).

On Latent Space, Lambert said: "be careful going into the RLHF can of worms". — Nathan Lambert on Latent Space, RLHF 201 (January 11, 2024).

On Latent Space, Lambert said: "instruction tuning has the fewest slides, even though it's the most practical thing for most people". — Nathan Lambert on Latent Space, RLHF 201 (January 11, 2024).

On Latent Space, Lambert said: "it's easy to overfit if you're training and focusing on them as a developer". — Nathan Lambert on Latent Space, RLHF 201 (January 11, 2024).

On Latent Space, Lambert said: "that's the type of thing that academia still can do and can do really well and hopefully continues to do". — Nathan Lambert on Latent Space, RLHF 201 (January 11, 2024).

On Latent Space, Lambert said: "I think the claims of chat GPT level are long overblown in most of the things in open source". — Nathan Lambert on Latent Space, RLHF 201 (January 11, 2024).

On ChinaTalk, Lambert said: "If the industry commits to this, it's really important that it works and isn't just like paper over a problem". — Nathan Lambert on ChinaTalk, ModelTalk: Pacing the Frontier (September 2026).

On ChinaTalk, Lambert said: "the US labs are taking longer and longer to release, at least at the frontier". — Nathan Lambert on ChinaTalk, ModelTalk: Pacing the Frontier (September 2026).

On ChinaTalk, Lambert said: "There's not a lot of history in the industry for the people building the models to give a ton of extended time to release partners". — Nathan Lambert on ChinaTalk, ModelTalk: Pacing the Frontier (September 2026).