Lessons from Donald Knuth

Computer scientist and mathematician Donald Knuth defined the study of algorithms with The Art of Computer Programming and built the TeX typesetting system to format math with exact precision. This collection gathers his thoughts on writing code for humans, requiring mathematical rigor, and deliberately avoiding modern distractions.

Part 1: The Art and Science of Programming

  1. On Programming as Art: "Computer programming is an art, because it applies accumulated knowledge to the world, because it requires skill and ingenuity, and especially because it produces objects of beauty." — Source: [Computer Programming as an Art]
  2. On Science vs. Art: "Science is what we understand well enough to explain to a computer. Art is everything else we do." — Source: [Things a Computer Scientist Rarely Talks About]
  3. On the Purpose of Computing: "The best programs are written so that computing machines can perform them quickly and so that human beings can understand them clearly." — Source: [Selected Papers on Computer Science]
  4. On Machine Instructions: "A computer is a very stupid machine; it only does exactly what you tell it to do." — Source: [The Art of Computer Programming]
  5. On Joy in Work: "I try to write programs that will bring pleasure to the people who read them." — Source: [Literate Programming]
  6. On Human Control: "Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do." — Source: [The Computer Journal]
  7. On the Definition of a Programmer: "A programmer is ideally an essayist who works with traditional aesthetic and literary forms as well as mathematical concepts." — Source: [Literate Programming]
  8. On Complexity: "Computer programs are the most complex things that humans have ever created." — Source: [Lex Fridman Podcast #62]
  9. On Tooling: "If you find that you're spending almost all your time on theory, start turning some attention to practical things; it will improve your theories. If you spend most of your time on practice, start turning some attention to theory; it will improve your practice." — Source: [Turing Award Lecture]
  10. On Theory and Practice: "Theory and practice are not mutually exclusive; they are intimately connected. The best theory is inspired by practice, and the best practice is guided by theory." — Source: [Computer History Museum Oral History]

Part 2: The Philosophy of Code Quality and Optimization

  1. On Premature Optimization: "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil." — Source: [Structured Programming with go to Statements]
  2. On the 3 Percent: "Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code." — Source: [Structured Programming with go to Statements]
  3. On Untested Code: "Beware of bugs in the above code; I have only proved it correct, not tried it." — Source: [Memo to Peter van Emde Boas]
  4. On Perfection: "If you optimize everything, you will always be unhappy." — Source: [Lex Fridman Podcast #219]
  5. On Profiling: "It is often a mistake to make a priori judgments about what parts of a program are really critical, since the universal experience of programmers who have been using measurement tools has been that their intuitive guesses fail." — Source: [Structured Programming with go to Statements]
  6. On Code Maintenance: "An undocumented program is a worthless program." — Source: [Computer History Museum Interview]
  7. On Software Architecture: "The structure of a program should reflect the structure of the problem it solves." — Source: [The Art of Computer Programming, Vol 1]
  8. On Design: "I prefer to write programs that have a clear, logical structure, rather than programs that are a collection of clever tricks." — Source: [Masaryk University Q&A]
  9. On Errors: "I define a bug as simply an error in the programmer's intention." — Source: [Lex Fridman Podcast #219]
  10. On Reliability: "Software must be built on a foundation of solid mathematical understanding if it is to be reliable." — Source: [Turing Award Lecture]

Part 3: Literate Programming and Communication

  1. On Literature: "When you write a program, think of it primarily as a work of literature. You're trying to write something that human beings are going to read." — Source: [Andrew Binstock Interview]
  2. On the Reader: "Let us change our traditional attitude to the construction of programs: Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do." — Source: [The Computer Journal]
  3. On Documentation: "The programmer who writes a program with the mindset of a teacher will produce much better code." — Source: [Literate Programming Book]
  4. On Web (the tool): "I chose the name WEB partly because it was one of the few three-letter words of English that hadn't already been applied to computers." — Source: [The WEB System of Structured Documentation]
  5. On Clarity: "By treating code as an essay, we are forced to clarify our own thoughts before the compiler ever sees them." — Source: [CLB Interview 1993]
  6. On Code Formatting: "A well-formatted program is not only easier to read, it is easier to verify and maintain." — Source: [Literate Programming]
  7. On Self-Explanatory Code: "There is no such thing as self-documenting code. Code explains the 'what' and 'how,' but human language is needed to explain the 'why'." — Source: [Masaryk University Q&A]
  8. On Writing: "The process of writing the documentation alongside the code acts as a natural feedback loop, improving the quality of both." — Source: [Computer History Museum Oral History]
  9. On Typography in Code: "I realized that the way programs were typeset in journals was terrible. I wanted my code to look as beautiful on the page as mathematics does." — Source: [Lex Fridman Podcast #62]
  10. On Communication: "Our primary audience when writing software is always other people, even if those people are just our future selves." — Source: [Coders at Work]

Part 4: Typography, TeX, and Aesthetics

  1. On TeX's Origin: "I wrote TeX because I couldn't stand the way my books looked when they were printed." — Source: [TUGboat Interview]
  2. On Font Design: "Designing a font with METAFONT taught me that a letter is not just a shape, it is a dynamic concept that varies with its context." — Source: [Computer Modern Typefaces]
  3. On Beauty: "A computer scientist should be concerned with the aesthetic aspects of their work, because beauty is a strong indicator of truth and utility." — Source: [Computer Programming as an Art]
  4. On Timelessness: "My goal with TeX was to create a system that would produce the same exact output on any computer, now and fifty years from now." — Source: [The TeXbook]
  5. On Versioning: "TeX is not being developed further. Its version number asymptotically approaches Pi. It is a finished work." — Source: [Knuth's TeX page]
  6. On Detail: "To get a layout exactly right, you have to care about the spacing between every single pair of letters, the so-called kerning." — Source: [Lex Fridman Podcast #219]
  7. On Mathematics Typesetting: "Math is a two-dimensional language, and rendering it correctly requires an algorithm that understands the hierarchical relationships of the symbols." — Source: [Digital Typography]
  8. On Automation: "We shouldn't automate typography to the point of removing human judgment, but we should automate the repetitive tasks that get in the way of art." — Source: [Computer History Museum Lecture]
  9. On Perfectionism: "I stopped writing my book for ten years because I had to solve the typesetting problem first. Some called it a distraction; I called it a prerequisite." — Source: [Stanford Engineering Lectures]

Part 5: Algorithms and Problem Solving

  1. On Algorithms: "An algorithm must be seen to be believed. You cannot truly understand it until you have traced its execution step by step." — Source: [The Art of Computer Programming, Vol 1]
  2. On Big-O Notation: "Big-O notation is essential, but it hides the constant factors. In practice, a simpler algorithm with a worse asymptotic bound often wins for realistic input sizes." — Source: [Lex Fridman Podcast #62]
  3. On Data Structures: "The choice of data structure is the most important decision in designing an algorithm. Once the data structures are laid out, the algorithms tend to fall into place." — Source: [The Art of Computer Programming, Vol 1]
  4. On Hard Problems: "The best way to solve a hard problem is to find a simpler problem that contains the essence of the difficulty, and solve that first." — Source: [Numberphile Podcast]
  5. On Estimation: "I have always been a very bad estimator of how long it will take to finish a project, because I invariably discover fascinating sub-problems along the way." — Source: [Numberphile Podcast]
  6. On Elegance: "An elegant algorithm is one that achieves a complex result with remarkably few steps or a surprisingly simple conceptual model." — Source: [ACM ByteCast Episode 2]
  7. On Randomness: "Generating random numbers is too important to be left to chance. It requires rigorous mathematical algorithms to guarantee uniform distribution." — Source: [The Art of Computer Programming, Vol 2]
  8. On Optimization Strategies: "When trying to speed up an algorithm, the first question should not be 'How can I do this faster?' but 'Do I need to do this at all?'" — Source: [Stanford Christmas Lectures]
  9. On Sorting: "Sorting is not just a problem to be solved; it is a fundamental lens through which we can study the efficiency of different computational paradigms." — Source: [The Art of Computer Programming, Vol 3]

Part 6: The History and Culture of Computer Science

  1. On the Evolution of Computing: "The field of computer science has moved so fast that we often forget to look back and understand the foundational ideas that made today's technology possible." — Source: [Lex Fridman Podcast #219]
  2. On Reading Old Code: "To understand how computing evolved, you must read the source code of early systems. It reveals the constraints and the genius of the pioneers." — Source: [Coders at Work Interview]
  3. On Email: "Email is a wonderful thing for people whose role in life is to be on top of things. But not for me; my role is to be on the bottom of things." — Source: [Knuth versus Email]
  4. On Focus: "I don't have time to keep up with the latest fads. I want to spend my life focusing on things that will still be relevant a hundred years from now." — Source: [Knuth's Home Page]
  5. On Academic Silos: "The best computer science sits at the intersection of mathematics, engineering, and art. We lose something when we separate these disciplines too rigidly." — Source: [Masaryk University Q&A]
  6. On Historical Context: "If you don't know the history of an algorithm, you only know its mechanics, not its soul or the specific human problem it was invented to solve." — Source: [Twenty Questions for Donald Knuth]
  7. On John von Neumann: "Von Neumann's insight that data and instructions could share the same memory space is the conceptual leap that made modern programming possible." — Source: [Computer History Museum Oral History]
  8. On Collaboration: "Science advances not just through individual brilliance, but through the careful, critical reading of each other's work. I offer checks to anyone who finds an error in my books to encourage this." — Source: [Knuth Reward Checks]
  9. On AI and Machine Learning: "Artificial Intelligence has made incredible strides, but it still struggles with the kind of rigorous, symbolic logic that humans use to reason about algorithms." — Source: [Lex Fridman Podcast #219]

Part 7: Faith, Life, and Personal Philosophy

  1. On Faith and Science: "Science is about discovering how God created the universe. Religion is about understanding why." — Source: [Things a Computer Scientist Rarely Talks About]
  2. On Music: "Music, like programming, relies on patterns, structures, and an underlying mathematical logic, yet its ultimate goal is to evoke a human emotional response." — Source: [ACM ByteCast Episode 2]
  3. On the Pipe Organ: "Playing the pipe organ requires coordinating the hands and feet across multiple manuals. It is a physical manifestation of parallel processing." — Source: [Stanford News Interview]
  4. On the Rhinoceros Attitude: "When I was young, I developed a 'rhinoceros attitude'—I simply charged forward at problems, ignoring anyone who told me they were too difficult." — Source: [Numberphile Podcast]
  5. On Mysteries: "There are things in the universe that we will never be able to compute or formalize, and I find great comfort in that mystery." — Source: [Talks at Google: Faith and Science]
  6. On Solitude: "Deep work requires long, uninterrupted blocks of time. You cannot discover fundamental truths in fifteen-minute increments between meetings." — Source: [Knuth versus Email]
  7. On Humility: "The more I learn about computer science, the more I realize how much is left to discover. Every answered question spawns three new ones." — Source: [Computer History Museum Oral History]
  8. On Legacy: "I measure my success not by how much money I've made, but by how many people have found my work useful or beautiful." — Source: [Lex Fridman Podcast #62]
  9. On Translation: "Translating the Bible was a fascinating exercise in understanding text—it shares similarities with parsing a complex, context-sensitive grammar." — Source: [3:16 Bible Texts Illuminated]

Part 8: Writing and The Art of Computer Programming

  1. On Ambition: "When I started writing The Art of Computer Programming, I thought it would take me a few years to summarize everything known about algorithms. I was delightfully naive." — Source: [Computer History Museum Oral History]
  2. On Drafts: "I write all my drafts in pencil. The physical act of writing slows down my thoughts just enough to let me evaluate the logic before I commit it to paper." — Source: [Lex Fridman Podcast #219]
  3. On Completeness: "I feel a responsibility to read the original source material for every concept I write about. I cannot summarize a paper I haven't studied myself." — Source: [Twenty Questions for Donald Knuth]
  4. On Revisions: "A book is never truly finished. It merely reaches a point where the author decides to release it to the world, knowing there are still improvements to be made." — Source: [The Art of Computer Programming, Preface]
  5. On Exercises: "The exercises in my books are as important as the main text. You cannot learn to program by reading alone; you must wrestle with the problems." — Source: [The Art of Computer Programming, Preface]
  6. On Bill Gates' Quote: "If you can read the entirety of The Art of Computer Programming and solve all the exercises, you are an exceptional programmer, and you should probably send Bill Gates your resume." — Source: [TAOCP Jacket Endorsement]
  7. On the Meaning of 'Art': "I called it 'The Art of Computer Programming' because writing software isn't just mechanical engineering; it's a creative process akin to composing music or writing poetry." — Source: [Computer Programming as an Art]
  8. On Future Volumes: "I am currently working on Volume 4, and I will continue working on it for as long as I have the energy and clarity of mind to do so." — Source: [Knuth's Home Page]
  9. On Knowledge: "My goal is simply to organize what is known, so that the next generation of programmers can build upon a solid, well-documented foundation." — Source: [Andrew Binstock Interview]