Visual summary of operating lessons from Emily Bender.

Emily M. Bender is a University of Washington linguistics professor whose research examines language technology and its societal effects. Her work with coauthors distinguishes linguistic form from communicative meaning, calls for better documentation of NLP data, and argues for naming the languages being studied. — Climbing towards NLU.

Part 1: The Mechanics of Form Versus Meaning

  1. Distinguish Form from Meaning: Bender and Koller argue that predicting linguistic forms from text is not the same as learning how those forms connect to communicative intent. — Climbing towards NLU.
  2. Look for Communicative Intent: Their account defines linguistic meaning in relation to what a speaker intends to communicate; a model trained only to predict strings lacks that kind of grounding in its training task. — Climbing towards NLU.
  3. Language Uses Shared Context: Human communication depends on a listener connecting words to the speaker's situation and intent, not merely recognizing a pattern of symbols. — Climbing towards NLU.
  4. Understand the Octopus Test: Bender and Koller's octopus thought experiment illustrates why matching the patterns in two people's messages does not by itself give an observer their connection to the world. — Climbing towards NLU.
  5. Notice Human Projection: Readers can make sense of fluent machine text by supplying context and intent themselves, even when the system has not communicated an intention. — Climbing towards NLU.
  6. Protect Empathy from Anthropomorphic Design: In her Guardian essay, Bender argues that human-like chat interfaces can invite misplaced trust and calls for transparency about the systems behind them. — Bender's Guardian essay.
  7. Separate Training Text from Intent: Bender and Koller argue that a training task limited to linguistic form does not supply the nonlinguistic signal needed to learn a speaker's communicative intent. — Climbing towards NLU.
  8. Do Not Equate Fluency with Understanding: Bender's position is that fluent generated text should not be treated as proof of human-like understanding; she distinguishes successful text prediction from grounded meaning. — Climbing towards NLU.
  9. Notice How Users Shape the Answer: Using a Magic 8 Ball analogy, Bender says people learn to pose questions that elicit interpretable answers and then supply the context themselves; she cautions against taking such output as serious advice. — Because Language interview.
  10. Ask What the Model Was Trained On: The form-and-meaning argument concerns systems trained only on string prediction; their results do not alone establish grounded human-analogous language understanding. — Climbing towards NLU.

Part 2: The Stochastic Parrot

  1. Use the Stochastic-Parrot Metaphor: Bender and her coauthors use 'stochastic parrot' to describe language models generating plausible sequences of linguistic forms without grounding the output in communicative meaning. — Bender's university lecture.
  2. Distinguish Prediction from Thought: Bender describes a language model's text-generation task as choosing plausible continuations from patterns in training data, not forming a human thought. — Carnegie Council interview.
  3. Question the Hallucination Label: Bender objects that calling fabricated output a 'hallucination' anthropomorphizes a system that generates text without human-like experience. — Carnegie Council interview.
  4. RAG Does Not Guarantee Truth: In a university lecture, Bender argues that retrieval-augmented generation can still recombine relevant source documents into unreliable output; adding good documents does not by itself make the generated answer trustworthy. — Bender's university lecture.
  5. Resist the Impression of a Mind: Bender says readers reflexively imagine an intending speaker behind fluent text; evaluating a chatbot requires remembering that its output is not a person's communicative act. — Carnegie Council interview.
  6. Scale Is Not Grounding: Bender and Koller argue that adding more text to a form-only training task does not provide the nonlinguistic grounding their account requires for meaning. — Climbing towards NLU.
  7. Treat Meaning as Communicative: Bender and Koller analyze meaning as a relation between linguistic form and communicative intent in a shared situation, not as a property of strings alone. — Climbing towards NLU.
  8. See Why Fluent Output Misleads: Bender's lecture separates probabilistic text generation, a chat interface, and users' reflexive interpretation to explain why output can appear understanding-like. — Bender's university lecture.
  9. Check Task Fit Before a Text-Generation Fix: Bender urges adopters to ask whether a proposed automation's inputs can support its promised output and whether its evaluation matches the use case, rather than treating fluent synthetic text as a ready fix. — Tech Policy Press interview.

Part 3: Redefining AI as Automation

  1. Use More Precise Terms Than AI: Bender recommends setting aside the broad label 'artificial intelligence' and identifying the particular automation under discussion. — Tech Policy Press interview.
  2. Name the Actual Function: She favors descriptions such as pattern matching or automatic transcription that identify a system's task without implying a mind or a single general capability. — Carnegie Council interview.
  3. Ask What Is Being Automated: Calling a tool automation makes it easier, in Bender's view, to ask what task is being delegated to a machine and whether that task is appropriate. — Tech Policy Press interview.
  4. Ask Who Benefits and Who Can Appeal: Bender's evaluation questions include the input, output, purpose, beneficiaries, possible harms, and recourse when an automated system fails. — Tech Policy Press interview.
  5. Describe Synthetic Text Plainly: Bender calls chatbot output synthetic text and warns that conversational presentation can make plausible strings look like a human exchange. — Bender's university lecture.
  6. Watch Anthropomorphic Metaphors: Bender asks readers to examine the assumptions in words such as 'recognition' and 'hallucination,' because they can imply that software has experiences or a mind. — Carnegie Council interview.
  7. Treat AI as an Overbroad Label: Bender argues that placing transcription, protein-folding tools, and chatbots under one 'AI' label makes their distinct functions harder to understand. — Carnegie Council interview.
  8. Keep Human Decisions Visible: Bender says the facade of full automation can hide the people whose labor and choices make systems work, weakening scrutiny of who is responsible. — Tech Policy Press interview.
  9. Avoid Bias-Washing by Computer: Bender warns that a decision may seem objective merely because a computer produced it, even though machine-learning systems reproduce patterns in their training data. — Carnegie Council interview.

Part 4: Data Statements and Accountability

  1. Document Large Datasets: As training collections grow, Bender and her coauthors argue for curation and documentation rather than indiscriminately ingesting web text. — Stochastic Parrots paper.
  2. Provide Data Statements: Bender and Friedman propose data statements that explain how a dataset was assembled and which populations it represents, so readers can assess likely generalization and bias. — Data Statements paper.
  3. Curate Rather Than Ingest Everything: The Stochastic Parrots authors recommend investing in careful dataset curation and documentation rather than taking in everything available on the web. — Stochastic Parrots paper.
  4. Name Language Varieties: A data statement should identify the language varieties represented in a dataset, because a system trained on one variety may not work comparably for another. — Data Statements paper.
  5. Describe Speakers and Annotators: The data-statements proposal asks researchers to document relevant information about both the people who produced the language data and those who annotated it. — Data Statements paper.
  6. Treat Bias as a Dataset Question: Bender and Friedman argue that documenting represented populations and collection choices helps identify biases a language system may reflect; documentation alone does not eliminate those biases. — Data Statements paper.
  7. Make Results Easier to Reproduce: Clear dataset descriptions help other researchers understand how results might generalize and make NLP experiments more reproducible. — Data Statements paper.
  8. Understand Training Data as Context: Bender and Friedman treat a dataset's composition as necessary context for interpreting an NLP result, not as an interchangeable bucket of text. — Data Statements paper.
  9. Explain Curation Choices: Their proposed data statement includes a curation rationale: how and why particular language material was selected for the dataset. — Data Statements paper.

Part 5: The Bender Rule and Linguistic Bias

  1. Always Name the Language: The Bender Rule asks NLP researchers to name the language or languages they study, including when the language is English. — The #BenderRule essay.
  2. Avoid False Language Independence: Bender warns that leaving English unnamed can give work a false appearance of language independence when only one language was studied. — The #BenderRule essay.
  3. Do Not Treat English as the Default: She argues that NLP's habit of leaving English implicit contributes to overlooking the many languages and language varieties outside the most-studied set. — The #BenderRule essay.
  4. Acknowledge Language Specificity: Naming a studied language keeps open the possibility that a technique depends on properties of that language rather than generalizing automatically. — The #BenderRule essay.
  5. Test Cross-Language Claims: A strong result on English data alone does not establish that the same approach will work for languages with different writing systems, morphology, or word order. — The #BenderRule essay.
  6. Include Understudied Languages: Bender argues that concentrating research and resources on English and a few other languages can leave speakers of less-supported languages without useful language technology. — The #BenderRule essay.
  7. State the Scope of Accuracy: Reporting a result as measured on English data is more precise than presenting it as a general result for natural language without naming the test language. — The #BenderRule essay.
  8. Account for Morphology: English has relatively little inflectional morphology; Bender notes that methods can face additional data sparsity when words appear in many forms in a highly inflecting language. — The #BenderRule essay.
  9. Decenter English in NLP: For Bender, explicitly naming English is a first step toward treating it as one language among many rather than as a universal stand-in. — The #BenderRule essay.

Part 6: Dismantling AI Hype

  1. Question Hype That Flatters Leaders: Bender and Hanna argue that grand stories of imminent machine minds can let powerful people cast themselves as heroes while distracting from present harms. — The AI Con excerpt.
  2. Reject the Booster-Doomer Binary: They argue that utopian and apocalyptic AI stories share an assumption of imminent all-powerful systems, narrowing attention to real choices and current harms. — The AI Con excerpt.
  3. Look Behind the Automation Facade: Bender says portraying systems as fully automated hides the workers and decisions that make them operate. — Tech Policy Press interview.
  4. Follow the Incentives Behind Hype: Bender and Hanna describe AI hype as serving a commercial function: selling products, attracting attention, and encouraging adoption. — The AI Con excerpt.
  5. Notice Hype's Cultural Function: They argue that AI promotion connects commercial interests to familiar fantasies about sentient machines, shaping what people expect from the technology. — The AI Con excerpt.
  6. Resist the Urge to Be Impressed: Bender urges journalists and readers to challenge extraordinary AI claims and inspect their evidence instead of treating technical spectacle as proof. — Bender on AI Doomerism.
  7. Expect Another Hype Peak: In the preface to The AI Con, Bender and Hanna use 'the summit of bullshit mountain' to describe how each apparent peak of AI hype is followed by still grander claims. — The AI Con excerpt.
  8. Distinguish Writing from Synthetic Text: Bender argues that writing helps people refine and communicate ideas, whereas a language model is designed to produce plausible-sounding text. — Bender on AI Doomerism.

Part 7: Human Labor and Exploitation

  1. Make Hidden People Visible: Bender says systems marketed as fully automated still depend on people's labor, including data work and deployment decisions. — Tech Policy Press interview.
  2. Credit Data Workers: Bender and Hanna describe the often low-paid workers who label, filter, and review data and content that current AI systems depend on. — The AI Con labor excerpt.
  3. Question Nonconsensual Data Collection: Bender criticizes companies that gather large amounts of web and social-media material without meaningful transparency or consent, including personal information. — Carnegie Council interview.
  4. Check the Labor Behind 'Just Walk Out': Bender cited reporting that Amazon's Just Walk Out stores relied on workers reviewing video to help charge shoppers, complicating the image of a fully autonomous checkout. — Convergence interview.
  5. Count Environmental Costs: The Stochastic Parrots authors urge developers to weigh the environmental and financial costs of ever-larger language models before building them. — Stochastic Parrots paper.
  6. Interrogate 'Smart' Products: Bender recommends asking what data a product marketed as 'smart' collects, where the data goes, and how a person can opt out. — Convergence interview.
  7. Inspect Data for Harmful Patterns: Bender and coauthors warn that indiscriminate web-text collection can overrepresent dominant viewpoints and reproduce harmful patterns in generated output. — Stochastic Parrots paper.
  8. Keep Adoption a Human Choice: Bender and Hanna frame resistance to AI hype as a way for people and institutions to evaluate automation on their own terms rather than accept vendor narratives as inevitable. — The AI Con excerpt.
  9. Center Present-Day Harms: Bender asks critics to focus on observable harms such as bias, exploited labor, and environmental costs rather than only speculative future scenarios. — Bender on AI Doomerism.
  10. Demand Data Transparency and Control: Bender links transparency about collected data to privacy and argues that putting information online for people to read is not blanket consent for model training. — Carnegie Council interview.

Part 8: Policy, Education, and Moving Forward

  1. Do Not Require Chatbots for Learning: Bender argues universities should not promote, let alone require, chatbot use; students should be able to learn without being forced to use synthetic-text tools. — Bender on Higher Education.
  2. Educate the Public and Policymakers: Bender says informed public discussion and policymaker engagement are connected routes toward more sensible technology policy. — Tech Policy Press interview.
  3. Read the Footnotes: Bender and Hanna describe checking endnotes and source methods as a way to test spectacular AI claims instead of relying on a press release. — The AI Con excerpt.
  4. Protect Writing as Learning: Bender argues that students learn by writing and receiving genuine feedback on their own voice, which essay-shaped synthetic output cannot substitute for. — Bender on Higher Education.
  5. Treat Adoption as a Choice: Bender challenges messaging that students must use chatbots to be ready for work and urges institutions to decide whether the tools fit their educational mission. — Bender on Higher Education.
  6. Test the Use Case: Before adopting automation, Bender asks whether the inputs can support the promised output and whether evaluation matches the intended context. — Tech Policy Press interview.
  7. Put Evidence Behind Extraordinary Claims: Bender says vendors making strong claims about intelligent or safe systems should supply evidence open to scrutiny, rather than shifting that burden to the public. — Bender on AI Doomerism.
  8. Ask What Data Built the System: Bender's data-statements proposal makes training-data composition and collection choices visible so users can assess generalization, bias, and appropriateness. — Data Statements paper.
  9. Preserve Human Learning Communities: Bender argues that directing students to chatbots can displace valuable interaction with classmates, instructors, and librarians. — Bender on Higher Education.