Visual summary of operating lessons from Douwe Kiela.

Lessons from Douwe Kiela

Douwe Kiela is a coauthor of the 2020 research paper that introduced retrieval-augmented generation (RAG) and is cofounder and CEO of Contextual AI. His work spans research on retrieval and evaluation and the development of enterprise systems grounded in organizations’ own data. The lessons below distinguish his direct statements from claims made in coauthored papers and company publications. — Contextual AI — Platform Launch.

Part 1: The Origins and Evolution of RAG

  1. The Name Stuck: Kiela recalls that the original RAG team joked they should have found a better name, but the term stuck as generative AI gained traction. — Madrona — Douwe Kiela Interview.
  2. Retrieve Rather Than Memorize Everything: The original RAG work combined a generation model with retrieved external information, so answers could use knowledge beyond what was stored in model parameters. — RAG — Original Research Paper.
  3. Notice When a Research Idea Becomes Mainstream: Kiela observes that RAG moved from an unfamiliar research method into common developer practice, while warning that its popularity does not make every implementation production-ready. — O’Reilly — Douwe Kiela Interview.
  4. Combine Two Kinds of Memory: The 2020 RAG model paired a pretrained sequence-to-sequence model with a dense index of Wikipedia, linking parametric and non-parametric memory. — RAG — Original Research Paper.
  5. Give Generators Access to External Knowledge: Kiela and his coauthors presented RAG as a way to augment language generation with retrieved passages rather than relying on the generator as a closed knowledge store. — RAG — Original Research Paper.
  6. Train Retrieval and Generation for the Task: The original RAG study explored fine-tuning a generation model with access to a neural retriever and tested two ways to condition generated text on retrieved passages. — RAG — Original Research Paper.
  7. Refresh Knowledge Through Retrieval: Kiela says that the ability to supply updated external context was part of the original motivation for RAG, though getting a model to rely fully on its retriever proved difficult. — Madrona — Douwe Kiela Interview.
  8. Treat RAG as a Coauthored Research Advance: The 2020 paper introduced and evaluated retrieval-augmented generation on knowledge-intensive NLP tasks, with Kiela among twelve coauthors. — RAG — Original Research Paper.

Part 2: Contextual AI and the Enterprise Data Problem

  1. Expect Enterprise Data to Be Harder: Kiela contrasts answering questions over one clean document with retrieving reliable answers across millions of complicated enterprise files, including diagrams and specialized content. — Madrona — Douwe Kiela Interview.
  2. Offer Deployment Choices for Sensitive Data: In Kiela’s signed platform announcement, Contextual AI describes managed, VPC and on-premises options for enterprises with different security and regulatory needs. — Contextual AI — Platform Launch.
  3. Specialize for Expert Workflows: Kiela argues that general-purpose agents do not fully serve expert, domain-specific work and that enterprise RAG agents should be optimized for those workflows. — Contextual AI — Platform Launch.
  4. Engineer Context, Not Just Prompts: Kiela calls for connecting models to an organization’s specialized data and workflows through a unified context layer rather than relying on prompt wording alone. — Contextual AI — Introducing Agent Composer.
  5. Treat Security as a Production Requirement: Kiela’s platform announcement treats access control, encryption, audit trails and deployment options as prerequisites for enterprise AI rather than afterthoughts. — Contextual AI — Platform Launch.
  6. Make Generated Claims Traceable: Kiela describes fine-grained citations that point users to the relevant retrieved material so they can inspect the basis of an answer. — Contextual AI — Platform Launch.
  7. Evaluate Deployed AI Continuously: Kiela says teams deploying AI need to know whether a system works, where it fails and what risks remain; evaluation is central to trust in the deployment. — Madrona — Douwe Kiela Interview.
  8. Design Retrieval for Messy Enterprise Data: Kiela’s product account identifies extraction, multimodal and multihop retrieval, security and scaling as technical challenges that compound in real deployments. — Contextual AI — Platform Launch.
  9. Ground Agents in Company Context: Kiela describes Contextual AI’s platform as building specialized agents over enterprise data stores so they can reason about an organization’s own knowledge. — Contextual AI — Platform Launch.
  10. Target High-Value Specialist Tasks: Kiela argues for domain-specialized agents supporting expert work, such as technical research and engineering, rather than stopping at simple generic internal Q&A. — Contextual AI — Platform Launch.

Part 3: RAG 2.0: Moving Beyond Stitched-Together Systems

  1. Treat RAG as a System: Contextual AI describes conventional RAG as a collection of frozen embedding, retrieval and generation components whose individual quality does not guarantee a strong end-to-end answer. — Contextual AI — Introducing RAG 2.0.
  2. Optimize the Pipeline Together: Contextual AI’s RAG 2.0 approach trains and aligns the retriever and language model as parts of one system rather than accepting an unoptimized chain of components. — Contextual AI — Introducing RAG 2.0.
  3. Let Agents Decide When to Retrieve: Kiela describes retrieval as one tool available to an agentic planner, which can determine what information to fetch for a task. — O’Reilly — Douwe Kiela Interview.
  4. Measure End-to-End Answer Quality: Contextual AI reports evaluating its integrated RAG system on answer accuracy, faithfulness and freshness rather than treating retrieval and generation as isolated components. — Contextual AI — Introducing RAG 2.0.
  5. Look Beyond Glue-Code Orchestration: Contextual AI argues that stitching frozen models and a vector database together through prompting or orchestration can create brittle enterprise systems. — Contextual AI — Introducing RAG 2.0.
  6. Specialize Components When the Domain Demands It: Kiela describes tuning retrieval and generation components for particular use cases rather than assuming one generic configuration will meet every enterprise accuracy requirement. — Contextual AI — Platform Launch.
  7. Preserve Document Structure Before Retrieval: Kiela emphasizes parsing document hierarchy and layout before indexing, because extraction quality determines whether later retrieval has useful context. — O’Reilly — Douwe Kiela Interview.

Part 4: The Flaws in AI Benchmarks and Evaluation

  1. Question Saturated Static Benchmarks: Kiela and coauthors argue that high benchmark scores can obscure simple model failures because fixed test sets are incomplete proxies for real capabilities. — Dynabench — Original Research Paper.
  2. Be Careful What a Benchmark Score Signals: Kiela and coauthors warn that rapid benchmark saturation can make apparent AI progress seem more comprehensive than the underlying task evidence justifies. — Contextual AI — Plotting Progress in AI.
  3. Collect Dynamic Challenge Examples: The Dynabench team proposed a platform where people create examples that current models misclassify but other people can recognize. — Dynabench — Original Research Paper.
  4. Use People to Find Model Weaknesses: Dynabench asks annotators to challenge a current model, exposing edge cases that a static benchmark may miss. — Dynabench — Original Research Paper.
  5. Inspect Metric Sensitivity: Kiela and coauthors observe that small changes to evaluation metrics can reorder model rankings, so leaderboard position alone is a fragile measure of capability. — Contextual AI — Plotting Progress in AI.
  6. Do Not Equate a Benchmark Win With Robustness: The Dynabench paper notes that models can surpass narrow human benchmark baselines yet fail on simple challenge examples or real-world scenarios. — Dynabench — Original Research Paper.
  7. Keep Evaluation Moving With Models: Dynabench proposes repeatedly collecting data against the latest models so evaluation changes as model weaknesses change. — Dynabench — Original Research Paper.
  8. Turn Failure Cases Into Better Data: The Dynabench team argues that human-discovered model errors can provide training and assessment data for the next round of systems. — Dynabench — Original Research Paper.
  9. Build Evaluations Around the Actual Use Case: Kiela argues that companies should define their own evaluation cases using their data and institutional expertise, because generic examples cannot fully test the deployed task. — Madrona — Douwe Kiela Interview.

Part 5: Hallucinations, Trust, and Auditability in AI

  1. Ground Answers in Retrieved Evidence: The Contextual AI team argues that enterprise answers should be generated from relevant retrieved documents and assessed for whether they reflect those sources. — Contextual AI — Grounded Language Model.
  2. Make Citations Precise Enough to Audit: Kiela’s signed platform announcement describes citations that highlight specific supporting passages rather than merely naming an entire source document. — Contextual AI — Platform Launch.
  3. Do Not Assume Correct Retrieval Ends Hallucination: Contextual AI reports that general-purpose models can still disregard retrieved evidence, motivating models trained specifically to stay grounded in source material. — Contextual AI — Grounded Language Model.
  4. Treat Enterprise Errors as Consequential: Contextual AI gives customer support, finance and engineering examples in which a generated answer that misstates company knowledge could erode trust or cause harm. — Contextual AI — Grounded Language Model.
  5. Allow a Grounded Model to Say It Does Not Know: Kiela describes training a model for retrieval-grounded answers so it can decline to answer when the required information is absent from its context. — O’Reilly — Douwe Kiela Interview.
  6. Combine Retrieval and Fine-Tuning When Useful: Kiela rejects a rigid choice between retrieval and fine-tuning; he says the value of additional tuning depends on the performance gain required for the use case. — O’Reilly — Douwe Kiela Interview.
  7. Add Safeguards Around Retrieved Answers: Kiela’s platform account describes query guardrails and groundedness checks alongside retrieval and generation for enterprise deployments. — Contextual AI — Platform Launch.

Part 6: The "RAG is Dead" Misconception

  1. Question Repeated Obituaries for RAG: Kiela observes that RAG is repeatedly declared obsolete, especially when larger-context models arrive, and argues those declarations overlook retrieval’s continuing role. — O’Reilly — Douwe Kiela Interview.
  2. Retrieval Still Connects Models to Data: Kiela argues that systems still need a way to retrieve relevant external information when applying a generative model to an organization’s own data. — Madrona — Douwe Kiela Interview.
  3. Use Long Context With Retrieval: Kiela argues that a longer context window can hold a broader set of relevant search results, but does not replace selecting those results in the first place. — O’Reilly — Douwe Kiela Interview.
  4. Do Not Reprocess Entire Corpora for Every Query: Kiela uses the example of answering a simple question from a book without reading every volume each time to explain why targeted retrieval can be more efficient than filling the full context. — O’Reilly — Douwe Kiela Interview.
  5. Account for Latency and Cost of Reasoning: Kiela notes that reasoning-heavy models increase cost and latency, so a system should reserve that mode for queries that need it. — O’Reilly — Douwe Kiela Interview.

Part 7: Scaling AI for the Real World

  1. Plan for Million-Document Workloads: Kiela contrasts easy one-document demonstrations with the difficulty of searching millions of complex enterprise documents while keeping answers useful. — Madrona — Douwe Kiela Interview.
  2. Evaluate Retrieval Beyond Simple Similarity: Kiela describes enterprise search as a system combining extraction, multiple retrievers and reranking, because relevant information is not always found by one simple lookup. — O’Reilly — Douwe Kiela Interview.
  3. Engineer for Production, Not Just a Demo: Kiela says open-source extraction tools can help start a prototype but that production systems must handle complex layouts and very large PDF collections. — O’Reilly — Douwe Kiela Interview.
  4. Consider Hybrid Retrieval Trade-offs: Contextual AI reports testing hybrid semantic-and-keyword retrieval as one way to improve search coverage, while measuring its added latency. — Contextual AI — Making Search Agents Faster and Smarter.
  5. Budget for Query-Time Compute: Kiela warns that reasoning-heavy inference can add cost and latency, especially when a system handles repeated, high-volume queries. — O’Reilly — Douwe Kiela Interview.
  6. Parse Before Fine-Tuning Chunks: Kiela argues that accurate extraction and document hierarchy matter more than trying to rescue poor source parsing through chunking or embeddings. — O’Reilly — Douwe Kiela Interview.
  7. Rerank Retrieved Candidates: Kiela describes a retrieval pipeline that gathers candidates and then reranks them before a grounded language model generates an answer. — O’Reilly — Douwe Kiela Interview.
  8. Respect Document Permissions: Kiela’s signed platform announcement describes role-based access control so enterprise responses are grounded only in data the user may access. — Contextual AI — Platform Launch.

Part 8: The Future of AI Architectures and Open Source

  1. Make Research Reproducible When Possible: Kiela says open research should enable others to reproduce results, while acknowledging legal and practical difficulties in sharing some datasets. — Stanford CS224U — Douwe Kiela Interview.
  2. Extend Retrieval Beyond Text: Kiela says the retrieval concept can work over other forms of data, and his team explored image-grounded variants rather than limiting the idea to text. — Madrona — Douwe Kiela Interview.
  3. Lower Barriers to AI Tools: Kiela describes Hugging Face demos, models and datasets as ways to make AI accessible to people beyond specialist researchers and to support education. — Stanford CS224U — Douwe Kiela Interview.
  4. Specialize Grounded Models for the Task: Kiela describes using models trained for retrieval-grounded answers rather than relying on a general-purpose generator to follow source material automatically. — O’Reilly — Douwe Kiela Interview.
  5. Optimize the Whole Retrieval System: Kiela argues that a high-quality language model alone cannot compensate for a poorly working retrieval and data-processing system. — O’Reilly — Douwe Kiela Interview.
  6. Translate Research Into Working Systems: Kiela describes the move from the original RAG research to building integrated enterprise systems that can withstand real production data and use cases. — Madrona — Douwe Kiela Interview.
  7. Use Open, Dynamic Evaluation Platforms: Kiela and coauthors proposed Dynabench as an open platform for gathering new challenge examples and evaluating changing models. — Dynabench — Original Research Paper.
  8. Build Agents Around Grounded Retrieval: Kiela describes specialized agents that actively retrieve and reason over enterprise data to support complex knowledge work. — Contextual AI — Platform Launch.
  9. Bridge Research and Deployment Carefully: Kiela frames production AI as a systems problem involving retrieval, extraction, evaluation, security and deployment—not simply a better standalone language model. — Contextual AI — Platform Launch.