Lessons from Brendan Foody

Lessons from Brendan Foody

Brendan Foody is the co-founder and CEO of Mercor, a platform that recruits domain experts to evaluate and train frontier AI models. He popularized the concept of "agentic data," arguing that AI progress now depends on specialized human feedback rather than scraped text. This profile gathers his thoughts on the changing labor market, how advanced models are actually trained, and the early experiences that shaped how he builds companies.

Part 1: The Bottleneck of AI Evals

1. On the primary AI bottleneck: "The scarcest resource is knowing what good looks like." — Source: Conversations with Tyler
2. On evaluating models: Creating benchmarks requires human experts because you cannot automate the grading of reasoning at the frontier of human knowledge. — Source: Lenny's Podcast
3. On text limits: The internet has run out of high-quality, easily scraped text, forcing labs to generate bespoke data. — Source: Stanford eCorner
4. On why rubrics matter: AI labs need carefully constructed rubrics more than they need endless volumes of raw text. — Source: Conversations with Tyler
5. On automated grading: Using one AI model to grade another works for basic tasks but fails when assessing complex, subjective, or deeply technical responses. — Source: Lenny's Podcast
6. On edge cases: Finding the boundaries of what a model knows requires prodding by someone who understands the domain better than the model does. — Source: Stanford eCorner
7. On reasoning vs. memory: Evaluating a model's ability to memorize facts is cheap; evaluating its ability to trace a logical argument requires expensive human time. — Source: Lenny's Podcast
8. On the shift in data collection: We have moved from passively scraping the web to actively commissioning experts to write specific prompts and answers. — Source: Conversations with Tyler
9. On benchmarking progress: You can only improve a system as fast as you can accurately measure its current capabilities. — Source: Stanford eCorner
10. On data quality: Poor quality feedback actively degrades a model's performance during fine-tuning. — Source: Lenny's Podcast

Part 2: The Value of Human Expertise

1. On paying poets: Mercor pays poets $150 an hour because evaluating the aesthetic quality of generated verse requires deep, specialized human judgment. — Source: Conversations with Tyler
2. On aesthetic standards: When training models on art or poetry, we are effectively enshrining the aesthetic standards of past eras into the systems of the future. — Source: Conversations with Tyler
3. On domain specificity: You cannot hire a generic labeler to tell you if an AI-generated economic model is structurally sound. — Source: Lenny's Podcast
4. On average feedback: Training a model on average human feedback guarantees a mediocre model. — Source: Stanford eCorner
5. On evaluating subtlety: Nuance in language, humor, and tone can only be graded by humans who inherently understand cultural context. — Source: Lenny's Podcast
6. On the cost of expertise: High-end AI training data is expensive because the people qualified to create it have lucrative alternative career options. — Source: Stanford eCorner
7. On crowdsourcing limits: Mechanical Turk-style platforms work for identifying stop signs, not for debugging advanced Python code. — Source: Conversations with Tyler
8. On defining the truth: For many advanced prompts, there is no single right answer, only better and worse arguments that require an expert to rank. — Source: Lenny's Podcast
9. On human intuition: Even the best statistical models lack the intuition an expert relies on when solving an edge-case problem. — Source: Stanford eCorner

Part 3: Agentic Data and Model Training

1. On agentic data: Data is no longer a static asset you download; it is an active, iterative feedback loop between a model and a human. — Source: Stanford eCorner
2. On dynamic training: The most valuable training happens when an expert argues with the model, correcting its assumptions step by step. — Source: Lenny's Podcast
3. On building RL environments: Knowledge workers are transitioning from doing the work to building the reinforcement learning environments that train the models to do the work. — Source: Conversations with Tyler
4. On adversarial testing: You need people actively trying to break the model to find its hidden failure modes before deployment. — Source: Stanford eCorner
5. On the human-in-the-loop: The loop is tightening; experts now review model outputs in real-time to adjust the training trajectory. — Source: Lenny's Podcast
6. On defining good: "Good" is a moving target that shifts as models become more capable, requiring constantly updated human baselines. — Source: Conversations with Tyler
7. On continuous learning: A deployed model is never finished; it requires ongoing expert correction to prevent drift. — Source: Stanford eCorner
8. On data as a moat: Proprietary access to a network of top-tier experts creates a data advantage that competitors cannot easily replicate. — Source: Lenny's Podcast
9. On synthetic data: Synthetic data generated by models is useful, but it eventually requires a human anchor to prevent compounding errors. — Source: Conversations with Tyler

Part 4: The Future of Knowledge Work

1. On repetitive analysis: Jobs that rely entirely on processing structured information and returning a standard analysis will be fully automated. — Source: Stanford eCorner
2. On shifting roles: Workers will move from executing tasks to managing a fleet of specialized AI agents. — Source: Lenny's Podcast
3. On stumping experts: We are approaching a point where AI can reliably stump leading academics in their own narrow subfields. — Source: Conversations with Tyler
4. On junior employees: The traditional apprenticeship model is breaking down because models can now do the entry-level work cheaper and faster. — Source: Stanford eCorner
5. On wage premiums: There will be a massive wage premium for individuals who possess truly unique, un-modellable expertise. — Source: Lenny's Podcast
6. On prompt engineering: Prompt engineering is a temporary bridge; eventually, models will infer intent without needing carefully structured syntax. — Source: Conversations with Tyler
7. On economic value: Generating value will depend less on doing the work and more on deciding which work is worth doing. — Source: Stanford eCorner
8. On managing AI: The defining skill of the next decade is the ability to break a complex project into discrete tasks an AI can execute. — Source: Lenny's Podcast
9. On the pace of improvement: Models are improving at economically valuable tasks faster than most legacy corporations can adapt their workflows. — Source: Conversations with Tyler

Part 5: Rethinking Hiring and Talent

1. On avoiding vibes: Interviewers often fall for charisma and vibes rather than objectively measuring a candidate's ability to do the job. — Source: Conversations with Tyler
2. On cover letters: AI optimization means every cover letter will eventually look flawless, rendering them completely useless as a screening tool. — Source: Lenny's Podcast
3. On the return of nepotism: As traditional signaling mechanisms break down, companies may revert to high-trust, network-based hiring. — Source: Conversations with Tyler
4. On identifying intellect: Raw processing speed and adaptability matter more than specific legacy software skills. — Source: Stanford eCorner
5. On the inefficiency of resumes: A resume is a highly compressed, lossy format that fails to capture how someone actually thinks through a problem. — Source: Lenny's Podcast
6. On scaling discovery: Software allows us to find a brilliant systems architect in a rural town who would never pass a traditional corporate screen. — Source: Stanford eCorner
7. On meritocracy: A true meritocracy requires testing people on actual work output, not their ability to navigate an interview process. — Source: Lenny's Podcast
8. On adapting to change: The best hires are those who can completely abandon their old workflow when a better tool becomes available. — Source: Conversations with Tyler
9. On credentialism: Degrees matter less when you can administer a real-time, AI-graded test that proves exactly what a candidate can do. — Source: Stanford eCorner

Part 6: Entrepreneurship and Early Lessons

1. On middle school business: Running a donut operation in eighth grade taught the basics of supply, demand, and identifying a captive market. — Source: Conversations with Tyler
2. On driving out competition: If you provide a superior product at the exact moment of highest demand, incumbent competitors lose their leverage. — Source: Conversations with Tyler
3. On authority: Dealing with school administrators trying to shut down a student business was an early lesson in navigating bureaucracy. — Source: Lenny's Podcast
4. On unit economics: Profitability at a small scale enforces discipline that is critical when a business expands. — Source: Stanford eCorner
5. On hustle: Early entrepreneurship is entirely about a willingness to do unglamorous manual work until you can build a system to replace it. — Source: Lenny's Podcast
6. On direct sales: Looking someone in the eye and asking them to pay for something is a skill that transfers directly from selling pastries to enterprise software. — Source: Conversations with Tyler
7. On failure: Early missteps are cheap; the goal is to make all your fundamental mistakes before real capital is on the line. — Source: Stanford eCorner
8. On execution over ideas: Having a good idea is common, but showing up every morning at 5 AM to execute it is rare. — Source: Lenny's Podcast
9. On youth as an advantage: Being young means you have fewer preconceived notions about how an industry is supposed to work. — Source: Conversations with Tyler
10. On continuous iteration: A business model is a hypothesis; the market provides immediate, unforgiving feedback on whether you are right. — Source: Stanford eCorner

Part 7: Dyslexia and Thinking Differently

1. On cognitive diversity: Dyslexia forces a reliance on alternative methods of information processing, which can become a structural advantage. — Source: Conversations with Tyler
2. On visual thinking: When reading is difficult, you naturally gravitate toward visual and spatial problem-solving frameworks. — Source: Lenny's Podcast
3. On delegation: Knowing early on that certain tasks take you longer forces you to become exceptionally good at delegating to those who are faster. — Source: Conversations with Tyler
4. On building systems: If rote memorization is a weakness, you survive by building external systems and processes to track information for you. — Source: Stanford eCorner
5. On big-picture strategy: A difficulty with micro-details often correlates with a highly developed ability to see macro-patterns and strategic shifts. — Source: Lenny's Podcast
6. On entrepreneurship and dyslexia: There is a clear link between dyslexia and founding companies, likely driven by the early necessity of navigating a world built for a different cognitive profile. — Source: Conversations with Tyler
7. On alternative learning: Traditional schooling penalizes dyslexia, which pushes many to seek alternative, faster paths to acquiring practical knowledge. — Source: Stanford eCorner
8. On overcoming friction: Getting comfortable with constant friction in basic tasks builds a high tolerance for the general difficulty of running a startup. — Source: Lenny's Podcast
9. On communication: When reading large texts is slow, you learn to communicate ideas concisely and demand the same from your team. — Source: Conversations with Tyler

Part 8: Scaling and Building Mercor

1. On rapid growth: Scaling from $1 million to $500 million requires systems that don't just work well, but work seamlessly without human intervention. — Source: Lenny's Podcast
2. On the Thiel Fellowship: The fellowship provided validation and a network, operating effectively as an accelerant for an already fast-moving trajectory. — Source: Conversations with Tyler
3. On scaling the fellowship model: The goal is to take the concentrated talent identification of the Thiel Fellowship and scale it globally through software. — Source: Conversations with Tyler
4. On talent marketplaces: A successful marketplace requires perfectly balancing supply and demand; acquiring thousands of experts means nothing if you lack the labs to deploy them. — Source: Stanford eCorner
5. On operational intensity: Hypergrowth breaks every internal process every three months; you have to hire people who thrive in a constant state of rebuilding. — Source: Lenny's Podcast
6. On working with AI labs: Top labs move at a blistering pace and expect their partners to match their speed and technical rigor. — Source: Stanford eCorner
7. On finding product-market fit: Fit happened the moment labs realized their models had plateaued on scraped text and they were desperate for a new data source. — Source: Lenny's Podcast
8. On capital efficiency: Throwing money at a problem usually masks a deeper operational flaw that will eventually destroy margins. — Source: Stanford eCorner
9. On company culture: Culture during hypergrowth is defined entirely by who you fire and who you promote, not by what is written on a wall. — Source: Lenny's Podcast
10. On long-term vision: The ultimate aim is to map the world's expertise and dynamically route human intelligence to wherever algorithms need it most. — Source: Conversations with Tyler