Lessons from Jack Krawczyk

Lessons from Jack Krawczyk

Jack Krawczyk is a product manager who built consumer experiences at Pandora and WeWork before leading development on Google's Bard and Gemini. He defined a framework for managing "probabilistic products," which helps teams build AI tools that inherently guess and hallucinate. This profile collects his methods for structuring product teams, launching early versions, and designing interfaces around unpredictable technology.

Part 1: Product Management Philosophy

1. On the core role: "I think every great product leader needs to be a deep product marketer at their core." — Source: [First Round Capital]
2. On understanding users: You must spend more time asking open-ended questions like a customer would, rather than jumping straight to solutions. — Source: [First Round Capital]
3. On user research: Understanding the voice of the consumer is the foundation of building products that respect the user experience while hitting business goals. — Source: [ClickZ Live]
4. On documentation: Use an executive summary document to tier information and keep everyone aligned on high-level goals without drowning them in detail. — Source: [First Round Review]
5. On iterating: Formalize processes like data reviews so teams capture lessons from missteps without being overwhelmed by the bureaucracy. — Source: [First Round Capital]
6. On dogfooding: Build the first versions yourself and use them internally to understand the feel before releasing them to anyone else. — Source: [First Round Capital]
7. On speed: Get technology out to users quickly to learn how they interact with it, because you cannot perfect a product in a vacuum. — Source: [This Week in Startups]
8. On product mechanics: Learn the technical realities of your product, even if it means jumping in to manage ad insertion logic yourself. — Source: [First Round Capital]
9. On balancing priorities: The tension between user experience and monetization is inevitable, and a product manager's job is to make ad experiences palatable while driving brand awareness. — Source: [Worth]
10. On taking ownership: Treat your own team and direct reports with the same discipline and strategic thinking you apply to software products. — Source: [First Round Capital]

Part 2: Generative AI and Creativity

1. On defining AI: "I describe generative AI as an idea creator. It helps take an idea in your mind, helps you find the words to describe it, and makes it approachable for people." — Source: [Accel Spotlight]
2. On reducing friction: "Creativity is one of the most amazing parts of humanity like our ability to imagine is magical and we're reducing the friction to imagination." — Source: [This Week in Startups]
3. On user communication: When AI helps people articulate their thoughts, it gives their ideas legs so they can speak them into the world effectively. — Source: [Accel Spotlight]
4. On familiar interfaces: When a new technology emerges, users naturally try to use it for what is most familiar, much like early TV shows were just people reading radio scripts. — Source: [Accel Spotlight]
5. On guiding users: Builders must anticipate familiarity bias and design tools to help users transition from old habits to new capabilities. — Source: [Accel Spotlight]
6. On human control: The user must always remain in control of the AI rather than feeling like the system is dictating the final outcome. — Source: [This Week in Startups]
7. On drafting versus final product: AI is best positioned as a starting point for drafts and brainstorming, allowing the human to finish the thought and retain authorship. — Source: [Accel Spotlight]
8. On imagination: The ultimate goal of consumer AI is to make the act of imagining something and bringing it into reality as fast as possible. — Source: [This Week in Startups]
9. On early adoption: Early versions of generative products are less about solving a specific workflow and more about exposing the raw capability so users can find the utility. — Source: [This Week in Startups]
10. On creative block: Generative models serve as a direct counter to the blank page problem, giving people a thread to pull when they do not know where to start. — Source: [Accel Spotlight]

Part 3: Navigating Probabilistic Products

1. On shifting mindsets: Product managers must change how they work to account for the probabilistic nature of large language models. — Source: [First Round Capital]
2. On hallucinations: "If you're going at it from a path of like 'I'm going to build the most non-hallucinated answer' what you're going to get is 'we don't know' versus embracing the hallucination." — Source: [First Round Capital]
3. On paradigm shifts: "If you can find a way to harness that build the guard rails around probability-oriented products then it's going to be an incredibly fortuitous paradigm shift." — Source: [First Round Capital]
4. On confident inaccuracies: Language models are predictive and can be confidently wrong, meaning users have to maintain a critical eye rather than treating outputs as strict facts. — Source: [Platformer]
5. On consumer vs enterprise AI: Different sectors have varying tolerances for imprecision; the enterprise space requires high reliability, while consumer tools can be more experimental. — Source: [First Round Capital]
6. On building guardrails: You cannot demand perfect precision from a probabilistic model; success comes from building safety nets around its natural tendency to guess. — Source: [First Round Capital]
7. On probability as a feature: Instead of fighting the model's tendency to predict words, teams should view this probabilistic behavior as a feature rather than a bug. — Source: [First Round Capital]
8. On deterministic thinking: Legacy product management assumes a single action leads to a single outcome, but AI leads to a range of probable outcomes that require entirely new testing strategies. — Source: [First Round Capital]
9. On setting expectations: The interface must communicate to the user that the system is guessing so they know to verify the information. — Source: [Platformer]
10. On testing unpredictable software: Because outputs change constantly, teams must test for acceptable ranges of behavior rather than strict binary responses. — Source: [First Round Capital]

Part 4: Leadership and Team Empowerment

1. On managing managers: A senior product leader is a leader of leaders, and their primary job is to help product managers own their ideas. — Source: [First Round Capital]
2. On giving credit: The spotlight belongs to the product managers and cross-functional partners who execute the work, not the person managing them. — Source: [First Round Capital]
3. On idea generation: A manager should help their team articulate and amplify their own insights rather than stepping in to generate ideas themselves. — Source: [First Round Capital]
4. On engineering context: Engineers need a clear and current understanding of the logic behind product decisions so they can contribute creatively. — Source: [First Round Capital]
5. On coaching: Treating your team as your product means applying product development rigor to how you train and level up the people you manage. — Source: [First Round Capital]
6. On listening: The defining trait of a great product manager is the ability to listen quietly and interpret what is actually needed from the noise. — Source: [First Round Capital]
7. On guiding teams: Ask questions to guide reports to a conclusion, letting them build the logic rather than handing them the answer. — Source: [First Round Capital]
8. On autonomy: Empowering product managers means stepping back and letting them run meetings and drive strategy with stakeholders directly. — Source: [First Round Capital]
9. On shared context: Information silos kill momentum, making it a leader's job to ensure engineering, design, and business all have the exact same context. — Source: [First Round Capital]

Part 5: Organizational Scaling

1. On the Dunbar Number: Apply the Dunbar Number (the cognitive limit to stable social relationships) to how you structure teams so people do not overextend themselves. — Source: [First Round Capital]
2. On structural pillars: As teams scale, organize them around functional pillars (like listener experience or ad technology) to maintain focus. — Source: [First Round Capital]
3. On deep relationships: You can only maintain a limited number of real working relationships, so design org charts that respect human cognitive limits. — Source: [First Round Capital]
4. On Group PMs: Dedicating Group PMs to specific verticals ensures that strategy does not dilute as the company adds headcount. — Source: [First Round Capital]
5. On staying close to the work: Even as an executive, you must build systems that keep you connected to the actual product development cycle. — Source: [First Round Capital]
6. On operational drag: If you try to maintain relationships across too many distinct teams, you will slow everyone down and become a bottleneck. — Source: [First Round Capital]
7. On clear boundaries: Teams move faster when they know exactly where their scope ends and another team's scope begins. — Source: [First Round Capital]
8. On rallying a company: You cannot align a large company by commanding them; you have to rally them around shared, documented objectives. — Source: [First Round Capital]
9. On process: Implement enough process to capture learnings and prevent repeated mistakes, but stop before the process becomes the product. — Source: [First Round Capital]

Part 6: Cross-Functional Collaboration

1. On connecting teams: The product manager is the translation layer between engineering constraints and business requirements. — Source: [First Round Capital]
2. On engineering involvement: Include engineering in the strategy phase early so they understand the problem space before writing code. — Source: [First Round Capital]
3. On design relationships: Design and product must debate the user experience constantly to ensure the solution actually reduces friction. — Source: [First Round Capital]
4. On business goals: Product cannot ignore revenue; you have to build features that make business models like advertising work without ruining the app. — Source: [Worth]
5. On resolving conflict: Use data and direct user feedback to settle arguments between internal stakeholders. — Source: [ClickZ Live]
6. On marketing alignment: If you view your role as a product marketer, you will naturally pull the marketing team into the development cycle earlier. — Source: [First Round Capital]
7. On shared wins: Celebrate product launches by highlighting the engineers and designers who built it, removing the product manager from the center. — Source: [First Round Capital]
8. On avoiding silos: An executive summary document forces cross-functional leads to read and agree on the exact same one-pager. — Source: [First Round Capital]
9. On regular touchpoints: Schedule systematic check-ins with cross-functional partners so alignment happens predictably, not just during a crisis. — Source: [First Round Capital]

Part 7: Managing Risk and Trust

1. On speed versus trust: "You can release things to generate that speed of insight, but you can't generate that speed of insight at the expense of trust." — Source: [This Week in Startups]
2. On responsible release: Releasing AI products requires a measured approach where you intentionally look for where the system breaks before users do. — Source: [This Week in Startups]
3. On data transparency: Users must have transparency regarding how their data is used and how the AI is generating its responses. — Source: [This Week in Startups]
4. On intentionality: When building large language models, teams must be uniquely intentional about risk and establish strict guardrails early. — Source: [First Round Capital]
5. On handling errors: Acknowledge when systems produce inaccurate or biased results, and commit publicly to implementing corrections. — Source: [India Times]
6. On brand risk: Moving fast in consumer tech is normal, but moving fast with AI requires an extra layer of caution to protect the company's reputation. — Source: [This Week in Startups]
7. On user trust: Once users lose trust in a probabilistic product's basic safety, it is incredibly difficult to win them back to experiment with it. — Source: [This Week in Startups]
8. On mitigating bias: AI development requires constant tuning to prevent the model from reflecting the worst biases of its training data. — Source: [India Times]
9. On public scrutiny: Leaders building frontier technology must expect intense public attention and be prepared to navigate controversy by focusing on product fixes. — Source: [India Times]

Part 8: Startup Strategy and Execution

1. On shipping quickly: You cannot learn what an AI product should be without putting it in the hands of users and watching them break it. — Source: [First Round Capital]
2. On acting like a startup: Even within a massive corporation, teams building new technology need to operate with the speed and scrappiness of a startup. — Source: [First Round Capital]
3. On finding utility: Focus less on building a polished final workflow and more on exposing a raw capability that early adopters can shape. — Source: [This Week in Startups]
4. On perfect products: Seeking perfection before launch is a trap; aim for a version that is safe enough to learn from. — Source: [First Round Capital]
5. On the first version: If your team isn't using the product daily to do their own jobs, it is not ready for external users. — Source: [First Round Capital]
6. On adapting: Founders must accept that users will misuse their product initially based on their past habits with older technology. — Source: [Accel Spotlight]
7. On user feedback: True insights come from observing behavior directly rather than just reading support tickets or surveys. — Source: [ClickZ Live]
8. On building safely: You must balance the urge to launch quickly with the absolute requirement to mitigate ethical and technical risks. — Source: [This Week in Startups]
9. On momentum: Keep the team moving by treating mistakes as learning data rather than failures, provided the core trust with the user remains intact. — Source: [First Round Capital]