Dylan Patel founded SemiAnalysis, where he researches semiconductor supply chains and AI infrastructure. His work connects chip design and manufacturing constraints with the economics of large-scale AI systems. — Sequoia Interview.

Part 1: The Invisible Backbone — Semiconductor Fundamentals
- Industry Complexity: Patel describes semiconductors as an unusually complex industry involving millions of specialists. Even long experience in one part of the stack leaves much of the rest to learn. — The Inside View Interview.
- Why Chips Matter: Patel regards semiconductors as central to modern technological progress because improvements in chips enable advances across software, products, and other industries. — Latent Space Interview.
- Software's Physical Foundation: Computing ultimately depends on physical transistors that switch electrical current; progress in software therefore rests partly on the capabilities and limits of manufactured chips. — The Future of the Transistor.
- Node Transitions: Moving to leading-edge nodes requires more than smaller features: transistor designs must manage leakage and switching efficiency as conventional planar structures reach their limits. — The Future of the Transistor.
- Yield Management: Process tuning is a multivariable problem: yield, performance, and power all matter. A node that produces too few good chips can squander otherwise attractive design gains. — Dwarkesh Semiconductor Interview.
- Power Delivery and Routing: At advanced chip nodes, electrical routing can limit efficiency as narrow interconnects increase resistance and heat. Backside power delivery is one response to those physical constraints. — Clash of the Foundries.
- The Cost of Leading-Edge Fabs: In a 2025 interview, Patel estimated that a base leading-edge 3nm or 2nm fab block could cost more than $30–40 billion, with multiple blocks potentially needed. The capital hurdle helps explain why advanced manufacturing has concentrated. — Lex Fridman Podcast #459.
- The Economics of Scaling: Leading-edge scaling still improves some chip capabilities, but advanced fab capital costs have risen and SRAM scaling has slowed. Technical progress and lower cost per function should not be treated as the same thing. — Clash of the Foundries.
- Design and Verification Complexity: More complex AI chips, tighter schedules, and scarce engineering talent make design verification a major bottleneck. A chip can require costly new mask sets and months of delay if a design needs another stepping. — The EDA Primer.
- The Foundry Value Chain: As the cost of leading-edge fabs rose, many chip designers stopped manufacturing their own advanced chips and relied on specialist foundries such as TSMC. Design remains important, but advanced manufacturing capacity is concentrated in fewer hands. — Lex Fridman Podcast #459.
Part 2: The Nvidia Monopoly — Hardware, Software, and Networking
- Nvidia's Three-Part Advantage: Patel analyzes Nvidia's lead as a combination of accelerator hardware, a broad software ecosystem, and networking systems. Those layers reinforce one another, even as competitors improve. — No Priors Interview.
- The Software Ecosystem: Nvidia's developer tools and model ecosystem remain a competitive advantage, but Patel and his coauthors note that AMD's software has improved and the gap is not static. — SemiAnalysis: AMD 2.0.
- Networking as a Scaling Constraint: Large AI clusters depend on interconnect performance as well as accelerator speed; inadequate networking can prevent a GPU fleet from delivering its theoretical compute throughput. — Nvidia's Optical Ascent.
- Scale-Up and Scale-Out Fabrics: Nvidia's rack systems use NVLink for scale-up communication within a rack, while larger clusters still need a back-end scale-out network that can use InfiniBand or Ethernet. The fabrics solve different connectivity problems. — Nvidia's Optical Boogeyman.
- Cross-Layer Co-Design: Nvidia's systems advantage comes in part from optimizing silicon, networking and infrastructure software together rather than treating each layer as an isolated product. — Sequoia Interview.
- Competition from AMD: AMD has improved its AI accelerator software and offers credible hardware competition. Nvidia still benefits from its broader ecosystem and system-level integration, so the competitive balance depends on the workload and product generation. — SemiAnalysis: AMD 2.0.
- Blackwell at Rack Scale: Nvidia's GB200 NVL72 packages 72 GPUs with CPUs, NVSwitches, and multiple network interfaces in an integrated rack. This shifts evaluation from a standalone GPU toward the performance and deployment requirements of a whole system. — Nvidia's Optical Boogeyman.
- Upstream Supply Commitments: In its 2023 analysis, SemiAnalysis argued that Nvidia's early commitments for HBM and CoWoS packaging capacity helped it supply AI accelerators when rivals faced tight upstream availability. — Nvidia's Plans to Crush Competition.
- The Economics of GPU Scarcity: Patel distinguishes Nvidia's chip margins from the margins earned by clouds renting its GPUs. In his conditional H100 rental example, utilization, contract length, depreciation and scarcity all affect whether the cloud earns an attractive return. — Dwarkesh Interview.
- Annual Accelerator Updates: SemiAnalysis reported in 2023 that Nvidia was moving toward annual AI-GPU updates while competitors advanced custom accelerators and software alternatives. The shorter product cycle raised the pace at which the company and its customers had to evaluate new systems. — Nvidia's Plans to Crush Competition.
Part 3: The Economics of AI — The Trillion-Dollar Buildout
- Capital-Intensive AI Buildout: Frontier AI progress requires large, continuing investments in accelerators, data centers and electrical infrastructure. SemiAnalysis warned in 2025 that access to capital could limit the pace of that buildout. — SemiAnalysis: Tariff Armageddon?.
- Cost per Token in Context: The cost to generate a million tokens is a key inference measure, but it must be evaluated alongside model quality and response speed. Higher interactivity can reduce throughput and raise the cost per token. — InferenceMAX Benchmarking.
- Uncertain Returns on Early AI Spending: In 2024, SemiAnalysis noted that large GPU purchases were running ahead of proven AI revenue even as demand for compute remained strong. Future returns depended on whether model and application businesses could monetize the capacity. — OpenAI Is Doomed?.
- Scaling Needs Physical Capacity: Larger frontier training runs require not just more accelerators but sites with sufficient power, networking, permitting and construction progress. These constraints were already pushing model developers toward multi-data-center plans in 2024. — Multi-Datacenter Training.
- Inference as a Continuing Cost: Unlike a single model-training run, serving users creates continuing hardware and operating costs. The economics depend on how many useful tokens a system delivers at the latency customers need. — InferenceMAX Benchmarking.
- Measuring AI's Economic Value: A bubble judgment based only on AI infrastructure spending can miss value that appears as faster work rather than recorded AI revenue. Patel and a coauthor argue that conventional economic statistics may capture the costs sooner than the output. — AI Dark Output.
- Value Moves Across the Stack: SemiAnalysis described infrastructure capturing much of AI's early profit pool, then argued that stronger model and inference businesses were capturing more value as useful agentic applications grew. The margin picture changes with adoption and serving costs. — AI Value Capture.
- GPU Useful Life Is Conditional: Patel presents two views of GPU depreciation: newer chips can lower the rental price of older models, while supply scarcity and improving model utility can keep older GPUs economically useful. A universal two-to-three-year replacement cycle does not follow. — Dwarkesh Interview.
- Capital Conditions Matter: In its 2025 analysis, SemiAnalysis warned that rising long-term interest rates and tighter financing could slow the AI infrastructure buildout, even where the underlying technology demand remained strong. — SemiAnalysis: Tariff Armageddon?.
- Gulf Capital and AI Infrastructure: SemiAnalysis argued in 2025 that U.S. agreements with the UAE and Saudi Arabia could bring substantial capital and energy-backed capacity into the AI buildout. Gulf investment was one channel, not the sole source sustaining global spending. — AI Arrives in the Middle East.
Part 4: Geopolitical Chip Wars — Taiwan, China, and Export Controls
- Taiwan's Mature-Node Role: In a 2024 discussion, Patel estimated that Taiwan made about 80% of the world's 28nm chips, including components used in many everyday electronics. That made a Taiwan supply interruption consequential beyond the most advanced AI chips. — Dwarkesh Semiconductor Interview.
- A Taiwan Disruption Scenario: Patel argued that a war interrupting Taiwanese chip production could trigger a severe market shock and shortages of chips needed across consumer and industrial products. This was a scenario, not a prediction of an inevitable outcome. — Dwarkesh Semiconductor Interview.
- China's Domestic Chip Ramp: SemiAnalysis reported that SMIC and Huawei were expanding domestic AI-chip manufacturing, while advanced-node yields and access to key equipment remained constraints. The pace of catch-up depends on process improvements and capacity allocation rather than a fixed number of years. — Huawei Ascend Production Ramp.
- Export Controls and Their Gaps: SemiAnalysis concluded that U.S. controls had constrained Chinese AI-chip output, especially access to advanced memory, while uneven allied enforcement and stockpiling left important gaps. Their effect depends on implementation as well as the written rules. — Huawei Ascend Production Ramp.
- Huawei and SMIC's Integration: Huawei has sought more control over Ascend manufacturing, working with SMIC while investing in its own fabrication and supply chain. SemiAnalysis identified yield and high-bandwidth-memory availability as limits on how many complete AI accelerators could be shipped. — Huawei Ascend Production Ramp.
- ASML's EUV Supply Constraint: Patel has described ASML's EUV lithography tools as exceptionally complex and forecast their production capacity as a potential late-decade limit on leading-edge chip output. He also noted uncertainty about how quickly alternative domestic toolchains may develop. — Dwarkesh Interview.
- The CHIPS Act Is One Part of Industrial Policy: Patel argued that U.S. fab subsidies were a useful start, but that durable semiconductor capacity also required changes to tax incentives, permitting, research funding and the skilled-worker pipeline. — Why America Will Lose Semiconductors.
- National Access to AI Chips: In its 2025 analysis of proposed AI-diffusion export rules, SemiAnalysis showed how country tiers, licensing and purchase caps could change where AI accelerators were deployed. Access to compute had become a geopolitical policy question, not only a supplier allocation decision. — 2025 AI Diffusion Export Controls.
- Rebuilding Lost Taiwanese Capacity: Patel warned that moving engineers alone would not quickly replace Taiwan's fabs after a destructive disruption. Chipmaking equipment itself depends partly on Taiwanese components, so rebuilding production elsewhere would be a lengthy, difficult process with a potentially severe economic cost. — Dwarkesh Interview.
- Open Models as Commercial Strategy: Patel and a coauthor argued that Meta's open-model releases could reduce rivals' commercial advantage and help attract AI talent. Their stated mechanism was competition in model distribution, not a verified strategy to target China. — OpenAI Is Doomed?.
Part 5: Infrastructure Bottlenecks — Power, Packaging, and Cooling
- Power Infrastructure Gates Cluster Buildout: Patel identified electricity generation, transmission, substations and transformers as constraints on building the largest U.S. AI clusters. In 2025 he discussed a roughly 10% share of U.S. electricity for all data centers by 2028–2030 as a scenario, not a measured AI-only share. — Lex Fridman Podcast #459.
- CoWoS Packaging as a Supply Gate: In 2023, SemiAnalysis identified TSMC's CoWoS advanced-packaging capacity, alongside high-bandwidth memory, as a key constraint on H100 supply. Wafer production alone did not determine how many usable accelerators could ship. — AI Capacity Constraints.
- Why HBM Matters: H100-class AI accelerators place high-bandwidth-memory stacks close to the logic die through advanced packaging, giving the chip access to large amounts of fast memory. SemiAnalysis reported that HBM supply was one of the constraints on accelerator output in 2023. — AI Capacity Constraints.
- Cooling Follows Rack Density: SemiAnalysis contrasted roughly 125 kW liquid-cooled GB200 NVL72 racks with lower-density, roughly 40 kW designs intended for air cooling. Cooling requirements depend on the chosen system and available facility infrastructure, not a universal end to air cooling. — Nvidia's Blackwell Reworked.
- Planning for Gigawatt-Scale Campuses: Patel described announced multi-gigawatt data-center campuses intended to host tightly connected training clusters. In his 2025 discussion, he explicitly distinguished a site's proposed full buildout from capacity already constructed. — Lex Fridman Podcast #459.
- Copper and Optics Serve Different Links: SemiAnalysis described copper NVLink connections within an AI rack and optical transceivers for links between racks. As clusters expand, their scale-out networks need more optical links even when in-rack copper remains useful. — Nvidia's Optical Boogeyman.
- Packaging Complexity Can Delay a Ramp: SemiAnalysis reported that Blackwell's move to CoWoS-L brought package-design and manufacturing challenges, including warpage and precise bridge placement, that constrained its 2024 production ramp. Advanced packaging is a productization gate, not just a final assembly step. — Nvidia's Blackwell Reworked.
- Specialty Materials Are Supply Risks: In 2022, Patel identified neon gas and etchant inputs as exposed to the Russia–Ukraine conflict, while also noting inventory and other mitigations. Specialized gases and chemicals matter to fab continuity, but a supply shock does not automatically shut down production. — I, Semiconductor.
- High-Current Power Delivery: As AI compute boards draw more power, delivering electricity at a higher voltage and converting it close to the chips can reduce current and resistive losses. SemiAnalysis illustrated this trade-off in its analysis of Vera Rubin's compute tray. — Vera Rubin Extreme Co-Design.
- Fault Recovery at Cluster Scale: At 100,000-GPU scale, individual GPU, network and optical-link failures can interrupt synchronous training. Patel and a coauthor argued for spare nodes, checkpointing and faster memory reconstruction to keep work progressing; their numerical failure estimate depended on stated link-reliability assumptions. — 100,000 H100 Clusters.
Part 6: The Hyperscaler Strategy — TPUs, Trainium, and Vertical Integration
- Google's TPU Infrastructure: Patel and a coauthor argued in 2023 that Google's in-house TPU fleet and supporting systems gave it unusual control over AI compute capacity. Google also used GPUs, so its advantage was not complete independence from Nvidia hardware. — Google Gemini Eats the World.
- Trainium Fits Particular Workloads: SemiAnalysis found that AWS Trainium2 was weaker than Nvidia systems on several raw specifications but competitive on memory bandwidth per dollar of total cost for some workloads. Anthropic's involvement in Trainium design illustrates why a custom accelerator's value depends on the workload it serves. — Amazon's AI Resurgence.
- Microsoft's First-Generation Maia Constraint: SemiAnalysis reported in 2025 that Microsoft's first Maia accelerator had not reached high-volume production use and lacked memory bandwidth well suited to generative-AI inference. The example shows how many chip generations a hyperscaler may need before custom silicon materially reduces merchant-chip dependence. — Microsoft's AI Strategy Deconstructed.
- Model–Hardware Co-Design: Patel argues that AI gains often come from tuning the model, software kernels and silicon together, because optimizations at one layer affect the others. This is a co-design opportunity for labs and infrastructure providers, not an inevitable closed ownership loop. — Sequoia Interview.
- Interconnect Designs Can Converge: SemiAnalysis observed in 2025 that Trainium's all-to-all NeuronLinkv3 network was moving toward an Nvidia NVLink-like design. The comparison concerns system interconnects, where different accelerators can face similar scaling pressures. — Amazon's AI Resurgence.
- Broadcom's Custom-Silicon Role: SemiAnalysis reported in 2023 that Broadcom had design roles in Google's TPU program and Meta's MTIA, and that Google's TPU ramp was a material driver of its accelerator revenue. This is a specific customer and period, not a claim that Broadcom builds every custom AI chip. — Broadcom's Google TPU Revenue Explosion.
- Cloud Value Beyond GPU Rentals: A GPU cloud's economics depend on hardware cost, useful life, pricing and the services layered above raw compute. SemiAnalysis argued that enterprise integration could add value for Azure, while acknowledging uncertainty in hardware lifetimes and competitive pricing. — Microsoft's AI Strategy Deconstructed.
- GPU Cloud Operating Quality: GPU-cloud providers differ in network topology, storage, monitoring, fault recovery and managed software. SemiAnalysis's ClusterMAX comparison treats those operating qualities as important to usable training capacity; it does not reduce neoclouds to a temporary pricing arbitrage. — ClusterMAX GPU Cloud Ratings.
- Apple's Split AI Compute Strategy: In a 2024 analysis, SemiAnalysis described Apple's AI strategy as spanning on-device inference, its own data centers and partner-cloud capacity. Those locations offer different privacy, latency and compute trade-offs. — Apple's AI Strategy.
Part 7: The Battle for Models — GPU Rich vs. GPU Poor
- The GPU-Poor Divide: In 2023, Patel and a coauthor used GPU-poor to describe startups and open-model researchers working with much smaller compute fleets than frontier labs. Their limited training capacity shaped which experiments and model scales were practical; it did not mean they had to rely only on APIs. — Google Gemini Eats the World.
- Google's 2023 Compute Opportunity: Patel and a coauthor called Google a sleeping giant in 2023 because its expanding TPU systems gave it substantial capacity to train Gemini. They presented the resulting model-performance opportunity as a forecast. — Google Gemini Eats the World.
- Training Clusters Keep Scaling: In a 2025 interview, Patel described OpenAI's then-current roughly 100,000-GPU training scale and plans across the industry for much larger connected clusters. He distinguished deployed capacity from forecasts and said tightly coupled networks and power draw become more consequential as clusters grow. — Lex Fridman Podcast #459.
- Goodput, Not Just GPU Count: The useful output of a training cluster depends on scheduling, network health and fault recovery as well as the number of GPUs. SemiAnalysis's ClusterMAX evaluation describes managed orchestration and active monitoring as ways providers raise goodput. — ClusterMAX GPU Cloud Ratings.
- Distillation as a Compute Lever: SemiAnalysis described Microsoft's access to OpenAI reasoning traces as a way to distill capabilities into smaller models without repeating the full frontier training effort. That partnership gave Microsoft a specific path to lower-compute model development. — Microsoft's AI Strategy Deconstructed.
- Training-Cost Competition: Patel reported in 2022 that MosaicML aimed to cut the cost of training a GPT-3-quality model from roughly $450,000 to $100,000, while Cerebras challenged GPU economics on larger GPT-style models. The MosaicML figure was a target, not a measured result. — GPT Model Training Competition Heats Up.
- Beyond Public-Text Scaling: SemiAnalysis argued that high-quality public text was becoming harder to obtain relative to available training compute, making simple pre-training expansion less straightforward. The same analysis identified synthetic data, other modalities and post-training as additional routes for improvement, rather than a total data wall. — Scaling Laws and Reasoning Infrastructure.
- De-Risking a Large Training Run: Patel described testing architecture choices through smaller experiments before committing a large cluster to a major training run. Some ideas that work at small scale fail at large scale, so careful ablations reduce—but cannot eliminate—the risk of an expensive run. — Lex Fridman Podcast #459.
- Gulf Capital for AI Infrastructure: Patel and coauthors analyzed 2025 U.S. agreements with the UAE and Saudi Arabia that paired Gulf capital and power with planned data centers and accelerator deployments. The commitments expand potential AI infrastructure; they do not by themselves establish frontier-model capability. — AI Arrives in the Middle East.
- Open-Model Catch-Up Is Era-Specific: A 2026 SemiAnalysis comparison found that open models narrowed selected benchmark gaps with closed frontier models at different speeds across successive eras. The authors caution that benchmarks do not fully capture real-world usefulness, so no fixed six-to-twelve-month lead applies to every capability. — Are Open Models Catching Up?.
Part 8: The Future of Compute — Tokens, Inference, and Scaling Laws
- Prefill and Decode Can Be Split: InferenceMAX describes disaggregated serving that assigns prefill and decode stages to different GPU resources so the stages interfere less under heavy concurrency. The specialization is in how existing compute resources are allocated. — InferenceMAX Benchmarking.
- A Large Market for Inference: Patel argued in a direct interview that widespread token use and the value created from inference could make it one of the world's largest markets. That is his forward-looking market thesis, not a demonstrated outcome or a claim that one producer will control the 2030s economy. — Sequoia Interview.
- Reasoning Uses Test-Time Compute: Patel and a coauthor observed that reasoning models were improving partly by spending more compute at test time and sustaining longer chains of thought. The trend complements, rather than simply replaces, model training. — Scaling Reinforcement Learning.
- Long Context Stresses Memory: Longer context windows enlarge the KV cache and complicate inference memory placement. Patel's coauthored analyses discuss memory-tier and data-preparation strategies that can make long-context workloads more practical. — Vera Rubin Extreme Co-Design.
- On-Device and Cloud AI Trade-offs: SemiAnalysis's 2024 Apple analysis describes AI workloads split between on-device processing, Apple's own servers and partner clouds. Device-side execution can serve different latency and privacy needs, but the article does not quantify a 90% local-request target. — Apple's AI Strategy.
- Sparse Experts Trade Compute for Communication: Mixture-of-experts serving routes a token to selected expert tensors rather than activating every expert at once. Patel's coauthored analysis shows that this can change useful compute and memory placement while adding communication and scheduling costs; it is not the only possible scaling path. — Computation and Data Movement for Inference.
- Verify Synthetic Training Data: Patel's coauthored 2026 analysis notes that reinforcement-learning environments use CPUs to run code, calculate rewards and verify generated synthetic data. Verification is part of making those generated examples useful for training. — CPUs Are Back.
- Quantum Computing's Practical Barriers: In a 2023 essay, Patel argued that no useful quantum computer had yet been built and highlighted qubit noise, manufacturing and error-correction challenges. His assessment focused on the engineering barriers to practical systems. — Separating Reality from Hype: Quantum Computing.
- Measure Useful Inference Economics: InferenceMAX compares total cost per million tokens alongside user interactivity, rather than judging accelerators by peak FLOPs alone. This is a practical workload-level measurement framework, not a proposed universal decisions-per-watt unit. — InferenceMAX Benchmarking.
Part 9: Market Realities — Commoditization and Investment Strategy
- AI Input Costs Pressure Software Margins: The cost of model inference changes application economics: a software company has to account for the compute it consumes to serve each customer. Patel's 2025 interview framed that cost as a challenge to traditional high-margin software assumptions. — Inside the Trillion-Dollar AI Buildout.
- Compute Concentration Is a Forecast Risk: In a 2026 interview, Patel forecast that a small number of frontier labs could command a large share of usable AI compute if current trends continued. He treated that concentration as a concern, not an established permanent limit of five or six competitors. — Dwarkesh Patel Interview.
- Efficiency Supports GPU Margins: Patel contrasted Nvidia and AMD GPUs on performance, chip size, power efficiency and margins in recounting the earlier GPU market. The comparison illustrates how technical efficiency can support a supplier's economics without proving record pricing power across all industries. — Sequoia Interview.
- The Electrical Buildout Matters: Patel identifies generation, transmission, substations, transformers and cooling as constraints on building large AI clusters. Chip availability alone cannot determine deployment speed when those supporting systems are delayed. — Lex Fridman Podcast #459.
- Physical Capacity in the Chip Cycle: Patel's fab analysis tracks semiconductor revenue, equipment-backed capacity additions and existing wafer capacity, with revenue per area of silicon affecting margins. These physical measures help explain the supply cycle; the essay does not prescribe replacing equity valuation with wafer starts. — Semiconductor Fab Buildout Delays.
- Overbuild Risk Depends on Model Progress: Patel argued that AI compute expansion remains economical only if models keep enabling valuable work faster than infrastructure capacity grows. He acknowledged that the balance could reverse if model progress stalls; that is a conditional risk, not a documented training-versus-inference timing failure. — Sequoia Interview.
Part 10: Career and Analytical Philosophy — The SemiAnalysis Method
- Learning Through Hardware Forums: Patel traces his interest in hardware to repairing a broken Xbox 360 and then reading, posting and moderating technology forums while young. That is his documented origin story, without the alleged universal learning maxim. — Sequoia Interview.
- Technical and Financial Research Together: Patel describes SemiAnalysis as combining engineers across the supply chain with people trained in financial markets. Its research follows niche process inputs and equipment costs as well as the economics of the companies using them. — Sequoia Interview.
- Learn from Supply-Chain Specialists: Patel says he attends specialist conferences, asks researchers and suppliers about current practice, and uses those conversations to understand details that papers and public descriptions leave out. Direct contact complements published technical work. — Sequoia Interview.
- Curiosity Sustained the Research: Patel describes repeatedly reading about chips and AI, writing publicly about what interested him, and following new questions through conferences and supplier contacts. Sustained curiosity shaped his research practice. — Sequoia Interview.
- Keep Benchmarks Current: Patel says new models and inference-software releases rapidly change measured performance, making one-off benchmarks stale. SemiAnalysis's recurring tests respond to that pace. — Sequoia Interview.
- Test Claims Against Physical Trade-offs: Patel's research combines technical specialists with economic analysis and asks suppliers how present systems work and what their components cost. That method checks impressive performance claims against physical and financial trade-offs without presuming the claimant is dishonest. — Sequoia Interview.
- Post-Training Was Still Early: In a 2025 interview, Patel characterized post-training and reinforcement learning as still early in their development. That is a dated view about a particular phase of AI progress, not a claim that today's infrastructure will anchor the next hundred years. — Inside the Trillion-Dollar AI Buildout.