> ## Content Index
> Fetch the complete content index at: https://www.antoinebuteau.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Lessons from Akshat Bubna
- URL: https://www.antoinebuteau.com/lessons-from-akshat-bubna/
- Published: 2026-08-01T16:38:57.000Z
- Updated: 2026-08-01T17:14:16.000Z
- Description: Akshat Bubna is the co founder and CTO of Modal, a cloud platform for running AI models and massive batch jobs without the usual infrastructure overhead.
- Author: Antoine Buteau
- Tags: Profile, AI & Machine Learning Profiles

Akshat Bubna is the co-founder and CTO of Modal, a cloud platform for running AI models and massive batch jobs without the usual infrastructure overhead. After hitting the limits of existing deployment tools as an early engineer at Scale AI, he helped build a custom file system and container runtime that delivers sub-second spin-up times for GPU workloads. This profile covers his approach to systems engineering, the shift toward agent-driven sandboxes, and how he sources technical talent.

![Visual summary of operating lessons from Akshat Bubna.](https://www.antoinebuteau.com/content/images/2026/08/lessons-from-akshat-bubna-profile-infographic.webp)

### Part 1: Infrastructure and the Agent Experience

1. **On the agent cloud era:** The previous generation of cloud infrastructure assumed a human operator who could decipher YAML and read documentation when things broke. In the new era, programmatic infrastructure must be precise enough for agents to write code, inspect outputs, and self-correct. — *Reference:* [*Latent Space*](https://www.latent.space/p/modal2026?ref=antoinebuteau.com)
2. **On empowering agents:** "Pretty cool watching agents fully realize the dream of programmatic infra." — *Source:* [*Latent Space*](https://www.latent.space/p/modal2026?ref=antoinebuteau.com)
3. **On agent iteration speed:** "Fast iteration and feedback loops with all the necessary context are crucial for agents to operate properly." — *Source:* [*Latent Space*](https://www.latent.space/p/modal2026?ref=antoinebuteau.com)
4. **On software-defined environments:** Modal redesigned its approach from developer experience to agent experience, allowing agents to provision their own runtimes using decorator-based configurations instead of managing massive Kubernetes files. — *Reference:* [*Frontier Models*](https://frontiermodels.cc/video/the-100000-sandbox-problem-akshat-bubna-modal-cto/?ref=antoinebuteau.com)
5. **On scaling agent sandboxes:** Running complex tasks like reinforcement learning rollouts can require spinning up tens of thousands of secure, isolated sandboxes concurrently. — *Reference:* [*Modal Blog*](https://modal.com/blog/announcing-our-series-b?ref=antoinebuteau.com)
6. **On iterative debugging:** "They need a place to write code, run it, inspect the output, change the environment, debug failures, and try again." — *Source:* [*Latent Space*](https://www.latent.space/p/modal2026?ref=antoinebuteau.com)

### Part 2: Rethinking Serverless and GPUs

1. **On spiky workloads:** "Inference workloads are more variable and less predictable than the training workloads that previously dominated." — *Source:* [*Modal Blog*](https://modal.com/blog/truly-serverless-gpus?ref=antoinebuteau.com)
2. **On the limitations of static provisioning:** Relying on fixed, over-provisioned GPU allocations to handle unanticipated spikes in demand usually leads to poor utilization and economics for the majority of the time. — *Reference:* [*Modal Blog*](https://modal.com/blog/truly-serverless-gpus?ref=antoinebuteau.com)
3. **On true serverless design:** Real serverless computing only functions efficiently if new replicas can be spun up in seconds. Unoptimized inference servers can take tens of minutes to load models and ready the host, creating unacceptable bottlenecks. — *Reference:* [*Modal Blog*](https://modal.com/blog/truly-serverless-gpus?ref=antoinebuteau.com)
4. **On pooling compute:** "By pooling the world’s compute and managing the capacity at scale, we can drive efficiency and speed." — *Source:* [*Modal Blog*](https://modal.com/blog/announcing-our-series-b?ref=antoinebuteau.com)
5. **On optimizing boot times:** By utilizing cloud buffers, a custom filesystem, and both CPU and GPU checkpoint/restore mechanisms, replica spin-up times for inference applications were reduced from thousands of seconds to roughly fifty seconds. — *Reference:* [*Modal Blog*](https://modal.com/blog/truly-serverless-gpus?ref=antoinebuteau.com)
6. **On sharing knowledge:** "secrecy is a bad moat. And if more people learn how to use GPUs efficiently, there will be more available in the market for us!" — *Source:* [*Modal Blog*](https://modal.com/blog/truly-serverless-gpus?ref=antoinebuteau.com)
7. **On anticipating demand:** Modal added GPUs to their product a full year before ChatGPT was released, initially targeting classical computer vision and XGBoost use cases before generative AI took off. — *Reference:* [*Podscripts*](https://podscripts.co/podcasts/latent-space-the-ai-engineer-podcast/why-ai-infrastructure-must-evolve-for-agent-experience-akshat-bubna-modal-cto?ref=antoinebuteau.com)
8. **On usage-based models:** Cloud pricing should directly reflect actual compute consumption, eliminating idle costs for developers who only need resources for short, intensive bursts of work. — *Reference:* [*Sparkco*](https://sparkco.ai/blog/modal?ref=antoinebuteau.com)

### Part 3: Systems Engineering and Building from Scratch

1. **On avoiding legacy systems:** "Tools like Kubernetes weren’t designed for large-scale ML workloads or global GPU management. People keep trying to retrofit old systems for new use cases, but building from scratch lets you design for today’s needs." — *Source:* [*The Times of India*](https://timesofindia.indiatimes.com/technology/india-needs-a-stronger-olympiad-pipeline/articleshow/126157645.cms?ref=antoinebuteau.com)
2. **On the AI infrastructure gap:** "AI-native companies need AI-native infrastructure." — *Source:* [*Modal Blog*](https://modal.com/blog/announcing-our-series-b?ref=antoinebuteau.com)
3. **On Docker's limitations:** Docker proved too slow for their container system because extracting large tar files over a network before mounting took an unacceptable amount of time. — *Reference:* [*YouTube*](https://www.youtube.com/watch?v=pLBxrY8RX6w&ref=antoinebuteau.com)
4. **On building a custom file system:** By using FUSE, it became possible to present a container image as if it were local, allowing the system to lazily load only the necessary files over the network as the runtime demands them. — *Reference:* [*YouTube*](https://www.youtube.com/watch?v=pLBxrY8RX6w&ref=antoinebuteau.com)
5. **On iterating infrastructure:** Building complex file systems evolved incrementally, starting with a basic network mount, prototyping it in Python, and finally rewriting it in Rust when safety and high performance became strictly necessary. — *Reference:* [*YouTube*](https://www.youtube.com/watch?v=pLBxrY8RX6w&ref=antoinebuteau.com)
6. **On acquiring adjacent systems expertise:** The best technical acquisitions can bring in teams that independently solved complementary problems. Jamsocket's work on stateful runtimes, real-time backends, and AI code execution made its team a natural fit for Modal's platform. — *Reference:* [*Modal Blog*](https://modal.com/blog/jamsocket-is-joining-modal?ref=antoinebuteau.com)
7. **On shifting compute profiles:** Modern AI workloads frequently bounce between CPU and GPU phases. Managing this effectively requires infrastructure that can seamlessly collocate and shift between these resources without extreme latency. — *Reference:* [*Frontier Models*](https://frontiermodels.cc/video/the-100000-sandbox-problem-akshat-bubna-modal-cto/?ref=antoinebuteau.com)

### Part 4: Operating the Agent Cloud at Scale

1. **On observability in agent-written systems:** When agents produce more of the code, operators can spend less time reading every implementation detail, but they need stronger observability to understand behavior, investigate failures, and make judgment calls. — *Reference:* [*Latent Space*](https://www.latent.space/p/modal2026?ref=antoinebuteau.com)
2. **On aggregating fragmented compute:** A software layer spanning 17 cloud and bare-metal providers can turn scattered global capacity into one pool, giving workloads more places to find the right mix of CPUs, GPUs, and memory. — *Reference:* [*Latent Space*](https://www.latent.space/p/modal2026?ref=antoinebuteau.com)
3. **On separating queue guarantees:** A single queue should not be forced to optimize for incompatible goals. Modal split its input plane into a low-latency, at-most-once path and a durable path that can hold large backlogs and retry work at least once. — *Reference:* [*Amplify Partners*](https://www.youtube.com/watch?v=pLBxrY8RX6w&ref=antoinebuteau.com)
4. **On making multi-tenancy defensible:** Containers alone expose too much attack surface for untrusted workloads. Modal uses gVisor to intercept system calls and sanitize memory behind a stronger isolation boundary that behaves more like a virtual machine. — *Reference:* [*Amplify Partners*](https://www.youtube.com/watch?v=pLBxrY8RX6w&ref=antoinebuteau.com)
5. **On designing abuse controls into the runtime:** Infrastructure providers can detect fraud below the application layer by inspecting sensitive system calls, network destinations, and executable signatures; Modal used this approach to suppress recurring crypto-mining abuse. — *Reference:* [*Amplify Partners*](https://www.youtube.com/watch?v=pLBxrY8RX6w&ref=antoinebuteau.com)
6. **On removing hidden round trips:** At infrastructure scale, performance gains often come from eliminating trips to the kernel or disk, serving data from memory, warming connections, and tuning congestion control rather than relying on a single dramatic optimization. — *Reference:* [*Amplify Partners*](https://www.youtube.com/watch?v=pLBxrY8RX6w&ref=antoinebuteau.com)
7. **On abstracting distributed networking:** Multi-node training requires more than placing GPUs together. Each cloud exposes a different RDMA fabric, so a portable platform must normalize provider-specific drivers, networking, authentication, and secure workload startup. — *Reference:* [*Amplify Partners*](https://www.youtube.com/watch?v=pLBxrY8RX6w&ref=antoinebuteau.com)
8. **On treating inference as a systems problem:** Moving KV caches and transferring weights between training and inference GPUs are fundamentally questions of memory movement and scheduling, not merely model-serving configuration. — *Reference:* [*Latent Space*](https://www.latent.space/p/modal2026?ref=antoinebuteau.com)
9. **On automating forward-deployed engineering:** Modal's internal auto-inference harness lets an agent sweep configurations, run NVIDIA's profiler, tune settings, and even change GPU types, turning repeated customer optimization work into a reusable system. — *Reference:* [*Latent Space*](https://www.latent.space/p/modal2026?ref=antoinebuteau.com)
10. **On using scheduling as a pricing lever:** Strong capacity planning and control of the scheduling stack can improve unit economics while enabling lower-cost batch tiers for customers whose workloads can trade latency for price. — *Reference:* [*Latent Space*](https://www.latent.space/p/modal2026?ref=antoinebuteau.com)
11. **On granting agents bounded capability:** Production agent infrastructure should pair powerful execution environments with granular permissions, so customers can give agents the capabilities they need without granting blanket access. — *Reference:* [*Modal Blog*](https://modal.com/blog/modal-series-c?ref=antoinebuteau.com)
12. **On shared responsibility for sandbox security:** Strong platform isolation does not make an unauthenticated customer endpoint safe. The rogue-agent incident showed that infrastructure boundaries and application-level access controls remain separate responsibilities. — *Reference:* [*Simon Willison*](https://simonwillison.net/2026/Jul/28/akshat-bubna/?ref=antoinebuteau.com)

### Part 5: Teams, Talent, and Company Building

1. **On human capital as the true bottleneck:** "Despite all the talk about AI replacing engineers, you still need very strong people to build these systems." — *Source:* [*The Times of India*](https://timesofindia.indiatimes.com/technology/india-needs-a-stronger-olympiad-pipeline/articleshow/126157645.cms?ref=antoinebuteau.com)
2. **On competitive programming backgrounds:** Actively recruiting software engineers from Olympiad and competitive programming networks yields developers with a particularly high ceiling for solving complex system design problems. — *Reference:* [*The Times of India*](https://timesofindia.indiatimes.com/technology/india-needs-a-stronger-olympiad-pipeline/articleshow/126157645.cms?ref=antoinebuteau.com)
3. **On the problem-solving mindset:** "Programming competitions teach you that most problems are solvable," — *Source:* [*The Times of India*](https://timesofindia.indiatimes.com/technology/india-needs-a-stronger-olympiad-pipeline/articleshow/126157645.cms?ref=antoinebuteau.com)
4. **On iterative development:** "You also learn to work incrementally – start with something simple that works, then improve it over time. That’s exactly how we’ve built Modal." — *Source:* [*The Times of India*](https://timesofindia.indiatimes.com/technology/india-needs-a-stronger-olympiad-pipeline/articleshow/126157645.cms?ref=antoinebuteau.com)
5. **On company culture:** "We like solving hard problems and building things we can be proud of." — *Source:* [*Latent Space*](https://www.latent.space/p/modal2026?ref=antoinebuteau.com)
6. **On building software moats:** Rather than investing heavily in owned data centers or hardware differentiation, a modern infrastructure provider can remain capital-light by prioritizing deep software abstractions across multiple cloud environments. — *Reference:* [*Frontier Models*](https://frontiermodels.cc/video/the-100000-sandbox-problem-akshat-bubna-modal-cto/?ref=antoinebuteau.com)

### Part 6: Scale AI, Career, and Ecosystem Perspective

1. **On foundational experiences:** Before launching an infrastructure company, seeing the same persistent operational and deployment issues repeatedly at Scale AI cemented the realization that legacy cloud abstractions were insufficient for machine learning. — *Reference:* [*The Times of India*](https://timesofindia.indiatimes.com/technology/india-needs-a-stronger-olympiad-pipeline/articleshow/126157645.cms?ref=antoinebuteau.com)
2. **On turning infrastructure depth into product speed:** Building filesystems, networking, asynchronous queues, and multi-cloud GPU orchestration as one integrated stack can accelerate how quickly AI products move from experiments to production scale. — *Reference:* [*Akshat Bubna on LinkedIn*](https://www.linkedin.com/posts/akshat-bubna-188885103%5Fat-modal-weve-built-every-layer-of-the-ai-activity-7364327855691866112-RsoT?ref=antoinebuteau.com)
3. **On self-belief:** When advising young aspirants in technical fields, the most fundamental step is possessing the confidence to attempt extremely difficult problems. — *Reference:* [*The Times of India*](https://timesofindia.indiatimes.com/technology/india-needs-a-stronger-olympiad-pipeline/articleshow/126157645.cms?ref=antoinebuteau.com)
4. **On closing systemic gaps:** Addressing the lack of localized deep-tech research infrastructure in places like India is vital to retaining top technical talent that otherwise migrates to find adequate computing resources. — *Reference:* [*The Times of India*](https://timesofindia.indiatimes.com/technology/india-needs-a-stronger-olympiad-pipeline/articleshow/126157645.cms?ref=antoinebuteau.com)