Philip Kiely is a software engineer and developer relations professional focused on technical writing and AI infrastructure. He wrote Writing for Software Developers straight out of college and recently published Inference Engineering based on his time at Baseten. His career offers a practical look at how developers can write more clearly and scale generative AI models in production.

Part 1: The Craft of Technical Writing

  1. On treating writing like code: Instructing a computer requires precise programming languages; writing technical content for humans requires that same level of specificity and technical detail. — Reference: share.transistor.fm
  2. On defining the audience: Just as you write code for a specific interpreter, you must write content tailored to your readers, allowing you to make safe assumptions about their baseline knowledge. — Reference: share.transistor.fm
  3. On the developer stereotype: The perception that programmers are inherently bad at writing is flawed, as the growing diversity of backgrounds in the field brings strong baseline communication skills. — Reference: share.transistor.fm
  4. On finding a niche: Operating at the intersection of writing code and writing words is a highly effective way to differentiate yourself in the tech industry. — Reference: share.transistor.fm
  5. On teaching as a learning tool: Technical writing offers a structured way to teach others the exact concepts you previously struggled to understand yourself. — Reference: share.transistor.fm
  6. On generating ideas: Software developers can use their own personal projects as an endless source of inspiration for technical content. — Reference: philipkiely.com
  7. On structural clarity: High-quality technical writing relies heavily on providing clear direction and explicit signposting for the reader. — Reference: philipkiely.com
  8. On working with publishers: Partnering with established publications and editors is a practical strategy for amplifying your voice and reaching a global audience. — Reference: share.transistor.fm

Part 2: Career Strategy and Early Success

  1. On launching without an audience: It is possible to successfully launch a book and generate significant revenue—like making over $15,000 in a day—with almost no Twitter followers by tapping into existing developer communities. — Reference: share.transistor.fm
  2. On writing as a resume: Publishing a technical book can act as a high-leverage career asset, directly leading to leadership roles like a Head of Marketing position immediately after college. — Reference: writerontheside.com
  3. On job applications: Sending authentic, targeted emails to real people yields much better career outcomes than mindlessly submitting hundreds of resumes to job boards. — Reference: philipkiely.com
  4. On proving competence: Candidates should use social proof and tangible proof of work to establish credibility when networking. — Reference: philipkiely.com
  5. On career compounding: Small, deliberate successes in writing and outreach eventually compound into disproportionately large career opportunities. — Reference: philipkiely.com
  6. On navigating graduation: Crafting a clear professional identity and understanding startup environments are critical first steps for new graduates entering the technology sector. — Reference: career.grinnell.edu
  7. On side income: Technical writing can serve as a flexible, high-paying alternative to typical student jobs while simultaneously padding a resume. — Reference: share.transistor.fm
  8. On documenting problems, not features: As a company matures, developer relations should evolve from explaining product features to documenting the problems and real-world use cases customers actually face. — Reference: Open Source CEO

Part 3: The Discipline of Inference Engineering

  1. On defining inference: In the generative AI lifecycle, inference encompasses everything that occurs between a user submitting a prompt and the model returning an answer. — Reference: opensourceceo.com
  2. On the infrastructure gap: A few years ago, many assumed AI training was the hard part and inference was trivial; in reality, serving models at scale is a complex discipline. — Reference: opensourceceo.com
  3. On full-stack optimization: Inference engineers must operate across the entire technology stack, solving deep technical problems from CUDA down to Kubernetes. — Reference: philipkiely.com
  4. On real-time constraints: Delivering AI in production means meeting strict service-level agreements where latency is measured in hundreds of milliseconds. — Reference: opensourceceo.com
  5. On the optimization toolkit: Hitting production latency targets requires custom kernels, post-training quantization, speculative decoding, and efficient KV cache reuse. — Reference: opensourceceo.com
  6. On quantization benefits: Quantizing more layers of a model can sometimes preserve downstream quality while boosting throughput, because errors introduced in different areas may cancel each other out. — Reference: latent.space
  7. On career timing: Because inference engineering is a relatively young field, developers who enter the space now can become experts very quickly. — Reference: philipkiely.com
  8. On turning research into product: Understanding the specific knobs of model serving allows teams to move applied AI research into production environments in a matter of hours. — Reference: twimlai.com
  9. On separating latency goals: Time to first token and sustained tokens per second are different performance problems, so teams should choose inference optimizations according to the specific user-facing service-level objective. — Reference: Open Source CEO

Part 4: Scaling AI and Multi-Model Systems

  1. On true AI-native software: Building a real AI product means moving past thin wrappers around single APIs and composing multiple models to achieve a specific user goal. — Reference: se-radio.net
  2. On the definition of agentic AI: An agentic framework is simply an AI that transitions from answering questions to taking direct action via tool use and function calling. — Reference: se-radio.net
  3. On geographic scaling: Serving millions of users requires moving beyond single regions and replicating complex inference architectures across multiple cloud providers. — Reference: opensourceceo.com
  4. On prototyping strategy: Teams looking to add intelligence to an existing product should start by experimenting with off-the-shelf models before investing in custom AI pipelines. — Reference: se-radio.net
  5. On software fundamentals: AI engineering is ultimately just software engineering, meaning teams must still prioritize observability, evaluation, and strict safety guardrails. — Reference: se-radio.net
  6. On open-source independence: The proliferation of millions of open-source models gives companies the ability to truly own the intelligence embedded in their applications. — Reference: opensourceceo.com
  7. On hardware nondeterminism: Deploying models at scale often exposes unique GPU hardware constraints and kernel race conditions that cause unpredictable system failures. — Reference: latent.space
  8. On continuous experimentation: The best approach to building multi-agent AI systems is to adopt an iterative process focused on rapid, small-scale experimentation. — Reference: se-radio.net
  9. On multi-model unit economics: A multi-model product is viable only when its orchestration remains fast enough for users and economical enough for the business at production scale. — Reference: Software Engineering Radio

Part 5: Navigating the AI Infrastructure Landscape

  1. On infrastructure personalization: Selecting a serving framework should not be based on generic advice; it must be tailored to the specific model, traffic scale, and organizational constraints. — Reference: inferenceengineering.tech
  2. On dedicated vs. shared APIs: As products mature, deploying models on dedicated infrastructure eventually becomes more reliable and cost-effective than using shared API endpoints. — Reference: latent.space
  3. On separating workloads: To maximize GPU efficiency, inference systems are increasingly separating the prefill and decode phases onto different hardware configurations. — Reference: latent.space
  4. On local vs. data-center AI: Local AI development is generally focused on making smaller models smarter, while data-center engineering is focused on making massive models run faster. — Reference: latent.space
  5. On specialized runtimes: When performance is the absolute priority, engineering teams must abandon generalized servers in favor of highly specialized, workload-specific runtimes. — Reference: twimlai.com
  6. On the training-inference loop: The industry is moving toward continuous learning setups where data from model inference feeds directly back into post-training improvement loops. — Reference: latent.space
  7. On optimizing model fidelity: It is entirely possible to implement aggressive inference optimizations that speed up the system without degrading the model's actual downstream intelligence. — Reference: latent.space
  8. On architectural retrofits: To push hardware limits, engineers are now grafting specialized components—such as vision encoders from other architectures—directly onto existing language models. — Reference: latent.space
  9. On cache-aware routing: Inference requests should be routed partly according to reusable KV-cache state, not only generic load balancing, so systems can avoid recomputing prior context. — Reference: Latent Space Inference Engineering Masterclass