Charity Majors writes and speaks about observability, engineering management, and the systems that shape software teams. — Kubernetes Podcast #230.

Infographic for "Lessons from Charity Majors".

On Observability

  1. Defining Observability: For software engineers, observability means being able to infer what a system is doing internally by asking new questions of its external outputs. — Observability Is a Many-Splendored Definition.
  2. Explore Unknown Unknowns: Observability is designed to let engineers investigate unexpected behavior and ask questions they could not predict when they instrumented the system. — Observability Is a Many-Splendored Definition.
  3. Keep Context Connected: Telemetry becomes more useful when the context around an event stays connected rather than being split into isolated data streams. — Kubernetes Podcast #230.
  4. Page Less, Debug Better: Charity argues that teams over-page partly because they do not trust their tools to diagnose production problems reliably. — InfoQ Interview.
  5. Know How a Change Could Break: Before merging code, engineers should be able to say how they will detect a breakage and what they will inspect after deployment. — On-Call Talk.
  6. Instrument for Emergent Systems: As systems become distributed and unpredictable, teams need rich instrumentation to investigate behavior they cannot fully reason out in advance. — Sociotechnical Teams Talk.
  7. Build Resilience to Failure: Charity urges teams to maintain high quality while practicing how to detect, investigate, and recover from failures instead of assuming they can prevent every failure. — Intercom Interview.
  8. Use Dashboards as Starting Points: Static dashboards encode past questions; Charity favors interactive investigation when a new problem demands a question the dashboard cannot answer. — Intercom Interview.
  9. Preserve Request Context in Events: For observability, Charity recommends wide structured events that retain the full context of each service's handling of a request. — Observability Is a Many-Splendored Definition.
  10. Bring Observability into Development: Charity's observability 2.0 framing extends beyond operating software to instrumenting, deploying, and verifying code during development. — Kubernetes Podcast #230.

On Engineering Management

  1. Management Is a Different Profession: Charity argues that moving from engineering to management is a change of profession, not a promotion on the same ladder. — Engineer/Manager Pendulum.
  2. Choose Management for People Work: The strongest reason to manage is wanting to support people and tackle their problems, not seeking title or power. — Intercom Interview.
  3. Manage Up as Well as Down: Managers must communicate the team's work and needs to leaders above them, not only support direct reports. — It Shipped That Way.
  4. Keep Technical Context Fresh: Charity values managers who have recent hands-on engineering experience because technical skills and production context can become stale. — Intercom Interview.
  5. Develop the Managerial Craft: Teams deserve a manager who wants to manage, understands sociotechnical process, and invests in colleagues' careers. — LeadDev Pendulum Slides.
  6. Keep Engineers Connected to Context: Rather than acting as a blanket filter of difficult information, managers should give engineers the context they need to understand the business and make sound decisions. — Swarmia Interview.
  7. Management Skills Travel Beyond Work: Charity says the interpersonal skills learned in management can improve relationships outside the workplace as well as work itself. — Why Become an Engineering Manager.
  8. Build an Engineering Foundation First: Before trying management, Charity advises engineers to gain confidence and substantial experience in their technical craft. — LeadDev Pendulum Slides.
  9. Make Career Goals Discussable: Managers should ask engineers about their career goals and offer chances to practice management to those who express interest. — How to Hire an Engineering Manager.
  10. Frontline Managers Shape Growth: A frontline manager who crafts challenging work for each engineer can support learning and loyalty, benefiting the organization over time. — Intercom Interview.

On the Engineer/Manager Pendulum

  1. Identify as a Technologist: Charity encourages engineers and managers to see themselves as technologists who can learn both technical and people-management skills. — LeadDev Pendulum Slides.
  2. Move Between Tracks: Charity argues that senior technical leadership can be strengthened by experience on both engineering and management tracks. — Engineer/Manager Pendulum.
  3. Refresh Hands-On Skills: Extended time away from coding can erode a manager's technical fluency, so Charity recommends returning to hands-on engineering while that path remains viable. — LeadDev Pendulum Slides.
  4. Let Engineers Try Management: A period in management can give senior engineers empathy for organizational work and a clearer sense of how technical choices connect to business needs. — Intercom Interview.
  5. Respect Different Attention Demands: Engineering often needs uninterrupted focus; managing people requires availability and responsiveness. Trying to do both jobs fully at once can undermine each. — LeadDev Pendulum Slides.
  6. Management Can Develop Staff Engineers: Charity suggests that a senior engineer's stint in management can strengthen the perspective needed for staff-level technical leadership. — Why Become an Engineering Manager.
  7. Carry Institutional Memory Across Tracks: When people move between engineering and management, they can preserve institutional knowledge and leadership continuity on both sides. — InfoQ Pendulum Talk.
  8. Give the First Management Tour Time: Charity recommends a meaningful first management stint—roughly two years in her example—to learn the basics before judging the fit. — LeadDev Pendulum Slides.
  9. Build a Varied Career: Charity argues that moving among engineering and management roles can make a long career more adaptable rather than narrower. — LeadDev Pendulum Slides.
  10. Separate Leadership from Management: Technical leadership is valuable even without a management title; management and leadership are not synonymous. — InfoQ Pendulum Talk.

On Team Culture and High-Performing Teams

  1. Avoid Dependence on a Hero Engineer: A healthy engineering organization should not depend on one exceptionally productive person to understand or rescue its software. — In Praise of Normal Engineers.
  2. Teams Shape Engineers: Charity argues that engineering organizations develop stronger engineers by creating conditions for learning, teaching, and safe delivery. — In Praise of Normal Engineers.
  3. Treat Team Performance as Sociotechnical: Engineering performance emerges from the interaction of people, practices, and tools—not technical skill alone. — Sociotechnical Teams Talk.
  4. Improve the System Around the Engineer: How quickly and safely an engineer ships depends heavily on team feedback loops, tooling, and operational practices. — On-Call Talk.
  5. Hire for the Team's Work: Charity recommends hiring people who share the team's values and have needed skills, then deliberately building the team rather than assuming star hires alone create performance. — Sociotechnical Teams Talk.
  6. Learn Through Small, Safe Shipments: Frequent, small deployments and production feedback help engineers learn how their software actually behaves. — Shipping Software Should Not Be Scary.
  7. Mix Disciplines to Avoid Monoculture: Charity favors interaction among engineering, operations, sales, and other functions because cross-pollination can produce creative ideas and broader understanding. — Intercom Interview.
  8. Adapt On-Call to Context: Charity warns that on-call practices must be adapted to each organization's circumstances; few rules apply without context. — On-Call Guide for Managers.
  9. Link Team Health with Customer Outcomes: Charity frames production excellence as a way to improve life for both customers and engineering teams, rather than treating either group as expendable. — Sociotechnical Teams Talk.

On On-Call and Operations

  1. Close the Production Feedback Loop: On-call ownership connects the engineers who build a service to its behavior and users outside normal working hours. — On-Call Talk.
  2. Make On-Call a Two-Way Compact: If engineers own production support, managers owe them time to improve the system, training, alert tracking, and humane conditions. — On-Call Guide for Managers.
  3. Protect the On-Call Week: Charity recommends freeing the on-call engineer from normal roadmap work so they can repair the system and reduce future pages. — On-Call Talk.
  4. Make Opt-In On-Call an Aspiration: Charity believes an on-call rotation can become meaningful enough that engineers might volunteer for it, but treats this as a design aspiration. — On-Call Guide for Managers.
  5. Spend Reliability Budget Deliberately: Charity recommends using service-level objectives to balance resilience and experimentation rather than treating any risk or downtime as forbidden. — On-Call Talk.
  6. Make Alerts Actionable: Do not page someone who lacks the access, training, or authority to act on an alert. — On-Call Guide for Managers.
  7. Own the Whole Software Lifecycle: Engineering ownership extends from building code to deploying, observing, operating, and improving it. — On-Call Guide for Managers.
  8. Design On-Call for Humans: On-call should respect sleep, family commitments, training needs, and the real human cost of unreliable systems. — On-Call Talk.
  9. Name the Rotation Owner: A sustainable on-call rotation needs explicit responsibility for who supports a service outside working hours and how the rotation is maintained. — On-Call Talk.
  10. Do Not Add Complexity Without Need: Charity advises using a simpler monolith, LAMP stack, and basic monitoring when those tools meet a system's real requirements. — InfoQ Interview.

Learn more:

  1. Charity Majors — Observability Is a Many-Splendored Definition
  2. Charity Majors — The Engineer/Manager Pendulum
  3. Kubernetes Podcast from Google — #230: Observability & Engineering Management with Charity Majors
  4. InfoQ — Charity Majors on Observability and Understanding the Operational Ramifications of a System
  5. Charity Majors — The Sociotechnical Path to High-Performing Teams (Speaker Deck)
  6. Intercom — Honeycomb’s Charity Majors on Operations Engineering
  7. Charity Majors — How to Hire an Engineering Manager
  8. It Shipped That Way — The Culture You Want to Build with Charity Majors (Episode 19)
  9. LeadDev — The Engineer/Manager Pendulum and You (Charity Majors slides)
  10. Charity Majors — Becoming an Engineering Manager Can Make You Better at Life and Relationships
  11. Swarmia — Charity Majors on Engineering Management and Context
  12. InfoQ — The Engineer/Manager Pendulum (Charity Majors)
  13. Charity Majors — In Praise of Normal Engineers
  14. Charity Majors — Shipping Software Should Not Be Scary
  15. Charity Majors — On Call Shouldn’t Suck: A Guide for Managers
  16. Charity Majors — Being On Call Does Not Have to Suck (Speaker Deck)