Visual summary of operating lessons from Dan Shiebler.

Lessons from Dan Shiebler

Dan Shiebler is co-founder and CTO of Artemis, after leading machine learning at Abnormal Security and working on embeddings, vector search, and ads ML at Twitter. His own writing and interviews connect production ML, behavioral security, AI-native product design, and mathematical research. — Dan Shiebler — About Me.

Part 1: Cybersecurity and AI-Native Defense

  1. Build AI into the company's operating system: Shiebler says Artemis developed its codebase and internal processes around AI-native tooling, allowing the team to move from concept to prototype to customer feedback quickly. — First Round In Depth — Artemis.
  2. Build detection around behavior: Shiebler describes Abnormal's goal as understanding behavior well enough to detect attacks precisely and remain resilient as attacker behavior changes. — First Round In Depth — Artemis.
  3. Organize logs into usable context: Shiebler says security agents need data architecture that lets them find relevant logs, entities, relationships, and context instead of treating telemetry as undifferentiated text. — Cyber Conversations — Dan Shiebler.
  4. Match machine-speed threats with better defense: Shiebler says increasingly capable attacker tooling raises the bar for automated, context-rich defensive systems. — Innovate Cybersecurity — Dan Shiebler.
  5. Learn normal identity behavior: Shiebler describes building baselines for employees and vendors from communication patterns, sign-in events, and other attributes to spot suspicious deviations. — Authority Magazine — Dan Shiebler.
  6. Design defense for a faster attacker: Shiebler says attackers now have better tools and move more quickly, requiring security products to respond with stronger behavioral understanding. — First Round In Depth — Artemis.
  7. Assume attackers can use strong AI too: Shiebler says powerful models are widely available to attackers as well as defenders, making data foundations and integrations a differentiator for defense. — Cyber Conversations — Dan Shiebler.
  8. Combine signals rather than isolate events: Shiebler says Abnormal assesses messages and accounts using sender, recipient, conversation history, behavioral patterns, and compromise signals together. — Bigeye Observatory Interview.
  9. Expect more personalized AI-assisted phishing: Shiebler says generative AI can automate personalization of social-engineering messages, making familiar template and indicator checks less reliable. — SuperDataScience 717 — Dan Shiebler.
  10. Ground agent findings in queryable evidence: Shiebler says agents need tools to retrieve the relevant records and entity relationships before interpreting an apparent security signal. — Cyber Conversations — Dan Shiebler.

Part 2: The Limits of Traditional Security

  1. Fix the data foundation beneath security AI: Shiebler argues that AI-native security requires an architecture for ingesting, normalizing, querying, and analyzing telemetry, not merely a new assistant interface. — Cyber Conversations — Dan Shiebler.
  2. Legacy foundations can bottleneck AI: Shiebler says AI layered over older logic is constrained when components must communicate through the old system's representations. — First Round In Depth — Artemis.
  3. Use LLMs to help evolve detection rules: In his signed essay, Shiebler explores how language models can propose detection rules that are audited and improved against noisy examples. — Generating Detection Rules with LLMs.
  4. Precision prevents customer work: Shiebler says false positives force security teams to hunt for missing messages or locked-out accounts, so detector precision must be monitored. — Bigeye Observatory Interview.
  5. More telemetry needs better organization: Shiebler says the hard part is making large, costly-to-move datasets accessible at the right resolution and with the right contextual relationships. — Cyber Conversations — Dan Shiebler.
  6. Do not rely only on static indicators: Shiebler says attackers can cheaply change domains, IPs, links, and other indicators, so Abnormal also looks for deviations from normal communication. — SuperDataScience 717 — Dan Shiebler.
  7. Monitor each detector's unique contribution: Shiebler describes specialized detectors feeding a decision layer and says teams should measure the precision each detector adds uniquely. — Bigeye Observatory Interview.
  8. Treat data movement as an architecture cost: Shiebler says moving large operational datasets into formats an agent can query quickly is expensive and must be designed deliberately. — Cyber Conversations — Dan Shiebler.
  9. An AI wrapper does not remove legacy bottlenecks: Shiebler says an AI layer on top of older rule-based foundations remains limited by how those components exchange information. — First Round In Depth — Artemis.
  10. Give analysts the relevant history quickly: Shiebler's Abnormal examples require access to conversation threads, account behavior, and earlier messages to judge an apparent attack. — Bigeye Observatory Interview.

Part 3: AI Agents and Data Handling

  1. Keep logs queryable but give agents context: Shiebler stresses that logs must remain accessible and queryable, while agents also need entity and semantic context to interpret them; he does not say agents should never see raw logs. — Cyber Conversations — Dan Shiebler.
  2. Prepare data for agent reasoning: Shiebler says combining telemetry with registries, HR systems, and IT-management context helps agents retrieve the right inputs for a security decision. — Cyber Conversations — Dan Shiebler.
  3. Let agents investigate and tune detectors: Shiebler describes agents writing detection rules, auditing false positives, back-testing changes, and reporting their actions. — Innovate Cybersecurity — Dan Shiebler.
  4. Make agent correctness an architectural goal: Shiebler says Artemis structures systems and monitoring to make agent decisions more likely to be correct rather than relying on model capability alone. — First Round In Depth — Artemis.
  5. Design query architecture around specific entities: Shiebler says agents must be able to slice operational data for the user, device, IP, and time window relevant to the task. — Cyber Conversations — Dan Shiebler.
  6. Generate detection rules with LLMs carefully: Shiebler's signed essay studies how LLM-labeled examples can yield useful rules in noisy domains, with validation against model error. — Generating Detection Rules with LLMs.
  7. Move from concept to customer feedback quickly: Shiebler says Artemis's AI-native codebase and processes reduce manual steps between prototype, customer use, and product iteration. — First Round In Depth — Artemis.
  8. Structure tools to improve agent reliability: Shiebler says the data and tool setup around agents influences whether their decisions are correct; the original numerical hallucination-reduction claim is not established. — First Round In Depth — Artemis.
  9. Monitor and adapt detector portfolios: Shiebler describes watching individual detector precision and retraining/adapting models as customer behavior and attacker tactics change. — Bigeye Observatory Interview.

Part 4: Machine Learning Architecture and Scale

  1. Build reusable embedding infrastructure: Shiebler says his Twitter Cortex team built core embedding and vector-search capabilities used across Twitter products; his bio does not establish the original keyword-versus-semantic performance claim. — Dan Shiebler — About Me.
  2. Train ML systems for failures: Shiebler argues that production models should be trained and designed to cope with missing features, service failures, and changing data rather than assuming clean inputs. — Resilient Machine Learning.
  3. Prepare for distribution drift: Shiebler distinguishes feature outages from changing data relationships and recommends training and monitoring that expose models to realistic shifts. — Resilient Machine Learning.
  4. Model noisy smartphone-sensor signals: Shiebler says his work at Truemotion used phone motion-sensor data to predict driving behavior for insurance; the original irregular-sampling architecture detail is unverified. — Authority Magazine — Dan Shiebler.
  5. Invest in the system around a model: Shiebler argues that improving a production model often requires infrastructure work because offline gains can fail when training and serving assumptions differ. — Improving an ML System — Part 1.
  6. Make historical context available for detection: Shiebler explains that Abnormal's backend must retrieve previous messages and behavior quickly enough to inform a current security decision. — Bigeye Observatory Interview.
  7. Keep training and serving features aligned: Shiebler warns that online and offline feature pipelines can produce different values or freshness, so production ML work must manage that gap explicitly. — Improving an ML System — Part 2.

Part 5: Adversarial Machine Learning and Threat Detection

  1. Expect AI-assisted personalized attacks: Shiebler says accessible generative AI tools make personalized malicious messages and code easier for attackers to create; he does not specifically attribute the original claim to DeepSeek. — Authority Magazine — Dan Shiebler.
  2. Build for an AI-versus-AI security environment: Shiebler says he has faced adversarial automation from Twitter spam through Abnormal email attacks and now applies those lessons to Artemis. — First Round In Depth — Artemis.
  3. Detect deviations from known-good behavior: Shiebler says Abnormal builds employee and vendor baselines using communication patterns, sign-ins, and other attributes to spot anomalous email behavior. — Authority Magazine — Dan Shiebler.
  4. Expect attackers to evade known patterns: Shiebler says attackers can cheaply change indicators and use generative AI to personalize attacks, so defenses must look beyond known-bad signatures. — SuperDataScience 717 — Dan Shiebler.
  5. Do not rely on obvious phishing templates: Shiebler says AI-assisted personalization can make malicious messages less like known phishing templates; the original claim about grammar being the 'most reliable' indicator is not supported. — SuperDataScience 717 — Dan Shiebler.
  6. Balance false negatives and false positives: Shiebler says missed attacks can be catastrophic, while too many false alarms cause customers to disable or ignore protections. — SuperDataScience 717 — Dan Shiebler.
  7. Use behavioral metadata with message content: Shiebler says sign-in behavior, sender/network information, communication patterns, and email text together help distinguish impersonation from normal activity. — Authority Magazine — Dan Shiebler.

Part 6: Leadership and Building ML Teams

  1. Hire builders willing to use AI well: Shiebler says Artemis looks for open-minded builders eager to use AI-native tools, with references and practical work also important in hiring. — First Round In Depth — Artemis.
  2. Tie ML evaluation to product value: Shiebler says specialized ML teams can drift away from end users, so leaders must ask how models benefit the actual product and customers. — Authority Magazine — Dan Shiebler.
  3. Reduce the gap between model teams and product teams: Shiebler says he structured Abnormal's ML teams to stay close to customer-facing product work instead of being separated by multiple organizational layers. — Authority Magazine — Dan Shiebler.
  4. Grow from individual contribution into leadership: Shiebler credits a Twitter manager with helping him move from IC to leadership and describes organizing specialized ML teams around customer value. — Authority Magazine — Dan Shiebler.
  5. Treat production behavior as part of model design: Shiebler shows why a model that improves offline can disappoint online when serving distributions, feedback loops, or coupled systems differ. — Improving an ML System — Part 1.
  6. Keep learning as AI practice changes: Shiebler recommends continued education through papers, conferences, news, and internal reading/discussion programs rather than relying only on an initial qualification. — Authority Magazine — Dan Shiebler.
  7. Maintain feature pipelines deliberately: Shiebler details online/offline feature discrepancies, time travel, and freshness as recurring production problems that require deliberate engineering. — Improving an ML System — Part 2.
  8. Use tooling to broaden engineering capability: Shiebler says AI coding tools let engineers work across unfamiliar code areas and help Artemis move from prototypes to customer-visible changes rapidly. — First Round In Depth — Artemis.

Part 7: Startup Strategy and Founder Lessons

  1. Stay personally reachable to customers: Shiebler says he wants Artemis customers to feel comfortable texting the founders directly about problems even as the company grows. — First Round In Depth — Artemis.
  2. In founder-led sales, make the ask: Shiebler says early sales taught him to ask clearly for the next meeting, introduction, or connection while also listening and building trust. — First Round In Depth — Artemis.
  3. Build advantage into AI-native processes: Shiebler says Artemis's advantage is partly in code and internal processes designed around AI-native tooling, not merely access to models. — First Round In Depth — Artemis.
  4. Use stealth to delight early customers: Shiebler says a startup need not leave stealth to serve its first customers; public launch helps with wider acquisition and hiring after that early trust is earned. — First Round In Depth — Artemis.
  5. Launch when capabilities and market timing align: Shiebler says starting Artemis earlier would have limited AI-native building, while waiting longer might have missed the market's pace. — First Round In Depth — Artemis.

Part 8: Advanced Mathematics and Academic Research

  1. Use category theory to reason about ML structure: Shiebler's signed essay uses categorical ideas to formalize how machine-learning transformations compose and preserve relationships. — Compositionality and Functoriality in ML.
  2. Use unsupervised transformations as building blocks: Shiebler describes unsupervised feature transformations as one way to compose simpler components into a more capable ML system; the original claim about label scarcity is broader than his discussion. — Compositional Structures in ML.
  3. Study optimizer invariance, not just convergence: Shiebler analyzes how different continuous optimization algorithms behave under transformations of the parameter space; the original sweeping efficiency constraint is unsupported. — Transformation-Invariant Optimizers.
  4. Neuroscience drew him into modeling: Shiebler says analyzing Parkinson's-related brain recordings during college introduced him to analytic modeling, rather than asserting that neuroscience broadly yields novel AI architectures. — Authority Magazine — Dan Shiebler.
  5. Functoriality can formalize composition: Shiebler uses functoriality to study transformations that preserve the structural relationships of ML systems. — Compositionality and Functoriality in ML.
  6. Dimensionality reduction preserves selected structure: Shiebler contrasts PCA's emphasis on global pairwise relationships with Laplacian Eigenmaps' focus on nearby points; compression does not preserve every kind of semantic relationship. — PCA vs Laplacian Eigenmaps.
  7. A manifold assumption changes the embedding method: Shiebler explains that Laplacian Eigenmaps assumes data lie on a low-dimensional manifold, while PCA makes fewer such assumptions. — PCA vs Laplacian Eigenmaps.
  8. Compose simpler models into more capable systems: Shiebler surveys ensembles, feature transformations, and end-to-end training as ways to combine simpler models for complex tasks. — Compositional Structures in ML.
  9. Adapt models as distributions change: Shiebler distinguishes feature/data drift from concept drift and describes training and monitoring approaches to keep production models useful as inputs shift. — Resilient Machine Learning.