AI Quality Systems
Source-grounded pack with Define an AI Quality Bar, Build an AI Eval Suite, and Review AI Workflow Drift, plus 1 more.
Use this pack when
Choose this pack when the work touches AI Quality Systems and you need a bounded workflow with explicit inputs, a defined output, and human review gates.
Included workflows
Choose the job in front of you.
4 source-grounded skills. Load one workflow at a time.
Define an AI Quality Bar
Use this workflow when defining what acceptable AI workflow performance means, how calibrated reviewers judge it, which confidence bands route actions, who owns the bar, and what evidence blocks or permits release.
Output Return an aiqualitybar with workflow, consequences, dimensions, scoreanchors, criticalfailures, reviewercalibration, routingthresholds, releasegates, stopconditions, qualityowner, domainapprovers, and version.
Build an AI Eval Suite
Use this workflow when turning an approved AI quality bar into a living gold set and regression suite of representative, edge, adversarial, exception, and critical cases with traceable judgments and release thresholds.
Output Return an evalsuitespec with unit, caseinventory, coverage, provenance, expectedproperties, rubricversion, developmentsplit, heldoutsplit, baseline, thresholds, criticalcases, maintenancerules, and owner.
Review AI Workflow Drift
Use this workflow when AI workflow quality changes after model, prompt, tool, data, policy, traffic, or reviewer changes and operators need to isolate the broken assumption, measure regressions, and decide whether to monitor, repair, roll back, or stop.
Output Return a driftreview with signal, measurementcheck, changediff, failureclusters, brokenassumptions, reproduction, blastradius, criticalfailures, decision, repair, regressioncases, and owner.
Respond to an AI Quality Incident
Use this workflow when an AI workflow quality escape or release risk requires containment, user-impact assessment, rollback, evidence preservation, owner coordination, corrective changes, regression coverage, and a governed return-to-service decision.
Output Return a qualityincidentplan with severity, owners, containment, evidence, affectedscope, statecorrection, communications, rootassumption, correctivechange, newregressions, returngate, and prevention.
Human review
Keep consequential decisions explicit.
These workflows structure analysis and artifacts; they do not authorize autonomous external, personnel, financial, legal, security, or other high-impact action.
Source and files
Grounded in the complete AI Quality Systems series.
10 source essays · 4 usable skills · reviewed Jul 2026