AI Quality Systems
Source-grounded pack with Define an AI Quality Bar, Build an AI Eval Suite, and Review AI Workflow Drift, plus 1 more.
Get this packGrounded in the complete AI Quality Systems series.
Open the pack, choose a workflow, and follow its setup and input instructions. You can also open each workflow below.
10 source essays · 4 usable skills · reviewed Jul 2026
Use this pack when
Choose this pack when the work touches AI Quality Systems and you need a bounded workflow with explicit inputs, a defined output, and human review gates.
Choose the job in front of you.
4 source-grounded skills. Load one workflow at a time.
Define an AI Quality Bar
Use this workflow when defining what acceptable AI workflow performance means, how calibrated reviewers judge it, which confidence bands route actions, who owns the bar, and what evidence blocks or permits release.
Output specification
Return an ai_quality_bar with workflow, consequences, dimensions, score_anchors, critical_failures, reviewer_calibration, routing_thresholds, release_gates, stop_conditions, quality_owner, domain_approvers, and version.
Build an AI Eval Suite
Use this workflow when turning an approved AI quality bar into a living gold set and regression suite of representative, edge, adversarial, exception, and critical cases with traceable judgments and release thresholds.
Output specification
Return an eval_suite_spec with unit, case_inventory, coverage, provenance, expected_properties, rubric_version, development_split, held_out_split, baseline, thresholds, critical_cases, maintenance_rules, and owner.
Review AI Workflow Drift
Use this workflow when AI workflow quality changes after model, prompt, tool, data, policy, traffic, or reviewer changes and operators need to isolate the broken assumption, measure regressions, and decide whether to monitor, repair, roll back, or stop.
Output specification
Return a drift_review with signal, measurement_check, change_diff, failure_clusters, broken_assumptions, reproduction, blast_radius, critical_failures, decision, repair, regression_cases, and owner.
Respond to an AI Quality Incident
Use this workflow when an AI workflow quality escape or release risk requires containment, user-impact assessment, rollback, evidence preservation, owner coordination, corrective changes, regression coverage, and a governed return-to-service decision.
Output specification
Return a quality_incident_plan with severity, owners, containment, evidence, affected_scope, state_correction, communications, root_assumption, corrective_change, new_regressions, return_gate, and prevention.
Keep consequential decisions explicit.
These workflows structure analysis and artifacts; they do not authorize autonomous external, personnel, financial, legal, security, or other high-impact action.