Enterprise AI — Course 3: Evaluating AI Quality

Lesson list

1. Success and consequential errorsThis lesson defines useful outcomes and distinguishes errors by their consequences before comparing scores.
2. Representative evidenceThis lesson designs coverage for intended work and keeps stress tests distinct from estimates of ordinary performance.
3. References and judgement qualityThis lesson checks the basis of correctness and calibrates evaluation rules before trusting their scores.
4. Metrics and denominatorsThis lesson calculates essential rates and separates answered-case quality from useful outcomes across all attempts.
5. Uncertainty and limited evidenceThis lesson distinguishes what a finite result shows from broader claims and chooses evidence that addresses the remaining uncertainty.
6. Fair experiments and leakageThis lesson distinguishes useful comparisons from unsupported attribution and protects evidence reserved for evaluation.
7. Acceptance and regressionThis lesson turns credible evaluation evidence into a decision under joint requirements and human capacity limits.
8. Challenging an evaluation claimThis lesson audits a complete comparison and revises a recommendation when its evidence changes.
9. Making an evidence-based evaluation decisionThis final assessment applies evaluation concepts to unfamiliar records and comparisons.