Methodology & Scoring
1. Our approach
The Workbench's analytical tools (System Intake, Fairness Scanner, Drift & Data Quality, Doc Gap Scanner,
the readiness score) are all deterministic rule engines written in plain JavaScript
(engines/risk.js, engines/fairness.js, engines/drift.js,
engines/remediation.js) — the same inputs always produce the same outputs, and every output
carries a human-readable reason. None of these five tools call a language model. Separately, a handful of
features are explicitly AI-assisted (the Governance Copilot, Agent Loop, and the blog) — see §7 for exactly
which ones and how that differs.
2. Risk-tier classification
The System Intake tool maps your answers to an EU AI Act risk tier — prohibited, high, limited, or minimal — using a fixed decision table, not a probabilistic model:
- Prohibited — you flagged a practice listed under EU AI Act Art. 5 (e.g. social scoring, subliminal manipulation, untargeted biometric scraping).
- High — the system operates in one of the eleven Annex III high-risk domains (employment, credit/finance, insurance, healthcare, essential services, critical infrastructure, biometric ID, law enforcement, migration, justice, education) and materially affects a decision about a person.
- Limited — the system uses generative AI or interacts directly with people (e.g. a chatbot), triggering the Art. 50 transparency duty, without meeting the High-risk criteria.
- Minimal — none of the above conditions are met.
Every classification returns a reasons[] list stating exactly which answer(s) drove the outcome —
shown to you in the tool itself, not hidden.
3. Readiness score (0–100)
The readiness score sums four independently-capped components — it is not a machine-learning prediction, just arithmetic:
| Component | Max points | How it's computed |
|---|---|---|
| Documentation completeness | 35 | (Doc Gap Scanner completeness % ÷ 100) × 35. If you haven't run the scanner yet, a flat 8-point placeholder is used instead of assuming full or zero coverage. |
| Fairness | 25 | 25 if the Fairness Scanner passes; otherwise 25 − (8 × number of failing groups), floored at 0. If you haven't run it yet: a flat 12-point placeholder. |
| Drift & data quality | 20 | 20 − (5 × number of drift/quality flags), floored at 0. If neither tool has been run: a flat 9-point placeholder. |
| Remaining process burden by tier | 20 | A fixed allowance reflecting how much obligation work is inherent to your tier: Prohibited 0, High 8, Limited 14, Minimal 20. |
The four components are summed and rounded to the nearest whole number, then clamped to 0–100. If your system is classified Prohibited, the score is additionally capped at 25 regardless of the other components — a prohibited practice cannot be scored as "ready." The resulting number maps to a band: 80+ Strong, 60–79 Developing, 35–59 Early, below 35 At risk.
4. Fairness Scanner
The scanner computes each group's disparate impact ratio — that group's selection rate divided by the highest-selection-rate group's rate — for every protected attribute in your data. It applies the standard four-fifths rule: any group with a ratio below 0.80 fails. The tool passes only if every group clears that bar; each failing group is reported by name with its exact ratio.
5. Drift & Data Quality
For numeric features, drift is measured with the Population Stability Index (PSI) computed across 10 quantile bins derived from your baseline dataset; categorical features use a PSI variant across the union of observed categories. A feature is flagged as significant drift when PSI ≥ 0.20 — the conventional industry threshold. Data-quality checks separately flag missing values, schema changes, and outlier concentrations using standard IQR-based bounds.
6. Doc Gap Scanner
Completeness is ((fully-present controls + 0.5 × partially-present controls) ÷ total controls) × 100,
rounded to the nearest whole percent, against a fixed checklist of documentation controls (intended use,
training data, evaluation, limitations, human oversight, monitoring, incident response, and more).
7. Where AI is actually used
Everything above runs without any AI model. The features that do send your input to a third-party LLM provider are the Governance Copilot (chat answers), the Governance Agent / Agent Loop (drafted remediation steps), and the AI-written blog — each one is labeled with a "Third-party AI processing" badge in the Workbench (see our subprocessor list). If no provider is configured on a given deployment, these features fall back to a deterministic, server-only answer rather than silently failing.
8. Limitations & disclaimers
- These are educational risk-triage aids, not legal, compliance, or professional advice — see Terms §3.
- Scores and classifications are only as accurate as the answers and data you provide; we do not independently verify your inputs.
- The Fairness Scanner and Drift tools analyze whatever sample you load — a small or unrepresentative sample will understate real-world disparities or drift.
- Regulatory mappings reflect our curated reading of public sources as of each entry's last-verified date; they are not a substitute for legal review, and proposed legislation is explicitly labeled "Proposed" rather than treated as binding — see the status badges on each matched regulation in the System Intake and Compliance Dossier panels of the Workbench.
- Trust Passports and readiness scores reflect self-attested inputs unless a specific engagement says otherwise — see Terms §8.
9. Contact
Questions about how a specific number was calculated: governance@autogovern.io.