Risk & Compliance for AI / Automation

Run automation and AI like production systems that touch money, customers, and controls. Set policy, classify risk, place guardrails, and keep evidence. Use one data story for operations, finance, and audit.

Overview

Risk management for AI/automation is not theory. It is policy, design, and runtime discipline. Classify the work, restrict what the system can touch, log every meaningful action, and prove value with dated evidence. If a change goes wrong, roll back in minutes.

Frameworks & standards

Core references

  • NIST AI Risk Management Framework (govern, map, measure, manage) — nist.gov
  • ISO/IEC 23894:2023 (AI risk management) — iso.org
  • ISO/IEC 42001:2023 (AI management system) — iso.org
  • ISO/IEC 27001 (information security) — iso.org
  • OWASP Top-10 for LLM Apps — owasp.org
  • COBIT & ITIL for change/governance — isaca.org · axelos.com

Regulatory touchpoints

  • GDPR (EU) data rights & DPIA — EUR-Lex
  • EU AI Act (risk-based obligations) — EU AI Act
  • HIPAA (US healthcare) — hhs.gov

Governance & policy

Operating model

  • Roles: product owner, model owner, data steward, security, compliance
  • Decision catalog & Delegations of Authority (DoA) for high-impact actions
  • Change windows, approvals, and rollback plans

Use policy

  • Allowed use cases; prohibited inputs; redaction rules
  • Human-in-the-loop thresholds and disclosure rules

Repository

  • One source of truth: maps, controls, SOPs, model cards, system cards
  • Version history; access by role; immutable logs

Risk assessment & classification

Classify

  • Impact: customer-visible, financial, safety, regulatory
  • Likelihood: model uncertainty, data drift, dependency risk
  • Assign risk level → control strength and approval tier

DPIA / AI risk record

  • Purpose, data types, storage, retention, processors
  • Mitigations and residual risk; review cadence

Data protection & privacy

Minimize & protect

  • Data minimization; approved corpora; contextual RAG only
  • PII/PHI redaction; encryption in transit/at rest
  • Retention & deletion by policy; residency where required

Rights & transparency

  • Subject requests: access, correction, deletion
  • User notices for AI-assisted outputs when policy requires

Identity, access & segregation of duties

Access

  • RBAC/ABAC on data, prompts, tools, and deployments
  • OAuth 2.0 / OIDC for service access
  • Secrets in vaults; short-lived tokens

SoD

  • Developer ≠ Deployer; Operator ≠ Reconciler
  • Approver ≠ Requester for high-impact actions
  • Quarterly access reviews with evidence

Model/Automation lifecycle controls

Design → deploy

  • Maps (BPMN/CMMN), controls, data contracts, RACI
  • Model/robot tests; change approvals; staged releases

Operate → improve

  • Drift checks; retraining gates; rollback
  • Retire or quarantine failing components

Documentation

  • Model and system cards (purpose, data, limits, evals)
  • Changelogs; deprecation and sunset dates

Testing, evaluation & red-teaming

Automation

  • Unit → contract → end-to-end; canary or blue/green
  • Rollback in minutes; runbooks with owners

AI/LLM

  • Offline evals: accuracy, groundedness, robustness
  • Online: override rate, safety flags, approval latency
  • Adversarial prompts; jailbreak/poison tests

References

Monitoring, logging & incident response

Observability

  • Distributed tracing; correlation IDs; SIEM integration
  • SLIs/SLOs: latency, error rate, retries, safety hits
  • Alerts on drift, data breaks, tool misuse

Incidents

  • Kill switch and rollback; severity rules
  • Post-incident review; control updates; evidence

Logs to keep

  • Prompts, retrieved sources, tool calls, outputs
  • Approvals/overrides (who/what/when)
  • Data access and retention events

Third-party & supply chain

Vendors & models

  • DPA and security posture; data residency and retention
  • SLOs, rate limits, quotas, and cost controls
  • API contracts; deprecation and change notices

Artifacts

  • SBOM/Model card if available
  • Pen test or SOC 2/SOC 3 summaries where appropriate

Documentation & transparency

Keep current

  • Purpose, scope, owners, and change log
  • Data sources, eval results, known limits
  • User notices and escalation paths

Templates

  • Model/system cards; DPIA/AI risk record
  • Decision catalog & DoA; control tests

Controls matrix & KCIs

Suggested controls

  • Policy, access, data minimization, retention, encryption
  • Prompt/retrieval templates; input/output filters
  • Approvals for high-impact actions; audit trail
  • Change control; staged releases; rollback
  • Monitoring; incident response; post-incident review

Key control indicators (KCIs)

  • Late access reviews; orphaned accounts
  • Safety/override flags per 1k requests
  • Log coverage; missing approvals; failed reconciliations
  • Mean time to rollback; incident count by severity

90-day starter

Days 0–30: Set the guardrails

  • Publish policy and role model; classify top use cases
  • Stand up logging (prompts, tools, outputs); add kill switch

Days 31–60: Prove control

  • Run first evals/red-team; fix weak controls
  • Add SLOs; wire alerts to SIEM; dry-run rollback

Days 61–90: Operate & review

  • Start monthly risk review and quarterly access/SoD
  • Publish KPIs/KCIs and a short assurance pack

References

Guard value with controls you can explain and evidence you can prove.

If you want a control matrix starter (policy → access → lifecycle → monitoring), ask for a copy.

Contact us