Continuous AI governance: automated risk reassessment and production drift alerts

Most AI risk programs run one thorough assessment at the point a system is approved and leave it there. Then the policies change, regulations change, the system’s own architecture and functionality changes, and the assessment on file describes a system that no longer exists. Redoing it by hand every time there’s a change is prohibitively expensive across dozens of live systems, so it does not happen. This keynote demonstrates two new Pacific AI capabilities built to close this gap.

The first is ongoing risk assessment in Governor. When a policy, a regulatory requirement, or a governed system’s architecture changes, Pacific AI reads the change, generates updated risk scores, risks and mitigation tasks, and shows the reviewer what moved so they can accept or override each item. Prior human decisions are preserved rather than recomputed from scratch. Take the 2025 AI impersonation statutes (Delaware HB 191 and California AB 489) for example: the AI Policy Suite updates on its quarterly cycle, every governed system the new requirement touches is reassessed automatically, and the affected ones receive new mitigation tasks, such as testing the LLM against the requirement and monitoring for it in production (with the new test suite built-in). Humans review, adjust and approve, which is what the regulation requires anyway, and they are reviewing drafts from software that has read the current policy library. Every change is versioned with a full audit trail across the system’s lifecycle, with no organizational project for each new law.

The second is monitoring and alerting on production systems in Guardian. Alerts are defined against the baseline a system established in Gatekeeper before release. A bias test that scored 0.80 pre-production can carry an alert that fires if it drops more than 5% from that baseline. Multiple alerts run at once, so a deployed system is watched across accuracy, safety, bias, reliability, fairness and escalation, using the same test suites, the same scoring and the same thresholds that gated its release. The Guardian Agent runs inside the customer’s own AWS or Azure tenant, so production probes and drift findings never leave their VPC.

This session describes how the Pacific AI platform supports organizations in meeting the requirements of the regulations named. It is not legal advice. Organizations subject to these rules should consult their own compliance counsel.

About the speaker

Julio Bonis

Principal Data Scientist at Pacific AI

Julio Bonis is a data scientist working on NLP & LLM for Healthcare at John Snow Labs. Julio has broad experience in software development and design of complex data products within the scope of Real World Evidence (RWE) and Natural Language Processing (NLP).

He also has substantial clinical and management experience – including entrepreneurship and Medical Affairs. Julio is a medical doctor specialized in Family Medicine (registered GP), has an Executive MBA – IESE, an MSc in Bioinformatics, and an MSc in Epidemiology.