Navigating AI Uncertainty in Healthcare: Orchestrating Task‑Specific Language Models for Medico‑Legal Workflows

Healthcare organizations are under pressure to adopt generative AI while managing uncertainty around cost, reliability, privacy, vendor lock‑in, and accountability in clinical and medico‑legal workflows. My talk presents a real‑world case study from the Canadian Medical Protective Association, where we designed and deployed an internally hosted architecture of orchestrated, task‑specific small language models (SLMs) for medical‑legal advice workflows. The system automates Note‑to‑File summarization and multi‑label classification of medico‑legal issues across thousands of physician advice cases, achieving high-quality performance comparable to, and in some areas exceeding, human coders, with quasi‑deterministic behavior suitable for governance‑critical use. I will share our end‑to‑end approach, including de‑identification, fine‑tuning, validation against human agreement, human‑in‑the‑loop review, and operational monitoring. The session is designed to give healthcare leaders and technical teams a concrete blueprint for building regulatory‑grade, on‑premises AI that balances innovation, risk management, and strategic autonomy.

About the speaker

Shilin Zhao

Manager, AI & Advanced Analytics at Canadian Medical Protective Association

Shilin Zhao, MA, CRM, FRM, CFA, is a healthcare AI leader focused on turning advanced models into safe, production‑grade systems across clinical, medical‑legal, and operational workflows. He leads AI and advanced analytics initiatives at the Canadian Medical Protective Association, applying LLMs, clinical NLP, and multimodal methods to real‑world physician and patient‑care challenges. His work spans enterprise healthcare data platforms, retrieval‑augmented generation, and model governance, with a strong emphasis on privacy, security, and regulatory‑grade reliability. A hands‑on builder, Shilin codes daily in Python and modern ML stacks, bridging research, engineering, and compliance. He is particularly interested in how applied AI can measurably improve care quality, workflow efficiency, and risk management in complex healthcare environments.