IndicSafe: Evaluating Multilingual LLM Safety in South Asia
As large language models (LLMs) are deployed in multilingual settings, their safety behavior in culturally diverse, low-resource languages remains poorly understood. We present the first systematic evaluation of LLM safety across 12 Indic languages, spoken by over 1.2 billion people but underrepresented in LLM training data. Using a dataset of 6,000 culturally grounded prompts spanning caste, religion, gender, health, and politics, we assess 10 leading LLMs on translated variants of the prompt.
Our analysis reveals significant safety drift: cross-language agreement is just 12.8%, and SAFE rate variance exceeds 17% across languages. Some models over-refuse benign prompts in low-resource scripts, overflag politically sensitive topics, while others fail to flag unsafe generations. We quantify these failures using prompt-level entropy, category bias scores, and multilingual consistency indices.
Our findings highlight critical safety generalization gaps in multilingual LLMs and show that safety alignment does not transfer evenly across languages.
About the speaker
Priyaranjan Pattnayak
Sr. Principal Scientist at Oracle
Priyaranjan (“Priyan”) Pattnayak is a strong AI 100 candidate because he combines production AI leadership, rigorous AI research, and community-building in responsible multilingual AI. As a Senior Principal Data Scientist at Oracle Cloud AI, Priyan builds enterprise-scale generative AI and NLP systems for real customer and operational workflows, including RAG-based assistants, hybrid response routing, multi-turn conversational orchestration, AI observability, and evaluation infrastructure for production AI agents. A public Oracle case study on the OCI GenAI-powered support chatbot shows the scale and business impact of this work: OCI support receives more than 600,000 annual interactions, and after the GenAI chatbot deployment, deflection increased from 52% in FY24 to 64% in FY25Q1, supported by a dynamic retrieval pipeline processing 30GB of documents daily. What makes Priyan distinctive is that his production work and research reinforce each other. His research targets the failure modes that determine whether AI works beyond demos: enterprise RAG, hard-negative retrieval, multilingual safety, low-resource language modeling, agent evaluation, and multimodal robustness. His Hybrid AI paper introduced a confidence-based routing and feedback-adaptation architecture for enterprise conversational AI, reporting 95% accuracy and 180ms latency. His ACL Industry Track work on hard-negative mining improved enterprise retrieval reranking by 15% MRR@3 and 19% MRR@10. His IndicSafe benchmark evaluates multilingual LLM safety across 12 Indic languages and 6,000 culturally grounded prompts, exposing safety drift for languages used by more than 1.2B people. Repello AI has used IndicSafe to evaluate CREST, its multilingual AI safety guardrail system. His MVTamperBench work, published in ACL Findings, released code, data, and benchmark resources for evaluating video-tampering robustness in vision-language models, and has been integrated into the broader VLM evaluation ecosystem. Priyan also contributes to the AI community as ACL Industry Track Area Chair, reviewer for major AI/NLP venues, mentor to students and early-career researchers, and co-organizer of RLEval at ACM CAIS 2026, a workshop focused on methods, RL environments, benchmarks, and real-world case studies for evaluating AI agents. With 13+ years of AI/ML experience, 15+ peer-reviewed publications, and 11+ filed patents, Priyan represents the AI 100 builder-researcher profile: someone translating frontier AI into reliable enterprise systems while advancing safer, more inclusive AI for multilingual and low-resource communities.