5 papers
With Great Capabilities Come Great Responsibilities: Introducing the Agentic Risk & Capability Framework for Governing Agentic AI Systems
Shaun Khoo, Jessica Foo, Roy Ka-Wei Lee
Agentic AI systems present both significant opportunities and novel risks due to their capacity for autonomous action, encompassing tasks such as code execution, internet interacti…
Measuring What Matters: A Framework for Evaluating Safety Risks in Real-World LLM Applications
Jia Yi Goh, Shaun Khoo, Nyx Iskandar +3
Most safety testing efforts for large language models (LLMs) today focus on evaluating foundation models. However, there is a growing need to evaluate safety at the application lev…
Know Or Not: a library for evaluating out-of-knowledge base robustness
Jessica Foo, Pradyumna Shyama Prasad, Shaun Khoo
While the capabilities of large language models (LLMs) have progressed significantly, their use in high-stakes applications have been limited due to risks of hallucination. One key…
Safe at the Margins: A General Approach to Safety Alignment in Low-Resource English Languages -- A Singlish Case Study
Isaac Lim, Shaun Khoo, Roy Ka-Wei Lee +3
Ensuring the safety of Large Language Models (LLMs) in diverse linguistic settings remains challenging, particularly for low-resource languages. Existing safety alignment methods a…
LionGuard: Building a Contextualized Moderation Classifier to Tackle Localized Unsafe Content
Jessica Foo, Shaun Khoo
As large language models (LLMs) become increasingly prevalent in a wide variety of applications, concerns about the safety of their outputs have become more significant. Most effor…