From the 1 of 7 linked papers with an AI index.
7 papers
Project Kaleidoscope: Contextual, Human-Aligned Evaluation for Real-World AI Applications
Leanne Tan, Rohan Jaggi, Shaun Khoo +1
The paper introduces Kaleidoscope, an integrated workflow that combines persona‑based test generation, contextual rubrics, and human review with LLM‑based judges to provide reliabl…
Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook
Jaehyeok Lee, Xiaoyuan Yi, Jing Yao +4
As LLMs are globally deployed, aligning their cultural value orientations is critical for safety and user engagement. However, existing benchmarks face the Construct-Composition-Co…
SEAHateCheck: Functional Tests for Detecting Hate Speech in Low-Resource Languages of Southeast Asia
Ri Chi Ng, Aditi Kumaresan, Yujia Hu +1
Hate speech detection relies heavily on linguistic resources, which are primarily available in high-resource languages such as English and Chinese, creating barriers for researcher…
Lost in Stories: Consistency Bugs in Long Story Generation by LLMs
Junjie Li, Xinrui Guo, Yuhao Wu +3
What happens when a storyteller forgets its own story? Large Language Models (LLMs) can now generate narratives spanning tens of thousands of words, but they often fail to maintain…
Lost in Localization: Building RabakBench with Human-in-the-Loop Validation to Measure Multilingual Safety Gaps
Gabriel Chua, Leanne Tan, Ziyu Ge +1
Large language models (LLMs) often fail to maintain safety in low-resource language varieties, such as code-mixed vernaculars and regional dialects. We introduce RabakBench, a mult…
LionGuard 2: Building Lightweight, Data-Efficient & Localised Multilingual Content Moderators
Leanne Tan, Gabriel Chua, Ziyu Ge +1
Modern moderation systems increasingly support multiple languages, but often fail to address localisation and low-resource variants - creating safety gaps in real-world deployments…