3 citations · 3 across the 10 of their papers we have counts for
10 papers
Lower-Resource, Higher Scores: Language Bias in LLM Evaluators
Ej Zhou, Lucas Resck, Zheng Hui +1
LLM evaluators (trained reward models and prompted LLM-as-a-Judge) are routinely validated via pairwise accuracy. In a multilingual setting, this operates under the premise that hi…
Dial HEALTHDIAL for Advice: A Multilingual and Multi-Parallel Spoken Dialogue Dataset for Knowledge-Grounded Information Seeking
Songbo Hu, Yinhong Liu, Ej Zhou +5
Creating spoken dialogue datasets is methodologically challenging, and these challenges are amplified when the goal is to build multilingual, multi-parallel datasets at scale. This…
Artificial intelligence is creating a new global linguistic hierarchy
Giulia Occhini, Kumiko Tanaka-Ishii, Anna Barford +10
Artificial intelligence (AI) has the potential to transform healthcare, education, governance and socioeconomic equity, but its benefits remain concentrated in a small number of la…
CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data
Pedro Ortiz Suarez, Laurie Burchell, Catherine Arnett +94
Language identification (LID) is a fundamental step in curating multilingual corpora. However, LID models still perform poorly for many languages, especially on the noisy and heter…
Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models
Ej Zhou, Caiqi Zhang, Tiancheng Hu +4
Confidence calibration, the alignment of a model's predicted confidence with its actual accuracy, is crucial for the reliable deployment of Large Language Models (LLMs). However, t…
Safe-SAIL: Towards a Fine-grained Safety Landscape of Large Language Models via Sparse Autoencoder Interpretation Framework
Jiaqi Weng, Han Zheng, Hanyu Zhang +6
Sparse autoencoders (SAEs) enable interpretability research by decomposing entangled model activations into monosemantic features. However, under what circumstances SAEs derive mos…
MSRS: Evaluating Multi-Source Retrieval-Augmented Generation
Rohan Phanse, Yijie Zhou, Kejian Shi +4
Retrieval-augmented systems are typically evaluated in settings where information required to answer the query can be found within a single source or the answer is short-form or fa…
Bias Beyond English: Evaluating Social Bias and Debiasing Methods in a Low-Resource Setting
Ej Zhou, Weiming Lu
Social bias in language models can potentially exacerbate social inequalities. Despite it having garnered wide attention, most research focuses on English data. In a low-resource s…
ODSum: New Benchmarks for Open Domain Multi-Document Summarization
Yijie Zhou, Kejian Shi, Wencai Zhang +3
Open-domain Multi-Document Summarization (ODMDS) is a critical tool for condensing vast arrays of documents into coherent, concise summaries. With a more inter-related document set…
Revisiting Cross-Lingual Summarization: A Corpus-based Study and A New Benchmark with Improved Annotation
Yulong Chen, Huajian Zhang, Yijie Zhou +8
Most existing cross-lingual summarization (CLS) work constructs CLS corpora by simply and directly translating pre-annotated summaries from one language to another, which can conta…