4 papers
Beyond Semantic Accuracy: Consequence-Aware Evaluation for Safety-Critical Language Understanding
Yujing Chang, Thinh Pham, Van-Phat Thai +4
Can language models be trusted in safety- critical operations? In such settings, strong per- formance on semantic metrics does not guaran- tee operational reliability: a misread al…
Safety-Oriented Evaluation of Language Understanding Systems for Air Traffic Control
Yujing Chang, Yash Guleria, Duc-Thinh Pham +4
Air Traffic Control (ATC) is a safety-critical domain in which incorrect interpretation of instructions may lead to severe operational consequences. While large language models (LL…
Benchmarking Retrieval-Augmented Multimodal Generation for Document Question Answering
Kuicai Dong, Yujing Chang, Shijie Huang +3
Document Visual Question Answering (DocVQA) faces dual challenges in processing lengthy multimodal documents (text, images, tables) and performing cross-modal reasoning. Current do…
MMDocIR: Benchmarking Multimodal Retrieval for Long Documents
Kuicai Dong, Yujing Chang, Xin Deik Goh +3
Multimodal document retrieval aims to identify and retrieve various forms of multimodal content, such as figures, tables, charts, and layout information from extensive documents. D…