8 papers
Culture In a Frame: CB as a Comic-Based Benchmark for Multimodal Culturally Awareness
Yuchen Song, Andong Chen, Wenxin Zhu +4
Cultural awareness capabilities have emerged as a critical capability for Multimodal Large Language Models (MLLMs). However, current benchmarks lack progressed difficulty in their…
An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4
Hui Huang, Xingyuan Bu, Hongli Zhou +5
Recently, there has been a growing trend of utilizing Large Language Model (LLM) to evaluate the quality of other LLMs. Many studies have fine-tuned judge models based on open-sour…
Look Before You Leap: Enhancing Attention and Vigilance Regarding Harmful Content with GuidelineLLM
Shaoqing Zhang, Zhuosheng Zhang, Kehai Chen +4
Despite being empowered with alignment mechanisms, large language models (LLMs) are increasingly vulnerable to emerging jailbreak attacks that can compromise their alignment mechan…
LLM-based Discriminative Reasoning for Knowledge Graph Question Answering
Mufan Xu, Kehai Chen, Xuefeng Bai +3
Large language models (LLMs) based on generative pre-trained Transformer have achieved remarkable performance on knowledge graph question-answering (KGQA) tasks. However, LLMs ofte…
Make Imagination Clearer! Stable Diffusion-based Visual Imagination for Multimodal Machine Translation
Andong Chen, Yuchen Song, Kehai Chen +3
Visual information has been introduced for enhancing machine translation (MT), and its effectiveness heavily relies on the availability of large amounts of bilingual parallel sente…
LLM-based Translation Inference with Iterative Bilingual Understanding
Andong Chen, Kehai Chen, Yang Xiang +5
The remarkable understanding and generation capabilities of large language models (LLMs) have greatly improved translation performance. However, incorrect understanding of the sent…