12 papers
GameCommBench: A Unified Benchmark and Type-Aware Evaluation for AI-Generated Game Commentary
Qirui Zheng, Zhengteng Lin, Yunyi Xiao +7
Game commentary is an open-ended generation task requiring multimodal perception, strategic reasoning, and contextual knowledge. Existing AI-Generated Game Commentary (AI-GGC) stud…
BotzoneBench: Scalable LLM Evaluation via Graded AI Anchors
Lingfeng Li, Yunlong Lu, Yuefei Zhang +7
Large Language Models (LLMs) are increasingly deployed in interactive environments requiring strategic decision-making, yet systematic evaluation of these capabilities remains chal…
COMPKE: Complex Question Answering under Knowledge Editing
Keyuan Cheng, Zijian Kan, Zhixian He +5
Knowledge Editing, which efficiently modifies the knowledge in large language models, has gathered great attention. Current benchmarks primarily use multi-hop question answering to…
From Multimodal Perception to Strategic Reasoning: A Survey on AI-Generated Game Commentary
Qirui Zheng, Xingbo Wang, Keyuan Cheng +5
The advent of artificial intelligence has propelled AI-Generated Game Commentary (AI-GGC) into a rapidly expanding research area, offering advantages such as scalable availability…
CODEMENV: Benchmarking Large Language Models on Code Migration
Keyuan Cheng, Xudong Shen, Yihao Yang +6
Large language models (LLMs) have shown remarkable capabilities across various software engineering tasks; however, their effectiveness in code migration, adapting code to run in d…
Code-Vision: Evaluating Multimodal LLMs Logic Understanding and Code Generation Capabilities
Hanbin Wang, Xiaoxuan Zhou, Zhipeng Xu +7
This paper introduces Code-Vision, a benchmark designed to evaluate the logical understanding and code generation capabilities of Multimodal Large Language Models (MLLMs). It chall…
DrivingDojo Dataset: Advancing Interactive and Knowledge-Enriched Driving World Model
Yuqi Wang, Ke Cheng, Jiawei He +5
Driving world models have gained increasing attention due to their ability to model complex physical dynamics. However, their superb modeling capability is yet to be fully unleashe…
Locate-then-edit for Multi-hop Factual Recall under Knowledge Editing
Zhuoran Zhang, Yongxiang Li, Zijian Kan +3
The locate-then-edit paradigm has shown significant promise for knowledge editing (KE) in Large Language Models (LLMs). While previous methods perform well on single-hop fact recal…
Leveraging Logical Rules in Knowledge Editing: A Cherry on the Top
Keyuan Cheng, Muhammad Asif Ali, Shu Yang +7
Multi-hop Question Answering (MQA) under knowledge editing (KE) is a key challenge in Large Language Models (LLMs). While best-performing solutions in this domain use a plan and so…
Prompt-SAW: Leveraging Relation-Aware Graphs for Textual Prompt Compression
Muhammad Asif Ali, Zhengping Li, Shu Yang +8
Large Language Models (LLMs) have shown exceptional abilities for multiple different natural language processing tasks. While prompting is a crucial tool for LLM inference, we obse…
Multi-hop Question Answering under Temporal Knowledge Editing
Keyuan Cheng, Gang Lin, Haoyang Fei +5
Multi-hop question answering (MQA) under knowledge editing (KE) has garnered significant attention in the era of large language models. However, existing models for MQA under KE ex…
Dialectical Alignment: Resolving the Tension of 3H and Security Threats of LLMs
Shu Yang, Jiayuan Su, Han Jiang +5
With the rise of large language models (LLMs), ensuring they embody the principles of being helpful, honest, and harmless (3H), known as Human Alignment, becomes crucial. While exi…