11 papers
Beyond Literal Mapping: Benchmarking and Improving Non-Literal Translation Evaluation
Yanzhi Tian, Cunxiang Wang, Zeming Liu +5
Large Language Models (LLMs) have significantly advanced Machine Translation (MT), applying them to linguistically complex domains-such as Social Network Services, literature etc.…
Incorporating Self-Rewriting into Large Language Model Reasoning Reinforcement
Jiashu Yao, Heyan Huang, Shuang Zeng +6
Through reinforcement learning (RL) with outcome correctness rewards, large reasoning models (LRMs) with scaled inference computation have demonstrated substantial success on compl…
SafeToolBench: Pioneering a Prospective Benchmark to Evaluating Tool Utilization Safety in LLMs
Hongfei Xia, Hongru Wang, Zeming Liu +3
Large Language Models (LLMs) have exhibited great performance in autonomously calling various tools in external environments, leading to better problem solving and task automation…
RepoDebug: Repository-Level Multi-Task and Multi-Language Debugging Evaluation of Large Language Models
Jingjing Liu, Zeming Liu, Zihao Cheng +7
Large Language Models (LLMs) have exhibited significant proficiency in code debugging, especially in automatic program repair, which may substantially reduce the time consumption o…
PRIM: Towards Practical In-Image Multilingual Machine Translation
Yanzhi Tian, Zeming Liu, Zhengyang Liu +4
In-Image Machine Translation (IIMT) aims to translate images containing texts from one language to another. Current research of end-to-end IIMT mainly conducts on synthetic data, w…
HomeBench: Evaluating LLMs in Smart Homes with Valid and Invalid Instructions Across Single and Multiple Devices
Silin Li, Yuhang Guo, Jiashu Yao +2
Large language models (LLMs) have the potential to revolutionize smart home assistants by enhancing their ability to accurately understand user needs and respond appropriately, whi…