4 papers
Executable but Unfaithful: Mock Object, Stub, and Assertion Generation by LLMs and Coding Agents
Hao Ning, Jiahui Dong, Zhen Yang +7
Mocking is a foundational practice in unit testing, enabling developers to isolate the code under test by replacing real dependencies with simulated alternatives. Automating the ge…
VTC-Bench: Evaluating Agentic Multimodal Models via Compositional Visual Tool Chaining
Xuanyu Zhu, Yuhao Dong, Rundong Wang +9
Recent advancements extend Multimodal Large Language Models (MLLMs) beyond standard visual question answering to utilizing external tools for advanced visual tasks. Despite this pr…
RealUnify: Do Unified Models Truly Benefit from Unification? A Comprehensive Benchmark
Yang Shi, Yuhao Dong, Yue Ding +22
The integration of visual understanding and generation into unified multimodal models represents a significant stride toward general-purpose AI. However, a fundamental question rem…
TransLibEval: Demystify Large Language Models' Capability in Third-party Library-targeted Code Translation
Pengyu Xue, Kunwu Zheng, Zhen Yang +11
In recent years, Large Language Models (LLMs) have been widely studied in the code translation field on the method, class, and even repository levels. However, most of these benchm…