4 papers
You're Hired: Strategic Model Selection for LLM Collaboration
Zongwan Cao, Ziyuan Yang, Shangbin Feng +5
While multi-agent and model collaboration algorithms gain traction to combine the strengths of diverse Large Language Models (LLMs), existing systems remain bottlenecked on pre-def…
Illusions of the Gold Standard: A Large-scale Analysis of Human Evaluation Protocols for Long-form Text Generation
Katelyn Xiaoying Mei, Yi-Li Hsu, Minjoon Choi +5
Human evaluation plays a critical role in assessing the quality of generated text. However, the reliability and reproducibility of these evaluations depend on transparent and well-…
AgenticVBench: Can AI Agents Complete Real-World Post-Production Tasks?
Zongheng Cao, Yi Zheng, Rui Song +1
Video production workflows offer a rich and demanding arena for evaluating multimodal AI agents: they require composite capabilities across text, image, audio, and video understand…
Clarify or Answer: Reinforcement Learning for Agentic VQA with Context Under-specification
Zongwan Cao, Bingbing Wen, Lucy Lu Wang
Real-world visual question answering (VQA) may be context-dependent: an image-question pair could be under-specified, such that the correct answer depends on external information t…