13 papers
Knowledge before Reasoning: EC-Reason-Bench, a Training-Free Diagnostic Benchmark for LLM Enzyme Classification
Linyu Li, Zhi Jin, Yichi Zhang +6
The paper introduces EC-Reason-Bench, a training-free diagnostic benchmark that evaluates why general large language models struggle with detailed enzyme classification and how per…
From Chat to Interview: Agentic Requirements Elicitation with an Experience Ontology
Dongming Jin, Zhi Jin, Yaotian Yang +5
Requirements elicitation interviews are crucial and time-consuming in requirements engineering, but heavily rely on the experience of requirements analysts. Although recent advance…
When Modalities Remember: Continual Learning for Multimodal Knowledge Graphs
Linyu Li, Zhi Jin, Yichi Zhang +5
Real-world multimodal knowledge graphs (MMKGs) are dynamic, with new entities, relations, and multimodal knowledge emerging over time. Existing continual knowledge graph reasoning…
Your Inference Request Will Become a Black Box: Confidential Inference for Cloud-based Large Language Models
Chung-ju Huang, Huiqiang Zhao, Yuanpeng He +5
The increasing reliance on cloud-hosted Large Language Models (LLMs) exposes sensitive client data, such as prompts and responses, to potential privacy breaches by service provider…
BrowseComp-: A Visual, Vertical, and Verifiable Benchmark for Multimodal Browsing Agents
Huanyao Zhang, Jiepeng Zhou, Bo Li +22
Multimodal large language models (MLLMs), equipped with increasingly advanced planning and tool-use capabilities, are evolving into autonomous agents capable of performing multimod…
ReqElicitGym: An Evaluation Environment for Interview Competence in Conversational Requirements Elicitation
Dongming Jin, Zhi Jin, Zheng Fang +4
With the rapid improvement of LLMs' coding capabilities, the bottleneck of LLM-based automated software development is shifting from generating correct code to eliciting users' req…