13 citations · 26 across the 20 of their papers we have counts for
21 papers
Position: Vector Prompt Interfaces Should Be Exposed to Enable Customization of Large Language Models
Liangwei Yang, Shiyu Wang, Haolin Chen +12
As large language models (LLMs) transition from research prototypes to real-world systems, customization has emerged as a central bottleneck. While text prompts can already customi…
VoiceAgentRAG: Solving the RAG Latency Bottleneck in Real-Time Voice Agents Using Dual-Agent Architectures
Jielin Qiu, Jianguo Zhang, Zixiang Chen +12
We present VoiceAgentRAG, an open-source dual-agent memory router that decouples retrieval from response generation. A background Slow Thinker agent continuously monitors the conve…
AudioCapBench: Quick Evaluation on Audio Captioning across Sound, Music, and Speech
Jielin Qiu, Jianguo Zhang, Zixiang Chen +13
We introduce AudioCapBench, a benchmark for evaluating audio captioning capabilities of large multimodal models. \method covers three distinct audio domains, including environmenta…
LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineering
Jielin Qiu, Zuxin Liu, Zhiwei Liu +18
As large language models (LLMs) evolve into sophisticated autonomous agents capable of complex software development tasks, evaluating their real-world capabilities becomes critical…
GeoGNN: Quantifying and Mitigating Semantic Drift in Text-Attributed Graphs
Liangwei Yang, Jing Ma, Jianguo Zhang +11
Graph neural networks (GNNs) on text--attributed graphs (TAGs) typically encode node texts using pretrained language models (PLMs) and propagate these embeddings through linear nei…
Test-Time Adaptation for LLM Agents via Environment Interaction
Arthur Chen, Zuxin Liu, Jianguo Zhang +6
Large language model (LLM)-based agents struggle to generalize to novel and complex environments, such as unseen websites or new sets of functions, due to a fundamental mismatch be…
ToolLibGen: Scalable Automatic Tool Creation and Aggregation for LLM Reasoning
Murong Yue, Zhiwei Liu, Liangwei Yang +8
Large Language Models (LLMs) equipped with external tools have demonstrated enhanced performance on complex reasoning tasks. The widespread adoption of this tool-augmented reasonin…
LoCoBench: A Benchmark for Long-Context Large Language Models in Complex Software Engineering
Jielin Qiu, Zuxin Liu, Zhiwei Liu +14
The emergence of long-context language models with context windows extending to millions of tokens has created new opportunities for sophisticated code understanding and software d…
MCPEval: Automatic MCP-based Deep Evaluation for AI Agent Models
Zhiwei Liu, Jielin Qiu, Shiyu Wang +9
The rapid rise of Large Language Models (LLMs)-based intelligent agents underscores the need for robust, scalable evaluation frameworks. Existing methods rely on static benchmarks…
UserBench: An Interactive Gym Environment for User-Centric Agents
Cheng Qian, Zuxin Liu, Akshara Prabhakar +9
Large Language Models (LLMs)-based agents have made impressive progress in reasoning and tool use, enabling them to solve complex tasks. However, their ability to proactively colla…
LAM SIMULATOR: Advancing Data Generation for Large Action Model Training via Online Exploration and Trajectory Feedback
Thai Hoang, Kung-Hsiang Huang, Shirley Kokane +12
Large Action Models (LAMs) for AI Agents offer incredible potential but face challenges due to the need for high-quality training data, especially for multi-steps tasks that involv…
APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay
Akshara Prabhakar, Zuxin Liu, Ming Zhu +12
Training effective AI agents for multi-turn interactions requires high-quality data that captures realistic human-agent dynamics, yet such data is scarce and expensive to collect m…
ActionStudio: A Lightweight Framework for Data and Training of Large Action Models
Jianguo Zhang, Thai Hoang, Ming Zhu +13
Large Action models are essential for enabling autonomous agents to perform complex tasks. However, training such models remains challenging due to the diversity of agent environme…
PersonaBench: Evaluating AI Models on Understanding Personal Information through Accessing (Synthetic) Private User Data
Juntao Tan, Liangwei Yang, Zuxin Liu +11
Personalization is critical in AI assistants, particularly in the context of private AI models that work with individual users. A key scenario in this domain involves enabling AI m…
Bridging the Data Provenance Gap Across Text, Speech and Video
Shayne Longpre, Nikhil Singh, Manuel Cherep +40
Progress in AI is driven largely by the scale and quality of training data. Despite this, there is a deficit of empirical analysis examining the attributes of well-established data…
LATTE: Learning to Think with Vision Specialists
Zixian Ma, Jianguo Zhang, Zhiwei Liu +9
While open-source vision-language models perform well on simple question-answering, they still struggle with complex questions that require both perceptual and reasoning capabiliti…
ToolScan: A Benchmark for Characterizing Errors in Tool-Use LLMs
Shirley Kokane, Ming Zhu, Tulika Awalgaonkar +15
Evaluating Large Language Models (LLMs) is one of the most critical aspects of building a performant compound AI system. Since the output from LLMs propagate to downstream steps, i…
PRACT: Optimizing Principled Reasoning and Acting of LLM Agent
Zhiwei Liu, Weiran Yao, Jianguo Zhang +13
We introduce the Principled Reasoning and Acting (PRAct) framework, a novel method for learning and enforcing action principles from trajectory data. Central to our approach is the…
xLAM: A Family of Large Action Models to Empower AI Agent Systems
Jianguo Zhang, Tian Lan, Ming Zhu +19
Autonomous agents powered by large language models (LLMs) have attracted significant research interest. However, the open-source community faces many challenges in developing speci…
Personalized Multi-task Training for Recommender System
Liangwei Yang, Zhiwei Liu, Jianguo Zhang +5
In the vast landscape of internet information, recommender systems (RecSys) have become essential for guiding users through a sea of choices aligned with their preferences. These s…
Consent in Crisis: The Rapid Decline of the AI Data Commons
Shayne Longpre, Robert Mahari, Ariel Lee +46
General-purpose artificial intelligence (AI) systems are built on massive swathes of public web data, assembled into corpora such as C4, RefinedWeb, and Dolma. To our knowledge, we…