9 citations · 41 across the 49 of their papers we have counts for
53 papers
Fine-Tuning on Self-Generated and Reward-Weighted Data: Learning Dynamics, Convergence Rates, and Benefits of Off-Policyness
Zhiwei Wang, Yanxi Chen, Yaliang Li +1
We study the learning dynamics of fine-tuning a policy model on self-generated and reward-weighted data, with particular focus on a generalized version of REINFORCE -- referred to…
Context as an Environment: Programmatic Context Management for Long-Horizon Agents
Yin Lin, Elaine Ang, Erkang Zhu +2
LLM agents increasingly take on long-running tasks whose history grows far beyond a single model context window. Existing approaches compress earlier interactions or extract select…
Private Direct Preference Optimization for LLM Alignment
Yangfan Jiang, Fei Wei, Ergute Bao +3
Direct preference optimization (DPO) is now a standard method for aligning large language models (LLMs) using human preference data. Each DPO example contains a prompt and a pair o…
From Atomic Actions to Standard Operating Procedures: Iterative Tool Optimization for Self-Evolving LLM Agents
Haipeng Ding, Yuexiang Xie, Zhewei Wei +2
Tool utilization enables Large Language Model (LLM) agents to interact with the real world and resolve complex tasks. However, existing agent frameworks predominantly rely on stati…
Connect the Dots: Training LLMs for Long-Lifecycle Agents with Cross-Domain Generalization Via Reinforcement Learning
Yanxi Chen, Weijie Shi, Yuexiang Xie +6
This work presents a general framework for training large language models (LLMs) to "Connect the Dots" (CoD), a meta-capability required by long-lifecycle agents: as an LLM-based A…
Beyond Domains: Reusing Web Skills via Transferable Interaction Patterns
Shiqi He, Yue Cui, Feijie Wu +5
Large language model (LLM) web agents are usually deployed as tool callers: each turn, the model reads a fresh page observation and emits one structured tool action. When every act…
Retrieval Heads are Dynamic
Yuping Lin, Zitao Li, Yue Xing +6
Recent studies have identified "retrieval heads" in Large Language Models (LLMs) responsible for extracting information from input contexts. However, prior works largely rely on st…
Accurate Table Question Answering with Accessible LLMs
Yangfan Jiang, Fei Wei, Ergute Bao +4
Given a table T in a database and a question Q in natural language, the table question answering (TQA) task aims to return an accurate answer to Q based on the content of T. Recent…
Leveraging LLM-based agents for social science research: insights from citation network simulations
Jiarui Ji, Runlin Lei, Xuchen Pan +8
The emergence of Large Language Models (LLMs) demonstrates their potential to encapsulate the logic and patterns inherent in human behavior simulation by leveraging extensive web d…
Grounded in Reality: Learning and Deploying Proactive LLM from Offline Logs
Fei Wei, Daoyuan Chen, Ce Wang +5
Large Language Models (LLMs) excel as passive responders, but teaching them to be proactive, goal-oriented partners, a critical capability in high-stakes domains, remains a major c…
BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuning
Qianli Shen, Daoyuan Chen, Yilun Huang +4
Reinforcement finetuning (RFT) is a key technique for aligning Large Language Models (LLMs) with human preferences and enhancing reasoning, yet its effectiveness is highly sensitiv…
Branch-and-Browse: Efficient and Controllable Web Exploration with Tree-Structured Reasoning and Action Memory
Shiqi He, Yue Cui, Xinyu Ma +3
Autonomous web agents powered by large language models (LLMs) show strong potential for performing goal-oriented tasks such as information retrieval, report generation, and online…
Group-Relative REINFORCE Is Secretly an Off-Policy Algorithm: Demystifying Some Myths About GRPO and Its Friends
Chaorui Yao, Yanxi Chen, Yuchang Sun +5
Off-policy reinforcement learning (RL) for large language models (LLMs) is attracting growing interest, driven by practical constraints in real-world applications, the complexity o…
AgentScope 1.0: A Developer-Centric Framework for Building Agentic Applications
Dawei Gao, Zitao Li, Yuexiang Xie +20
Driven by rapid advancements of Large Language Models (LLMs), agents are empowered to combine intrinsic knowledge with dynamic tool use, greatly enhancing their capacity to address…
On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weighting
Wenhao Zhang, Yuexiang Xie, Yuchang Sun +5
Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) are two prominent post-training paradigms for refining the capabilities and aligning the behavior of Large Language Mod…
ReDiSC: A Reparameterized Masked Diffusion Model for Scalable Node Classification with Structured Predictions
Yule Li, Yifeng Lu, Zhen Wang +3
In recent years, graph neural networks (GNN) have achieved unprecedented successes in node classification tasks. Although GNNs inherently encode specific inductive biases (e.g., ac…
Respecting Temporal-Causal Consistency: Entity-Event Knowledge Graphs for Retrieval-Augmented Generation
Ze Yu Zhang, Zitao Li, Yaliang Li +2
Retrieval-augmented generation (RAG) based on large language models often falters on narrative documents with inherent temporal structures. Standard unstructured RAG methods rely s…
PC-MoE: Memory-Efficient and Privacy-Preserving Collaborative Training for Mixture-of-Experts LLMs
Ze Yu Zhang, Bolin Ding, Bryan Kian Hsiang Low
Mixture-of-Experts (MoE) has been gaining popularity due to its successful adaptation to large language models (LLMs). In this work, we introduce Privacy-preserving Collaborative M…
Enhancing Tool Learning in Large Language Models with Hierarchical Error Checklists
Yue Cui, Liuyi Yao, Shuchang Tao +4
Large language models (LLMs) have significantly advanced natural language processing, particularly through the integration of external tools and APIs. However, their effectiveness…
Efficient Leave-one-out Approximation in LLM Multi-agent Debate Based on Introspection
Yue Cui, Liuyi Yao, Zitao Li +3
Multi-agent systems based on large language models (LLMs) advance automatic task completion in various fields, where debate is a common cooperation form for agents to solve complic…
Enhancing Latent Computation in Transformers with Latent Tokens
Yuchang Sun, Yanxi Chen, Yaliang Li +1
Augmenting large language models (LLMs) with auxiliary tokens has emerged as a promising strategy for enhancing model performance. In this work, we introduce a lightweight method t…
Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement Fine-Tuning of Large Language Models
Xuchen Pan, Yanxi Chen, Yushuo Chen +11
Trinity-RFT is a general-purpose, unified and easy-to-use framework designed for reinforcement fine-tuning (RFT) of large language models. It is built with a modular and decoupled…
Automated Profile Inference with Language Model Agents
Yuntao Du, Zitao Li, Bolin Ding +4
Impressive progress has been made in automated problem-solving by the collaboration of large language model (LLM) based agents. However, these automated capabilities also open aven…
Tree-based Models for Vertical Federated Learning: A Survey
Bingchen Qian, Yuexiang Xie, Yaliang Li +2
Tree-based models have achieved great success in a wide range of real-world applications due to their effectiveness, robustness, and interpretability, which inspired people to appl…