13 papers
Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editing
Tianci Liu, Zihan Dong, Tianchun Li +8
Large language models (LLMs) achieve remarkable performance across natural language tasks, yet they are trained on static corpora and their knowledge quickly becomes outdated in a…
Personalization as Inverse Planning: Learning Latent Design Intents for Agentic Slide Generation via Structural Denoising
Tianci Liu, Zihan Dong, Linjun Zhang +5
Slide design requires personalizing both deck themes and page layouts. Yet, current AI agent-based methods struggle with fine-grained, page-level design. Solely relying on prespeci…
RIMS: Preference Optimization via Smoothed Multi-pair Aggregation for Small-Scale LLM Retrieval-Augmented Generation
Pei Tian, Zihan Dong, Tianci Liu +2
Small-scale language models (SLMs) are attractive for retrieval-augmented generation (RAG) in resource-constrained settings, but their limited capacity makes them highly sensitive…
Evidence Over Plans: Online Trajectory Verification for Skill Distillation
Yang Zhou, Zihan Dong, Zhenting Wang +7
Agent skills can remarkably improve task success rates by using human-written procedural documents, but their quality is difficult to assess without environment-grounded verificati…
RUBRIC-ARROW: Alternating Pointwise Rubric Reward Modeling for LLM Post-training in Non-verifiable Domains
Haoxiang Jiang, Zihan Dong, Tianci Liu +5
Pointwise reward modeling offers critical signals for LLM post-training, yet struggles with absolute scoring in subjective, non-verifiable settings. Rubric-based methods address th…
On Safety Risks in Experience-Driven Self-Evolving Agents
Weixiang Zhao, Yichen Zhang, Yingshuo Wang +8
Experience-driven self-evolution has emerged as a promising paradigm for improving the autonomy of large language model agents, yet its reliance on self-curated experience introduc…
Alternating Reinforcement Learning for Rubric-Based Reward Modeling in Non-Verifiable LLM Post-Training
Ran Xu, Tianci Liu, Zihan Dong +6
Standard reward models typically predict scalar scores that fail to capture the multifaceted nature of response quality in non-verifiable domains, such as creative writing or open-…
Labels or Preferences? Budget-Constrained Learning with Human Judgments over AI-Generated Outputs
Zihan Dong, Xiaotian Hou, Ruijia Wu +1
The increasing reliance on human preference feedback to judge AI-generated pseudo labels has created a pressing need for principled, budget-conscious data acquisition strategies. W…
Secret-Protected Evolution for Differentially Private Synthetic Text Generation
Tianze Wang, Zhaoyu Chen, Jian Du +3
Text data has become extremely valuable on large language models (LLMs) and even lead to general artificial intelligence (AGI). A lot of high-quality text in the real world is priv…
AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforced Self-Play
Ran Xu, Yuchen Zhuang, Zihan Dong +7
Search-augmented LLMs often struggle with complex reasoning tasks due to ineffective multi-hop retrieval and limited reasoning ability. We propose AceSearcher, a cooperative self-p…
RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimization
Tianci Liu, Haoxiang Jiang, Tianze Wang +5
Large language models (LLMs) have achieved impressive performance but face high computational costs and latency, limiting their deployment in resource-constrained settings. In cont…
MPO: An Efficient Post-Processing Framework for Mixing Diverse Preference Alignment
Tianze Wang, Dongnan Gui, Yifan Hu +2
Reinforcement Learning from Human Feedback (RLHF) has shown promise in aligning large language models (LLMs). Yet its reliance on a singular reward model often overlooks the divers…
Mitigating Heterogeneous Token Overfitting in LLM Knowledge Editing
Tianci Liu, Ruirui Li, Zihan Dong +6
Large language models (LLMs) have achieved remarkable performance on various natural language tasks. However, they are trained on static corpora and their knowledge can become outd…