7 papers
KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing
Mufei Li, Shikun Liu, Dongqi Fu +5
Post-hoc context erasing over the KV cache is challenging because a local edit has a global consequence: once a span has been processed, its influence propagates into the cached st…
Implicit Turn-Wise Policy Optimization for Proactive User-LLM Interaction
Haoyu Wang, Yuxin Chen, Liang Luo +3
Multi-turn human-AI collaboration is fundamental to deploying interactive services such as adaptive tutoring, conversational recommendation, and professional consultation. However,…
Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluation
Mufei Li, Dongqi Fu, Limei Wang +10
Modern long-context large language models (LLMs) perform well on synthetic "needle-in-a-haystack" (NIAH) benchmarks, but such tests overlook how noisy contexts arise from biased re…
Struc-EMB: The Potential of Structure-Aware Encoding in Language Embeddings
Shikun Liu, Haoyu Wang, Mufei Li +1
Text embeddings from Large Language Models (LLMs) have become foundational for numerous applications. However, these models typically operate on raw text, overlooking the rich stru…
Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning
Eli Chien, Haoyu Wang, Ziang Chen +1
Machine unlearning has raised significant interest with the adoption of laws ensuring the ``right to be forgotten''. Researchers have provided a probabilistic notion of approximate…
Model Generalization on Text Attribute Graphs: Principles with Large Language Models
Haoyu Wang, Shikun Liu, Rongzhe Wei +1
Large language models (LLMs) have recently been introduced to graph learning, aiming to extend their zero-shot generalization success to tasks where labeled graph data is scarce. A…