4 papers · 1 filter
Understanding Sparse Attention Selectivity in Long-Context Foundation Models via Counterfactual Evaluation
Xingyu Ren, Youran Sun, Chugang Yi +1
Sparse attention is widely deployed in long-context serving stacks, yet no framework audits how discarding blocks changes the influence of specific content on model output. We firs…
PerspectiveGap: A Benchmark for Multi-Agent Orchestration Prompting
Youran Sun, Xingyu Ren, Kejia Zhang +2
Real-world LLM applications are moving beyond single-agent workflows toward orchestrated multi-agent systems, yet current models still struggle to determine what each sub-agent nee…
Correcting Mean Bias in Text Embeddings: A Refined Renormalization with Training-Free Improvements on MMTEB
Xingyu Ren, Youran Sun, Haoyu Liang
We find that current sentence-embedding models produce outputs with a consistent bias: every embedding decomposes as , where the mean is near-identical acro…
Decoupling Strategy and Execution in Task-Focused Dialogue via Goal-Oriented Preference Optimization
Jingyi Xu, Xingyu Ren, Zhoupeng Shou +2
Large language models show potential in task-oriented dialogue systems, yet existing training methods often rely on token-level likelihood or preference optimization, which poorly…