4 citations · 4 across the 12 of their papers we have counts for
12 papers
Dynamic Linear Attention
Xin Wang, Hui Shen, Boyuan Zheng +7
The scalability of Large Language Models (LLMs) to long contexts is fundamentally constrained by the quadratic complexity of standard attention, motivating the adoption of linear a…
How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMs
Zhichen Dong, Yang Li, Yuhan Sun +9
Token-level credit assignment remains a key obstacle for reinforcement learning (RL) in large language models (LLMs), where RL recipes typically treat all tokens equally, failing t…
PoC: Performance-oriented Context Compression for Large Language Models via Performance Prediction
Runsong Zhao, Shilei Liu, Jiwei Tang +8
While context compression can mitigate the growing inference costs of Large Language Models (LLMs) by shortening contexts, existing methods that specify a target compression ratio…
COMI: Coarse-to-fine Context Compression via Marginal Information Gain
Jiwei Tang, Shilei Liu, Zhicheng Zhang +4
Large Language Models (LLMs) have demonstrated exceptional capabilities across diverse tasks. However, their deployment in long context scenarios remains hindered by computational…
Read As Human: Compressing Context via Parallelizable Close Reading and Skimming
Jiwei Tang, Shilei Liu, Zhicheng Zhang +9
Large Language Models (LLMs) demonstrate exceptional capability across diverse tasks. However, their deployment in long-context scenarios is hindered by two challenges: computation…
PretrainRL: Alleviating Factuality Hallucination of Large Language Models at the Beginning
Langming Liu, Kangtao Lv, Haibin Chen +8
Large language models (LLMs), despite their powerful capabilities, suffer from factual hallucinations where they generate verifiable falsehoods. We identify a root of this issue: t…
Data Distribution Matters: A Data-Centric Perspective on Context Compression for Large Language Model
Kangtao Lv, Jiwei Tang, Langming Liu +7
The deployment of Large Language Models (LLMs) in long-context scenarios is hindered by computational inefficiency and significant information redundancy. Although recent advanceme…
CoMeT: Collaborative Memory Transformer for Efficient Long Context Modeling
Runsong Zhao, Shilei Liu, Jiwei Tang +8
The quadratic complexity and indefinitely growing key-value (KV) cache of standard Transformers pose a major barrier to long-context processing. To overcome this, we introduce the…
How to inject knowledge efficiently? Knowledge Infusion Scaling Law for Pre-training Large Language Models
Kangtao Lv, Haibin Chen, Yujin Yuan +5
Large language models (LLMs) have attracted significant attention due to their impressive general capabilities across diverse downstream tasks. However, without domain-specific opt…
Multi-task Offline Reinforcement Learning for Online Advertising in Recommender Systems
Langming Liu, Wanyu Wang, Chi Zhang +6
Online advertising in recommendation platforms has gained significant attention, with a predominant focus on channel recommendation and budget allocation strategies. However, curre…
Weight Spectra Induced Efficient Model Adaptation
Chongjie Si, Xuankun Yang, Muqing Liu +5
Large-scale foundation models have demonstrated remarkable versatility across a wide range of downstream tasks. However, fully fine-tuning these models incurs prohibitive computati…
NAN: A Training-Free Solution to Coefficient Estimation in Model Merging
Chongjie Si, Kangtao Lv, Jingjing Jiang +6
Model merging offers a training-free alternative to multi-task learning by combining independently fine-tuned models into a unified one without access to raw data. However, existin…