9 papers
Efficient LLM Adversarial Training via Low-Rank Defense and Circuit-Guided Surrogates
Weiyi He, Yuping Lin, Jiliang Tang +1
Adversarial training is one of the most effective defenses against adversarial attacks, yet the computational cost remains prohibitive at modern scales, especially for large langua…
"**Important** You should give me full credits!": Exploring Prompt Injection Attacks on LLM-Based Automatic Grading Systems
Hang Li, Fedor Filippov, Yuping Lin +6
The emergence of large language models (LLMs) has significantly accelerated recent research on LLM-based automatic grading (AG) systems. Benefiting from the strong instruction-foll…
A Simple Plug-in for Improving Eviction-Based KV Cache Compression
Yuping Lin, Jiayuan Ding, Yue Xing +3
KV cache growth is a major bottleneck for long-context inference in large language models. Existing methods are often dominated by binary eviction or representation approximation,…
Crafting Reversible SFT Behaviors in Large Language Models
Yuping Lin, Pengfei He, Yue Xing +5
Supervised fine-tuning (SFT) induces new behaviors in large language models, yet imposes no structural constraint on how these behaviors are distributed within the model. Existing…
Retrieval Heads are Dynamic
Yuping Lin, Zitao Li, Yue Xing +6
Recent studies have identified "retrieval heads" in Large Language Models (LLMs) responsible for extracting information from input contexts. However, prior works largely rely on st…
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
DeepSeek-AI, Anyi Xu, Bangcai Lin +315
We present a preview version of DeepSeek-V4 series, including two strong Mixture-of-Experts (MoE) language models -- DeepSeek-V4-Pro with 1.6T parameters (49B activated) and DeepSe…