3 papers
cs.LG2026
WinQ: Accelerating Quantization-Aware Training of Language Models Around Saddle Points
Dongyue Li, Zechun Liu, Kai Yi +6
Quantization-aware training (QAT) is widely adopted to quantize language models by training full-precision weights using gradients from the quantized model. The main bottleneck is…
cs.LG2026
Attention Drift: What Autoregressive Speculative Decoding Models Learn
Doğaç Eldenk, Payal Mohapatra, Yigitcan Comlek +3
Speculative decoding accelerates LLM inference by drafting future tokens with a small model, but drafter models degrade sharply under template perturbation and long-context inputs.…
cs.CL2023
Gradient-Based Word Substitution for Obstinate Adversarial Examples Generation in Language Models
Yimu Wang, Peng Shi, Hongyang Zhang
In this paper, we study the problem of generating obstinate (over-stability) adversarial examples by word substitution in NLP, where input text is meaningfully changed but the mode…