10 citations · 10 across the 9 of their papers we have counts for
6 papers · 1 filter
The Functional Structure of Post-Compression Recovery in Low-Rank LLMs
Zishan Shao, Liang Tian, Georgiy Zemlevskiy +10
Different low-rank compression methods can produce compressed LLMs that respond differently to the same post-compression recovery procedure, and relative advantages observed betwee…
LowRankArena: A Standardized Evaluation Platform for SVD-Based LLM Compression
Zishan Shao, Lixun Zhang, Kangning Cui +10
SVD-based low-rank compression has become a fast-growing direction for reducing the memory and computational cost of large language models (LLMs). However, meaningful comparison ac…
DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions
Zishan Shao, Lixun Zhang, Kangning Cui +10
Large language models (LLMs) handle many tasks with one set of parameters, but under KV-cached inference it is unclear what task-general structure, if any, is used at decode time r…
Multi-Stream Perturbation Attack: Breaking Safety Alignment of Thinking LLMs Through Concurrent Task Interference
Fan Yang
The widespread adoption of thinking mode in large language models (LLMs) has significantly enhanced complex task processing capabilities while introducing new security risks. When…
The Curse and Blessing of Mean Bias in FP4-Quantized LLM Training
Hengjie Cao, Zhendong Huang, Mengyi Chen +15
FP4 training promises substantial memory and compute savings for large language models, but remains fragile because blockwise quantization is dictated by extreme activation magnitu…
Spectra: Rethinking Optimizers for LLMs Under Spectral Anisotropy
Zhendong Huang, Hengjie Cao, Fang Dong +14
Gradient signals in LLM training are highly anisotropic: recurrent linguistic structure concentrates energy into a small set of dominant spectral directions, while context specific…