10 citations · 10 across the 9 of their papers we have counts for
10 papers
The Functional Structure of Post-Compression Recovery in Low-Rank LLMs
Zishan Shao, Liang Tian, Georgiy Zemlevskiy +10
Different low-rank compression methods can produce compressed LLMs that respond differently to the same post-compression recovery procedure, and relative advantages observed betwee…
LowRankArena: A Standardized Evaluation Platform for SVD-Based LLM Compression
Zishan Shao, Lixun Zhang, Kangning Cui +10
SVD-based low-rank compression has become a fast-growing direction for reducing the memory and computational cost of large language models (LLMs). However, meaningful comparison ac…
DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions
Zishan Shao, Lixun Zhang, Kangning Cui +10
Large language models (LLMs) handle many tasks with one set of parameters, but under KV-cached inference it is unclear what task-general structure, if any, is used at decode time r…
Multi-Stream Perturbation Attack: Breaking Safety Alignment of Thinking LLMs Through Concurrent Task Interference
Fan Yang
The widespread adoption of thinking mode in large language models (LLMs) has significantly enhanced complex task processing capabilities while introducing new security risks. When…
The Curse and Blessing of Mean Bias in FP4-Quantized LLM Training
Hengjie Cao, Zhendong Huang, Mengyi Chen +15
FP4 training promises substantial memory and compute savings for large language models, but remains fragile because blockwise quantization is dictated by extreme activation magnitu…
Spectra: Rethinking Optimizers for LLMs Under Spectral Anisotropy
Zhendong Huang, Hengjie Cao, Fang Dong +14
Gradient signals in LLM training are highly anisotropic: recurrent linguistic structure concentrates energy into a small set of dominant spectral directions, while context specific…
Metis: Training LLMs with FP4 Quantization
Hengjie Cao, Mengyi Chen, Yifeng Yang +13
This work identifies anisotropy in the singular value spectra of parameters, activations, and gradients as the fundamental barrier to low-bit training of large language models (LLM…
Denoising Reuse: Exploiting Inter-frame Motion Consistency for Efficient Video Latent Generation
Chenyu Wang, Shuo Yan, Yixuan Chen +10
Video generation using diffusion-based models is constrained by high computational costs due to the frame-wise iterative diffusion process. This work presents a Diffusion Reuse MOt…
Train Faster, Perform Better: Modular Adaptive Training in Over-Parameterized Models
Yubin Shi, Yixuan Chen, Mingzhi Dong +10
Despite their prevalence in deep-learning communities, over-parameterized models convey high demands of computational costs for proper training. This work studies the fine-grained,…
Can Large Language Models Be Good Companions? An LLM-Based Eyewear System with Conversational Common Ground
Zhenyu Xu, Hailin Xu, Zhouyang Lu +11
Developing chatbots as personal companions has long been a goal of artificial intelligence researchers. Recent advances in Large Language Models (LLMs) have delivered a practical s…