6 papers
FlattenGPT: Depth Compression for Transformer with Layer Flattening
Ruihan Xu, Qingpei Guo, Yao Zhu +3
Recent works have indicated redundancy across transformer blocks, prompting the research of depth compression to prune less crucial blocks. However, current ways of entire-block pr…
Focus-Scan-Refine: From Human Visual Perception to Efficient Visual Token Pruning
Enwei Tong, Yuanchao Bai, Yao Zhu +2
Vision-language models (VLMs) often generate massive visual tokens that greatly increase inference latency and memory footprint; while training-free token pruning offers a practica…
RAPTOR: Ridge-Adaptive Logistic Probes
Ziqi Gao, Yaotian Zhu, Qingcheng Zeng +4
Probing studies what information is encoded in a frozen LLM's layer representations by training a lightweight predictor on top of them. Beyond analysis, probes are often used opera…
Generation then Reconstruction: Accelerating Masked Autoregressive Models via Two-Stage Sampling
Feihong Yan, Peiru Wang, Yao Zhu +4
Masked Autoregressive (MAR) models promise better efficiency in visual generation than autoregressive (AR) models for the ability of parallel generation, yet their acceleration pot…
FlyLoRA: Boosting Task Decoupling and Parameter Efficiency via Implicit Rank-Wise Mixture-of-Experts
Heming Zou, Yunliang Zang, Wutong Xu +2
Low-Rank Adaptation (LoRA) is a widely used parameter-efficient fine-tuning method for foundation models, but it suffers from parameter interference, resulting in suboptimal perfor…
Kelp: A Streaming Safeguard for Large Models via Latent Dynamics-Guided Risk Detection
Xiaodan Li, Mengjie Wu, Yao Zhu +5
Large models (LMs) are powerful content generators, yet their open-ended nature can also introduce potential risks, such as generating harmful or biased content. Existing guardrail…