14 papers
The Functional Structure of Post-Compression Recovery in Low-Rank LLMs
Zishan Shao, Liang Tian, Georgiy Zemlevskiy +10
Different low-rank compression methods can produce compressed LLMs that respond differently to the same post-compression recovery procedure, and relative advantages observed betwee…
Preserving Mathematical Reasoning in Compressed Diffusion Language Models via Trajectory-Aware Low-Rank Approximation
Tian Liang, Zishan Shao, Yiran Chen
Diffusion language model (dLLM) compression faces a known challenge because calibration is typically performed on clean, fully visible activations, whereas inference traverses part…
Joint Branch-Space Transform Coding for Diffusion Activation Quantization with Classifier-Free Guidance
Mingrun Jiang, Yuejia Liu, Zishan Shao +11
Post-training quantization for diffusion models increasingly exploits timestep, feature, and layer structure. While recent work has begun incorporating CFG structure into diffusion…
LowRankArena: A Standardized Evaluation Platform for SVD-Based LLM Compression
Zishan Shao, Lixun Zhang, Kangning Cui +10
SVD-based low-rank compression has become a fast-growing direction for reducing the memory and computational cost of large language models (LLMs). However, meaningful comparison ac…
DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions
Zishan Shao, Lixun Zhang, Kangning Cui +10
Large language models (LLMs) handle many tasks with one set of parameters, but under KV-cached inference it is unclear what task-general structure, if any, is used at decode time r…
ELDOR: A Dataset and Benchmark for Illegal Gold Mining in the Amazon Rainforest
Kangning Cui, Surendra Bohara, Suraj Prasai +12
Illegal gold mining in the Amazon rainforest causes deforestation, water contamination, and long-term ecosystem disruption, yet remains difficult to monitor at fine spatial scales.…
FlashSVD v1.5: Making Low-Rank Transformers Inference Actually Fast
Wenhao Wu, Zishan Shao, Kangning Cui +5
SVD-based Low-rank compression reduces transformer parameters and nominal FLOPs, but these savings often translate poorly into real LLM serving speedups. We show that this gap is l…
ZEUS: Accelerating Diffusion Models with Only Second-Order Predictor
Yixiao Wang, Ting Jiang, Zishan Shao +6
Denoising generative models deliver high-fidelity generation but remain bottlenecked by inference latency due to the many iterative denoiser calls required during sampling. Trainin…
T2S-Bench & Structure-of-Thought: Benchmarking and Prompting Comprehensive Text-to-Structure Reasoning
Qinsi Wang, Hancheng Ye, Jinhee Kim +12
Think about how human handles complex reading tasks: marking key points, inferring their relationships, and structuring information to guide understanding and responses. Likewise,…
Enhanced Cyclic Coordinate Descent Methods for Elastic Net Penalized Linear Models
Yixiao Wang, Zishan Shao, Ting Jiang +1
We present a novel enhanced cyclic coordinate descent (ECCD) framework for solving generalized linear models with elastic net constraints that reduces training time in comparison t…
FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models
Zishan Shao, Yixiao Wang, Qinsi Wang +6
Singular Value Decomposition (SVD) has recently seen a surge of interest as a simple yet powerful tool for large language models (LLMs) compression, with a growing number of works…
SADA: Stability-guided Adaptive Diffusion Acceleration
Ting Jiang, Yixiao Wang, Hancheng Ye +7
Diffusion models have achieved remarkable success in generative tasks but suffer from high computational costs due to their iterative sampling process and quadratic attention costs…
Scalable Dual Coordinate Descent for Kernel Methods
Zishan Shao, Aditya Devarakonda
Dual Coordinate Descent (DCD) and Block Dual Coordinate Descent (BDCD) are important iterative methods for solving convex optimization problems. In this work, we develop scalable D…
PalmProbNet: A Probabilistic Approach to Understanding Palm Distributions in Ecuadorian Tropical Forest via Transfer Learning
Kangning Cui, Zishan Shao, Gregory Larsen +8
Palms play an outsized role in tropical forests and are important resources for humans and wildlife. A central question in tropical ecosystems is understanding palm distribution an…