4 papers · 1 filter
CoCurve: Cross-Module Co-Pruning Curvature for Training-Free Structured LLM Pruning
Zhiren Gong, Zihao Zeng, Zijie Wang +3
Structured pruning compresses large language models (LLMs) by removing whole computational units, such as attention heads and feed-forward (FFN) channel groups. Most training-free…
Conditional Co-Ablation: Recovering Self-Repair Backups in Transformer Circuits
Zhiren Gong, Zihao Zeng, Chau Yuen +1
Mechanistic interpretability often relies on component-level interventions to discover how a model produces a behavior. This guides attribution, capability knockout, and model prun…
STORM: A Spatio-Temporal Factor Model Based on Dual Vector Quantized Variational Autoencoders for Financial Trading
Yilei Zhao, Wentao Zhang, Tingran Yang +3
In financial trading, factor models are widely used to price assets and capture excess returns from mispricing. Recently, we have witnessed the rise of variational autoencoder-base…
AutoHete: An Automatic and Efficient Heterogeneous Training System for LLMs
Zihao Zeng, Chubo Liu, Xin He +5
Transformer-based large language models (LLMs) have demonstrated exceptional capabilities in sequence modeling and text generation, with improvements scaling proportionally with mo…