1 citations · 1 across the 8 of their papers we have counts for
4 papers · 1 filter
CoCurve: Cross-Module Co-Pruning Curvature for Training-Free Structured LLM Pruning
Zhiren Gong, Zihao Zeng, Zijie Wang +3
Structured pruning compresses large language models (LLMs) by removing whole computational units, such as attention heads and feed-forward (FFN) channel groups. Most training-free…
Conditional Co-Ablation: Recovering Self-Repair Backups in Transformer Circuits
Zhiren Gong, He Lu, Zihao Zeng +6
Mechanistic interpretability seeks to explain transformer behavior through circuits: sets of internal components that causally support a behavior. However, self-repair creates a bl…
Which Data Attributes Stimulate Math and Code Reasoning? An Investigation via Influence Functions
Siqi Kou, Qingyuan Tian, Hanwen Xu +2
Large language models (LLMs) have demonstrated remarkable reasoning capabilities in math and coding, often bolstered by post-training on the chain-of-thoughts (CoTs) generated by s…
MatryoshkaKV: Adaptive KV Compression via Trainable Orthogonal Projection
Bokai Lin, Zihao Zeng, Zipeng Xiao +5
KV cache has become a de facto technique for the inference of large language models (LLMs), where tensors of shape (layer number, head number, sequence length, feature dimension) a…