3 papers
cs.LG2026
Scalable Kronecker-Fisher Approximation: Efficient Hessian Analysis for Billion-Parameter Language Models Compression
Viacheslav Yusupov, Daria Cherniuk, Evgeny Frolov
In this paper, we propose a scalable Kronecker-based approximation that captures cross-layer interactions without storing the entire Fisher matrix, enabling practical Hessian analy…
cs.CL2025
CoRoVA: Compressed Representations for Vector-Augmented Code Completion
Daria Cherniuk, Nikita Sukhorukov, Danil Gusak +4
Retrieval-augmented generation has emerged as one of the most effective approaches for code completion enhancement, especially when repository-level context is important. However,…
cs.LG2025
Generalized Fisher-Weighted SVD: Scalable Kronecker-Factored Fisher Approximation for Compressing Large Language Models
Viktoriia Chekalina, Daniil Moskovskiy, Tatiana Matveeva +2
The Fisher information is a fundamental concept for characterizing the sensitivity of parameters in neural networks. However, leveraging the full observed Fisher information is too…