8 papers
Super-Tuning: From Activation-Aware Pruning to Sparse Fine-Tuning
Ivan Ilin, Philip Zmushko, Peter Richtárik
Large language models (LLMs) remain expensive to fine-tune because full-parameter updates require substantial memory, compute, and per-task storage. We study whether saliency signa…
URSA: Chemistry-Aware Benchmark for Utilitarian Retrosynthesis Assessment
Bogdan Zagribelnyy, Ivan Ilin, Nikita Bondarev +7
Synthesis planning aiming to find pathways of reactions for a target molecule is one of the most important and challenging tasks in drug discovery. Recent progress has produced bot…
When Single Answer Is Not Enough: Rethinking Single-Step Retrosynthesis Benchmarks for LLMs
Bogdan Zagribelnyy, Ivan Ilin, Maksim Kuznetsov +10
Recent progress has expanded the use of large language models (LLMs) in drug discovery, including synthesis planning. However, objective evaluation of retrosynthesis performance re…
MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery
Maksim Kuznetsov, Zulfat Miftahutdinov, Rim Shayakhmetov +17
General-purpose large language models (LLMs) that rely on in-context learning do not reliably deliver the scientific understanding and performance required for drug discovery tasks…
Thanos: A Block-wise Pruning Algorithm for Efficient Large Language Model Compression
Ivan Ilin, Peter Richtarik
This paper presents Thanos, a novel weight-pruning algorithm designed to reduce the memory footprint and enhance the computational efficiency of large language models (LLMs) by rem…
Hessian of Perplexity for Large Language Models by PyTorch autograd (Open Source)
Ivan Ilin
Computing the full Hessian matrix -- the matrix of second-order derivatives for an entire Large Language Model (LLM) is infeasible due to its sheer size. In this technical report,…