14 citations · 20 across the 4 of their papers we have counts for
4 papers
Mashup Learning: Faster Finetuning by Remixing Past Checkpoints
Sofia Maria Lo Cicero Vaina, Artem Chumachenko, Max Ryabinin
Finetuning on domain-specific data is a well-established method for enhancing LLM performance on downstream tasks. Training on each dataset produces a new set of model weights, res…
Distributed Inference and Fine-tuning of Large Language Models Over The Internet
Alexander Borzunov, Max Ryabinin, Artem Chumachenko +5
Large language models (LLMs) are useful in many NLP tasks and become more capable with size, with the best open-source models having over 50 billion parameters. However, using thes…
Petals: Collaborative Inference and Fine-tuning of Large Models
Alexander Borzunov, Dmitry Baranchuk, Tim Dettmers +5
Many NLP tasks benefit from using large language models (LLMs) that often have more than 100 billion parameters. With the release of BLOOM-176B and OPT-175B, everyone can download…
Weight Squeezing: Reparameterization for Knowledge Transfer and Model Compression
Artem Chumachenko, Daniil Gavrilov, Nikita Balagansky +1
In this work, we present a novel approach for simultaneous knowledge transfer and model compression called Weight Squeezing. With this method, we perform knowledge transfer from a…