4 papers
Exploring the Hidden Capacity of LLMs for One-Step Text Generation
Gleb Mezentsev, Ivan Oseledets
A recent study showed that large language models (LLMs) can reconstruct surprisingly long texts - up to thousands of tokens - via autoregressive generation from just one trained in…
Scalable Cross-Entropy Loss for Sequential Recommendations with Large Item Catalogs
Gleb Mezentsev, Danil Gusak, Ivan Oseledets +1
Scalability issue plays a crucial role in productionizing modern recommender systems. Even lightweight architectures may suffer from high computational overload due to intermediate…
SparseGrad: A Selective Method for Efficient Fine-tuning of MLP Layers
Viktoriia Chekalina, Anna Rudenko, Gleb Mezentsev +3
The performance of Transformer models has been enhanced by increasing the number of parameters and the length of the processed text. Consequently, fine-tuning the entire model beco…
RECE: Reduced Cross-Entropy Loss for Large-Catalogue Sequential Recommenders
Danil Gusak, Gleb Mezentsev, Ivan Oseledets +1
Scalability is a major challenge in modern recommender systems. In sequential recommendations, full Cross-Entropy (CE) loss achieves state-of-the-art recommendation quality but con…