5 papers
Structure Aware Neural Architecture Search for Mixture of Experts
Petr Babkin, Oleg Bakhteev
Neural Architecture Search (NAS) has so far rarely been applied to Mixture-of-Experts (MoE) models, and existing MoE designs leave the alignment between experts and the structure o…
Surrogate assisted diversity estimation in neural ensemble search
Alexandr Udeneev, Petr Babkin, Oleg Bakhteev
Ensembles are a standard way to improve the performance and robustness of deep neural networks, but their effectiveness crucially depends on both the quality and the diversity of i…
Markovian Compression: Looking to the Past Helps Accelerate the Future
Andrey Veprikov, Vladimir Solodkin, Mikhail Rudakov +2
This paper deals with distributed optimization problems that use compressed communication to achieve efficient performance and mitigate communication bottleneck. We propose a famil…
Perturb Your Data: Paraphrase-Guided Training Data Watermarking
Pranav Shetty, Mirazul Haque, Petr Babkin +3
Training data detection is critical for enforcing copyright and data licensing, as Large Language Models (LLM) are trained on massive text corpora scraped from the internet. We pre…
Towards Effectively Leveraging Execution Traces for Program Repair with Code LLMs
Mirazul Haque, Petr Babkin, Farima Farmahinifarahani +1
Large Language Models (LLMs) show promising performance on various programming tasks, including Automatic Program Repair (APR). However, most approaches to LLM-based APR are limite…