1 citations · 1 across the 4 of their papers we have counts for
5 papers
SLEM: Spectrum-Aware Latent Expert Merging
Dohyeon Kim, Bedionita Soro, Bruno Andreis +1
Mixture-of-Experts (MoE) models scale model capacity efficiently by activating only a small subset of experts for each input, while still requiring all parameters to be stored in m…
Robust Molecular Property Prediction via Densifying Scarce Labeled Data
Jina Kim, Jeffrey Willette, Bruno Andreis +1
A widely recognized limitation of molecular prediction models is their reliance on structures observed in the training data, resulting in poor generalization to out-of-distribution…
Instruction-Guided Autoregressive Neural Network Parameter Generation
Soro Bedionita, Bruno Andreis, Song Chong +1
Learning to generate neural network parameters conditioned on task descriptions and architecture specifications is pivotal for advancing model adaptability and transfer learning. E…
Diffusion-Based Neural Network Weights Generation
Bedionita Soro, Bruno Andreis, Hayeon Lee +4
Transfer learning has gained significant attention in recent deep learning research due to its ability to accelerate convergence and enhance performance on new tasks. However, its…
Co-training and Co-distillation for Quality Improvement and Compression of Language Models
Hayeon Lee, Rui Hou, Jongpil Kim +4
Knowledge Distillation (KD) compresses computationally expensive pre-trained language models (PLMs) by transferring their knowledge to smaller models, allowing their use in resourc…