1 citations · 1 across the 4 of their papers we have counts for
5 papers
SLEM: Spectrum-Aware Latent Expert Merging
Dohyeon Kim, Bedionita Soro, Bruno Andreis +1
Mixture-of-Experts (MoE) models scale model capacity efficiently by activating only a small subset of experts for each input, while still requiring all parameters to be stored in m…
Distribution-Consistent Inference for Dynamic Sparse Mixture-of-Experts
Dohyeon Kim, Bedionita Soro, Sung Ju Hwang
Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling model capacity while preserving efficient inference in large foundation models. However, most…
Instruction-Guided Autoregressive Neural Network Parameter Generation
Soro Bedionita, Bruno Andreis, Song Chong +1
Learning to generate neural network parameters conditioned on task descriptions and architecture specifications is pivotal for advancing model adaptability and transfer learning. E…
Diffusion-Based Neural Network Weights Generation
Bedionita Soro, Bruno Andreis, Hayeon Lee +4
Transfer learning has gained significant attention in recent deep learning research due to its ability to accelerate convergence and enhance performance on new tasks. However, its…
Set-based Neural Network Encoding Without Weight Tying
Bruno Andreis, Soro Bedionita, Philip H. S. Torr +1
We propose a neural network weight encoding method for network property prediction that utilizes set-to-set and set-to-vector functions to efficiently encode neural network paramet…