4 papers
REAM: Merging Improves Pruning of Experts in LLMs
Saurav Jha, Maryam Hashemzadeh, Ali Saheb Pasand +3
Mixture-of-Experts (MoE) large language models (LLMs) are among the top-performing architectures. The largest models, often with hundreds of billions of parameters, pose significan…
Celo2: Towards Learned Optimization Free Lunch
Abhinav Moudgil, Boris Knyazev, Eugene Belilovsky
Learned optimizers are powerful alternatives to hand-designed update rules like Adam, yet they have seen limited practical adoption since they often fail to meta-generalize beyond…
M^4olGen: Multi-Agent, Multi-Stage Molecular Generation under Precise Multi-Property Constraints
Yizhan Li, Florence Cloutier, Sifan Wu +5
Generating molecules that satisfy precise numeric constraints over multiple physicochemical properties is critical and challenging. Although large language models (LLMs) are expres…
(Almost) Free Modality Stitching of Foundation Models
Jaisidh Singh, Diganta Misra, Boris Knyazev +1
Foundation multi-modal models are often designed by stitching of multiple existing pretrained uni-modal models: for example, an image classifier with an text model. This stitching…