5 papers · 1 filter
MoRE: Mixture of Reused Experts
Eric S. Qiu, Utku Umur Acikalin, Justin Lovelace +4
Mixture-of-Experts (MoE) architectures decouple model capacity from computational cost, yet incur high memory footprints as parameters grow linearly with the number of experts. Rec…
Optimize Your Sampling: Tuned Diffusion Sampling with Bayesian Optimization
Travis Zhang, Christian Belardi, Justin Lovelace +4
Sampling from a diffusion model typically requires many forward passes through a large neural network, making generation computationally expensive. While much work has focused on e…
Adaptive Moments are Surprisingly Effective for Plug-and-Play Diffusion Sampling
Christian Belardi, Justin Lovelace, Kilian Q. Weinberger +1
Guided diffusion sampling relies on approximating often intractable likelihood scores, which introduces significant noise into the sampling dynamics. We propose using adaptive mome…
Learning from Synthetic Data Improves Multi-hop Reasoning
Anmol Kabra, Yilun Yin, Albert Gong +6
Reinforcement Learning (RL) has been shown to significantly boost reasoning capabilities of large language models (LLMs) in math, coding, and multi-hop reasoning tasks. However, RL…
PhantomWiki: On-Demand Datasets for Reasoning and Retrieval Evaluation
Albert Gong, Kamilė Stankevičiūtė, Chao Wan +6
High-quality benchmarks are essential for evaluating reasoning and retrieval capabilities of large language models (LLMs). However, curating datasets for this purpose is not a perm…