1 citations · 1 across the 5 of their papers we have counts for
3 papers · 1 filter
Random Recursive Models
Jama Hussein Mohamud, Mirco Ravanelli
Recursive models create computational depth through parameter reuse, offering a parameter-efficient alternative to increasing model size. However, most recursive models repeatedly…
Adaptive Order Policies for Masked Diffusion
Jama Hussein Mohamud, Mohsin Hasan, Mirco Ravanelli +1
Masked diffusion models have seen great success in capturing data distributions over discrete sequences in domains such as text and proteins. These models generate data by iterativ…
Self-Routing: Parameter-Free Expert Routing from Hidden States
Jama Hussein Mohamud, Drew Wagner, Mirco Ravanelli
Mixture-of-Experts (MoE) layers increase model capacity by activating only a small subset of experts per token, and typically rely on a learned router to map hidden states to exper…