11 citations · 20 across the 13 of their papers we have counts for
4 papers · 1 filter
Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction Tuning
Ted Zadouri, Ahmet Üstün, Arash Ahmadian +3
The Mixture of Experts (MoE) is a widely known neural architecture where an ensemble of specialized sub-models optimizes overall performance with a constant computational cost. How…
-models: Effective Decision-Aware Reinforcement Learning with Latent Models
Claas A Voelcker, Arash Ahmadian, Romina Abachi +2
The idea of decision-aware model learning, that models should be accurate where it matters for decision-making, has gained prominence in model-based reinforcement learning. While p…
Intriguing Properties of Quantization at Scale
Arash Ahmadian, Saurabh Dash, Hongyu Chen +5
Emergent properties have been widely adopted as a term to describe behavior not present in smaller models but observed in larger models. Recent work suggests that the trade-off inc…
Pseudo-Inverted Bottleneck Convolution for DARTS Search Space
Arash Ahmadian, Louis S. P. Liu, Yue Fei +2
Differentiable Architecture Search (DARTS) has attracted considerable attention as a gradient-based neural architecture search method. Since the introduction of DARTS, there has be…