19 citations · 26 across the 11 of their papers we have counts for
4 papers · 1 filter
EBGAN-MDN: An Energy-Based Adversarial Framework for Multi-Modal Behavior Cloning
Yixiao Li, Julia Barth, Thomas Kiefer +1
Multi-modal behavior cloning faces significant challenges due to mode averaging and mode collapse, where traditional models fail to capture diverse input-output mappings. This prob…
Finding Fantastic Experts in MoEs: A Unified Study for Expert Dropping Strategies and Observations
Ajay Jaiswal, Jianyu Wang, Yixiao Li +6
Sparsely activated Mixture-of-Experts (SMoE) has shown promise in scaling up the learning capacity of neural networks. However, vanilla SMoEs have issues such as expert redundancy…
NoWag: A Unified Framework for Shape Preserving Compression of Large Language Models
Lawrence Liu, Inesh Chakrabarti, Yixiao Li +3
Large language models (LLMs) exhibit remarkable performance across various natural language processing tasks but suffer from immense computational and memory demands, limiting thei…
IDEA Prune: An Integrated Enlarge-and-Prune Pipeline in Generative Language Model Pretraining
Yixiao Li, Xianzhi Du, Ajay Jaiswal +4
Recent advancements in large language models have intensified the need for efficient and deployable models within limited inference budgets. Structured pruning pipelines have shown…