1 citations · 1 across the 8 of their papers we have counts for
6 papers · 1 filter
AutoResearch-RL: Perpetual Self-Evaluating Reinforcement Learning Agents for Autonomous Neural Architecture Discovery
Nilesh Jain, Rohit Yadav, Sagar Kotian +1
We present AutoResearch-RL, a framework in which a reinforcement learning agent conducts open-ended neural architecture and hyperparameter research without human supervision, runni…
Mamba-Shedder: Post-Transformer Compression for Efficient Selective Structured State Space Models
J. Pablo Muñoz, Jinjie Yuan, Nilesh Jain
Large pre-trained models have achieved outstanding results in sequence modeling. The Transformer block and its attention mechanism have been the main drivers of the success of thes…
Low-Rank Adapters Meet Neural Architecture Search for LLM Compression
J. Pablo Muñoz, Jinjie Yuan, Nilesh Jain
The rapid expansion of Large Language Models (LLMs) has posed significant challenges regarding the computational resources required for fine-tuning and deployment. Recent advanceme…
MultiPruner: Balanced Structure Removal in Foundation Models
J. Pablo Muñoz, Jinjie Yuan, Nilesh Jain
Recently, state-of-the-art approaches for pruning large pre-trained models (LPMs) have demonstrated that the training-free removal of non-critical residual blocks in Transformers i…
SQFT: Low-cost Model Adaptation in Low-precision Sparse Foundation Models
Juan Pablo Muñoz, Jinjie Yuan, Nilesh Jain
Large pre-trained models (LPMs), such as large language models, have become ubiquitous and are employed in many applications. These models are often adapted to a desired domain or…
Shears: Unstructured Sparsity with Neural Low-rank Adapter Search
J. Pablo Muñoz, Jinjie Yuan, Nilesh Jain
Recently, several approaches successfully demonstrated that weight-sharing Neural Architecture Search (NAS) can effectively explore a search space of elastic low-rank adapters (LoR…